marquee background
Improve Click Through Rate Model Performance

Increase click through rate and improve model performance

There are a number of problems that can arise in your click through rate models after they are shipped into production. From untrained domains to bad input data, poor performance in CTR models can be challenging to resolve. ML observability helps bridge the gap between model performance issues and time to resolution to maximize CTR, increase click probability, and decrease cost per click.

Get Started →

Ad optimization via CTR

See how Arize can help you improve your CTR model performance in production by comparing model behavior across all model environments and versions, automatically surface model issues, and decrease time to root cause analysis with our feature performance heat map.

Download Notebook →

Proactively monitor new and unseen trends before they significantly impact model performance
Identify drifting features to actively improve models in production
Manage bad, corrupt, or missing key input data to easily retrain poor performing models
Get ML observability in minutes.

What is Click-Through Rate (CTR)

Click-through rate (CTR) is one measure used in online advertising to quantify the effectiveness of an advertising campaign. CTR is calculated by dividing the number of clicks by the number of impressions and expressing the result as a percentage. For example, if a company runs an advertising campaign that generates 5,000 impressions and 200 users click through to the advertised product or link, the CTR of that campaign would be 4%.

According to recent benchmarks from Google AdWords, the average CTR in AdWords across all industries is 3.17% on the search network and 0.45% in the display network. The search network is typically text based ads displayed on top of search results. The display network shows ads based on images as a user opens a page in the browser, for example a banner at the top or sides of a page.

The Ad Selection Process

  1. Taking the example of real-time online bidding for a page position, the process can be summarized with the following steps:
  2. A user visits a web page or search engine that has dedicated ad space
  3. A request is sent with information such as domain, position, matching keywords, etc. from the publisher to an ad exchange, which submits it to multiple advertisers (Demand-side Platforms, or DSPs).
  4. Advertisers that have a matching ad are entered in the auction by issuing a bid
  5. The auction takes place and the impression goes to the highest bidder
  6. The ad is served on the page
  7. The user clicks (click-through) the ad if they are interested

Challenges with CTR Models

  • Low Probabilistic Nature: Click-through ratios of only a few percent or less require a large sample size
  • New and unseen trends: CTR models are often trained on historical data on a specific set of websites along with other inputs. If an ad exchange starts sending bid requests for new popular websites that were not part of the original training data, an ad could end up getting placed on a page that does not align with the context of the ad, resulting in a lower CTR.
  • Time of day or seasonal fluctuations: Some search keywords or display ads may be more or less relevant during certain periods of the year, or less likely to get a click during the morning versus the afternoon. For example, users may be more likely to click on a search ad for a sushi restaurant during lunch or dinner hours than on their morning commute to work. Given this, tracking how a model performs or degrades over specific time periods is critical.
  • Bad, corrupt, or missing key input data: CTR models are often impacted by data quality issues – including missing data. If a model’s inputs include device information, the absence or corruption of this feature is likely to impact performance. More specifically, an ad with a “download app” call-to-action may be more relevant to someone on a mobile device and result in a low CTR if delivered to someone on a computer where it is less contextually relevant. Similarly, location information is particularly relevant to mobile advertisers when users are out and about, perhaps already shopping. An example of location targeting in mobile advertising would be the distance from a user to the nearest store. In these cases, unreliable location data may have a negative impact on CTR performance.

What Can Cause Low CTR?

There are several factors that can influence CTR including people’s behaviors, interests, market conditions, etc. Over the years Machine Learning systems have become more sophisticated in their ability to account for a wide range of features and effectively predict the probability of an ad’s relevance to a user. However as these systems are rolled out into production, new factors like changing market conditions, behaviors, and/or anomalies may lead to lower CTR performance than expected, for example:

  • New popular sites appear: An Ad Exchange is sending bid requests for sites that were not part of the original training data. An ad could end up getting placed on a page that does not align with the context of the ad, resulting in lower CTR.
  • Time/Season Fluctuation: Some keywords and associated ads may be more or less relevant during certain periods of the year such as holidays. Or some ads may have less likelihood of a click during different hours of the day, e.g. morning vs night. Tracking how the model performs or degrades over time is critical.
  • Bad/Corrupt Data: A movie ad might be more relevant to a tablet user than someone busy at work on their desktop. Receiving requests with no device information or some cryptic value is likely to affect the performance of a model that was trained with a complete dataset including device aspect ratios.
  • Change in Location Information: According to a study published by Verve Mobile, location based targeting can double CTR performance versus regular strategies. An example of location targeting in mobile advertising would be the distance from a user to the nearest store. This is particularly relevant to mobile advertising where users are out and about and perhaps already looking at shopping. Unreliable location data is certain to have a negative impact on CTR performance and an important metric to prioritize. Other examples of location targeting include be country, state, and zip code.

Importance of CTR

Click-through rate is a general indicator of how relevant users find an ad or webpage campaign to be. Advertisers work to attract people by designing eye-catching ads with carefully selected keywords to entice a user to take the next step and click on the ad presented. Online publishers have carefully placed dedicated website banners on their sites for advertisements. Publishers get paid per click (PPC) and therefore strive to maximize the number of clicks by presenting ads that are most relevant to the user. They may also choose to show premium ads if a user was identified with a high probability of clicking on an ad or particular category of ads.

Click-through rate also directly affects Quality Score and consequently ad pricing. Quality Score is a rating of the quality and relevance for keywords and pay-per-click (PPC) ads. For example, it is used to determine the cost per click (CPC) and factored-in to the maximum bid to determine the ad rank in an auction process. Online advertising platforms like Google Ads offer discounts for ads with high-Quality Scores. When auctioning ad space, decisions are not determined solely on how much a company is willing to pay for a click, but also the probability of a click-through. For example a $0.10 ad with a 10% CTR can generate significantly higher ad revenue than a $0.20 ad with a 2% CTR. Similarly, the higher the Quality Score for keywords associated with an ad, the less an advertiser pays for a given ad position. Publishers and advertisers both have an interest in maximizing CTR to maximize revenue.

Recommended Resources