Google participates in Stanford AdFraud 2007 workshop

Over the past several years, Google has encouraged academic research and participation in the area of click fraud detection. Our Ad Traffic Quality team is excited to continue this support and recently participated in the AdFraud 2007 workshop at Stanford University. The workshop was attended by more than one hundred participants from academia and industry, and provided an open forum for discussing the technical and social aspects of fighting click fraud.

Dr. Kourosh Gharachorloo, who leads Google's Ad Traffic Quality engineering team, presented the opening talk at the workshop. Kourosh's talk focused primarily on the economics of online advertising and click fraud. The talk presented two frameworks modeling advertiser spend against Google's incentives to help advertisers achieve better ROI. The first framework illustrates how Google's incentives are aligned with those of our advertisers - i.e. our click fraud detection techniques improve advertiser ROI, which then leads to increased advertiser success with AdWords. The second framework shows how low-quality sources of traffic in the Google Network directly reduce Google's revenue along with the revenue of our content partners. The combined frameworks demonstrate that Google has strong economic incentives to fight click fraud, in addition to the extremely important goal of earning and maintaining advertiser trust.

Kourosh's talk also included an overview of Google's approach to detecting invalid clicks. In addition, he described the limitations of the metrics used by the click fraud detection industry to evaluate the impact of click fraud. He concluded the talk by discussing the additional data that Google discloses to advertisers, which uniquely enables AdWords users to reconcile their web server logs with the statistics in their AdWords accounts.

The Ad Traffic Quality team will continue to collaborate with the academic community on research projects and events such as the Stanford AdFraud workshop. For the latest on what this team is up to, please visit the Ad Traffic Quality Resource Center where you'll find Kourosh's presentation [PDF] and a 70-minute video of his talk.

Introducing the Ad Traffic Quality Resource Center

Over the past couple of years, the Click Quality team has launched many new initiatives and shared a great deal of information about how they combat click fraud. To help keep track of their various presentations, blog posts, articles, and more, they've created a new website to serve as the single source for all click fraud and ad traffic quality related information. The new site is called the Ad Traffic Quality Resource Center and can be found at http://www.google.com/adtrafficquality.

If you're just starting to learn about click fraud -- what it is and what we're doing about it -- start with the Overview section. If you'd like more information, visit the Resource Center's Help Center for detailed FAQs and multimedia presentations. If, on the other hand, you follow the industry closely, you may want to check out Technical Talk which features in-depth articles and blog posts written by our engineering team and other experts in the field.

Click fraud remains an important issue to online advertisers (you may have noticed that Yahoo! has also launched a similar effort for their advertisers), so we hope you find the new website useful. Much of the information was created based on the questions and concerns heard from our advertisers, so let the team know what you think. The Ad Traffic Quality Resource Center is a work in progress and the team will continue to add information, including videos of events, presentations, and articles, as the content becomes available.

Finally, if you're interested in hearing more about click fraud directly from the experts and are attending the Search Engine Strategies conference in San Jose, CA, be sure to check out the Search Engines on Click Fraud panel taking place Tuesday, August 21st -- Shuman Ghosemajumder, Business Product Manager for Google Trust & Safety, will be speaking.

What’s new in the world of invalid clicks

AdWords is a product that’s constantly evolving. To help keep you up-to-date, we like to check-in with individual product teams to see what they’re up to. This time around, I got an update from the Click Quality team, a group of folks dedicated to protecting advertisers from invalid clicks and click fraud. Here’s what I learned:

There’s no “silver bullet” when combating invalid clicks and click fraud, so the team is continuously improving their techniques. For obvious reasons, the team couldn’t go into detail about every initiative they’re working on, but they were able to share some of their more recent efforts.

Recent product launches
In June, the team launched Internet Protocol (IP) address exclusion. IP address exclusion allows advertisers to exclude up to 20 IP addresses, or ranges of addresses, where advertisers don’t want their ads to appear. It's important to note that some large Internet Service Providers (ISPs) use IP address ranges for all their users. If you’re thinking of using IP exclusion, be sure you’ve carefully selected the IP addresses you'd like to exclude so you don't end up blocking a large amount of relevant and potentially profitable traffic.

The team also continues to develop new filters and signals (these are some of the top secret things they couldn't go into too much detail on) used to detect invalid clicks. Constantly updating and improving our systems enable them to stay ahead of some very sophisticated fraudsters.

Increased transparency
In addition to product enhancements and features, the Click Quality team has been working to provide more information about what Google does to detect invalid clicks and protect advertisers from click fraud. Earlier this year, the team disclosed the invalid click metrics across the Google network here on the Inside AdWords blog. That post was soon followed up by a post on how to request a click quality investigation and a paper on click fraud botnets.

In May, the Click Quality team held the first-ever Invalid Clicks Advertiser Forum at Google headquarters in Mountain View, CA. The forum was a half-day event offering presentations from Shuman Ghosemajumder, Business Product Manager for Trust & Safety, a Q&A session with the click quality engineering team, and a review of a few invalid clicks case studies presented by the click quality investigation team. The event was a great way for advertisers to express their concerns directly to the team and also get their questions answered by the team's experts. Future forums in different cities are in the planning stages, so stay tuned.

Shuman has also been racking up the frequent flier miles by speaking with advertisers at events in the US, Canada and Europe. So far this year, Shuman presented at Search Engine Strategies conferences in New York, Toronto, and London, as well as at Search Insider Summit (Bonita Springs, FL) and Search Marketing World (Dublin, Ireland). If you're planning on attending Search Engine Strategies San Jose in August, be sure to drop in on the "Search Engines On Click Fraud" session where Shuman is scheduled as a speaker.

Well, sounds like the team's been keeping themselves pretty busy these days. While reviewing some of the team's past efforts, I found a very helpful FAQ in the AdWords Help Center that lists some of the things advertisers can do to monitor or prevent invalid clicks on ads. Advertisers can also use the following tools and features to monitor and control the traffic they receive from their AdWords ads:
  1. Auto-tagging: This feature that allows AdWords advertisers to easily see how their keywords are performing from click to conversion. Auto-tagging is often used by Google Analytics users, but you do not have to use Google Analytics to use auto-tagging.

  2. Site exclusion: This tool can be used to prevent ads from appearing on certain Google content network websites that advertisers don't feel are appropriate for their ads. Advertisers can get an idea of where their ads are showing by running a Placement Performance report.

  3. Invalid clicks reporting: This advanced reporting option allows advertisers to review the number and percentage of clicks determined to be invalid and automatically filtered out by the AdWords system. This data is available in Campaign Performance and Account Performance reports by including the "Invalid Clicks" and "Invalid Clicks Rate" advanced options.
I hope this review of the Click Quality team's recent work has provided some insight into the efforts that go on everyday to improve and maintain the AdWords system. Stay tuned to the Inside AdWords blog for future AdWords product team updates.

A new case study on botnet-based click fraud

In the last year, you've probably read a lot about how Google not only manages the issue of click fraud, but also protects our advertisers against the many ways in which criminals attempt click fraud -- including clicking on ads themselves, hiring low-cost workers to click on ads all day, and "high-tech" approaches such as botnets (i.e. a collection of software robots, or bots, which run autonomously).

As part of our continuing efforts to provide greater transparency, Google's Click Quality and Security Teams have published a paper (available as a PDF) entitled "The Anatomy of Clickbot.A" for the HotBots 2007 workshop which took place earlier today in Boston, MA. Neil Daswani, a software engineer and contributing author of "The Anatomy of Clickbot.A," is here to tell us more:

Clickbot.A is the name of a botnet that Google's Click Quality and Security Teams investigated last year. Using our findings, we published "The Anatomy of Clickbot.A" - a detailed case study on botnet-based click fraud for the benefit of the technical research community.

Clickbot.A is an example of a botnet operator attempting a click fraud attack against syndicated search engines. Google was able to identify clicks on our advertisers' ads that exhibited Clickbot.A-like patterns and flagged them as invalid. While Clickbot.A is a specific example of a botnet application that conducted click fraud, botnets can also be used for keylogging, distributed denial of service (DDoS), and other types of attacks.

Due to the potential for misuse and the inherent loss of control that can result from having a machine participate in such a botnet, we hope "The Anatomy of Clickbot.A" will help facilitate further collaboration between search engines, Internet Service Providers (ISPs), anti-virus vendors, and other parties on the Internet in managing botnets and similar threats.

You can find the PDF version of "The Anatomy of Clickbot.A" in its entirety here. For more information on how Google detects and fights click fraud, visit the Invalid Clicks section of the AdWords Help Center.

Meet the Click Quality team

Over the past year, we’ve written about invalid clicks to let you know about the new invalid clicks report and provide answers to some of the most frequently asked invalid click questions. Now, we want to introduce you to the Click Quality team, who investigates your invalid click concerns. This team has been around for over three years and they provide protection in addition to our automated filters, which proactively identify invalid clicks and discard them before they make it into your account. Today, Julian from the Click Quality team joins us to let you know how you can request an invalid clicks investigation and shares some of the common suspicious click concerns that his team sees and how you can monitor them:

Working on the Click Quality team, I’m one of a number of people who investigate suspicious traffic to determine if invalid click activity has occurred. These thorough investigations analyze a number of factors, including IP address, duplicate clicks, and various other clicking patterns. It is rare that invalid clicks are not detected by our automatic filters, but when we find them we immediately credit the advertiser's account. If you'd like us to investigate suspicious activity in your account, please fill out this form. If and when you do, providing detailed information, such as your web logs, will greatly assist with our investigation, so we ask that you include any reports indicating suspicious IP addresses, referrers, or requests. Our team will respond to you as soon as we complete our investigation of your account. And, please be patient, as a thorough investigation can take 3-5 business days.

Our job is to look into advertiser concerns and complaints and we’re happy to answer any questions you have. However, before submitting an investigation request, you may want to do a little checking on your own because, over the years, we've found that in many cases the activity that the advertiser has pointed out isn't actually due to click fraud. I've provided a list of some of the more common concerns below, along with tips and tools to help you better understand and monitor suspicious activity in your account.

Concern: I’ve seen huge fluctuations in my spend, traffic, or ROI.
  • Review your account to see if you’ve made recent changes to your daily budget, maximum CPCs, or ad distribution preferences. Making changes to any of these can increase your reach and change your conversion rate as new visitors may have different buying patterns.
  • Traffic on the Google content network can also fluctuate from day to day as your ads match new sites. To better control your spend on the content network, make sure that you implement content bids or set up a separate campaign that is targeted just to the content network.
  • If you're not currently tracking your conversions, we recommend that you use a conversion tracking tool, such as Google conversion tracking or Google Analytics, in order to monitor your ROI.
Concern: There are discrepancies in my web logs.
  • You may have noticed that the clicks in your web logs are different than the number of clicks that are reported in your account. If you find that there are fewer clicks reported in your web logs than in your account, we suggest that you run an invalid clicks report because many of your clicks may have been automatically filtered.
  • If you've seen more clicks in your account than in your web logs, make sure that your tracking software is counting clicks from the Google Network. You can can make sure your tracking URLs are set up correctly here. Or, you can use auto-tagging to more easily identify AdWords clicks in your web logs.
Concern: I have multiple clicks coming from the same IP address.
  • There are several legitimate reasons that you may see multiple clicks from a single IP address in your web logs. For example, if a user refreshes their browser after clicking on your ad, this may appear as multiple ad clicks in your logs, however, you are only charged for one click. Also, some service providers, such as AOL, assign the same IP address to a large number of users, so multiple users visiting your page may appear as a single IP (or a few very similar IPs).
  • In order to determine if clicks from the same IP address are unique, you can use auto-tagging. Auto-tagging will show you if multiple clicks were the result of a user refreshing their browser or if they represent a new click on your ad.
Concern: My competitors are clicking on my ads.
  • Our filters are able to recognize the vast majority of invalid clicks that come from a competitor who is repeatedly clicking on your ads. These clicks should be filtered out before they reach your account so you will not be charged.
  • To determine the number of clicks that we've automatically filtered from your account, you can run an invalid clicks report.
I hope this list will help you better determine if suspicious clicks in your account are legitimate, or invalid. As I mentioned earlier, if, at any point, you'd like us to investigate suspicious activity in your account or have any questions, please get in touch with us.