Downloading reports for lots of client accounts

Reporting has changed significantly with v201109. We’d like to take this opportunity to discuss how to download reports for lots of clients simultaneously. To download reports for a large number of client accounts, we recommend requesting the download of these reports simultaneously.

We recommend requesting no more than 10 reports concurrently and to use one thread for each concurrent download. If for some reason the report download fails, we recommend you retry after a short wait. The length of the wait should increase with the number of times the request has been tried (known as Exponential Backoff). This backoff algorithm helps prevent making too many requests to the server in a short period of time.

You should check the specific reason the download failed and only retry if it was a transient error (such as exceeding the rate limit) instead of retrying if the report definition XML was invalid. The HTTP Status of the response provides useful information:

• HTTP Status 500 indicates there is a transient server side issue and the request can be retried after a delay. We retrying no more than 5 times for any given report.
• HTTP Status 400 indicates there was a problem with the request (or the report itself) and retrying will not help.
• HTTP Status 200 means the report download was successful.

With v201109 you should use Ad Hoc reports to download new reports. With this new feature, there is no need for stored ReportDefinitions. The XML describing a Report Definition is sent directly with report download. A developer token is required, but the API call accrues no costs (report downloads are now free!).

Another benefit of downloading reports in this fashion is you get data more quickly than with legacy cross-client reports. As soon as a report download succeeds, you can start processing the data by putting it into a database or performing further logic on the results.

We hope this discussion of best practices will help you with writing report download code in the future. We’ve published a reference implementation showing how to do this in java here. If you have any questions about downloading reports, you can ask us on the forum.

Kevin Winter, AdWords API Team

Enhanced documentation for selectors and reports

We understand the importance of documentation when navigating a system as complex as the AdWords API; for this reason we’ve been working to expand our reference materials to include even more information.

First off, we’ve published a new Selector Fields page that provides information about the fields that can be used in generic selectors. This information was always available on an object’s individual reference page, but now we’ve aggregated it together across entire services for quicker lookups.


Additional information on how to migrate from the service-specific selectors used in versions v200909 through v201008 is available in the Selector Migration Reference.

Next, we’ve added more notes to the Report Types page to indicate which fields will prevent zero-impression rows from being returned in the downloaded report. In general, any field that is a property of an impression or click (such as the date, network, etc) will prevent rows with no impressions from being returned, but we decided to explicitly mark these fields so that the behavior is clear.


Finally, we’ve added new tips to our Best Practices Guide on topics such as deleting entities and using gzip compression. The guide is a great resource for developers starting an AdWords API project, but is also useful for established developers trying to increase the efficiency of their applications.

We’re always looking to expand and improve our documentation, so if you have any feedback for us please post it to the forum.

- Eric Koleda, AdWords API Team

Discover v2009: Do more with mutate

In previous versions of the API, services had separate methods for adding, updating, and removing items in an AdWords account. The v2009 version changed this paradigm and combined the functionality into a single method called mutate. The mutate method works on operations, where each operation defines the action to perform (operator) and the item to perform it against (operand).

While it may not be immediately obvious, the mutate method is designed to process many operations in one call. For example, the following method uses the PHP client library to add multiple keywords in one request.

function addKeywords($keywords, $adGroupId, $adGroupCriterionService) {
  $operations = array();

  foreach ($keywords as $keyword) {
    // Create biddable ad group criterion.
    $adGroupCriterion = new BiddableAdGroupCriterion();
    $adGroupCriterion->adGroupId = $adGroupId;
    $adGroupCriterion->criterion = $keyword;

    // Create operation.
    $operation = new AdGroupCriterionOperation();
    $operation->operand = $adGroupCriterion;
    $operation->operator = 'ADD';

    // Add operation to array.
    $operations[] = $operation;
  }

  // Make mutate request.
  $results = $adGroupCriterionService->mutate($operations);

  return $results;
}

It’s important to note that performing multiple operations in one request can be substantially faster then performing them one at a time. Making any request to the AdWords API comes with a certain amount of overhead, so performing more work per request can dramatically improve the performance of your application. For example, in a simple test I ran adding 500 keywords in a single request was over 100x faster than performing 500 separate requests.

Also of note is that the operations in a mutate request can contain any mixture of operators and span any number of campaigns, ad groups, etc. That means you can add a keyword to one ad group and update a keyword in a different ad group in the same request.

More information on the mutate method of a service and the operations it supports can be found in the developer documentation, and any questions you have can be posted to the forum.

- Eric Koleda, AdWords API Team

AdWords API sandbox - Best practices

The API sandbox is open to anyone who wants to test code during development, and it's completely free. We recommend using the sandbox as much as possible to catch your bugs during development so they won't hit your production account later. Here are some tips on using the sandbox effectively:

Initialize your sandbox account

Make sure that the first call to the sandbox is a get() call from AccountService, preferably AccountService's getClientAccounts(). Assuming you use user@domain.com as the login email, this call will initialize the sandbox and create 5 test client accounts, namely client_1+user@domain.com to client_5+user@domain.com for your sandbox account. While making this call you should not provide the clientEmail or clientCustomerId headers, since this call expects an MCC account to succeed. 

While making a getClientAccounts() call to the sandbox, set the SOAP headers as follows:

email: address of any google account, such as a Gmail account
password: password of the account
useragent: set to any identifying string
developerToken: set to the value for email, with the string '++USD' (or the currency code of your choice) appended
applicationToken: ignored, and could be set to anything or left out
clientEmail: should be either set to the empty string or left out

clientCustomerId: should be either set to the empty string or left out

The sandbox database is usually refreshed once every month, wiping out all client accounts. Making getClientAccounts() as your application's first API call ensures that all your calls happen against a properly initialized sandbox account.

Simulate error responses

Because a good application needs to gracefully handle errors thrown by the AdWords API, the sandbox provides a mechanism for simulating error messages. If you use user@domain.com++errorCode as the email for a call to the sandbox, the sandbox will return an error response with error code set as the desired errorCode, and a text message indicating that an error response has been forced by the sandbox. 

The error code should be set to one of the valid error codes. For instance, by providing login email as user@domain.com++17, I can simulate error code 17 (account exceeded maximum number of allowed campaigns). 

The SOAP response looks as follows:

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <soapenv:Body>
    <soapenv:Fault>
      <faultcode>soapenv:Server.userException</faultcode>
      <faultstring>Error forced by the request (Sandbox)</faultstring>
      <detail>
        <ns1:fault xmlns:ns1="https://adwords.google.com/api/adwords/v13">
          <ns1:code>17</ns1:code>
          <ns1:message>Error forced by the request (Sandbox)</ns1:message>
          <ns1:trigger>user@domain.com++17</ns1:trigger>
        </ns1:fault>
      </detail>
    </soapenv:Fault>
  </soapenv:Body>
</soapenv:Envelope>

Similarly, you can also test for some policy violations in ads and keywords. For instance, providing repeated punctuation in the ad or keyword text, such as "Hello World!!" as the ad title, should trigger a policy violation error. However, the sandbox doesn't have all of the policy checks in place, so you can't test your application against every possible policy violation, e.g. trademark-related violations. 

Generate reports

Since generating reports is one of the most common calls in the AdWords API, the sandbox provides features to test each stage of your report generation logic. In the sandbox, you can create either a report whose status does not change, or a report whose status changes from pending to deleted. If you include any one of the following strings in the report name field while creating it, it will create a report with the specified status.

• fixstatus=Pending
• fixstatus=InProgress
• fixstatus=Completed
• fixstatus=Failed

If the name doesn't contain any of the above codes, the sandbox will create a report whose status changes as follows:

• During the first minute since the report job was scheduled, the status is Pending.
• During the second minute, the status is InProgress.
• During the third minute, the status is Completed.
• After the third minute, the report is deleted.

Also, a call to getAllJobs will return four reports, one for each value of ReportJobStatus. Sandbox reports do not return any data related to any live accounts. So if you want to test your report parsing logic against some live data, you can generate a report of the desired type in the Reports Center of your production account, download it in xml format, and use it to develop and test your code.  

Handling account limits

As with a live account, an AdWords sandbox account can hold a maximum of 25 active campaigns, with up to 100 active ad groups per campaign. If your program runs into these limits, then you can change the status of some campaigns or ad groups to "deleted."

TrafficEstimatorService and KeywordToolService behavior

Both TrafficEstimatorService and KeywordToolService are available for use in AdWords API sandbox. However, both these services return fake data. TrafficEstimatorService will give you fake data for Position, CPC, and Clicks per day. KeywordToolService can give you fake Seed-based keyword suggestions when using getKeywordVariations. It can also give you content-based keyword suggestions and SiteKeyword statistics when using getKeywordsFromSite.

The AdWords API sandbox can greatly help you with your development by preventing bugs from hitting production account during early stages of development. You can find more information on AdWords API sandbox at http://code.google.com/apis/adwords/docs/developer/adwords_api_sandbox.html.

-Anash P. Oommen, AdWords API Team