Downloading reports for lots of client accounts

Reporting has changed significantly with v201109. We’d like to take this opportunity to discuss how to download reports for lots of clients simultaneously. To download reports for a large number of client accounts, we recommend requesting the download of these reports simultaneously.

We recommend requesting no more than 10 reports concurrently and to use one thread for each concurrent download. If for some reason the report download fails, we recommend you retry after a short wait. The length of the wait should increase with the number of times the request has been tried (known as Exponential Backoff). This backoff algorithm helps prevent making too many requests to the server in a short period of time.

You should check the specific reason the download failed and only retry if it was a transient error (such as exceeding the rate limit) instead of retrying if the report definition XML was invalid. The HTTP Status of the response provides useful information:

• HTTP Status 500 indicates there is a transient server side issue and the request can be retried after a delay. We retrying no more than 5 times for any given report.
• HTTP Status 400 indicates there was a problem with the request (or the report itself) and retrying will not help.
• HTTP Status 200 means the report download was successful.

With v201109 you should use Ad Hoc reports to download new reports. With this new feature, there is no need for stored ReportDefinitions. The XML describing a Report Definition is sent directly with report download. A developer token is required, but the API call accrues no costs (report downloads are now free!).

Another benefit of downloading reports in this fashion is you get data more quickly than with legacy cross-client reports. As soon as a report download succeeds, you can start processing the data by putting it into a database or performing further logic on the results.

We hope this discussion of best practices will help you with writing report download code in the future. We’ve published a reference implementation showing how to do this in java here. If you have any questions about downloading reports, you can ask us on the forum.

Kevin Winter, AdWords API Team

Upcoming Reports Changes: August 19

We want to give developers a heads-up about an upcoming change affecting AdWords reports. If you retrieve reports via the AdWords API's getReportDownloadUrl() or getGzipReportDownloadUrl() method then these changes may potentially affect the data in your reports. These changes impact a small number of columns and should not fundamentally change the way you parse report data.

We plan on rolling out these changes on Tuesday, August 19.

The values returned by the following columns will be changing. Please note that some of these columns are only available for reports scheduled via the AdWords web interface, not the AdWords API, but we're listing them here in case you use the AdWords API to retrieve and process such reports.

• CustomerTimeZone: the time zone information will be returned as, e.g., the previous "America/New_York" will change to "(GMT-05:00) Eastern Time".
  
• Time aggregation columns: the columns representing the aggregation time period (e.g. weekStart) will represent date ranges in a different format. E.g. the previous "2008-06-01-2008-06-01" will now be returned as "6/1/08 - 6/1/08".
• Keyword: values for category-based website criteria will change from, e.g., "docvertical::3" to "Animals >> Endangered Species". Values returned for keyword or non-category-based website criteria should not change.
  
• Interactions: the name of reported interactions will change from, e.g., "VideoPlayTime25" to "Video Play Reached 25%."

There are also fixes to edge cases in which values for the impressions and the position preferences columns that were previously returned as invalid values should be more accurate.

If you would like to test your application against these changes ahead of time, please visit the AdWords web interface and log in as an account in which you normally would run reports. Visit the "Reports" or "Client Reports" tab, select an existing report from the list, and when viewing the report, use the "Export Report" button to download the report as a .xml file. This XML should match the report XML that you will retrieve via the AdWords API after the changes are put in place.