Fine grained partial failure in the BulkMutateJobService

Since its introduction the BulkMutateJobService has provided support for partial failures, allowing some operations to succeed even though others have failed. Over the next few weeks we'll be rolling out improvements to this functionality, allowing for more granular failures when possible.

Bulk mutate jobs process operations in batches, and a failed operation will cause the entire batch to fail, but not the rest of the job. The results for a failed batch start with a FailureResult that holds all the errors generated by the operations in that batch. It is followed by a series of BatchFailureResults which act as placeholders for the rest of the operations in the batch. More information about how these failures are reported can be found in a previous blog post we did on the topic.

In v201008 we enabled partial failure support for the synchronous AdGroupCriterionService. Unlike the behavior in the BulkMutateJobService, this version of partial failure allows individual operations to fail without affecting successful ones. This method of partial failure is preferred as it cuts down on the number of retries needed.

While we can't update the BulkMutateJobService to process all operations this way, we are able to enable it for operations that use an underlying service that does support granular partial failures. Starting August 1st, operations of type AdGroupCriterionOperation and AdGroupAdOperation processed by the BulkMutateJobService may start failing granularly instead of in batches.

Let's see how this will change the results returned.

Existing partial failure behavior

Operation	Result
AdGroupCriterionOperation	FailureResult cause.errors[0].fieldPath = "operations[1]..." cause.errors[1].fieldPath = "operations[2]..."
AdGroupCriterionOperation (Invalid)	BatchFailureResult operationIndexInBatch = 1
AdGroupCriterionOperation (Invalid)	BatchFailureResult operationIndexInBatch = 2
AdGroupCriterionOperation	BatchFailureResult operationIndexInBatch = 3

Before the change the invalid operations will cause other valid operations to fail. The FailureResult is the first result in the batch, even if the operation at the position was valid. The field path of the errors indicates the index of the invalid operations relative to the start of the batch.

New granular partial failure behavior

Operation	Result
AdGroupCriterionOperation	ReturnValueResult
AdGroupCriterionOperation (Invalid)	FailureResult cause.errors[0].fieldPath = "operations[0]..."
AdGroupCriterionOperation (Invalid)	FailureResult cause.errors[0].fieldPath = "operations[0]..."
AdGroupCriterionOperation	ReturnValueResult

After the change invalid operations will not affect other valid operations in the same batch. Each invalid operation will have a corresponding FailureResult. There may still be cases where BatchFailureResults are returned for these operations, such as if there is a general error with the account that cannot be attributed to any specific operation.

So how does this affect code you've already written to process BulkMutateJob results? The good news is that if you've written your code to allow for a variable batch size (no hard coded batch size) then no changes are needed! Invalid operations are essentially processed as batches of one, and the same logic that you use to process the results today should work perfectly with these new granular failures. If your application is currently expecting a batch of a specific size then you should update it to detect the start and end of batches dynamically.

If you have any questions about how these new granular failures work, or feedback on how we can further improve the error handling in the API, add a post to the AdWords API forum.

Update 2011-08-25
This change has been fully deployed and all BulkMutateJobs will be processed with granular partial failures enabled.

- Eric Koleda, AdWords API Team

Discover v201101: Experiment Reports

Back in September of 2010, we added support to manage AdWords Campaign Experiments (ACE) via the API. With the release of v201101, we are extending the API to allow you to fully manage your experiments, as we’ve included new experiment fields for use in reports. You can now segment your report statistics by the experiment splits as well as obtain experiment statistical significance on every statistical field (i.e. clicks, impressions, conversions, etc).

Segmentation

A new field called AdvertiserExperimentSegmentationBin is now available for you to segment your report rows in OUTSIDE_OF_EXPERIMENT, CONTROL and EXPERIMENT. Rows reported as OUTSIDE_OF_EXPERIMENT refer to statistics captured outside the experiment lifespan, which will occur when your report dateRange covers a broader time period than your experiment. Data reported as part of the CONTROL segment includes statistical values that belong to the percentage of traffic that you have configured as part of your control split. Likewise, values for the experimental split are reported with the EXPERIMENT value.

Keep in mind that adding AdvertiserExperimentSegmentationBin to your reports may potentially multiply their size by three, so make sure you use report predicates to control exactly what you want back and to define your reports in one of the GZIP compressed formats, if possible.

Statistical Significance

For every available statistical field (i.e. Clicks, Impressions, Conversions, CPC, CPM, etc) a Statistical Significance field pair is now available for you to use in some reports. These new field pairs have, with a few exceptions, the form of [field singular name]Significance, so for example Clicks has ClickSignificance as its pair. For more information about the available Statistical Significance fields per report see our Report Types page.

Significance fields have a specific range of values that goes from -3 to 3, values that correspond to three
, two and one (down and up) arrows respectively in the AdWords interface, while 0 is represented by a grayed up and down arrow.  To learn more about  Statistical Significance and its meaning visit our Help Center FAQ page. Also keep in mind that negative values don’t mean your experiment isn’t going as expected since fields that imply cost, like CPC, you most likely want to have a negative variation.

Life of your experiment data

Your experiment segmented statistics will exist as long as your experiment does, so as soon as your experiment is either promoted or deleted you will no longer able to retrieve experiment statistics. Hence, we recommend you download a last set of experiment reports right before the experiment gets promoted or deleted.

Reports Types Included

AdWords Campaign Experiments are applied at the Campaign level, therefore reports that apply to Campaigns, AdGroups, Ads and Keywords have included experiment specific fields. Specifically, the following reports have been enhanced with experiment fields: Ad Performance, Ad Group Performance, Campaign Performance, Keyword Performance and Managed Placements Performance.

As always, feel free to contact us at the AdWords API Forum.

- David Torres, AdWords API Team

Client Library Updates

We often make changes to the AdWords API client libraries that do not merit a full blog post. We’d like to start publishing a summary blog post every other week to highlight changes that might otherwise go unnoticed.

Python

• Added support for OAuth as an authentication mechanism. This will allow developers to access the API without requiring a username and password. See the new code example for more information on how to integrate OAuth with your application.
• Additionally, a number of issues were fixed: 115, 142, 144, and 145.

Perl

• Fixed some bugs that prevented usage of multiple Client objects.
• Full XPath support was added; XPath queries can now be done through the entire tree of results.
• OAuth support was also added.

Ruby

• Improved support for UTF reports in download extensions.
• New logger interface.
• Updates have been made to the code examples.

PHP

• Updated OAuth support to allow for alternative OAuth libraries, including the bundled Andy Smith library.
• The logging functionality has been updated to support filtering for levels. The default logging behavior has been changed to only log SOAP XML if an error occurs.
• New methods were added to AdWordsUser for additional logging configuration. The AdWords Get*Service() methods were replaced with a single GetService() method. Backwards compatibility was maintained using __call(). All examples and tests were updated to use the new method.
• Additionally, 2 issues were fixed: 55 and 59.

You can download updated versions of the AdWords API client libraries from their respective sites, or see a list of all the client libraries using this link.

If you have any questions or would like to discuss these changes, please post on the forum.

Kevin Winter, AdWords API Team

Introducing MCC accounts multiple links feature

We’ve just released a new My Client Center (MCC) feature called multiple links, which makes it easier for agencies, SEMs, AdWords API developers, other 3rd party providers and clients to work together while taking advantage of the benefits of the MCC structure. It enables up to five MCCs to link to and manage a single AdWords account as necessary. Multiple links also removes the distinction that Google currently has between UI and API and API-only links.

Note this change affects Advertiser accounts only. MCC accounts can have only one manager as before.

Changes for the API users

There are a few API changes introduced by this new feature:

• All new Account links are now of the UI_AND_API type. Existing API_ONLY links were not updated and are still valid as defined.
• Every managing account with an API token will now receive 100% of free units (if eligible) for each linked client account based on its monthly spend. Previously, UI_AND_API type of links provided only 10% of free units quota.

For FAQ and billing-related information check out the AdWords Blog post.


Danial Klimkin, AdWords API Team.

Enhanced documentation for selectors and reports

We understand the importance of documentation when navigating a system as complex as the AdWords API; for this reason we’ve been working to expand our reference materials to include even more information.

First off, we’ve published a new Selector Fields page that provides information about the fields that can be used in generic selectors. This information was always available on an object’s individual reference page, but now we’ve aggregated it together across entire services for quicker lookups.


Additional information on how to migrate from the service-specific selectors used in versions v200909 through v201008 is available in the Selector Migration Reference.

Next, we’ve added more notes to the Report Types page to indicate which fields will prevent zero-impression rows from being returned in the downloaded report. In general, any field that is a property of an impression or click (such as the date, network, etc) will prevent rows with no impressions from being returned, but we decided to explicitly mark these fields so that the behavior is clear.


Finally, we’ve added new tips to our Best Practices Guide on topics such as deleting entities and using gzip compression. The guide is a great resource for developers starting an AdWords API project, but is also useful for established developers trying to increase the efficiency of their applications.

We’re always looking to expand and improve our documentation, so if you have any feedback for us please post it to the forum.

- Eric Koleda, AdWords API Team

AdWords API JavaScript library - Getting started

Recently, we launched the AdWords API JavaScript Client library. This blog post discusses its basic usage by explaining the GetAllCampaigns example.

Including required scripts

AdWords API JS library comes with the following compiled scripts that you can include in your web page:

• awapi.js: This is the AdWords API JS library, combined into a single script.
• v201101.js: This is the compiled script for AdWords API v201101. There are compiled scripts for each supported API version.
• config.js: This is optional; if included, you can define your application configuration in this file.

Importing the required types

To use any type provided by the AdWords API JS library, you need to import it. For instance, to import the CampaignService object, use

<script type="text/javascript">
  goog.require('google.ads.adwords.v201101.CampaignService');
</script>

All the goog.require statements should be put together in a single <script> tag.

Creating a User and obtaining a service instance

In the AdWords API JS library, each AdWords account is represented using a User object. The User object allows you to create various services provided by the AdWords API and make calls against the AdWords account associated with this user. The code to do this is given below:

var user = new google.ads.adwords.User();
var campaignService = user.getService(google.ads.adwords.AdWordsService.v201101.
    CampaignService);

The user takes a config object as its constructor argument. If you don't provide one, then the config definition from config.js is used.

Calling the method and displaying results

The code to construct a selector and get all the campaigns is shown below:

// Create a selector.
var selector = new google.ads.adwords.v201101.Selector();
selector.fields = ["Id", "Name", "Status"];

campaignService.get(selector,
    goog.bind(function(page) {
      if (page && page.entries && page.entries.length > 0) {
        for (var i = 0, len = page.entries.length; i < len; i++) {
          var campaignValue = page.entries[i];
          alert(goog.string.format('Campaign with id = "%s", name = "%s" ' +
              'and status = "%s" was found.', campaignValue.id,
              campaignValue.name, campaignValue.status));
        }
      } else {
        alert('No campaigns were found.');
      }
      callback();
    }, this),
    goog.bind(function(soapException) {
      alert(goog.string.format('Failed to get campaign(s). Soap Fault ' +
         'says "%s"', soapException.getInnerException().getFaultString()));
         callback();
    }, this)
);

The important thing to note here is that unlike other languages, JavaScript client library implements all API calls as asynchronous methods. Each method accepts an onSuccess callback and an onFailure callback which will be called when a call succeeds or fails respectively. The prototype of onSuccess and onFailure callbacks are as shown below:

function onSuccess(results) {
}

function onFailure(soapException) {
  // soapException is an object of type ApiException
}

Setting up an HttpWebTransport

To make the AdWords API calls, the client library needs to POST xml data to the AdWords API Server. The exact mechanism for making these calls depend on the environment your code is running. You can set an appropriate HttpWebTransport in your config.js to tell the client library how to make its calls to the AdWords API server. The relevant snippet is given below:

HttpWebTransport: 'google.system.net.HttpWebTransportForServerProxy',
HttpWebTransportSettings: {
   ProxyUrl: '/awapi-js-lib/proxy.php'
}

When writing code for a normal website, you need to use HttpWebTransportForServerProxy as transport mechanism. This HttpWebTransport forwards your calls through a server-side proxy script. This is required since all modern browsers implement same origin policy that prevents the library from making a cross-domain XmlHttpRequest.

When writing code for a Chrome extension, you can make cross-domain xhr calls, if the domain is specified in the manifest file. In such cases, you can use HttpWebTransportForExtensions instead. You can refer to the README file for details on various HttpWebTransports provided by the library.

Please join us on the AdWords API forum to discuss the new client library and the Chrome extension and report any bugs on the project’s issue tracker or forum.

  --Anash P. Oommen, AdWords API Team

Statistics in reports

A common question for the AdWords API Team is “when is the best time to run reports” (and for that matter, when to retrieve statistical data via the API). The quick answer is: it depends on the type of data and report. Read on for details.

The reporting system is very complex and statistical data is not available in realtime. For most reports and API services, the statistical data for a given date is expected to stabilize three hours after midnight in the account's local time zone. The information below applies specifically to post-V13 reports.

For some reports, the underlying statistical data is populated in daily batch operations. These operations take a while to finish, which means the statistical data won’t be available as quickly. These reports will have their data ready at 6pm PST the following day and include the following:

• Automatic Placements Performance
• Demographic Performance
• Geo Performance
• Search Query Performance
• URL Performance

All other reports should have their statistical data ready three hours after midnight in the account’s local time zone.

Conversions are a special case. Every day we start a batch process to recalculate all statistical data up until the job begins. This process will be complete by 6pm PST but will only include data up until 10am PST (when the process starts). Please note that since conversions can occur weeks later, these will never be completely stable, but will always take into account the latest data.

Note that due to ongoing operations such as click fraud detection, some of the data, such as clicks, may change for up to a month later. Statistical data is “frozen” on the first day of every month and will not change after this time, except for conversions that do not incur cost.

Please see this page for more information about reports. If you have any questions about using reports with the API, you can ask us on the forum.

Kevin Winter, AdWords API Team