Concurrency Management in the AdWords API

Has a call from your application ever returned a CONCURRENT_MODIFICATION error but you can’t figure out why? Let’s look at concurrency management in the API to understand what might have happened.

The AdWords API uses an optimistic concurrency control mechanism.  When the server begins to process a request, it checkpoints the current state of the relevant account, then executes the business logic and validation rules in the request.  Before committing the requested change, the server again checkpoints the account. If the “before and after” checks don’t match, the server returns a concurrency error to ensure that the request doesn’t introduce data inconsistency.

A concurrency issue occurs when two different applications or two different instances or threads of the same application attempt to change a single piece of data at the same time. Let’s say that request1 and request2 both want to operate on the same original version of some data. If request1 changes the data first and is committed, request2 becomes invalid, because it’s trying to operate on data that is now out-of-date.

A concurrency issue doesn’t occur only when your application tries to modify an entity that’s changed during processing. A problem can also occur if the entity that your request modifies has a dependency on another entity that’s been changed.  For example, an ad group entity references a campaign. If your request modifies an ad group and the referenced campaign is modified while your request is executing, you’ll get an error.   

In an exception to this general rule, we’ve recently introduced the ability to make concurrent changes to some services, such as AdGroupCriterionService and AdGroupAdService. However, the principle still applies at the field level: field-level changes are exclusive to one request at a time.

In the past, you might have gotten a CONCURRENT_MODIFICATION error even when using a single-threaded application to modify one account at a time. The problem was likely due to background processing, such as approvals, which also modified the account’s entities. We’ve made some changes that should reduce the likelihood of these errors occurring

To summarize, when you get a concurrency error, consider whether the application has done one of the following:

• Run multiple threads that attempted to modify the same entity
• Modified an entity on which another entity is dependent 

As always, if you have more questions, feel free to post them to AdWords API Forum.

Posted by Dan H. and Nina Silk, AdWords API Team

Bid simulator results have moved to the DataService

We introduced the BidLandscapeService in v201003 as a read-only service that allowed developers to retrieve the bid simulator results available for keywords, then extended it to include ad group results in v201008. Looking ahead we foresaw that additional forms of metadata would need to be exposed via the API, so we decided to consolidate these types of features into a single service: the DataService. Today the DataService provides the ability to retrieve bid simulator results and we plan to expand it to include other functionality in future versions of the API.

In addition to relocating the bid simulator feature to a new service, we updated it to support generic selectors. The returned objects haven't changed however, so much of your application logic can remain the same. It's also worth noting that while the BidLandscapeService had a single getBidLandscape() method which would return both criteria and ad group bid simulation results, the DataService has now broken this functionality out into two methods: getAdGroupBidLandscape() and getCriterionBidLandscape().

As a reminder, to get criteria bid simulation results using the v201008 BidLandscapeService you can use the following PHP code:

// Old v201008 code.
$selector = new CriterionBidLandscapeSelector();

$idFilter = new BidLandscapeIdFilter();
$idFilter->adGroupId = $adGroupId;
$idFilter->criterionId= $keywordId;
$selector->idFilters = array($idFilter);

$bidLandscapes = $bidLandscapeService->getBidLandscape($selector);

To accomplish the same task with the v201101 DataService use the following code:

// New v201101 code.
$selector = new Selector();

$selector->fields = array('AdGroupId', 'CriterionId', 'StartDate',
    'EndDate', 'Bid', 'LocalClicks', 'LocalCost', 'MarginalCpc',
    'LocalImpressions');

$adGroupIdPredicate = 
    new Predicate('AdGroupId', 'IN', array($adGroupId));
$criterionIdPredicate =
    new Predicate('CriterionId', 'IN', array($criterionId));
$selector->predicates =
    array($adGroupIdPredicate, $criterionIdPredicate);

$page = $dataService->getCriterionBidLandscape($selector);

Like with all generic selectors you must explicitly list all of the fields you wish to have returned, the names of which can be found in the documentation for CriterionBidLandscape (or AdGroupBidLandscape) and its sub-objects. Also notice that the functionality of the BidLandscapeIdFilter can be replicated by using a combination of predicates, which are always ANDed together.

If you have any questions about the DataService, or how to migrate your bid simulator code, reach out to us on the forum.

 - Eric Koleda, AdWords API Team

Discover v201101: Generic selectors

AdWords API v201101 changes the way you retrieve data using the get() method of most services. This blog post discusses the new feature in detail and highlights the changes you need to make in your code to migrate to generic selectors.

Introduction

Prior to v201101 of AdWords API, every service had its own selector to retrieve data using the get() method. For instance, to retrieve all active ad group criteria in an ad group, you would write your code as

// OLD CODE - v201008.
// Create selector.
AdGroupCriterionSelector selector = new AdGroupCriterionSelector();
selector.userStatuses = new UserStatus[] {UserStatus.ACTIVE};
 
// Create id filter.
AdGroupCriterionIdFilter idFilter = new AdGroupCriterionIdFilter();
idFilter.adGroupId = adGroupId;
 
selector.idFilters = new AdGroupCriterionIdFilter[] {idFilter};
 
// Get all ad group criteria.
AdGroupCriterionPage adGroupCriterionPage =
    adGroupCriterionService.get(selector);

While simple to use, since each service had its own selector class, you needed to know about selectors of each service with which you worked. Also, since filtering options were specified as member fields of the selector, adding support for new filtering involved changing the selector definition in the service. and hence could be done properly only across versions.

On the other hand, the selector model used by reports is much more flexible - you can specify a list of fields, provide any number of filtering conditions on any Filterable field using predicates, use multiple operators (unlike the implicit EQUALS operator in the code above) and so on. Also, since field names aren’t tied to a SOAP schema, it is easy to add new selectable and filterable fields. Additionally, the code you need to write to generate and download any report remains the same. In v201101, we decided to take this design and apply it across all services.

Using v201101 of AdWords API, you can retrieve all active ad group criterion in an ad group as follows:

// NEW CODE - v201101.
// Create a selector.
Selector selector = new Selector();
selector.fields = new string[] {"Id", "AdGroupId", "KeywordText", "Status"};
 
// Set filter conditions.
Predicate adGroupPredicate = new Predicate();
adGroupPredicate.field = "AdGroupId";
adGroupPredicate.@operator = PredicateOperator.EQUALS;
adGroupPredicate.values = new string[] {adGroupId.ToString()};
 
Predicate statusPredicate = new Predicate();
statusPredicate.field = "Status";
statusPredicate.@operator = PredicateOperator.EQUALS;
statusPredicate.values = new string[] {UserStatus.ACTIVE.ToString()};
 
selector.predicates = new Predicate[] {adGroupPredicate, statusPredicate};
 
// Get all ad group criteria.
AdGroupCriterionPage adGroupCriterionPage =
    adGroupCriterionService.get(selector);

Understanding selector properties

The main properties of the selector are:

Fields: When using generic selectors to retrieve an object, the server will not return every field of the object by default; it will return only the requested set of fields. Other fields in the object will be set as undefined/null depending on the programming language you use. You can use the Fields property to specify the list of fields to be returned. You can use any Selectable field from the entity here. For example, when using CampaignService, you can use any Selectable field from Campaign. Campaign.name is marked as a Selectable field, and its selector field name is specified as “Name”, so you can use that as an entry for Fields.

Note that when an object has a field that is a reference to another object, then that property may not be marked as Selectable. In such cases, its sub fields will be marked as Selectable. For example, Campaign.budget is a property of type Budget, so Campaign.budget is not a Selectable field. To retrieve a campaign’s budget fields, you have to request Selectable fields from the Budget object.

Predicates: You can use predicates to filter the returned data by field values. You can use any Filterable field from the entity in a predicate. You can also use various operators to specify the filtering condition. Also, when using multiple predicates in a single selector, the predicates are ANDed to evaluate the effective filter condition.

Each service provides a set of service-specific fields and generic selector fields that can be used with Predicates. E.g. you can use CampaignId as a predicate field with AdGroupAdService, even though CampaignId is not a member of AdGroupAd. For the list of all the generic selector fields, you can refer to the selector migration guide.

DateRange: You can provide an optional DateRange to control the date range for which the data is retrieved. Specifying a date range only affects the stats returned by the server (if applicable), not the actual entities being returned by the server. The date format for min and max fields have the format yyyymmdd. The date range should be in the range [19700101, 20380101].

Ordering: You can use the ordering field to specify the fields on which you want to sort the data (ascending or descending). The order in which you specify the OrderBy objects in the ordering field is significant; the first element specifies the primary sort order, the second element specifies the secondary sort order and so on.

Paging: You can use the paging field to specify the position from which to start returning results and the number of results to return per page.

Migrating from service-specific selectors to generic selectors

Service-specific selectors (e.g. CampaignSelector) aren’t supported from v201101 onwards, so you will have to modify your code to use generic selectors when migrating to v201101 of AdWords API. You can refer to the selector migration guide to get the field mapping for each service-specific selector. Also note that a few services do not use generic selectors yet. This will happen in a future version of AdWords API.

We have added support for generic selectors in all of our client libraries, so please take advantage of this new feature and share your feedback with us on the forum.

  --Anash P. Oommen, AdWords API Team

AdWords Downtime: March 20, 3am-7am PT

We'll be performing routine system maintenance on Sunday, March 20 from approximately 3:00am to 7:00am PT. You won't be able to access AdWords or the API during this time frame, but your ads will continue to run as normal.

Best,

- Eric Koleda, AdWords API Team 

Reminder about company info

We would like to share a friendly reminder for all AdWords API developers to keep company information up to date in your AdWords API Center. It should take less than a minute to update and allows us to deliver better support as well as provide you with critical product updates in a timely manner.

The company information, such as contact email, can be modified in the “AdWords API Center” tab of your My Client Center (MCC) account. Navigate to the “Company Information” section of the page and follow the “Edit” link to make changes.

As always, please post any questions and feedback to the AdWords API Forum.

--Stan Grinberg, AdWords API Team

Update to reporting service in AdWords API v201101

Last week we announced AdWords API v201101, which included support for My Client Center (MCC) cross-client reports.

This feature did not go live as expected, so is currently only available to a limited set of developers. We hope to make it generally available in the coming weeks.

Please note that ReportDefinitionService will continue to work for non-cross-client reports as it did before.

If you have any questions please post them on the AdWords API forum.

Posted by Katie Wasilenko, AdWords API Team

New demo launch: AdWords Command Line Interface (AWCLI)

Today we’re releasing a new AdWords API sample application: AWCLI (AdWords Command Line Interface) intended to demonstrate a way for using the API. It’s a tool for AdWords interaction which was designed for power users and administrators to be able to inspect AdWords objects from a command line. It could also be used in various scripts and cron jobs.

The current code is work in progress. We’re sharing this code as open source to provide a starting point for new developers and to demonstrate some of the core functionality in the API.

AWCLI features

• Shell-like command line interface with tab-completion.
• Ability to list (ls), navigate (cd), retrieve (cat), edit (edit) and delete (rm) AdWords objects (campaigns, ad groups etc).
• Search (find) function.
• Local objects cache for easy referencing.
• Environment (env), history (history) and other utility functions.

Getting AWCLI

AWCLI is available from the Google Code page and is part of AdWords Java API project. It can be obtained from the downloads page: http://code.google.com/p/google-api-adwords-java/downloads/. You can find installation and running instructions as well as usage examples on the project wiki page.

Please join us to discuss the tool and report any bugs on the project page or forum.


Danial Klimkin, AdWords API Team.

AdWords API v201101 launch -- providing enhanced reporting, generic selectors, support for filtering and more

We’re pleased to announce the launch of AdWords API v201101, which allow you to more efficiently run reports, as well as implement campaign experiments and other recently released advertising features at scale. Through the AdWords API forum and developer events we’ve heard frequent requests for the newly available services. We’ve highlighted some of the new features below. A complete list of changes is available in the v201101 release notes.

v201101 highlights:

• Easily run reports across clients: ReportDefinitionService now supports My Client Center (MCC) cross-client reports, which are fetched asynchronously. The ReportDefinition type has two new fields with which you specify retrieval of a cross-client report, and all report types have new fields for use in cross-client reporting. In addition, the following new reports have been added: Geo Performance, Demographic Performance, Ad Extensions Performance,  Destination URL and Creative Conversion. See Report Types for more information.

• Greater data control and filtering with a single generic selector: The get operations for a number of services now use a single generic selector, rather than service-specific selectors. The generic selector provides more control over the data returned, lets you filter on almost any field, and specify sorting and paging. For information about migrating your code, see the Selector Migration Reference.

• Try out campaign experiments: A campaign can now try out sets of ads experimentally by using the ExperimentData in the AdGroupAd type. A new set of report fields provide information about how an advertiser experiment performed. The Ad Group Performance, Campaign Performance, Keywords Performance, and Managed Placements Performance reports each have a field that lets you segment on the control arm or experiment arm. In addition, a report field for each statistic indicates the significance in the change of that statistic in the experiment arm. For more information about campaign experiments, see the AdWords Campaign Experiments overview.

• Differentiate locations for desired targeting: A new setting for the Campaign type --  GeoTargetTypeSetting -- lets you specify how to apply geo targeting. The setting lets you differentiate between the user’s physical location and the location that’s targeted in the search. 

• Run interest-based advertising at scale (coming soon): Adwords API will fully support interest-based advertising via the ConversionTrackerService, which enables you to create a new conversion event and get the Javascript tag to create new lists. This service will be released in the coming weeks.

Deprecation timeline for previous versions
With the release of equivalent functionality in v201101, the following versions and services will be deprecated:

• API versions v13 (ReportService and TrafficEstimatorService only), v200909, v201003, v201008
• BidLandscapeService (being moved to the DataService)

We will be sunsetting these versions and services in August 2011.

As with every new version of the AdWords API, we encourage you to review the resources in the AdWords API client libraries. If you have any questions please post them on the AdWords API forum.

Posted by Katie Wasilenko, AdWords API Team