Discover v201003: Learn from the past with the Bid Simulator

Wednesday, July 28, 2010


Have you ever wondered how your keywords could have performed with a different bid? The bid simulator feature released last year allows you to do just that. Using data from the last seven days, it calculates the impressions, clicks, and cost your keywords could have received if different maximum CPC bids were used. This information was previously only exposed in the AdWords web interface, but is now available in the v201003 version of the API with the BidLandscapeService.

The getBidLandscape() method is used to retrieve the bid simulation results, also known as bid landscapes. You can use the selector to filter the results by campaign, ad group, or keyword ID. For example, to retrieve the results for a single keyword using the PHP client library you would use the following code:


  // Create selector.
  $selector = new CriterionBidLandscapeSelector();

  // Create id filter.
  $idFilter = new BidLandscapeIdFilter();
  $idFilter->adGroupId = $adGroupId;
  $idFilter->criterionId= $keywordId;
  $selector->idFilters = array($idFilter);

  // Get bid landscape for keyword.
  $bidLandscapes = $bidLandscapeService->getBidLandscape($selector);

The returned BidLandscape contains a series of LandscapePoints, each of which represents a single simulation for that keyword. It isn’t possible to configure which bids are used in the simulations, but the points are chosen automatically by the system to give an interesting range of data. Each landscape point contains the impressions, clicks, and cost that the keyword could have received at that bid. In addition, it contains the field marginalCpc, which contains the incremental cost-per-click (ICC) at the simulated bid. More information on ICC values and how you can use them is outlined in a video by Google's Chief Economist, Hal Varian. Please note that depending on the recent performance of the keyword, only some of this information may be populated in the landscape point.

Bid simulations are currently only available for campaigns that use ManualCPC bidding and target the GOOGLE_SEARCH or SEARCH_NETWORK networks. Additional restrictions are listed in the AdWords Help Center. The bid simulations are performed automatically in regular intervals, and new keywords or those with no impressions in the last seven days won’t have any associated results. No errors will be thrown when this happens, but the response will not contain a BidLandscape for that keyword.

Example code using the BidLandscapeService is available in the client libraries. Please post any questions about this service to the forum. Additional information about the bid simulator can be found in the Help Center.

- Eric Koleda, AdWords API Team

Discover v201003: Migrating your Account Structure Reports

Tuesday, July 20, 2010


Account structure report is one of the popular report formats in v13 of the AdWords API. This report allows you to download all the account attributes in a single report without performance details. A common use of account structure reports is to sync a local database against the AdWords server without using more expensive get() API calls.

Despite being a useful report format, account structure report is a specialized report format available only through the AdWords API. We designed the new API such that standard report formats can support your needs rather than creating additional specialized report types. Therefore, the new AdWords API doesn’t have a separate account structure report type, but you can use the existing report types supported by ReportDefinitionService to download the same data. This blog post discusses how this can be done, so that you can migrate your existing account structure reports to the v201003 version of the AdWords API. This blog post assumes you are familiar with the usage of ReportDefinitionService, if not, we suggest that you read our introductory blog post first.

Retrieving campaign structure information

You can download all campaigns in an account in the v201003 version of the AdWords API using CAMPAIGN_PERFORMANCE_REPORT. The xml request for adding the report definition is shown below:

<mutate xmlns="https://adwords.google.com/api/adwords/cm/v201003">
  <operations>
    <operator>ADD</operator>
    <operand>
      <selector>
        <fields>Id</fields>
        <fields>Name</fields>
        <fields>Status</fields>
        <fields>Amount</fields>
        </selector>
      <reportName>Campaign performance report #1277301277814.61</reportName>
      <reportType>CAMPAIGN_PERFORMANCE_REPORT</reportType>
      <dateRangeType>TODAY</dateRangeType>
      <downloadFormat>XML</downloadFormat>
    </operand>
  </operations>
</mutate>
Retrieving the ad group structure information

You can download all adgroups in an account in the v201003 version of the AdWords API using ADGROUP_PERFORMANCE_REPORT. The xml request for adding the report definition is shown below:
<mutate xmlns="https://adwords.google.com/api/adwords/cm/v201003">
  <operations>
    <operator>ADD</operator>
    <operand>
      <selector>
        <fields>Id</fields>
        <fields>Name</fields>
        <fields>Status</fields>
        <fields>CampaignId</fields>
      </selector>
      <reportName>AdGroup performance report #1277302184111.48</reportName>
      <reportType>ADGROUP_PERFORMANCE_REPORT</reportType>
      <dateRangeType>TODAY</dateRangeType>
      <downloadFormat>XML</downloadFormat>
    </operand>
  </operations>
</mutate>

If required, you can restrict the report to contain ad groups from only a particular campaign by setting a Predicate. A code example showing the usage of Predicate can be found here.

Retrieving the ad structure information


You can download all ads in an account in the v201003 version of the AdWords API using AD_PERFORMANCE_REPORT. The xml request for adding the report definition is shown below:
<mutate xmlns="https://adwords.google.com/api/adwords/cm/v201003">
  <operations>
    <operator>ADD</operator>
    <operand>
      <selector>
        <fields>Id</fields>
        <fields>Headline</fields>
        <fields>Description1</fields>
        <fields>Description2</fields>
        <fields>DisplayUrl</fields>
        <fields>Url</fields>
        <fields>Status</fields>
        <fields>AdGroupId</fields>
      </selector>
      <reportName>Ad performance report #1277303749892.73</reportName>
      <reportType>AD_PERFORMANCE_REPORT</reportType>
      <dateRangeType>TODAY</dateRangeType>
      <downloadFormat>XML</downloadFormat>
    </operand>
  </operations>
</mutate>
If required, you can restrict the report to contain ads from only a particular ad group by setting a Predicate.

Retrieving keyword and placement structure information


You can download all keywords in an account in the v201003 version of the AdWords API using KEYWORD_PERFORMANCE_REPORT. Similarly, all placements can be downloaded using MANAGED_PLACEMENTS_PERFORMANCE_REPORT. The xml requests for adding the keyword performance report definition are shown below:
<mutate xmlns="https://adwords.google.com/api/adwords/cm/v201003">
  <operations>
    <operator>ADD</operator>
    <operand>
      <selector>
        <fields>AdGroupId</fields>
        <fields>Id</fields>
        <fields>KeywordText</fields>
        <fields>KeywordMatchType</fields>
        <fields>IsNegative</fields>
      </selector>
      <reportName>Keyword performance report #1277353190509.28</reportName>
      <reportType>KEYWORDS_PERFORMANCE_REPORT</reportType>
      <dateRangeType>TODAY</dateRangeType>
      <downloadFormat>XML</downloadFormat>
    </operand>
  </operations>
</mutate>

The xml for placement report is identical to the one shown above, except for the reportType field. Note that have to use the KeywordText field to get the placement url while running MANAGED_PLACEMENTS_PERFORMANCE_REPORT.

Putting it all together

Here’s a code example that puts everything together to download your account structure using reports. The code snippet for parsing xmls and copying streams has been excluded for brevity.

Click here to expand

We've included support for ReportDefinitionService in all of our client libraries to help get you started, so please try it out and share your feedback with us on the forum or the projects' issue trackers.

-- Anash P. Oommen, AdWords API Team

Introduction to ReportDefinitionService

Tuesday, July 13, 2010


Reporting is an integral part of most AdWords API applications. To help you create and manage reports related to your AdWords campaigns, we introduced the ReportDefinitionService in the v201003 version of the AdWords API. In this post, we’ll cover the basics of working with the ReportDefinitionService. If you’ve used the v13 ReportService, you’ll notice that the new ReportDefinitionService differs in many ways from its v13 counterpart.

Retrieving the report fields

To create a report in the v201003 version of AdWords API, you pick a report type of your choice and then retrieve the list of supported fields using getReportFields. The Java code below shows how to retrieve the report fields:

// Get report fields.
ReportDefinitionField[] reportDefinitionFields = 
    reportDefinitionService.getReportFields(
        ReportDefinitionReportType.KEYWORDS_PERFORMANCE_REPORT);
// Display report fields.
System.out.println("Available fields for report:");
for (ReportDefinitionField reportDefinitionField : reportDefinitionFields) {
  System.out.print("\t" + reportDefinitionField.getFieldName() + "("
      + reportDefinitionField.getFieldType() + ")");
}

This feature is quite convenient if you want to display the list of supported fields for a user and allow the users to pick the desired fields. For those who prefer static documentation like in v13, we are working on it, and it will be made available in the near future.

Defining the report

To create a report in the v201003 version of AdWords API, you have to create a report definition first. This is different from the v13 version of API, where you could schedule a report without creating any definitions first. The following Java code snippet creates a report definition:

// Create ad group predicate.
Predicate adGroupPredicate = new Predicate();
adGroupPredicate.setField("AdGroupId");
adGroupPredicate.setOperator(PredicateOperator.EQUALS);
adGroupPredicate.setValues(new String[] {adGroupId});
 
// Create selector.
Selector selector = new Selector();
selector.setFields(new String[] {"AdGroupId", "Id", "KeywordText",
    "KeywordMatchType", "Impressions", "Clicks", "Cost"});
selector.setPredicates(new Predicate[] {adGroupPredicate});
selector.setDateRange(new DateRange(startDate, endDate));
 
// Create report definition.
ReportDefinition reportDefinition = new ReportDefinition();
reportDefinition.setReportName("Keywords performance report”);
reportDefinition.setDateRangeType(ReportDefinitionDateRangeType.CUSTOM_DATE);
reportDefinition.setReportType(ReportDefinitionReportType.KEYWORDS_PERFORMANCE_REPORT);
reportDefinition.setDownloadFormat(DownloadFormat.XML);
reportDefinition.setSelector(selector);
 
// Create operations.
ReportDefinitionOperation operation = new ReportDefinitionOperation();
operation.setOperand(reportDefinition);
operation.setOperator(Operator.ADD);
ReportDefinitionOperation[] operations = new ReportDefinitionOperation[]
    {operation};
 
// Add report definition.
ReportDefinition[] result = reportDefinitionService.mutate(operations);

When successful, the API call creates a report definition under the report section of your account’s Control Panel and Library. You may delete or modify a report definition using the mutate method, with operators as REMOVE and SET respectively. You can also retrieve all report definitions in your account using the get method.

If you’d like to inexpensively validate the report definition before adding it, you can call the mutate method as shown above with the validateOnly header set. This works similar to the validateReportJob method in v13 version of AdWords API. You can refer to our earlier blog post for details on how to use validateOnly headers to validate API calls.

The v201003 version of ReportDefinitionService introduces a new feature called predicates. You can use predicates to filter the results by any field with canFilter=true.. You can also use various operators to define the filtering condition. This is an improvement over v13 where the data could only be filtered using certain predefined fields like campaignId or adGroupId.

The v201003 version of ReportDefinitionService allows you to generate reports for predefined date ranges. This is very useful if you need to download a predefined type of report on a regular basis. For instance, you can create a single report definition with dateRangeType as YESTERDAY, and then use that report definition to download a daily report. This is an improvement over v13 where you needed to schedule a new report every day to accomplish the same task. If you need to download reports for a custom period, you can set dateRangeType as CUSTOM_DATE.

The v201003 ReportDefinitionService currently has some limitations: it lacks support for aggregation types and cross client reports. We are working to add support for these features in a future version of the AdWords API.

Generating and downloading a report

To download a report in the v201003 version of AdWords API, you need to issue a HTTP GET to https://adwords.google.com/api/adwords/reportdownload?__rd=reportDefinitionId. You can get the report definition id from the result returned by the server when you add a report definition as shown in the code sample above. In addition, you need to specify authToken and clientLogin (or clientCustomerId) as http headers to authorize the report download. Note that the clientLogin or clientCustomerId must be the same as the one used while creating the report definition. The sample Java code below downloads a report:

String url = "https://adwords.google.com/api/adwords/reportdownload?__rd="
    + reportDefinitionId;
 HttpURLConnection urlConn = 
    (HttpURLConnection) new URL(url).openConnection();
urlConn.setRequestMethod ("GET");
urlConn.setRequestProperty("Authorization", "GoogleLogin auth="
    + user.getRegisteredAuthToken());
if (user.getClientCustomerId() != null) {
  urlConn.setRequestProperty("clientCustomerId", user.getClientCustomerId());
} else if (user.getClientEmail() != null) {
  urlConn.setRequestProperty("clientEmail", user.getClientEmail());
} else {
  urlConn.setRequestProperty("clientEmail", user.getEmail());
}
urlConn.connect();
copyStream(urlConn.getInputStream(), new FileOutputStream(
    new File(outputFileName)));

The raw HTTP message when executing this code is as shown below:

GET /api/adwords/reportdownload?__rd=XXXXXX HTTP/1.1
Host: adwords.google.com
Accept: */*
Authorization: GoogleLogin auth=XXXXXX
clientEmail: XXXXXX

A major difference between this approach and the v13 approach is that the report generation is inherently synchronous, unlike in v13 where you had to poll regularly using getReportJobStatus to see if a report job completed, and later use getReportDownloadUrl to get the report download url. Also, the report generation in v201003 is much faster than in v13, and most reports complete in a few seconds. The request to download reports will time out after 3 minutes, so we recommend that you use gzipped csv format to minimize transfer size of potentially large reports. GZipped XML format is not yet supported, but we’re working to include this feature in a future version of the AdWords API.

We've included support for ReportDefinitionService in all of our client libraries to help get you started, so please try it out and share your feedback with us on the forum or the projects' issue trackers.

-- Anash P. Oommen, AdWords API Team

Discover v2009: Working with AuthTokens

Thursday, July 01, 2010


Authorization tokens were introduced with v2009 as part of the authorization mechanism adopted for the new version, ClientLogin. This new approach allows us to separate authorization from API requests, making it a two step process: first retrieve a token, and then include it with your requests.


How ClientLogin works


ClientLogin was designed as a standalone authentication and authorization procedure that returns a token, if successful. This token is then supplied together with the requests as a form of proof that the user is who they claim to be, and that they are allowed access to the API. Since both the authorization and the requests happen over SSL connections, there is no risk of anyone retrieving the token and impersonating the user.

Explaining how to implement a full ClientLogin handler is outside the scope of this blog post, but for the full details on the protocol please consult the documentation. Here is a small example of how to log in using the AuthToken module included with the Ruby client library (which can be used outside of it, since its only dependency is the httpclient gem):

require 'authtoken'

auth_token = AdWords::AuthToken.get_token(email, password,

'www.google.com', 443, true)


Other client library projects include NoClientLib pages in their wikis detailing how to perform the ClientLogin step (as well as all other steps in a successful request) without the client library.

If you’re using the client libraries, the ClientLogin step is usually handled for you, although you’ll still need to keep an eye out for any errors. The Readme file included with your client library has more details.

Remember to reuse the authorization tokens


This is a point of some confusion among developers, and illustrates one of the key differences in authorization between v13 and v2009: In v13, you authenticated and authorized every request by sending your account credentials as part of the header; in v2009, you authenticate with ClientLogin beforehand and then authorize your request to the AdWords API by supplying the token generated by ClientLogin.

A naive migration from v13 to v2009 would generate a new token for every request and insert it into the header. This is a bad idea. Such an implementation will very likely cause your software to run into CAPTCHAs and block your accounts until the CAPTCHAs are resolved.

A correct implementation would generate the token, cache it, and reuse it for all subsequent requests with that email address, even against the sandbox.

Handling expiration and errors


Tokens don’t last forever; as a security mechanism, they only last up to two weeks. Do bear in mind, though, that this doesn’t mean they’ll last a full two weeks: Authorization tokens can at any time be revoked for any number of reasons (usually for security). If a token has been revoked, you’ll get back an AuthenticationError with the reason GOOGLE_ACCOUNT_COOKIE_INVALID.

Most of these errors can be handled simply by requesting a new authorization token, but if you get back a CaptchaRequired error when performing the ClientLogin interactions, you’ll need the user to resolve the CAPTCHA before you’re allowed generation of new tokens (or wait approximately 5 minutes, after which it will work again). The error message will include a URL to the image and a CAPTCHA token, so you can embed this directly into your application, without having to redirect the user to a web page. Check the ClientLogin documentation for details.

What to keep in mind when implementing your application


When implementing your application, keep these best practices in mind:
  • Generate a new token before the first request your application makes. You’ll need a token for every different email header that you use, so:
    • if you log in by setting the email header to your MCC (My Client Center) account and then change the clientEmail header to the different accounts you’re accessing, you’ll only need a single token;
    • if you log in by setting the email and password headers directly to those of the different accounts you’re accessing, you’ll need a new token for each.
  • Reuse this token in subsequent requests.
  • If you receive an AuthenticationError back, take a look at the reason:
    • if it indicates an expired token, try generating a new one and proceeding;
    • if it indicates an unrecoverable error, present the information to the user, and let them decide how to proceed.


Finally, if you’re using one of our client libraries, then we have good news for you: the first two steps are already taken care of! Just set your credentials according to the instructions for your specific client library, and it will manage both v13 and v2009 authorization for you.

Keep in mind that if you have any further questions or run into any other authorization issues, the documentation and the forum are the right place to go!


--Sérgio Gomes, AdWords API Team



The Broad Match Modifier and the API

Thursday, July 01, 2010


Back in May, we announced the new “broad match modifier” in AdWords, which lets you create keywords with more reach than phrase match and more control than broad match. For now, this feature is only available in the UK and Canada, but it will be rolled out to more countries this summer.

We’ve heard developers asking when this will be supported in the API, so we want to let you know that it already is. To use the broad match modifier, simply add plus signs (+) to your keyword text as desired. Because these keywords are still considered BROAD match, no additions to the KeywordMatchType enum are necessary.

Unless you prohibit the entry of the plus symbol (+) character in keyword text fields or reports, you may not need to change your code at all. However, we recommend testing to make sure.

For more information on the broad match modifier, visit the AdWords Help Center. And as always, you can post any questions you have regarding the API to the forum.

-Aaron Karp, AdWords API Team