A New Strategy for Testing Google AMP

In May, Google announced a beta rollout of Accelerated Mobile Pages (AMP) pages in paid search ads.

At a high-level, it’s all the more reason to build AMP versions of your mobile site since visits from Google could account for more than half a website’s traffic. More competitors will start rolling out AMP pages, which could be a competitive disadvantage.

AMP pages are lightning-fast. Quicker page load times means better user engagement. Customers expect fast mobile web experiences, and when they don’t get them, they abandon your site. In fact, a two-second delay in web page load time can increase bounce rates by 103 percent according to Akamai.

If you haven’t rolled out AMP pages, we’ve previously discussed the considerations needed for investing in AMP. This announcement could add another, likely better, method of using real conversion rate data to make the business case for AMP.

The AdWords environment is a good way to run controlled testing. It can allow you to compare one group of pages to another to see how AMP performs vs. non-AMP pages.

Attempting to pull off a test in the organic search results with AMP is not a great option; since it would be very difficult to implement and wouldn’t actually be controlled. Because of the way Google shows organic AMP results, a standard A/B testing tool wouldn’t cut it here.

Depending on how the beta is structured, it remains to be seen if Google will allow true A/B testing of their version of the AMP experience (which includes a bar at the top of the page linking back to Google) vs. your own paid search pages. If not, you could run them on the same domain by simply using your /amp/ version as a landing page. This should give you a good approximation of the bottom line conversion rate impact without a ton of setup.

For a dynamic site, building out a single template is one approach, but it can still be a large workload and a large hurdle for internal buy-in. An alternative is to simply build a set of static HTML AMP pages. But how would that work?

Select a sample of the pages you are going to work on (products, properties, etc.) Divide them into your control and test groups.

Pull in the data you’ll need to pre-populate the static HTML pages. This can be done by scraping the site or getting a structured data feed delivered.

AMP pages won’t have the exact look and feel of your current mobile pages. Build a mockup to be signed off on by the key decision-makers.

Write a script to churn out these pages in AMP format and validate that they work and look as expected.

Upload the pages to a destination on the site, such as an “/amp” directory.

Set up your AdWords campaign and review the results of the new pages vs. old pages.

Did AMP win? Now there is a business case to do a full development which can replace your organic and paid search results. Did your non-AMP mobile experience win? In this case, you’ve likely saved time and money on a full AMP build out.

While there is a bit of development work involved, it is a much easier task to write a simple script that can be run on a laptop compared to building a process that works within your existing site platform or CMS.