• About Us
    • New York
  • Work
  • Capabilities
  • Careers
  • Technology
  • Blog
  • Contact Us
  • About Us
    • New York
  • Work
  • Capabilities
  • Careers
  • Technology
  • Blog
  • Contact Us
October 31, 2018

Why Testing Matters

Posted by John Stewart

As a digital marketer, your concern is making the customer journey as frictionless as it can be but in order to accomplish this, you’ll need to try as many new strategies as possible. This, of course, sets up your digital testing strategy to lead the way for learning what makes the best experience for your consumers.

Why Testing?

Testing is an opportunity to explore something that we don’t already know the answer to, allowing us the safe environment to confirm or disprove. An infinite amount of hypothesis and outcomes exist with each test, which in turn allows us to gain additive learnings across all our programs.

But why do we test things?

It seems like a simple answer, but oftentimes, the answer is obscured by all the nuances behind how we test things. Test design, hypothesis, statistical significance, etc. (all of which are important to a successful test) are all causes for misleading initiatives. Ultimately, we test to learn.

Why Testing Matters

The point of every test we run should be to learn something about what we’re testing. Not just if it works, but how it works and why it aligns to our client’s strategic direction. All of this is becoming the norm and the new standard for capturing and maintaining customers. The best digital strategy is not an assortment of one-off campaigns, quick customer acquisition ideas or growth hacks. Sure, those might get you some quick wins but they will not stand the test of time. It’s this need to learn that drives the direction for a culture of testing.

PMG’s Approach to Testing

At PMG, we use testing as a gateway to finding iterative incremental efficiencies both in the digital media and across the strategic business objectives for our clients. To steal a line from the ever-quoted Simon Sinek: “You have to know why you are doing something not just how.”

Generally speaking, larger scale initiatives come from the top down, and instead of playing telephone, we like to get an understanding of the strategic business direction. This is because it is of the utmost importance for us to align our digital marketing efforts with the strategic direction of the overall business. Simply put — if we have no direction from the business objectives then why does a test even matter?

This sense of direction allows us to better choose business objectives mapped to the digital levers we have control over in our campaign management. By design, the initial conversations with client teams revolve around making sure we have a holistic understanding of the state of the business before we get started. Because of this, we are able to prioritize what will be the most impactful to overall business objectives.

Running A Clean Test & Understanding Results

A clean test is paramount to the effective outcome (whether positive or negative) of a test.

At a high level, we make sure we are asking the following questions before a launch:

  • Is there mutual exclusivity with what we are testing?
  • Are the test/control groups random?
  • Have we chosen a specific primary and secondary KPI?
  • Are we set up to measure the results correctly?

These questions ensure that whatever the results of the test are, we can trust the methodology behind them.

After the test is complete, our data science team looks at both the statistical significance of a result and the incremental percentage of the results. Understanding both metrics allows us to properly scope the expected impact of the test results. An important note here is that we make a hard distinction between statistical significance and practical significance.

Although we are diligent about having a clean statistical approach, we understand that at the end of the day, results are going to be subjective from the bottom up. Due to this, we believe it is okay to have statistically significant results that warrant further investigation.

Sometimes, we learn things we weren’t even looking or testing for but we take the knowledge the test delivers to us and use it to create more tests. Allowing these ancillary benefits steps us in the right direction for future iterations  — one of the most embraced tenants of PMG’s testing approach. Good luck and happy testing!

data analysisDigital Strategylearningtechnologytesting
Previous
Next

Latest White Papers

  • Shifting Plans for 2020 & Beyond
  • Game On: How Brands Can Log Into A Diverse Multi-Billion Dollar Industry
  • What CCPA Means For Brands
  • How Google is Improving Consumer Data Privacy
  • Ways to Prepare for the Cookieless Future
  • See all White Papers

Featured Posts

  • Ad Age Names PMG #1 Best Place to Work in 2021
  • Hindsight 2020 & Looking Ahead to 2021
  • Preparing for Streaming’s Growth & The Future of TV Buying
  • MediaPost Names PMG Independent Agency of the Year
  • PMG Client Portfolio Trends During Amazon Prime Day 2020

Categories

  • Consumer Insights
  • Content
  • Creative Design
  • Data Analytics
  • Development
  • Digital TV & Video
  • Ecommerce
  • Industry News
  • Local
  • Mobile
  • Paid Search
  • PMG Culture
  • Programmatic & Display
  • SEO
  • Social Media
  • Structured Data
Fort Worth

2845 West 7th Street
Fort Worth, TX 76107

Dallas

3102 Oak Lawn Avenue
Suite 650
Dallas, TX 75219

Austin

823 Congress Avenue
Suite 800
Austin, TX 78701

London

33 Broadwick Street
London
W1F 0DQ

New York

120 East 23rd Street
New York, NY 10010

Get in touch

(817) 420 9970
info@pmg.com

Subscribe to the PMG Newsletter
© 2021 PMG Worldwide, LLC, All Rights Reserved
  • Contact
  • Privacy Policy
 Tweet
 Share
 Tweet
 Share
 Tweet
 Share
 LinkedIn
We and our partners use cookies to personalize content, analyze traffic, and deliver ads. By using our website, you agree to the use of cookies as described in our Cookie Policy.