Why Testing Matters

As a digital marketer, your concern is making the customer journey as frictionless as it can be but in order to accomplish this, you’ll need to try as many new strategies as possible. This, of course, sets up your digital testing strategy to lead the way for learning what makes the best experience for your consumers.

Testing is an opportunity to explore something that we don’t already know the answer to, allowing us the safe environment to confirm or disprove. An infinite amount of hypothesis and outcomes exist with each test, which in turn allows us to gain additive learnings across all our programs.

But why do we test things?

It seems like a simple answer, but oftentimes, the answer is obscured by all the nuances behind how we test things. Test design, hypothesis, statistical significance, etc. (all of which are important to a successful test) are all causes for misleading initiatives. Ultimately, we test to learn.

The point of every test we run should be to learn something about what we’re testing. Not just if it works, but how it works and why it aligns to our client’s strategic direction. All of this is becoming the norm and the new standard for capturing and maintaining customers. The best digital strategy is not an assortment of one-off campaigns, quick customer acquisition ideas or growth hacks. Sure, those might get you some quick wins but they will not stand the test of time. It’s this need to learn that drives the direction for a culture of testing.

At PMG, we use testing as a gateway to finding iterative incremental efficiencies both in the digital media and across the strategic business objectives for our clients. To steal a line from the ever-quoted Simon Sinek: “You have to know why you are doing something not just how.”

Generally speaking, larger scale initiatives come from the top down, and instead of playing telephone, we like to get an understanding of the strategic business direction. This is because it is of the utmost importance for us to align our digital marketing efforts with the strategic direction of the overall business. Simply put — if we have no direction from the business objectives then why does a test even matter?

This sense of direction allows us to better choose business objectives mapped to the digital levers we have control over in our campaign management. By design, the initial conversations with client teams revolve around making sure we have a holistic understanding of the state of the business before we get started. Because of this, we are able to prioritize what will be the most impactful to overall business objectives.

A clean test is paramount to the effective outcome (whether positive or negative) of a test.

At a high level, we make sure we are asking the following questions before a launch:

Is there mutual exclusivity with what we are testing?

Are the test/control groups random?

Have we chosen a specific primary and secondary KPI?

Are we set up to measure the results correctly?

These questions ensure that whatever the results of the test are, we can trust the methodology behind them.

After the test is complete, our data science team looks at both the statistical significance of a result and the incremental percentage of the results. Understanding both metrics allows us to properly scope the expected impact of the test results. An important note here is that we make a hard distinction between statistical significance and practical significance.

Although we are diligent about having a clean statistical approach, we understand that at the end of the day, results are going to be subjective from the bottom up. Due to this, we believe it is okay to have statistically significant results that warrant further investigation.

Sometimes, we learn things we weren’t even looking or testing for but we take the knowledge the test delivers to us and use it to create more tests. Allowing these ancillary benefits steps us in the right direction for future iterations  — one of the most embraced tenants of PMG’s testing approach. Good luck and happy testing!