Part 2: Why Legacy Analytics Tools Are Failing Marketers

Exploring the relationship between experimentation and action

(Ed note: This is the second installment of a four-part series discussing how AI marketing technology disrupts the traditional analytics continuum with Amplero’s head of product Garrett Tenold. In “Part 1: Moving marketers out of neutral,” we covered the strengths and limitations of legacy analytics and business intelligence tools, as well as the current interplay between analytics and human input.)

In our last post, we reviewed how a marketer needs to focus on how they take action.

So perhaps the new challenge becomes: how do I as a marketer know which actions to take? How do I learn the right way to act or influence each individual customer such that (in aggregate) I drive maximal outcomes?

Traditionally, the marketer might assert they know the right thing to do. Experts. Highest paid person’s opinion (HiPPO)! They take those assumptions and execute—often by defining a segment or writing up a chain of marketing automation rules.

Or, as might be popular today, they succumb to the hype: “if only I get all my data into a data lake, something magical will happen.” But traditional models are not enough—often demonstrating correlations in the base behavior rather than whether the marketer’s actions are influencing the outcome.

Imagine a big data initiative gathering fine-grained data about your city. A data science team at a workbench might train a model which finds that on days where people are carrying umbrellas, it’s more likely to rain. This is very true and indeed useful. I can use the umbrella carrying ratio (UCR) to predict the likelihood of rain.

But this misses the action question: If I intervene and hand out umbrellas on a given day, does it change the actual probability of rain? Here, correlation does not equal causation.

Transitioning back to marketing, users who have a particular behavioral pattern are likely to migrate to the next feature adoption or subscription tier. In a financial services context, this could be a gold card. In gaming, this could be moving from player to payer. In telecommunications, we’re looking at larger data plans.

For example, what change in the naturally occurring feature adoption rate will occur if I give a 10% off discount, communicated on email template #10 to users on the silver plan?

The question is whether these users would migrate to the next tier without action on behalf of the marketer or will they need a nudge to do so? If it’s the former, it’s often better to let sleeping dogs lie.

When we want to learn about our actions and how they really influence the world, it’s critical to separate that which would have naturally occurred from what the marketer causes to occur.

When we want rigorous understanding of casuality, we run tests. Imagine we’re dealing with medicine—does a particular treatment cause a change in the outcome for population of patients, half of whom will receive the treatment and half whom will get a placebo?

Marketing experimentation

A randomized controlled test measures whether the intervention does in fact change the outcome

  • Isolate & measure
  • Identical setup except for the intervention
  • Observe the outcomes for users that were targeted as well as user’s who were left as a control.

Examining the Gartner framework demonstrating the ascending path from descriptive to prescriptive analytics, it’s crucial to use testing and experimentation to learn how to influence.

While most marketers understand the mandate for experimentation, there are considerable limitations to both the traditional approach to optimization strategy and conventional testing and personalization tools on the market, such as Adobe Target, Optimizely, Maxymiser (Oracle), and Monetate.

To summarize, the marketer’s job is to take action. To learn to do this more effectively, they need to use testing and experimentation to isolate and directly measure the impacts their marketing is causing. Stay tuned for the next installment of the series where we’ll look at why this is not just painful today, but actually getting worse.

About Garrett

As head of product for Amplero, Garrett dwells at the forefront of the machine learning-powered marketing revolution. Building the Amplero Digital Intelligence Platform since its inception, Garrett is a SaaS product leader focused on marketing transformation with deep experience across both MarTech and AdTech industries.

With more than a decade focused on product innovation, he’s built forward-focused technology and analytical solutions for brands such as XBox, Sprint, American Express, Citibank, and Ganett. Prior to Amplero, he held product leadership roles at Microsoft Advertising, AdReady, and aQuantive.

Contact him directly at

About Amplero

Headquartered in Seattle, Amplero is an Artificial Intelligence Marketing (AIM) company that enables business-to-consumer (B2C) marketers at global brands to optimize customer lifetime value at a scale that is not humanly possible.

Unlike traditional rules-based marketing automation systems, Amplero’s Artificial Intelligence Marketing Platform leverages machine learning and multi-armed bandit experimentation to dynamically test thousands of permutations to adaptively optimize every customer interaction and maximize customer lifetime value and loyalty.

With Amplero, marketers in competitive, customer-obsessed industries like telecom, banking, gaming and consumer tech are currently seeing measurable lift across key performance indicators—including 1-3% incremental growth in customer topline revenue and 3-5x lift in retention rates.

For more information, contact us today or follow us on Twitter.