6 tips on how to build a test & learn program

In short, if you are looking to build a test & learn program from within, and from scratch, here are 6 tips to consider.  LiftCentro’s Jake Sapirstein built and ran the test & learn program at a Top 25 global brand and, here, he distills 6 nuggets on how to establish brand-side experimentation programs. 

 

 

For those that prefer to read, below is a transcript of the above video.

 

Introduction

Hi, I’m Jake Sapirstein from LiftCentro.  LiftCentro is a digital marketing consultancy, and we focus heavily on helping brands build, and really scale out, test & learn – or “experimentation” – programs.

After having built and run a test & learn program for a large enterprise company for several years, I’m often asked for tips and techniques on how to build and scale out such a program.  What I thought I’d do in the next few minutes is package together 6 tips on how to build a test & learn program. Let’s dive in.

Tip 1 – Find Analytical Minds

Tip number 1 is to find analytical minds.  Now when you’re building a program from scratch to do experimentation work, you may not have open head count. You may have to be resourceful, and look from within to build a team. So the area you want to focus on are people that have an analytical inclination or way of thinking.

You can look at analytics teams – that’s a good starting place.  But these days, analytical minds are really scattered throughout the organization – different teams, different functions.  Some people just like to look at data more than others.  Some people have a comfort level with data that’s greater than others. You want to find those minds and see if you can pull in a couple of those people to help form the initial experimentation team.

Now why focus on analytical mindsets? One reason is analytics is part of experimentation.  So when you doing an experiment, you are actually producing data, and that data needs to be analyzed.  Also, when you are coming up with test concepts, different ideas for testing, the best ideas are really rooted in some form of data. So you want to have people that have a comfort level looking at data, being able to think of concepts of how to improve, where to improve, where to test – based on data.

The other, less obvious, reason you want to focus on people with analytical mindsets, is that people that are looking at data tend to be people that are focused on improvement – or how to improve.  They are looking at data, where the problems are, where the high bounce rates are, for example on a website – that’s just an example – but people that are focused on data are often looking at a snapshot of a baseline, of where you are at today in your journey to reach certain organizational goals.

So finding people with analytical mindsets is a great place to start.

Tip 2 – Develop a Core Centralized Team

Now tip number 2 is to build a core centralized team. So it’s really important, even from the get go, to have testing expertise.  Now, at the beginning, you may have to have external resources augment your internal team in order to accomplish that. So usually your initial core, centralized team is going to comprise of a mix of internal and external resources to drive the program.

You need to have that expertise from the beginning on how to set up an experiment, how to design an experiment, understand things like test sample size and duration, being able to come up with sound test concepts from actual data.  So mining quant and qual research to formulate hypotheses – to develop sound test concepts.  You want to have that expertise, for different people, from different parts of the company to turn to, to drive that testing execution.

Additional note: some de-centralization is recommended to expand programs in later phases, but to start a program, it’s important to develop a centralized core team.

Tip 3 – Develop T&L Ambassadors

So Tip number 3 – develop Test & Learn ambassadors.  So what do I mean by an ambassador?  An ambassador is not someone that gets paid to do experimentation. They have day jobs.  And they are in different parts and pockets of the larger organization. Now they do have a testing spirit.  That it is really important trait you want to look for before assigning people these roles – which happens to be in addition to their regular job.

So the idea of having these ambassadors,  is not only to bring test concepts to the centralized team, coming from the area of the company that they sit in – and that could be the design team, it could the development team.  It could be a different line of business or geography.  Really whatever pocket that they sit in, they are acting as a conduit to bring test concepts to the table.

In addition to that, their role is to evangelize and remind people of the practice, of the capability of experimentation.  And this is really important, because a lot of times, what happens is, people naturally think about testing think about experimentation late in a process.  So after a webpage has been deployed, we naturally will start to think about that as a baseline, and how to beat that baseline.  How to get better. How to improve. How to drive more engagement.  How to drive more sales from a product page.

Naturally, we think about testing more late in a process, after things have been deployed.  But really great programs and cultures are developed when you can start to get people thinking about experimentation early in a process.  So when requirements are being gathered and defined for a web page, or a product, or a launch.

You want to start thinking about ideas – alternative ways of doing things – early in a process, and having ambassadors planted throughout the organization, helps to remind people to think about the testing practice, the testing possibilities, early in processes.

Tip 4 – Be Goal-Centric

Point number 4 is to be goal-centric. And by goal-centric I don’t just mean having goals, but actually touting goals, again and again – over time.  That can be management cascading goals to the team doing execution work, on a monthly or quarterly basis.

And the idea is to remind people not only the presence of the goals, but what those goals are, and what the progress is, at that point in time, towards those goals.  Now why is this important in the context of experimentation?

When you are touting goals again and again, it instills a performance-oriented mindset.  And when people have that performance-oriented mindset, they are naturally going to be thinking about ideas, how to improve, how to meet those goals faster.

And when you start ideating – coming up with concepts, new ideas, thinking out of the box – it’s a natural segue to think about experimentation.  Because it is experimentation that tells us, with hard data, what tactics are working really well, what tactics are not working well, which ones we need to stop – right now.  It’s experimentation that guides us on how to reach our goals faster. So it all starts with goals and being goal focused.

Tip 5 – Communicate Learnings Broadly

Communicate learnings broadly.  This is really important to get the quant and qual research learnings beyond that research core, and into different pockets and corners of the company.  We think this is so important, we are actually building a platform called Datatinga™ to help brands cascade those learnings farther and wider across the organization.

The reason why this is so important is really twofold – one is more obvious, which is the more people that are aware of the insights and the underlying data, the more people are in position to take action on the data.  And more action is going to drive more incremental value from that research.

The other reason why it’s important to cascade insights is a little less obvious.  And that is when a lot of people are not familiar with the insights, not familiar with the research or even the programs – like experimentation – behind the research – what happens is they get surprised by some of the results (of tests for example).

A natural response to surprises is a to develop a sense of curiosity.  Curiosity starts to focus on the possibilities of what you could test, going forward.  So what I have found over the years is that as you as you cascade insights farther and broader across the organization, driving more reach, you are going to tend to get more engagement with experimentation going forward.

More engagement, means more experiments. More experiments means more business value.  It all really starts with cascading the learnings farther and wider across the organization.

Tip 6 – Test Everywhere

Tip number 6 is to test everywhere.  And by this, I don’t mean to test every copy block on every web page – or even every web page.  But what I mean is going beyond website testing.  Going beyond Marketing.  A lot of programs start in Marketing – focused on design and copy.  And these are fine places to start a testing program.

But when you want to build and scale out a really comprehensive testing program, it really comes down to developing a mindset of coming up with alternative approaches to doing everything that is done. To be able to experiment, to measure and to validate your way forward – that’s really the key.

Conclusion

So that’s a wrap.  Those are 6 tips on how to build a test & learn program. If you want more insights like this, we have a paper called 20 keys to building a data-driven culture and that can be found on LiftCentro.com.  Good luck on building out your test & learn program.