In short: If you’re looking to establish a culture of experimentation – a prevailing “we can always improve” mindset – that both fuels cross-team collaboration and continuously improves the bottom line, it’s important to think well beyond the centralized experimentation team.
Building a strong core group that facilitates experiments – often called a Lab or CoE – is important to building the capability and capacity to execute experiments. But building culture hinges on a broad-based approach to instilling a way of working, and to develop a flourishing culture many pockets outside of this core operation need to get in on the act.
Here are 10 ways to cement an experimentation culture across a company
1. Present testing capabilities as a resource for groups to maximize their own performance
For builders and designers of products, applications and websites, experimentation can be interpreted as some kind of an audit. A tire kicking of their own work. Since being questioned is often an uncomfortable feeling, a resistance to participate can naturally develop.
This feeling of being second-guessed must be nipped in the bud by presenting experimentation as a resource for each group to fine tune, to think big and to be at their best. It’s a friend – not an enemy. How testing is baked into different parts of the company and how the capability is offered and presented is all very meaningful in terms of driving long-lasting outcomes.
2. Catalog insights into a single, easily-searchable repository
When experiments are executed in different channels, teams and geographies, many valuable learnings are developed, but are often not shared broadly across different teams. This lack of knowledge sharing suppresses the return on investment for the execution work.
By using a centralized and broadly accessible insights repository that makes learnings easy to find and quick to understand, internal knowledge spreads and more of the workforce becomes positioned to take action on the insights, and their underlying data.
Further, cascading the background and results of experiments to those outside of the core experimentation circles, can inspire more people to engage with the experimentation practice. They see the possibilities and start to think through how the practice can be applied to their every day work. The result is more engagement with the program and practice.
3. Develop an experimentation advocate in as many teams as possible
There are many groups that are often outside of the testing CoE at a company that can and should develop a testing mentality. The content, design, development, promotions, pricing, social media and paid search teams are just some examples.
Advanced experimentation cultures extend into groups where testing hasn’t existed traditionally – in operations, corporate strategy, human resources, as some examples.
What’s important here to keep in mind is that experimentation is a mindset and a framework of measurement to see how one or more approaches of doing something stacks up against another. So developing this mindset is not limited to Marketing, or any one group.
4. Use feature flags for testing in applications
Whether it is mobile app testing, testing in a SaaS product or the code that powers the operations of your business, experimentation can be leveraged through feature flags. Feature flags are toggles that enable and disable feature-specific code blocks so that you can test features and functions similar to how a marketer might test different content on a campaign landing page.
Leveraging feature flags helps to assure a bug-free user experience, enables the analysis of how a feature is being adopted and its impact on overall application metrics. But beyond these direct benefits, it brings the practice and mindset of experimentation to new groups that were often not exposed to A/B testing in the past. VWO, GrowthBook and Flagsmith are just a few examples of platforms that offer feature flag testing.
5. Develop an ideation mechanism that cuts across teams
Covid changed the world in many ways. For most companies, whether it’s fully remote or hybrid, the workforce is more distributed geographically than it was previously. This can hinder camaraderie and communications. A way to counter these forces is to develop meaningful ways in which team members can collaborate with colleagues that are both on their own team, but also on other teams.
With common goals to focus on, having the courage to throw ideas around – debate ideas, build on others’ ideas, point out cautions or caveats that need to be considered – is all healthy in a multitude of ways. No matter where individuals are stationed – be it in attics, basements, cafes or headquarters – ideating towards common goals with others develops feelings of togetherness and cohesion.
It also helps to crowdsource and prioritize the best ideas and the best ideas have great impact on the bottom line. Experimentation as a practice can fuel this cross-team ideation process. Use tools like Datatinga, Trello and others to store, share, prioritize and take action on ideas.
6. Pick your spots, but experiment everywhere
A/B testing in the digital world was pioneered by platforms like Optimost (now part of OpenText) and Offermatica (now called Adobe Target) – both platforms that were heavily geared for Marketing in those early days. Over the last decade, thanks in part to the well-documented successes of experimentation programs at Google, Netflix, Airbnb, Booking.com, Microsoft and others, the practice of testing has permeated many functional and geographical units of a company.
Some will say “test everywhere and test all the time.” This is simply not practical. You still need to pick your spots. Testing that can be concluded in reasonable timeframes (2 weeks to 2 months – no more) require a minimum test population and expected number of “conversions” – whatever your experiment’s primary KPI is. So testing all the time and everywhere just does not make sense.
While we need to pick our spots, there is some benefit to broadly testing everywhere in terms of where company activity is. Every business unit, geography and channel should be able to come up with at least a few scenarios a year where different approaches are considered, executed and analyzed against each other.
It’s the spirit of recognizing that we don’t always have the answers unless and until we put different approaches into the wild. This mindset and practice can and should be everywhere. And when it is, a culture can flourish.
7. Recognize great experimentation use cases and behavior
A great way to fuel a pattern of behavior is to reward and recognize that good behavior. What works with influencing kids with awaiting treats and employee-of-the-month bulletin boards by store rest rooms also works for experimentation programs.
Recognizing and even financially rewarding individuals for participating in the practice of experimentation generates signals that energize others to follow suit. This newfound energy comes not just from incentives to be recognized and rewarded but to experience experimentation and demonstrate how it can make themselves and their team even stronger contributors to the company.
Having proof of direct impact on a company is gratifying. Experimentation stories – whether it’s about the experiment or the people that propelled the experiment (or both) spur curiosity. And curiosity is healthy for employee engagement and the company bottom line.
8. Develop an unrelenting focus on goals
Experiment ideas should be rooted in data. And the experiments themselves produce more data. All of this data must be tethered to a goal or set of goals. And the goal should roll up in some way to the most overarching company goals. To energize the workforce around experimentation, there must be a strong goal focus to start.
And it can’t stop where it starts. For example, sometimes goals are discussed at the beginning of a fiscal year and socialized a few times here and there over the course of the year. Great experimentation cultures have an unrelenting focus on goals, because to ensure experimentation has meaningful impact on outcomes, the north star goals – and the current progress towards them – must be presented like a stadium scoreboard on a regular basis.
This laser focus on goals is good for maximizing performance, but it’s also important for generating an experimentation mindset.
9. Commit some development team hours to the practice of experimentation
Development teams that focus on company websites, mobile apps or enterprise functions that power a company are a key cog in an organization’s business engine. Since there is so much demand on development it becomes convenient and sometimes easy for these teams to de-prioritize experimentation. When deadlines mount and scope creeps, experimentation may feel like a “nice to have” type of initiative.
But what separates the best companies from the rest is their insistence on ingraining experimentation into development circles and cycles, so that it never gets victimized by competing priorities. A good approach is to siphon off a certain percentage of development resource time – 10% would be a good starting point – into the experimentation practice in one way or another.
Ensure a minimum out of 1 out of every 10 tickets approved has experimentation implications. Ensure a minimum of 1 out of every 10 developers is directly involved in the experimentation practice in some way or another. This could be setting up experiments or working on user experiences that would serve as an alternative approach to the default, baseline experience. Or it could be interfacing with the core testing CoE to ensure a speedy transition from successful test to successful implementation of the winning experience.
Meet prioritization challenges head on by instituting a minimum level of experimentation focus from your development resources. It has to be systemically managed at an organizational level to ensure experimentation gets the minimum amount of attention for culture to truly spread – and stick.
10. Wireframe and design and write with an experimentation mindset
Whoever is producing the creative goods – be it information architects, designers or copywriters – have them craft their product with experimentation in mind – from the outset. Often companies develop test ideas weeks, months or years “after the fact” – and this creates a feeling of tire kicking and conflict.
Instead the creative producers should be scheming on alternative approaches early on in their creative process. Their alternatives are future test ideas. Regular communications between these teams and the centralized testing operation is key to ensuring this sustained mindset. This more proactive experimentation mindset removes the tire kicking feeling and helps to embed testing DNA into creative circles. Doing so also helps to instill this experimentation culture.