Experimentation
Culture Awards

Globally awarding your growth in experimentation culture
Sharing inspirational stories by the nominees of 2022
during the July 7th, 6pm UTC live virtual broadcast

During our live broadcast we will raffle free access to
Experimentation Culture Awards Lifetime Achievement winner
Ronny Kohavi his 5*2 hours live class on A/B-testing

THE 2022 AWARD NOMINEES

The case submit deadline for the 2022 experimentation culture awards was May 21st 2022. We received 54 case submissions.The 2022 jury went through all the cases and rated each of them on a 1 to 5 scale (1= should not win, 5 = should win). Jury members were not allowed to vote on cases by friends, family, colleagues or clients. The top 5 in each category (based on average score) became the finalists for the awards.Cases that did not end up in the top 5, but scored a 3.5 on average without a single jury member scoring below 3 will receive an honorable mention (finalists also need this score).All the finalists will be interviewed. Based on these interviews the jury will select a winner in each category and also select the winners for the community and the rising star award. The winners will be announced in the live broadcast of July 7th.Register your free ticket to be able to view the live broadcast.Looking for the 2021 nominees & winners? Find them here!

INDIVIDUAL NOMINEES

Experimentation Culture Awards 2022 Individual award nominations
  • Amy Hartwig (US) - For growing experimentation at Microsoft DevDiv.

  • Tom van den Berg (NL) - For CRO weekly: a curated newsletter to help increase the quality of experimentation throughout the market.

  • Ana Catarina Cizilio (BR) - For empowering A/B-testing in Brazil.

  • Bjarn Brunenberg (NL) - For experiment program spitting fire at TomTom.

  • Ellie Hughes (UK) - For becoming an experimentation evangelist at AND Digital

Honorable mention for:

  • Marianne Stjernvall (SE) - For sharing in-house lessons of building an experimentation culture.

Click on the nominee to read the submitted case descriptions
or see the 2021 individual award nominees & winner

TEAM NOMINEES

Experimentation Culture Awards 2022 Team award nominations
  • Vista (US) - For the Experimentation Hub: "Building a culture of experimentation in public".

  • SMG (CH) - For the CRO Team: "Inform, inspire, involve, iterate. Revive & expand a culture of experimentation".

  • Videoland (NL) - For the Experimentation Team: "From 5.5 experiments per month to a center of excellence".

  • Accor (FR) - For the Experimentation Department: "Making experimentation great(ly known/used)".

  • ING (NL) - For the Global Conversion Centre of Excellence and Pricing team: "Connecting to experiment: the virtual conversion workshops lead to 77% more tests in 1 year".

Honorable mentions for:

  • DPG Media (NL) - For the CRO-team: "From CRO-team to centre of excellence".

  • Thinkific (CA) - For the Growth team: "Reducing barriers to experimentation".

Click on the nominee to read the submitted case descriptions
or see the 2021 team award nominees & winner

ORGANIZATION-WIDE NOMINEES

Experimentation Culture Awards 2022 Organization-wide award nominations
  • Marks & Spencer (UK) - For culture, velocity & quality.

  • TL (CH) - For how do you energize the culture of experimentation in a 125 year old company?

  • ASOS (UK) - For embracing an experiment-oriented mindset.

  • Vattenfall Netherlands (NL) - For adopting data-driven customer experience optimization.

  • Beter Bed (NL) - For development accelerates experimentation.

Click on the nominee to read the submitted case descriptions
or see the 2021 organization-wide award nominees & winner

July 7th 2022 live broadcast

This live broadcast will have 120+ minutes of keynotes, interviews and winners
The live broadcast starts on July 7th, 6pm UTC

During our live broadcast we will raffle free access to Experimentation Culture Awards Lifetime Achievement winner and Conversion Hotel 2022 speaker Ronny Kohavi his upcoming 5*2 hours live class on A/B-testing.

THE 2021 NOMINEES & WINNERS

The nomination deadline for the 2021 experimentation culture awards was May 28th 2021. We received 62 case submissions.The 2021 jury went through all the cases and rated each of them on a 1 to 5 scale. They were not allowed to vote on cases by friends, family, colleagues or clients. The top 5 in each category (based on average score) became the finalists for the awards.All those finalists were interviewed. Based on these interviews the jury selected a winner in each category:- 2021 individual nominees & winner
- 2021 team nominees & winner
- 2021 organization-wide nominees & winner
In 2020 we had 6 out of 15 finalists (case owners) being female, in 2021 this went down to 4 out of 15 finalists. We are concerned about this. We also believe we need to work harder on the full diversity perspective of our selected finalists / cases.This is why we are reconsidering the way we collect cases and judge these cases to have a more diverse representation of our industry as finalists of our 2022 awards.Looking for the 2020 nominees and winners? Find them here!

INDIVIDUAL NOMINEES

Experimentation Culture Awards 2021 Individual award nominations
  • WINNER of the 2021 Individual AWARD is: Aleksander Fabijan (SI) - For it Takes a Flywheel to Fly: Leading Microsoft to share lessons in experimentation culture.

  • Alexandre Suon-Perhirin (FR) - For driving the adoption of A/B testing as a decision-making tool for strategic investments.

  • Peter Ernst (US) - For building a culture of experimentation in healthcare: find better ways to connect patients during COVID.

  • WINNER of the 2021 Community AWARD is: Rommil Santiago] (CA) - For launching Experiment Nation: a home for experimenters around the world, from beginner to pro.

  • Ruben de Boer (NL) - For building a worldwide CRO community with free coaching for charity and low barrier Udemy courses.

See the 2020 individual award nominees & winner

TEAM NOMINEES

Experimentation Culture Awards 2021 Team award nominations
  • WINNER of the 2021 Team AWARD is: bol.com (NL) - For Team Experimentation and the journey to make their colleagues smarter.

  • TOYOTA (FR) - For the CRO team. An eye on ROI: Executives and CRO experts nurture experimentation together.

  • ANWB (NL) - For the Web analytics & CRO-team and the CRO-training, onboarding and documentation of the CRO process.

  • iTech Media (UK) - For the Experimentation Team. Advocating their operational framework and increasing the velocity of experiments.

  • fonQ (NL) - For the CX Team. Rocket start CRO: from 0 to 30 A/B-tests in 6 months.

See the 2020 team award nominees & winner

ORGANIZATION-WIDE NOMINEES

Experimentation Culture Awards 2021 Organization-wide award nominations
  • Rabobank (NL) - For the growth from ad-hoc to structural experimentation and optimization.

  • BAM - Bamboo Clothing (UK) - For the growth and journey to experimentation informing smarter decisions.

  • Convoy (US) - For enabling experimentation at scale in a B2B Marketplace.

  • WINNER of the 2021 Rising Star AWARD is: The Royal Mint] (UK) - For embedding a culture of experimentation at a company with over 1,100 years of history.

  • WINNER of the 2021 Organization-wide AWARD is: ING (NL) - For moving towards continuous experimentation.

See the 2020 organzation-wide award nominees & winner

July 8th 2021 live broadcast

This live broadcast will have 150 minutes of keynotes, interviews and winners

THE 2020 NOMINEES & WINNERS

The nomination deadline for the 2020 experimentation culture awards was on July 31st 2020. A total of 65 cases were submitted through our website.The 2020 jury went through all the cases and rated every single one of them on a 1 to 5 scale. The top 5 in each category became the final nominees for the awards.All those finalists have been interviewed. Based on these interviews the jury selected a winner in each category.Looking for the 2021 nominees? You can find them here.

INDIVIDUAL WINNER & NOMINEES

Experimentation Culture Awards 2020 Individual award nominations
  • WINNER of the 2020 Individual AWARD is: Guido X Jansen (NL) - For evangelizing CRO in general and CRO culture in particular with the CRO Cafe Podcast.

  • Gabriela Denison (UK) - For building trust in fast, pain-free experimentation at ASOS.com.

  • Raoul Warren Doraiswamy (AU) - For building the CovidCRAP community. Supporting businesses affected by CoronaVirus for free.

  • Desiree van der Horst (NL) - For taking away the fear of sharing failures at Fingerspitz.

  • Stefan Thomke (US) - For writing "Experimentation Works", the book on experimentation for business leaders.

TEAM WINNER & NOMINEES

Experimentation Culture Awards 2020 Team award nominations
  • WINNER of the 2020 Team AWARD is: VodafoneZiggo (NL) - For the CRO Center of Excellence. Creating a culture of experimentation.

  • Taxfix (DE) - For the Experimentation taskforce. The path towards a strong experimentation culture.

  • Hallmark Cards Benelux (NL) - For the CRO team. Started from the bottom, now they are 100 tests further.

  • Microsoft (US) - For the Experimentation Platform Team. A/B Testing and Covid-19: Data-Driven decisions in times of uncertainty.

  • SAP (US) - For the Optimization Lab. Building a data-driven culture through widespread testing.

ORGANIZATION-WIDE WINNER & NOMINEES

Organization-wide award nominations
  • WINNER of the 2020 Organization-wide AWARD is: Farfetch (PT) - For the circle path from insights to learnings. Change a company experimentation mindset in a data-driven approach.

  • GlaxoSmithKline (US) - For building a culture and practice of experimentation.

  • MongoDB (US) - For going from 0 to 100 within 6 months. Implementing and accelerating a research and testing program.

  • AllClear Travel Insurance (UK) - For the democratization of experimentation.

  • RS Components (UK) - For full stack experimentation transformation. The 'binding agent' between Product & Engineering.

LIFETIME ACHIEVEMENT AWARD

The 2020 jury has decided to hand out an INDIVIDUAL Experimentation Culture lifetime achievement AWARD

  • The WINNER is: Ronny Kohavi (USA) - For all the research and publications that helped so many experimenters to understand how to conduct trustworthy digital experiments.

LIFETIME ACHIEVEMENT AWARD

The 2020 jury has decided to hand out an ORGANIZATION-WIDE Experimentation Culture lifetime achievement AWARD

  • The WINNER is: Booking.com (NL) - For being a big example organization for experimentation, known and quoted by many decision makers throughout the world.

September 24th 2020 broadcast

The 2020 liveshow has 180 minutes of inspirational talks and interviews

Marks and Spencer Experimentation Culture Award 2022 nomination

ORGANIZATION-WIDE NOMINEE:
Marks & Spencer

Marks & Spencer

Culture, Velocity & Quality"Experimentation at Marks & Spencer lies in the Digital & Data function whose goal is to accelerate and enable the digital transformation of the business. As part of this goal D&D aspire to develop a market-leading decentralised experimentation program at scale.The program has gone from strength-to-strength and in the financial year 21/22 M&S reached a new milestone in experimentation maturity and velocity with over 300 experiments completed.We have achieved this through enabling and democratising the experimentation process within the product teams who are all now autonomously experimenting and working hard to create a great user experience.Not only focusing on velocity we have looked to improve the quality and trustworthiness of our experimentation.Our experimentation test plans are once again democratised and shared where anyone can access the board, see what we're testing, where we're testing it and what we consider success.This creates a great platform for our internal knowledge shares and playbacks allowed for a cross-pollination of ideation. We want our product managers to share the wins, the losses and most importantly the learnings.Our monthly councils also have lunch and learn sessions covering statistically robust test plans, experimentation pitfalls, overlapping risk management - the list goes on!
Looking towards next year we have lofty ambitions to double our velocity and we can't wait to see what comes next!"

This live broadcast will have 120+ minutes of keynotes, interviews and winners
The live broadcast starts on July 7th, 6pm UTC

During our live broadcast we will raffle free access to Experimentation Culture Awards Lifetime Achievement winner and Conversion Hotel 2022 speaker Ronny Kohavi his upcoming 5*2 hours live class on A/B-testing.


TL Experimentation Culture Award nomination

ORGANIZATION-WIDE NOMINEE:
TL (Transports publics de la région Lausannoise)

TL

How do you energize the culture of exploration in a 125 year old company?"In 2017, our CEO, who had just come out of a training course on digital transformation at IMD, told us: "I want to have a culture of experimentation!" And then ... it's drama! Silence and forced laugh... Who hasn't experienced that feeling your boss asks you to think outside of the box with no idea how to do it?
We had several options: Recruiting hipsters from the silicon valley, Dressing up colleagues as experimenters, or using our collective intelligence. Finally, we gathered our creativity to become the first experimenters.
Our strategy was based on the snowball effect through 3 steps.Step 1 - Start with HR
- A new hiring process to attract talents who have this culture of experimentation and can spread it: design sprints, use of improvisational comedians for recruitment tests, etc.
- Impact: 55 applications received, 2 recruited, and retained 😉
Step 2 - First experiments
- To start the diffusion of the experimentation culture through co-creation, we launched several experiments in which collaborators took part.
- Impact: 49 experiments with a transformation rate of 25% 🥂
Step 3 - Make it sustainable
- We created a community of internal ambassadors trained on design thinking to bring the culture of experimentation into all tl layers.
- Impact : 22 experiments launched with 160 participants 💪
What's next?
Keeping anchoring the experimentation culture in the company and maybe hiring a Chief Experimentation Officer in a few years 😍 Dare, try, fail, learn and grow!"

This live broadcast will have 120+ minutes of keynotes, interviews and winners
The live broadcast starts on July 7th, 6pm UTC

During our live broadcast we will raffle free access to Experimentation Culture Awards Lifetime Achievement winner and Conversion Hotel 2022 speaker Ronny Kohavi his upcoming 5*2 hours live class on A/B-testing.


Asos Experimentation Culture Award nomination

ORGANIZATION-WIDE NOMINEE:
ASOS

Asos

Embracing an Experiment-Oriented Mindset"In the past year, an experimentation culture has been adopted more widely across the ASOS organisation. The number of Controlled Online Experiments is set to double in 2022, and we build innovative framework to speed up experimenting in Apps (https://medium.com/asos-techblog/android-and-ios-config-driven-experiments-that-dont-require-releases-droidcon-london-talk-7e64c03e84a9).Experimentation is embraced by our Tech teams and they are proud if it (https://medium.com/asos-techblog/embracing-an-experiment-oriented-mindset-at-asos-9f53551f692e).However the use of experimentation is being adopted in entirely new areas. We have a sophisticated program for Geo-Testing Marketing, built in-house (https://medium.com/asos-techblog/accelerating-geo-testing-at-asos-edbdf123a98b) and we are starting to run experiments in new off-line areas like Customer Care.We are working across different teams sharing our knowledge and ideas. After a few years of experimentation being principally the domain of Online Product, experimentation is now being advocated for by our most senior leaders in company-wide meetings and to external stakeholders. It has become a must-have for major initiatives and projects."

This live broadcast will have 120+ minutes of keynotes, interviews and winners
The live broadcast starts on July 7th, 6pm UTC

During our live broadcast we will raffle free access to Experimentation Culture Awards Lifetime Achievement winner and Conversion Hotel 2022 speaker Ronny Kohavi his upcoming 5*2 hours live class on A/B-testing.


Vattenfall Netherlands Experimentation Culture Award nomination

ORGANIZATION-WIDE NOMINEE:
Vattenfall Netherlands

Vattenfall

Adopting Data-Driven Customer eXperience OptimizationWhat we are most proud of in this case:1) The buy-in from the organisation is great. On multiple levels they understand the potential of the changes we are trying to make. This results in freeing up capacity on all levels, helping each other out, testing and inspiring each other to get things done. Everybody is so eager and enthusiastic!2) The way we tackle this entire process is based on feedback and testing rounds (love data). We don’t know everything in advance. For example: training modules are first tested by all members of the knowledge work stream, then by the overarching work stream testers and only then it’s being passed publicly throughout the organisation.3) The way this process is set-up is very well documented, accessible for everyone and very scalable. If this works out the way we planned it, we are an example for the other Vattenfall countries."

This live broadcast will have 120+ minutes of keynotes, interviews and winners
The live broadcast starts on July 7th, 6pm UTC

During our live broadcast we will raffle free access to Experimentation Culture Awards Lifetime Achievement winner and Conversion Hotel 2022 speaker Ronny Kohavi his upcoming 5*2 hours live class on A/B-testing.


Beter Bed Experimentation Culture Award nomination

ORGANIZATION-WIDE NOMINEE:
Beter Bed

Beter Bed

Development accelerates experimentation"BUILDING AN EXPERIMENTATION CULTURE
Last year, Beter Bed grew from an ROI driven experimentation program to supporting business decision making through validation. ROI is still being tracked, but this is not the main focus anymore. Beter Bed is growing from a discount-focused retailer to a leading sleep specialist. Experimentation became a mindset to validate this long term strategy iteratively. By shifting from an e-commerce CRO team to a new Owned & Operated department we can really make an impact. We scaled up our test velocity even more and educated the company by sharing all the learnings from experiments.
DEVELOPMENT ACCELERATES EXPERIMENTATION
The role of our own development team in close collaboration with the Owned & Operated department is essential for our success. For our development team experimentation is not a sideshow, but embedded in their daily activities. We don’t need to seek resources as the development team is part of our experimentation program. Instead of development blocking or slowing down the process, we managed to make the dedicated development resources accelerate the experimentation program. This leads to better quality and higher test velocity.
WHAT’S NEXT?
By sharing experimentation learnings the organization sees the added value of our experimentation program. For the coming year we will expand the dialogue with other departments and share relevant learnings to expand the experimentation culture even further in the organization."

This live broadcast will have 120+ minutes of keynotes, interviews and winners
The live broadcast starts on July 7th, 6pm UTC

During our live broadcast we will raffle free access to Experimentation Culture Awards Lifetime Achievement winner and Conversion Hotel 2022 speaker Ronny Kohavi his upcoming 5*2 hours live class on A/B-testing.


Vista Experimentation Culture Award nomination

TEAM NOMINEE:
Vista, Experimentation Hub

Vista

Building a culture of experimentation in public"Vista’s goal is to deliver jaw-dropping customer value to small business owners looking for design and marketing solutions. To confirm that our execution and delivery do indeed result in jaws being dropped, we have a strong desire to test the validity of every assumption and the impact of every change.To build the foundations for such a culture of experimentation, we have created a multidisciplinary team which is comprised of analysts, developers and data product managers. This Experimentation Hub has a shared objective to evolve the maturity of our culture of experimentation.In the past year, following the inception of Experimentation Hub, our internal experimentation Slack channel audience has quintupled in size, while the number of messages about experimentation in that channel has increased by an order of magnitude. The number of experiments has grown also; more than 100 A/B tests were launched and documented in the past year.To help the community at large, we have been writing about our journey publicly on our blog (https://vista.io/blog/building-a-culture-of-experimentation and https://vista.io/blog/organising-for-scaled-experimentation), presented at conferences, and will soon publish our first academic paper (https://conf.researchr.org/details/ease-2022/ease-2022-industrial-track/7/Automated-Sample-Ratio-Mismatch-SRM-detection-and-analysis).Clearly experimentation culture is growing at Vista; and we have only just begun!"

This live broadcast will have 120+ minutes of keynotes, interviews and winners
The live broadcast starts on July 7th, 6pm UTC

During our live broadcast we will raffle free access to Experimentation Culture Awards Lifetime Achievement winner and Conversion Hotel 2022 speaker Ronny Kohavi his upcoming 5*2 hours live class on A/B-testing.


SMG Experimentation Culture Award nomination

TEAM NOMINEE:
SMG (Swiss Marketplace Group), CRO Team

SMG

Inform. Inspire. Involve. Iterate - Revive & expand a Culture of Experimentation"Our CRO team, but especially Erica Chang - initially building and leading the Experimentation program for our Scout24 brands - achieved keeping AND improving the Culture of Experimentation under very difficult conditions in our org:
While we had some sort of foundation for experimentation before, a triangle of challenges arose: low trust, bottlenecks and technical complexity. The testing tool at the time had its share.
For this to tackle, one step back (e.g. tool migration) was necessary in order to move two steps forward.
Tremendous work did happen then led by Erica: From tool evaluation phase to POC, transition + onboarding: One team at a time really paid off for us. By getting up to speed again, it was easier to expand to the next team and the next.On the way we could hire two more Conversion Managers to leverage and accelerate the Onboarding & program further.
In parallel the CRO team tirelessly put our Experimentation Mantra “Inform. Inspire. Involve. Iterate” more serious to practise by e.g. identifying internal champions for each of our brands and product teams, building and maintaining an internal knowledge space, using specific channel(s) to trigger curiosity/discussions and hold regular Experimentation Insights sessions to share test highlights & key learnings with stakeholders.
By doing all of this, in 2021 this led to doubling the # experiments across all our marketplaces and getting the well-needed trust back to bring Experimentation back on track again!"

This live broadcast will have 120+ minutes of keynotes, interviews and winners
The live broadcast starts on July 7th, 6pm UTC

During our live broadcast we will raffle free access to Experimentation Culture Awards Lifetime Achievement winner and Conversion Hotel 2022 speaker Ronny Kohavi his upcoming 5*2 hours live class on A/B-testing.


Videoland Experimentation Culture Award nomination

TEAM NOMINEE:
Videoland, Experimentation Team

Videoland

From 5.5 experiments per month to a Center of Excellence"Jan‘20 - May‘21
5.5 experiments per month (decent given the manpower).
Skepticism for experimentation (especially amongst brand marketers, business analysts, and devs).
After May ‘21
Experimentation team strengthened with an analyst, UX designer, and CRO lead from Online Dialogue. The new team had two focus areas: improving the process (run at least 80 exp. with higher win rate) and organization culture.
Most impactful activities:
- Improvement of the entire process, including automation, the way of working, and automated, evidence-based prioritization.
- An idea competition resulting in 133 new test ideas from ~50% of the people in the company.
- Promote a customer problem framework for every step in the customer journey to all departments.
- Presentation of experiments (winners and losers) in product updates and Sprint reviews (with 80+ people present).
- Set up and tested server-side with in-stream advertising on web, app, and TV (this was entirely new for Videoland), including business case calculation and hypothesis framework.
Results
The team ran 9.7 experiments per month (based on MDE max is 10.8 & win rate improved by 40%).
This got the attention of higher management. With their support, we are now setting up a Center of Excellence to help Videoland and its parent (RTL). In the new Videoland platform, custom-made server-side experimentation is available. We are currently experimenting with marketing, and analysts are providing the team with test ideas."

This live broadcast will have 120+ minutes of keynotes, interviews and winners
The live broadcast starts on July 7th, 6pm UTC

During our live broadcast we will raffle free access to Experimentation Culture Awards Lifetime Achievement winner and Conversion Hotel 2022 speaker Ronny Kohavi his upcoming 5*2 hours live class on A/B-testing.


Accor Experimentation Culture Award nomination

TEAM NOMINEE:
Accor, Experimentation Department

Accor

Making experimentation great(ly known/used)"Part 1 - A new considerationEnd of 2021, Experimentation became its own entity (department) at Accor and now sits at the office function of the chief digital officer (same level as Product, Tech, Design etc…) - it is no longer a side function but an actually core team that is represented at the highest round tables/board. This is very important because within a company of 400 000 employees, being the closest to the top enables to push the best practices to the largest amount of projects. Experimentation strategy can now be applied in much wider range of projects including APIs performances, new tech stack, hotel rebranding etc…Part 2 - Investment in technologyAs our volume of abtest grew (predicted to 350+ this year) so our technology requirements.
It was the right time for Accor to build something custom to match its strong desire in performance but also to match its desire to make experimentation even more present within all its brands. Therefore a sub-team of 5 engineers and 1 Product manager has been created back in september within the Experimentation team (we call it ExP Platform) to build a platform 100% dedicated to “server-side testing”, feature flipping, and personalisation client side. Platform is build as API based with an interface in React where users can easily get their results without having to build reports themselves.
Its a game changer and true testament in Accor desire to push experimentation."

This live broadcast will have 120+ minutes of keynotes, interviews and winners
The live broadcast starts on July 7th, 6pm UTC

During our live broadcast we will raffle free access to Experimentation Culture Awards Lifetime Achievement winner and Conversion Hotel 2022 speaker Ronny Kohavi his upcoming 5*2 hours live class on A/B-testing.


ING Experimentation Culture Award nomination

TEAM NOMINEE:
ING, Global Conversion Centre of Excellence and Pricing team

ING

Connecting to experiment: The Virtual Conversion Workshops lead to 77% more tests in 1 year"The Virtual Conversion Workshops lead to 77% more tests in 1 yearThe ING Global Conversion Centre of Excellence and Pricing team have worked together to run over 20 Conversion workshops. Virtually working with ING teams in Countries like Belgium, Germany, Netherlands, Spain, Italy, Australia, Poland. This has been done in a Virtual Workshop setting where we ran 2 hour workshops to be able to enthuse and empower the local teams with test ideas, data, insights. Preparations for the workshops took an average of 2 days but the workshop itself can be described as a Inspiration Explosion. Having longer workshops runtimes was tested but as this felt more like a Inspiration Implosion we set ourselves a 2 hour max.Results: This has impacted over 100 colleagues who have jointly come up with >300 hypothesis to be tested. Not only AB tests but also mailings, marketing campaigns, inn app enticements and personalisation features have been tested on the back of this. This has resulted in a huge increase in enthusiasm from both Digital and product squads boosting the amount of test with 77%.The results of which are used to ‘spread the word’ by sharing them across all ING using 1 hour Virtual Global Conversion meetups & Newsletters. Which leads to test forming the inspiration for other country and product teams further boosting cross fertilisation.Virtually training >150 colleagues by peers and contracted specialists: no more ‘lets release it and hope for success culture"

This live broadcast will have 120+ minutes of keynotes, interviews and winners
The live broadcast starts on July 7th, 6pm UTC

During our live broadcast we will raffle free access to Experimentation Culture Awards Lifetime Achievement winner and Conversion Hotel 2022 speaker Ronny Kohavi his upcoming 5*2 hours live class on A/B-testing.


DPG Media Experimentation Culture Award nomination

HONORABLE MENTION - TEAM:
DPG Media, CRO Team

DPG Media

From CRO-team to Centre of Excellence"We're responsible for subscription sales. 12 months ago we found ourselves in a highly involved environment which was full of ideas, but mostly without any foundation in research or data, nor placing our visitor at the centre of the business questions.In the last 12 months things have really changed for the better. We started with a CRO team consisting of just Joshua Kreuger and myself, and right now around 18 people over different teams are practising CRO.We've taken on the role of a Centre of Excellence to provide the different teams with the right tools and approaches to experiment in the "right" way. We work according to a Problem - Solution - Result framework, and always base our experiments on data, insights or research.We document our tests in an experiment-base. This year we've improved by connecting our test results for deeper analysis. We label tests on many dimensions like: optimization strategy, psychological tactic, observation (consumer problems like "visitors find subscription terms not transparant'') and many more. This gives us a deeper understanding of all our test results and more importantly: this provides easier to understand insights to stakeholders outside of our CRO-team.We now see a lot more constructive involvement, discussions revolve around understanding the problem of our visitor instead of shooting ideas based on gut feel. Our next step is to foster better understanding of user problems and grow the culture even further."

This live broadcast will have 120+ minutes of keynotes, interviews and winners
The live broadcast starts on July 7th, 6pm UTC

During our live broadcast we will raffle free access to Experimentation Culture Awards Lifetime Achievement winner and Conversion Hotel 2022 speaker Ronny Kohavi his upcoming 5*2 hours live class on A/B-testing.


Thinkific Experimentation Culture Award nomination

HONORABLE MENTION - TEAM:
Thinkific, Growth Team

Thinkific

Reducing barriers to experimentation"The Growth team at Thinkific has transformed the culture of experimentation over the last couple years.When the team was founded in 2019, the entire business had one analyst running queries that supported already-made product decisions. But we worked hard at shifting the culture towards rapidly understanding ROI and causality when making decisions.Today, we have 3 growth teams that own day to day experimentation for acquisition, activation, and monetization, as well as widespread experimentation across core product and marketing teams.How did we get here?- We stuck to a rigorous pace of attacking the highest friction problems in our customer journey, and an even more rigorous pace of talking about our wins and losses to everyone- Wins included Advanced Reporting features, social sharing/viral loop features, and numerous pricing/paywall experiments. Losses included retention modals, feature prompts, and landing pages (among many others!)- We built a portal for experiments and beta features called Thinkific Labs, which encouraged core product teams to ship MVPs rather than perfect products- We started an internal Experimentation Guild, which invited growth experimentation practitioners from across all teams (not just Growth) to learn key tools, statistics, and hypothesis development from each other."

This live broadcast will have 120+ minutes of keynotes, interviews and winners
The live broadcast starts on July 7th, 6pm UTC

During our live broadcast we will raffle free access to Experimentation Culture Awards Lifetime Achievement winner and Conversion Hotel 2022 speaker Ronny Kohavi his upcoming 5*2 hours live class on A/B-testing.


Amy Hartwig Experimentation Culture Award nomination

INDIVIDUAL NOMINEE:
Amy Hartwig

Amy Hartwig

Growing experimentation at Microsoft DevDiv"Amy has helped grow the experimentation program in Microsoft's Developer Division (The team that builds VS Code, GitHub, and other developer tools that we all love to use) from a handful of A/B tests that the team ran 2 years ago to hundreds of experiments monthly.Thanks to her efforts, over 40 teams across the Developer Division today precisely measure expected and unexpected side effects of product changes and make decisions about how to best configure what they ship for specific audiences through A/B tests.Amy regularly shares learnings and experiment results, and empowers others at Microsoft to do the same, generating excitement about experimentation across the company.In addition to growing the experimentation program at DevDiv, Amy has partnered closely with the ExP team and influenced the product roadmap of Microsoft's Experimentation Platform team. Thanks to her awesome engagement in the ExP council, her ideas sparked many features in ExP that benefited teams across Microsoft!"

This live broadcast will have 120+ minutes of keynotes, interviews and winners
The live broadcast starts on July 7th, 6pm UTC

During our live broadcast we will raffle free access to Experimentation Culture Awards Lifetime Achievement winner and Conversion Hotel 2022 speaker Ronny Kohavi his upcoming 5*2 hours live class on A/B-testing.


Tom van den Berg Experimentation Culture Award nomination

INDIVIDUAL NOMINEE:
Tom van den Berg

Tom van den Berg

CRO Weekly: a curated newsletter to help increase the quality of experimentation throughout the market"We all know experimentation is not easy. There is a lot to consider, and mistakes can easily be made. However, why reinvent the wheel when others share their learnings? To benefit from those learnings, you have to be able to find them. This is where Tom's bi-weekly newsletter comes in.Tom is an experimentation enthusiast who wants to know everything there is to know about experimentation. Gaining knowledge is great, but sharing knowledge is even better. Tom, therefore, decided to use his passion and obsession for newsgathering to give something back to the CRO community: a curated newsletter.Thanks to Tom's bi-weekly newsletter (CRO Weekly), the worldwide community of experimentation enthusiasts keep their knowledge up to date with high-quality articles about CRO, data, tooling, experimentation maturity, and culture. Through his newsletter, Tom ensures that the quality of experimentation increases throughout the market.The newsletters contain the best articles from Medium, LinkedIn, and other places on the web. If there's something written about experimentation, Tom knows where to find it. All articles are in English (not his native language) to reach and help as many people as possible.Of course, Tom also experiments to find the best moment to send the newsletter.Over the last year, Tom added exciting job opportunities to his newsletter. All for free, without any profit motive."

This live broadcast will have 120+ minutes of keynotes, interviews and winners
The live broadcast starts on July 7th, 6pm UTC

During our live broadcast we will raffle free access to Experimentation Culture Awards Lifetime Achievement winner and Conversion Hotel 2022 speaker Ronny Kohavi his upcoming 5*2 hours live class on A/B-testing.


Ana Catarina Cizilio Experimentation Culture Award nomination

INDIVIDUAL NOMINEE:
Ana Catarina Cizilio

Ana Catarina Cizilio

Empowering A/B-testing in Brazil"The experimentation market seems to be focused on Northern America and Western Europe, while there is so much else going on the world of experimentation. This mostly stays covered because publications are not in English and the market is not always on the front-end of new experimentation development, but it's a huge market and experimentation is taking big steps. Ana is a linking pin in the experimentation market in Brazil, connecting to the English speaking world through her efforts for Experiment Nation and transferring it back to the local market in Brazil. I believe she should be nominated for her efforts to show the world experimentation is not just something for Northern America and Western Europe".

This live broadcast will have 120+ minutes of keynotes, interviews and winners
The live broadcast starts on July 7th, 6pm UTC

During our live broadcast we will raffle free access to Experimentation Culture Awards Lifetime Achievement winner and Conversion Hotel 2022 speaker Ronny Kohavi his upcoming 5*2 hours live class on A/B-testing.


Bjarn Brunenberg Experimentation Culture Award nomination

INDIVIDUAL NOMINEE:
Bjarn Brunenberg

Bjarn Brunenberg

Experiment Program Spitting Fire at TomTom"Hi my name is Bjarn, a Dutch lad who geeks out about CRO, A/B testing and Experimentation.Looking back to what I have achieved whit the team now compared to 12 months ago is quite a big change! What was first mainly spaghetti testing based on opinions, has now transformed into a strategic and data-backed experimentation program spitting fire! 🔥The biggest achievements are listed down below:Due to slack workflow and Airtable automation, everyone from different teams can add his/her experiment idea. This really got many excited about running experiments!
I have created a clear process when experiments move from draft to refinement to backlog etc. based on certain criteria (ex: no hypothesis or success criteria? No testing).
Created quality scorecards to keep quality of experimentation high (ex: backed by research +20p).Build a Growth Leaderboard keeping scores based on the contribution of each team member regarding adding new ideas, wins, fails and inconclusive test results.Created a learning template which is easy to share via slack automation - so everyone between teams is aware of our latest findings.I introduced Bi-weekly brainstorm sessions to specifically focused on problems based on research, instead of randomly testing.And finally - and the one I am most proud of - our growth team grew from 4 to 14 people! Improving this whole growth program resulted in breaking down silos between teams, acquisition, ecommerce and product all working together!"

This live broadcast will have 120+ minutes of keynotes, interviews and winners
The live broadcast starts on July 7th, 6pm UTC

During our live broadcast we will raffle free access to Experimentation Culture Awards Lifetime Achievement winner and Conversion Hotel 2022 speaker Ronny Kohavi his upcoming 5*2 hours live class on A/B-testing.


Ellie Hughes Experimentation Culture Award nomination

INDIVIDUAL NOMINEE:
Ellie Hughes

Ellie Hughes

Becoming an Experimentation Evangelist at AND Digital"Ellie joined AND Digital in order to found a new practice and centre of excellence for experimentation. Her mission is to be an Experimentation Evangelist within AND Digital and to make a unique contribution to the experimentation community.Experimentation is incredibly important to AND Digital’s clients; however, before Ellie joined the organisation, it was not a service that we offered. Ellie was starting from scratch: she had to introduce experimentation as a practice to the company and develop it into a highly valued service in AND’s wider offering.Ellie has been tireless in her efforts to bring experimentation to life at AND and for our clients. She has spoken at countless internal events including lunch & learns, communities of practice and team days. She recently won two Experimentation Elite awards for her work at AND and for her work with one of our UK clients Rightmove.co.uk, and has spoken at several external events including Measurecamp and the AND Data lunch and learn series.She has a rare ability to put across complex experimentation concepts to clients at a business level - so clients understand the value of what she does - and challenge at a technical level to demonstrate her deep understanding of experimentation.In just 6 months since Ellie started, her efforts have paid off: AND have secured experimentation projects with some of our largest clients (News UK, British Airways, Gousto and Rightmove) directly as a result of Ellie’s efforts."

This live broadcast will have 120+ minutes of keynotes, interviews and winners
The live broadcast starts on July 7th, 6pm UTC

During our live broadcast we will raffle free access to Experimentation Culture Awards Lifetime Achievement winner and Conversion Hotel 2022 speaker Ronny Kohavi his upcoming 5*2 hours live class on A/B-testing.


Marianne Stjernvall Experimentation Culture Award nomination

INDIVIDUAL NOMINEE:
Marianne Stjernvall

Marianne Stjernvall

Sharing inhouse lessons of building an experimentation culture"For experimentation consultants it makes sense to publish a lot of content on experimentation and to present their insights at conferences. Marketing and sales is the main driver here. But not so many inhouse specialists and managers have, take or get the time to do this. This does not stop Marianne. She loves to share her experimentation knowledge on stage and she does this frequently (https://www.linkedin.com/in/marianne-stjernvall-04983680/recent-activity/) even knowing this means that she has to invest her free time to be able to do so. It's refreshing to have theory applied by the practical perspective from someone in an inhouse experimentation team. I believe this should be awarded with a nomination, because Marianne is a great example to inspire other (female) inhouse specialists and managers to take the stage more often."

This live broadcast will have 120+ minutes of keynotes, interviews and winners
The live broadcast starts on July 7th, 6pm UTC

During our live broadcast we will raffle free access to Experimentation Culture Awards Lifetime Achievement winner and Conversion Hotel 2022 speaker Ronny Kohavi his upcoming 5*2 hours live class on A/B-testing.


SUBMIT YOUR 2022 AWARD CASE(S)

Extended deadline for submission was May 21st, 8pm UTC, which means that our case submit form is closed now.The 2022 jury was looking for inspirational stories on (your) growth in experimentation maturity. It's not about the current level of experimentation, we nominate and award for the growth steps that have been taken in the past 12 months.See our about page for more info on our award categories. If you were too late to submit your story: register for the #EXPCA2022 live broadcast and get notified when to submit your 2023 case.All we needed was a max. 1500 characters (approximately 350 words) case description which the jury will use to select the finalists. Once selected as finalist we will schedule a 30 minute videocall to have an in depth interview about the case. The jury will select the winners based on the interview recordings.You were able to submit submit your own case or you were able to nominate someone else. When a case from someone else is selected as a finalist, we will reach out to them on behalf of the case submitter and ask the potential finanlist if they accept the nomination and are willing to participate in the interview, before we announce anything about their case on our website.

"The Experimentation Culture Awards is not awarding that one experiment. This is about improving the process, structure, trustworthiness, democratization and motivation for experimentation in organizations."

July 7th 2022 live broadcast

This live broadcast will have 120+ minutes of keynotes, interviews and winners
The live broadcast starts on July 7th, 6pm UTC

During our live broadcast we will raffle free access to Experimentation Culture Awards Lifetime Achievement winner and Conversion Hotel 2022 speaker Ronny Kohavi his upcoming 5*2 hours live class on A/B-testing.

ABOUT THIS AWARDS EVENT

This is the 2022 about page.The Experimentation Culture Awards started in 2020 as an event to globally promote a culture of experimentation. Our goal is to help people who are trying to grow an evidence-based decision making culture, by giving them a push in the right direction.We do this by sharing inspirational stories of experimentation maturity growth and by nominating individuals, teams and organizations who deserve to be recognized for their work.In an experimentation culture you are free to try and fail or succeed, while the direction and the result of work is based on trustworthy gathered evidence. Grow fast, grow forward.

"It's not about that one experiment, it's about improving the process, structure, trustworthiness, democratization and motivation for experimentation and data-driven decision making."

In 2022 the jury be handing out at least 3 awards:

  • Individual award 2022: something 1 person has done in the past 12 months to keep or improve the culture of experimentation in an organization or even broader.

  • Team award 2022: something a team has done in the past 12 months to keep or improve the culture of experimentation in your organization or even broader.

  • Organization-wide award 2022: something an organization has done in the past 12 months to keep or improve the culture of experimentation in your organization or even broader.

On top of the three main categories we also have a community award to appreciate efforts that support the experimentation industry as a whole. Last but not least we have the rising star award for the individual, team or organization really going from nothing to something in the past 12 months.Both these extra awards will separately be handed out by the jury to entries in one of the three main categories.

THE AWARDS JURY OF 2022

This is our 2022 jury. Read more about our awards or our 2021 jury.

EXPCA 2021 Jury

After the submission deadline has passed every jury member will score each case entry on a 5 star rating scale (besides cases of friends, family, collegues and clients). The best scoring cases will become nominated as finalists and will be interviewed in-depth about their award entry. Based upon the extra information from the interview, the jury will select the winner among the nominees in all categories. The winners will be announced during the broadcast of July 7th, 6.00pm UTC - 8.30pm UTC.Question on this selection process: please use our contact form.

July 7th 2022 live broadcast

This live broadcast will have 120+ minutes of keynotes, interviews and winners
The live broadcast starts on July 7th, 6pm UTC

During our live broadcast we will raffle free access to Experimentation Culture Awards Lifetime Achievement winner and Conversion Hotel 2022 speaker Ronny Kohavi his upcoming 5*2 hours live class on A/B-testing.

ABOUT THIS AWARDS EVENT

This is the 2021 about & jury page.The Experimentation Culture Awards started in 2020 as an event to globally promote a culture of experimentation. Our goal is to help people who are trying to grow an evidence-based decision making culture, by giving them a push in the right direction.We do this by sharing inspirational stories of experimentation maturity growth and by nominating individuals, teams and organizations who deserve to be recognized for their work.In an experimentation culture you are free to try and fail or succeed, while the direction and the result of work is based on trustworthy gathered evidence. Grow fast, grow forward.

"It's not about that one experiment, it's about improving the process, structure, trustworthiness, democratization and motivation for experimentation and data-driven decision making."

In 2021 we are handing out 3 awards (but we do like surprises):

  • Individual award 2021: something 1 person has done in the past 12 months to keep or improve the culture of experimentation in an organization or even broader.

  • Team award 2021: something a team has done in the past 12 months to keep or improve the culture of experimentation in your organization or even broader.

  • Organization-wide award 2021: something an organization has done in the past 12 months to keep or improve the culture of experimentation in your organization or even broader.

THE AWARDS JURY OF 2021

This is our 2021 jury. Read more about our 2022 awards or our 2022 jury.

EXPCA 2021 Jury

After the submission deadline passed every jury member scored each case entry on a 5 star rating scale (besides cases of friends, family, collegues and clients). The top 5 in each category became nominated as finalists and was interviewed in-depth about their award entry. Based upon the extra information from the interview, the jury will select the winner among the 5 nominees in all 3 categories. The winners will be announced during the broadcast of July 8th, 6.00pm UTC - 8.30pm UTC.Question on this selection process: please use our contact form.

July 8th 2021 live broadcast

This live broadcast will have 150 minutes of keynotes, interviews and winners

ABOUT THIS AWARDS EVENT

This is the 2020 about & jury page.The Experimentation Culture Awards were scheduled as part of the new Outperform Conference in Amsterdam, June 2020. This event got postponed to June 2021 2022, due to the Covid virus.This outbreak changed a lot, it caused a massive shift for many organizations. We also went through digital transformation and moved the awards show to a virtual event.What have you, your team and your company done the past months to maintain a culture of experimentation, or to even grow it to a next level of maturity? This question is exactly the reason why we wanted to move forward with the Awards online!

On the deadline of July 31st 2020 we had received 65 amazing cases on evidence-based maturity growth in organizations. Stories that will inspire others in these strange times. Stories that inspire us all to keep on moving forward maintaining and improving a culture of experimentation.It's not about that one experiment, it's about improving the process, structure, trustworthiness, democratization and motivation for experimentation and data-driven decision making.

Read on for our awards, the jury and the nominee selection process

  • Individual award: something 1 person has done in the past months to keep or improve the culture of experimentation in an organization or even broader.

  • Team award: something a team has done in the past months to keep or improve the culture of experimentation in an organization or even broader.

  • Organization-wide award: something an organization has done in the past months to keep or improve the culture of experimentation in an organization or even broader.

THE AWARDS JURY OF 2020

This is our 2020 jury. Read more about our awards or our 2021 jury.

After the submission deadline of July 31st 2020, 6.00pm UTC, every jury member has scored each entry on a 5 star rating scale. The top 5 in each category has been contacted and interviewed about their award entry. Based upon that extra information the jury has selected the winners among the 5 nominees in every category and those winners were announced in the broadcast of September 24th, 6.00pm UTC.The live show is done, but you can still register your free event ticket to get access to the recording with 6 keynotes, 15 interviews, 3 awards and 2 bonus items. Truly inspirational content on experimentation culture.


REGISTER YOUR FREE TICKET

For the 2022 experimentation culture awards live broadcast.You need to register yourself to be able to login to the live streamThe live broadcast starts on July 7th 2022, 6pm UTC.

(which is Amsterdam: 8pm, London: 7pm, New York 2pm, San Francisco 11am)This live broadcast will have 120+ minutes of keynotes, interviews and winners

During our live broadcast we will raffle free access to Experimentation Culture Awards Lifetime Achievement winner and Conversion Hotel 2022 speaker Ronny Kohavi his upcoming 5*2 hours live class on A/B-testing.


GET FREE ACCESS TO THE 2021 LIVE AWARD SHOW RECORDING

Register yourself to be able the recording of the 2021 live show. This event has keynotes, interviews with the 15 finalist nominees and announces the winners. You can check the full schedule here.Clicking on the button below here will take you to our livestream platform, where you also can register a new account directly.

If you registered earlier on this site: check your email for login informationEmails are mainly send by info@experimentationcultureawards.com, but login info also comes from no-reply@editmysite.comUse "reset password" on our live stream platform to set a new pass.

2021 AWARD SHOW SCHEDULE

July 8th 2021

2020 keynote speakers

All times in the schedule are UTC times, which means it starts in/at:- Moscow: 9.00pm (+3 hours on UTC)
- Amsterdam / Berlin / Paris: 8.00pm (+2 hours on UTC)
- London / Dublin: 7.00pm (+1 hour on UTC)
- New York / Boston: 2.00pm (-4 hours on UTC)
- Silicon Valley: 11.00am (-7 hours on UTC)

  • 5.50pm UTC: Livestream starts with a 10 minute countdown - you can start to log in

  • 6.00pm UTC: Introduction of the 2020 Awards by jury chair & host of the this show Ton Wesseling

  • 6.15pm UTC: Individual award nominees case introductions & announcing the winner!

  • 6.45pm UTC: Keynote presentation by VodafoneZiggo - Winner of the Experimentation Culture Team award in 2020

  • 7.15pm UTC: Team award nominees case introductions & announcing the winner!

  • 7.45pm UTC: Keynote presentation by Farfetch - Winner of the Experimentation Culture Organization-wide award in 2020

  • 8.15pm UTC: Organization award nominees case introductions & announcing the winner!

  • 8.45pm UTC: Closing remarks by Ton Wesseling

2020 organization-wide nominees
2021 team nominees
2021 individual nominees

All nominations will be introduced during the event by the case owners. Do not forget to register to get access to the show and/or the recording!


GET FREE ACCESS TO THE RECORDING OF THE 2020 SHOW

Register yourself to be able to watch the recording of the live broadcast. The events has 6 keynotes, the 15 final nominees and presents the winners. You can check the full schedule here.

The full recording is accessible through our live stream platform
If you registered earlier: check your email for login information
(emails are mainly send by info@experimentationcultureawards.com, but login info also comes from no-reply@editmysite.com - use "Reset password")

2020 AWARD SHOW SCHEDULE

September 24th 2020

2020 keynote speakers

All times in the schedule are UTC: Moscow +3hrs, Paris +2hrs, London +1hr, New York -4hrs, Austin -5hrs, San Jose -7hrs

  • 5.50pm - 6.00pm: Livestream starts with a 10 minute countdown - you can start to log in

  • 6.00pm - 6.15pm: Introduction of the 2020 Awards by jury chair & host of the this show Ton Wesseling

  • 6.15pm - 6.30pm: Keynote presentation by Stephen Pavlovich - Tactics that could help to create a culture of experimentation

  • 6.30pm - 6.45pm: Keynote presentation by Marianne Stjernvall - How to build an experimental organization

  • 6.45pm - 7.05pm: Individual award nominees case introductions & announcing the winner!

  • 7.05pm - 7.20pm: Keynote presentation by Hazjier Pourkhalkhali - Who optimizes the optimizers

  • 7.20pm - 7.35pm: Keynote presentation by Casandra Cambell - Stop just optimizing; start deliberate learning instead

  • 7.35pm -7.45pm: Individual lifetime achievement award given by the 2020 jury to Ronny Kohavi

  • 7.45pm - 8.10pm: Team award nominees case introductions & announcing the winner!

  • 8.10pm - 8.20pm: Keynote presentation by Kelly Wortham - Experimentation Culture From individual to industry

  • 8.20pm - 8.30pm: Keynote presentation by Lukas Vermeer - Experimentation Culture at Booking.com

  • 8.30pm - 8.40pm: Organization-wide lifetime achievement award given by the 2020 jury to Booking.com

  • 8.40pm - 9.02pm: Organization award nominees case introductions & announcing the winner!

  • 9.02pm - 9.05pm: Closing by Ton Wesseling

All keynotes are presentations by the Award jury members.

2020 keynote speakers

All nominations will be introduced during the event by the case owners. Do not forget to register to get access to the show and/or the recording!


CONTACT FORM

Use the form below to ask us your question(s) about the awards and the judging process or to submit your help as ambassador or business partner. Our goal is to reply within 2 business days.

Emails are send through info@experimentationcultureawards.comYou will be contacted by a representative of the owner of the Experimentation Culture Awards: Help je Klant Holding BV, registered at the Chamber of Commerce in the Netherlands under number 30251897.


CONTACT FORM COMPLETED

Thank you for asking your question. Our goal is to get back to you within 2 business days. We will get in touch with you through info@experimentationculture.com

TICKET REGISTRATION DONE

Our welcome email should arrive in your email box in 2 minutes. It's send through info@experimentationcultureawards.com.Keep an eye for our emails on event announcements and the event livestream link (which will be emailed on July 6th, 2022)