Experimentation
Culture Awards
Globally awarding your growth in experimentation culture
Sharing inspirational stories by the nominees & winnersThe 2024 live broadcast is on May 23rd, 2024, 6 pm UTC
The live broadcast has 120+ minutes of keynotes, interviews and winners
THE 2024 AWARD NOMINEES
The case submit deadline for the 2024 experimentation culture awards was March 31st 2024. We received 55 case submissions.The 2024 jury went through all the cases and rated each of them on a 1 to 5 scale (1= should not be nominated, 5 = should be nominated). Jury members were not allowed to vote on cases by friends, family, colleagues or clients. The top 5 in each category (based on average score) became the finalists for the awards.Cases that did not end up in the top 5, but scored a 3.5 on average without a single jury member scoring below 3 have received an honorable mention (finalists also need this score).All the finalists will be interviewed through a video call. Based on these interviews, the jury will select a winner in each category.Looking for the 2023 nominees & winners? Find them here!
RISING STAR NOMINEES
For organizations (teams) that grew from almost no experimentation, to scaling up experimentation quickly.
Apoteket (SE) - For the Product division: "Cultivating confidence: the journey to experimentation success".
IU (DE) - For the CRO team: "Empowering growth: the experimantation journey at IU".
NS (NL) - For the CRO center of excellence: "From almost nothing to experimentation by 9 product teams on web and app".
Serko (NZ) - For the Product delivery team: "From zero to hero, experimentation culture now at the heart of the growth journey".
Telenet (BE) - For the OptiMice team: "Unleashing the powerful beast of experimentation".
Click on the organization name to read the case description
See the 2023 rising star nominees & winner
May 23rd 2024 live broadcast
TEAM NOMINEES
For organizations (teams) that grew from some people applying some CRO / Experimentation to a more solid set-up, and a strong team.
A1 Telekom Austria (AT) - For the Experimentation team: "From obligation to empowerment: the birth of experimentation culture".
Comcast Cable (US) - For Digital experience & platforms: "Driving evidence-based decision making one test at a time".
Enchanting Travels (US) - For the Web team: "From zero to data dynamo: our journey to precision testing and streamlined processes".
Pon Automotive (NL) - For the Digital marketing automation team: "Super Tuesday".
TomTom GO Navigation App (NL) - For the D2C team: "The Battle of Transformation: From Web to App Testing".
Honorable mentions for:
Butlin's (UK) - For the CRO team: "Nurturing a test & learn culture".
Intuit Quickbooks (US) - For the Online ecosystem team: "Ecosystem experimentation process evolution".
Click on the organization name to read the case description
See the 2023 team award nominees & winner
May 23rd 2024 live broadcast
ORGANIZATION-WIDE NOMINEES
For organizations that grew from a strong experimentation team to experimentation more broadly adopted throughout the organization.
ANWB (NL) - For solving scaling issues with their integrated serverside experimentation platform.
Carwow (UK) - For their experimentation culture transformation and cutting test alignment meetings in half.
Nerdwallet (US) - For enabling true democratization: any employee can launch an experiment.
Oda (NO) - For cultivating an experimentation culture to full bloom.
Sunweb Group (NL) - For propelling experimentation efforts within the organization: from processes, output to outcomes.
Honorable mentions for:
Boats Group (US) - Teams no longer test in isolation.
NatWest Retail (UK) - For reframing and evolving experimentation.
Click on the organization name to read the case description
See the 2023 organization-wide award nominees & winner
May 23rd 2024 live broadcast
COMMUNITY AWARD
The Community Award 2024, sponsored by Convert.com, is for an individual, a team, or an organization who really helps the broader experimentation community move forward.2024 is the first year we have a substantial list of potential award winners (who could only be nominated by someone else) of this separate award category we introduced last year.When the 2024 jury reviewed the cases, it did not make sense to have the jury select the nominees and winners as we do with the award for in-house teams and full organizations.The community should vote on the cases and select the winner. This is precisely what we will do with this year's Community Award.We will provide more details during the live broadcast on May 23.Attendees of the Experimentation Culture Awards will vote via email after the live broadcast, and we will announce the 2024 Community Award winner and the honorable mentions on June 27.
May 23rd 2024 live broadcast
This broadcast has 120+ minutes of keynotes, interviews and winners
The live broadcast starts on May 23rd, 6 pm UTC
THE 2023 AWARD NOMINEES
The case submit deadline for the 2023 experimentation culture awards was April 25th 2023. We received 69 case submissions.The 2023 jury went through all the cases and rated each of them on a 1 to 5 scale (1= should not be nominated, 5 = should be nominated). Jury members were not allowed to vote on cases by friends, family, colleagues or clients. The top 5 in each category (based on average score) became the finalists for the awards.Cases that did not end up in the top 5, but scored a 3.5 on average without a single jury member scoring below 3 have received an honorable mention (finalists also need this score).All the finalists will be interviewed through a video call. Based on these interviews, the jury will select a winner in each category.After we stopped with the individual category (experimentation is a team sport, was our reasoning, and we created a separate community award category), we added the Rising Star Award as a full category this year. The community award has no nominee overview; we will announce the winner in the live show.Looking for the 2022 nominees & winners? Find them here!
RISING STAR NOMINEES
For organizations (teams) that grew from almost no experimentation, to scaling up experimentation quickly.
Winner of the 2023 Rising Star award: Bupa Australia (AU) - For the Tiger Squad: "Proving the value of business experimentation through one product squad."
EDC Retail (NL) - For the CRO Team: "Lighting the experimentation fire in the boardroom".
Moneywise (CA) - For the Experiment Lab: "Finding leverage with an ambassador program structure for experimentation".
Gelato (NO) - For the Data Team: "Turning the tide to experimentation - challenging the status quo".
Gympass (BR) - For the Center of Excellence: "From zero to two in 6 months: scaling experimentation across marketing channels".
Honorable mention for:
Perlego (UK) - For the CRO Strategists: "How Perlego are trusting their growth to experimentation".
TEAM NOMINEES
For organizations (teams) that grew from some people applying some CRO / Experimentation to a more solid set-up, and a strong team.
DPG Media (NL) - For the Circulation Team: "From a CRO silo to experimentation in product teams".
ACLU (US) - For the Digital, Tech, and Analytics Team: "Scaling Experimentation at ACLU".
Heineken (NL) - For the E-business Team: "Embedding digital growth with their experimentation ecosystem".
Winner of the 2023 Team award: Tele2 Sweden (SE) - For the Experimentation Core Teams: "From a disengaged team to a fully focused machine".
ANWB (NL) - For the Web Analytics & CRO Team: "Facilitate central development of optimization through a center of excellence".
Honorable mentions for:
PVH (US) - For the CXO Team: "Breaking silos through journey-based optimization".
Smartphonehoesjes (NL) - For the CXO Team: "From CRO silo to CXO team".
ORGANIZATION-WIDE NOMINEES
For organizations that grew from a strong experimentation team to experimentation more broadly adopted throughout the organization.
Beter Bed (NL) - For how their experimentation program evolved from focusing on winners to gaining comprehensive insights.
Groover (FR) - For how they defined a testing framework to ensure alignment without compromising velocity.
Winner of the 2023 Organization-wide award: Drukwerkdeal (NL) - For how their CEO became their biggest fan.
Zillow (US) - For building the bridge between experimentation and business impact.
RTL Netherlands (NL) - For going from pushing experimentation in limited teams towards getting pulls from everywhere.
COMMUNITY AWARD
For an individual, a team, or an organization who really helps the broader experimentation community move forward. Note that the case owner can not submit these cases; they can only be nominated by someone else.
Winner of the 2023 Community Award:
Lucia van den Brink & Daphne Tideman (NL) - For starting the Female Speakers in Growth and Experimentation board and the Women in Experimentation community.
Honorable mention for:
Bhavik Patel (UK) - For running the CRAP Community with the Conversion Rate optimization, Analytics and Product meetups.
June 15th 2023 recording
This broadcast has 120+ minutes of keynotes, interviews and winners
The live broadcast started on June 15th, 6pm UTC
THE 2022 NOMINEES & WINNERS
The case submit deadline for the 2022 experimentation culture awards was May 21st 2022. We received 54 case submissions.The 2022 jury went through all the cases and rated each of them on a 1 to 5 scale (1= should not win, 5 = should win). Jury members were not allowed to vote on cases by friends, family, colleagues or clients. The top 5 in each category (based on average score) became the finalists for the awards.Cases that did not end up in the top 5, but scored a 3.5 on average without a single jury member scoring below 3 have received an honorable mention (finalists also need this score).All the finalists were interviewed through a video call. Based on these interviews the jury selected a winner in each category and also selected the winners for the community award and the rising star award.Looking for the 2021 nominees & winners? Find them here!
INDIVIDUAL NOMINEES
Amy Hartwig (US) - For growing experimentation at Microsoft DevDiv.
WINNER of the 2022 Community AWARD is: Tom van den Berg (NL) - For CRO weekly: a curated newsletter to help increase the quality of experimentation throughout the market.
Ana Catarina Cizilio (BR) - For empowering A/B-testing in Brazil.
WINNER of the 2022 Individual AWARD is: Bjarn Brunenberg (NL) - For experiment program spitting fire at TomTom.
Ellie Hughes (UK) - For becoming an experimentation evangelist at AND Digital
Honorable mention for:
Marianne Stjernvall (SE) - For sharing in-house lessons of building an experimentation culture.
TEAM NOMINEES
WINNER of the 2022 Team AWARD is: Vista (US) - For the Experimentation Hub: "Building a culture of experimentation in public".
WINNER of the 2022 Rising Star AWARD is: SMG (CH) - For the CRO Team: "Inform, inspire, involve, iterate. Revive & expand a culture of experimentation".
Videoland (NL) - For the Experimentation Team: "From 5.5 experiments per month to a center of excellence".
Accor (FR) - For the Experimentation Department: "Making experimentation great(ly known/used)".
ING (NL) - For the Global Conversion Centre of Excellence and Pricing team: "Connecting to experiment: the virtual conversion workshops lead to 77% more tests in 1 year".
Honorable mentions for:
DPG Media (NL) - For the CRO-team: "From CRO-team to centre of excellence".
Thinkific (CA) - For the Growth team: "Reducing barriers to experimentation".
ORGANIZATION-WIDE NOMINEES
Marks & Spencer (UK) - For culture, velocity & quality.
TL (CH) - For how do you energize the culture of experimentation in a 125 year old company?
WINNER of the 2022 Organization-wide AWARD is: ASOS (UK) - For embracing an experiment-oriented mindset.
Vattenfall Netherlands (NL) - For adopting data-driven customer experience optimization.
Beter Bed (NL) - For development accelerates experimentation.
July 7th 2022 live broadcast
This live broadcast had 120+ minutes of keynotes, interviews and winners
The live broadcast started on July 7th, 6pm UTC
THE 2021 NOMINEES & WINNERS
The nomination deadline for the 2021 experimentation culture awards was May 28th 2021. We received 62 case submissions.The 2021 jury went through all the cases and rated each of them on a 1 to 5 scale. They were not allowed to vote on cases by friends, family, colleagues or clients. The top 5 in each category (based on average score) became the finalists for the awards.All those finalists were interviewed. Based on these interviews the jury selected a winner in each category:- 2021 individual nominees & winner
- 2021 team nominees & winner
- 2021 organization-wide nominees & winnerIn 2020 we had 6 out of 15 finalists (case owners) being female, in 2021 this went down to 4 out of 15 finalists. We are concerned about this. We also believe we need to work harder on the full diversity perspective of our selected finalists / cases.This is why we are reconsidering the way we collect cases and judge these cases to have a more diverse representation of our industry as finalists of our 2022 awards.Looking for the 2020 nominees and winners? Find them here!
INDIVIDUAL NOMINEES
WINNER of the 2021 Individual AWARD is: Aleksander Fabijan (SI) - For it Takes a Flywheel to Fly: Leading Microsoft to share lessons in experimentation culture.
Alexandre Suon-Perhirin (FR) - For driving the adoption of A/B testing as a decision-making tool for strategic investments.
Peter Ernst (US) - For building a culture of experimentation in healthcare: find better ways to connect patients during COVID.
WINNER of the 2021 Community AWARD is: Rommil Santiago] (CA) - For launching Experiment Nation: a home for experimenters around the world, from beginner to pro.
Ruben de Boer (NL) - For building a worldwide CRO community with free coaching for charity and low barrier Udemy courses.
TEAM NOMINEES
WINNER of the 2021 Team AWARD is: bol.com (NL) - For Team Experimentation and the journey to make their colleagues smarter.
TOYOTA (FR) - For the CRO team. An eye on ROI: Executives and CRO experts nurture experimentation together.
ANWB (NL) - For the Web analytics & CRO-team and the CRO-training, onboarding and documentation of the CRO process.
iTech Media (UK) - For the Experimentation Team. Advocating their operational framework and increasing the velocity of experiments.
fonQ (NL) - For the CX Team. Rocket start CRO: from 0 to 30 A/B-tests in 6 months.
ORGANIZATION-WIDE NOMINEES
Rabobank (NL) - For the growth from ad-hoc to structural experimentation and optimization.
BAM - Bamboo Clothing (UK) - For the growth and journey to experimentation informing smarter decisions.
Convoy (US) - For enabling experimentation at scale in a B2B Marketplace.
WINNER of the 2021 Rising Star AWARD is: The Royal Mint] (UK) - For embedding a culture of experimentation at a company with over 1,100 years of history.
WINNER of the 2021 Organization-wide AWARD is: ING (NL) - For moving towards continuous experimentation.
July 8th 2021 live broadcast
This live broadcast will have 150 minutes of keynotes, interviews and winners
THE 2020 NOMINEES & WINNERS
The nomination deadline for the 2020 experimentation culture awards was on July 31st 2020. A total of 65 cases were submitted through our website.The 2020 jury went through all the cases and rated every single one of them on a 1 to 5 scale. The top 5 in each category became the final nominees for the awards.All those finalists have been interviewed. Based on these interviews the jury selected a winner in each category.Looking for the 2021 nominees? You can find them here.
INDIVIDUAL WINNER & NOMINEES
WINNER of the 2020 Individual AWARD is: Guido X Jansen (NL) - For evangelizing CRO in general and CRO culture in particular with the CRO Cafe Podcast.
Gabriela Denison (UK) - For building trust in fast, pain-free experimentation at ASOS.com.
Raoul Warren Doraiswamy (AU) - For building the CovidCRAP community. Supporting businesses affected by CoronaVirus for free.
Desiree van der Horst (NL) - For taking away the fear of sharing failures at Fingerspitz.
Stefan Thomke (US) - For writing "Experimentation Works", the book on experimentation for business leaders.
TEAM WINNER & NOMINEES
WINNER of the 2020 Team AWARD is: VodafoneZiggo (NL) - For the CRO Center of Excellence. Creating a culture of experimentation.
Taxfix (DE) - For the Experimentation taskforce. The path towards a strong experimentation culture.
Hallmark Cards Benelux (NL) - For the CRO team. Started from the bottom, now they are 100 tests further.
Microsoft (US) - For the Experimentation Platform Team. A/B Testing and Covid-19: Data-Driven decisions in times of uncertainty.
SAP (US) - For the Optimization Lab. Building a data-driven culture through widespread testing.
ORGANIZATION-WIDE WINNER & NOMINEES
WINNER of the 2020 Organization-wide AWARD is: Farfetch (PT) - For the circle path from insights to learnings. Change a company experimentation mindset in a data-driven approach.
GlaxoSmithKline (US) - For building a culture and practice of experimentation.
MongoDB (US) - For going from 0 to 100 within 6 months. Implementing and accelerating a research and testing program.
AllClear Travel Insurance (UK) - For the democratization of experimentation.
RS Components (UK) - For full stack experimentation transformation. The 'binding agent' between Product & Engineering.
LIFETIME ACHIEVEMENT AWARD
The 2020 jury has decided to hand out an INDIVIDUAL Experimentation Culture lifetime achievement AWARD
The WINNER is: Ronny Kohavi (USA) - For all the research and publications that helped so many experimenters to understand how to conduct trustworthy digital experiments.
LIFETIME ACHIEVEMENT AWARD
The 2020 jury has decided to hand out an ORGANIZATION-WIDE Experimentation Culture lifetime achievement AWARD
The WINNER is: Booking.com (NL) - For being a big example organization for experimentation, known and quoted by many decision makers throughout the world.
September 24th 2020 broadcast
The 2020 liveshow has 180 minutes of inspirational talks and interviews
ORGANIZATION-WIDE NOMINEE:
ANWB - Netherlands
Solving scaling issues by implementing an integrated serverside experimentation platform."After building CRO processes and culture and improving maturity, we started to run into scaling issues:• Our webanalytics & CRO team is responsible for experimentation and needed in every step of the process. We grew to a team of >25 people, but we can’t keep on growing forever.• There’s an increased demand top-down for measuring impact and making decisions data driven.• We got more and more disturbed by the client-side implementation of our experimentation platform.We combined this into one solution: a server-side experimentation platform, integrated with cms.• The server side implementation removes flickering. Beside the performance improvement, this is required in order to be able to run experiments on components like main navigation.• The integration with cms makes it easy to build experiments and thus makes experimentation accessible to a large group of (potential) users.• We moved ownership for building and monitoring experiment from webanalists to content marketers.• It’s a lot less error-sensitive.• We shortened the process in the building-phase.We use this project to boost experimentation by communicating about it in central communication points, through presentations in different chapters (Content, Product Owners, UX, Front-end) and giving several workshops.The coming year we’ll be working on automating processes. The new experimentation platform is the first step in our plan to scale from 120 to 850 experiments per year, our goal for 2026."
May 23rd 2024 live broadcast
This broadcast has 120+ minutes of keynotes, interviews and winners
The live broadcast starts on May 23rd, 6 pm UTC
ORGANIZATION-WIDE NOMINEE:
Carwow - United Kingdom
Experimentation culture transformation and cutting test alignment meetings in half."Carwow has gone through an experimentation culture transformation! We went from a company feeling that AB tests slow down product innovation, to seeing how making data driven decisions ultimately means we can ship the right things faster.The analytics team decided that we need to do more testing, as a large portion of our time was spent on pre/post analysis for changes released without testing, which also resulted in low confidence decision making. To ensure we built lasting change, we invested in incentives and time saving processes that led our stakeholders to testing rather than shaming them for releasing without measurement in mind.• First, we launched an experimentation platform to make reading tests easy and fun, we hosted talks on experimentation, and loudly celebrated experimentation wins.• We also helped stakeholders understand in hindsight where testing could have been helpful, and even went back and tested some of those changes!• Finally, we introduced, iterated on, and agreed to a practical decision framework and tool called “When To Test”. This framework was approved by leadership, thereby saving time and meetings to decide what gets tested and what doesn’t.We now have less than half as many hours spent discussing testing plans! Slowly, our teams have become more comfortable and at ease with testing and have a clear understanding the of the value of moving with velocity (in the direction of a better product) and not just the speed of releasing changes."
May 23rd 2024 live broadcast
This broadcast has 120+ minutes of keynotes, interviews and winners
The live broadcast starts on May 23rd, 6 pm UTC
ORGANIZATION-WIDE NOMINEE:
Nerdwallet - United States
Enabling true democratization: any employee can launch an experiment."This organization started at 6 experiments per quarter company wide and then developed a strong experimentation culture through a staged approach to achieve a 1900% increase in velocity and hundreds of thousands in net incremental revenue per day.The addition of a specialised team that centralised experiments for paid performance marketing channels resulted in 5X velocity and large growth of marketing spend to soak up the extra ROAS generated from conversion rate increases.With that success, the effectiveness of experimentation was respected and the specialist team was graduated to a centre of excellence team which toured their best practices to the rest of the company even though corporate restructuring discouraged centralisation.But by continuing to provide superior and persistent spearheading, (and the ability for any employee in the company to launch an experiment) the team has increased experimentation coverage to 95% of their entire population of consumers and 125 experiments per quarter (as of Q1).Initially only a web team effort, we have now expanded to apps, marketing, ML models, etc. while maintaining quality control, strategic influence and best practices by building them as the default configurations of the mechanisms for experimentation."
May 23rd 2024 live broadcast
This broadcast has 120+ minutes of keynotes, interviews and winners
The live broadcast starts on May 23rd, 6 pm UTC
ORGANIZATION-WIDE NOMINEE:
Oda - Norway
Cultivating an experimentation culture to full bloom."Over the past 12 months, Oda has seen a significant increase in cultural maturity surrounding experimentation and the growth of related practices. We went from a novice stage to intermediate with some parts of our experimentation program/culture stretching out on the advanced stage. Today, all new product development goes via gradual rollouts and A/B tests, with everyone in Product & Tech equipped to interpret test results.This transformation began with the adoption of an experimentation platform in early 2022, which proved to be cost-effective and user-friendly. Within 6 months, the incremental cost of running a new test approached 0. We further increased the robustness of the platform by enabling teams to run A/A tests and have recently implemented pre-experiment bias detection mechanisms, indicating a deeper level of maturity.By placing emphasis on documentation, templates, and guides, we enhanced teams' autonomy in testing, while bi-weekly community sessions facilitated knowledge sharing. The introduction of risk profiles increased the frequency of low-risk tests, and introducing guardrail metrics in assessing success of tests helped align experimentation with Oda's business strategy.Lastly, on organization, our hybrid setup for A/B testing, consisting of a Centre of Excellence alongside autonomous cross-functional teams, has empowered Product Managers to prioritize evidence-guided experimentation as a means of learning and optimizing our product."
May 23rd 2024 live broadcast
This broadcast has 120+ minutes of keynotes, interviews and winners
The live broadcast starts on May 23rd, 6 pm UTC
ORGANIZATION-WIDE NOMINEE:
Sunweb Group - Netherlands
Propelling experimentation efforts within the organization:
from processes, output to outcomes."In November 2023, Sunweb restructured, forming Team Conversion with customer-facing Product Owners, Product/UX designers, and CRO specialists, reshaping our approach to experimentation.Within three months, we surpassed 2023 and 2022's experimentation velocity (currently +60), affirming our target of +200 experiments this year with a +25% win rate and +70% conclusive rate.Conclusiveness, a key metric, propels further iterations despite outcomes. Additionally, we aim for a 20% CVR uplift EOY, tracked through revised revenue models considering effect over time and factors like false discovery rate.To meet our targets, we host bi-weekly hypotheses workshops using qualitative and quantitative analyses ("Product boost"), engage in bi-weekly user research, and hold monthly deep dives with our customer care team.Each product team communicates with at least one customer weekly, using opportunity solution trees. To maintain velocity, we've implemented a dual-track experiment pipeline—product teams handle development experiments, while the CRO guild conducts low or no-code experiments.Experimentation is now intrinsic to Sunweb's success, supported by transparency and accountability through automation in our improved Airtable setup, facilitating company-wide progress sharing.With dedication, collaboration, and a strong belief in experimentation, Team Conversion competes for the Experimentation Culture Award at Sunweb this year."
May 23rd 2024 live broadcast
This broadcast has 120+ minutes of keynotes, interviews and winners
The live broadcast starts on May 23rd, 6 pm UTC
HONORABLE MENTION - ORGANIZATION-WIDE:
Boats Group - United States
Teams no longer test in isolation."Mike Grabowski has seen it all. As the Chief Product Officer at Boats Group, the world's largest network of online boating and yachting marketplaces, including BoatTrader, yachtworld.com & boats.com and 13 leading European marketplaces, Mike knows the promise AND pitfalls of testing tools. He's learned that for experimentation-led growth to be successful, the teams (the people) responsible for growth –product, marketing, UX, data, etc.– must be independent while also pulling in the same direction.Is Mike testing? No, but he's a senior executive who has managed to successfully unite different teams at Boats Group to use experimentation to drive the business forward.Globally, Boats Group's teams now leverage hybrid (client and server-side) testing approaches to build controlled experiments while validating cross-functional hypotheses built from a wealth of experience and industry-leading data.Whereas global teams were previously limited in their shared learnings across similar platforms with different consumer bases, they now test and learn together - by analyzing the performance of shared KPIs.Compared to the previous 12 months, the Boats Group has increased its testing by 63% and jumped out to a hot start in Q1 achieving business outcome goals."
May 23rd 2024 live broadcast
This broadcast has 120+ minutes of keynotes, interviews and winners
The live broadcast starts on May 23rd, 6 pm UTC
HONORABLE MENTION - ORGANIZATION-WIDE:
NatWest Retail - United Kingdom
Reframing and evolving experimentation."Context: We set up from scratch as a CRO CoE within the Retail Bank in Jan of 21 and have since undertaken hunderds of experiments on web. We’ve laid down new processes, frameworks, and training to hunderds of colleagues. We immerse ourselves in product teams and work together with them to build optimisation plans and train colleagues.
Last year, we evolved to begin framing experimentation differently, as an insurance against bad decisions. Here’s how:We all love the happy path of finding a problem, fixing it, and basking in the win, right? But we observed an uptick in A/B testing ‘losses,’ whereby challenger variants were driving a downturn in sales. That was frustrating, but also an immensely healthy thing culturally. Firstly – we were finding these things out precisely because we’re experimenting in the first place. But secondly, we were halting the roll out of changes which damaged revenue.With this trend prevalent, we re-framed the way we talk to experimentation as an ‘insurance against bad decisions’ as much as being a method of driving incremental ROI. We also added an additional ‘sales saved’ metric to track the accumulative positive impact of not rolling these changes out to customers.In those same instances, we also made sure we followed up on what we did learn and pursued follow up phases of testing to try and either minimise the loss of ROI or, ideally, find a pathway to drive an uplift."
May 23rd 2024 live broadcast
This broadcast has 120+ minutes of keynotes, interviews and winners
The live broadcast starts on May 23rd, 6 pm UTC
TEAM NOMINEE:
A1 Telekom - Austria
Experimentation team
From obligation to empowerment: the birth of experimentation culture in A1."Until recently, A1 did “trials” and not experiments - performance was compared to the past instead of a control group and statistical significance was considered as something Nobel laureates should care about, but not us marketing folks! In 2022, 27 experiments were launched by a few colleagues, because it was something “we had to do” due to volume goals.Experimentation team (with 1.5 FTEs) focused on upskilling operational level and providing hands-on support with experiments to increase awareness & interest.With change management efforts on all levels, from people, tools, processes and to strategy, we democratized experimentation in 1 year! In 2023, almost 100 experiments were done by 12 (of 14) teams.We focused on:1) Mindset of management and educating them about their role in the experimentation game. We saw a drastic change; instead of “no need to test what we already know” to “Let’s test this, it’s really important”. Challenging HiPPO’s has never been more frequent (and fun!) as people are now empowered with data;2) Promoting value of iterative testing and incremental changes and regularly reporting performance incl fin. impact;3) Rewards Program developed to recognize the testing efforts (not results), enable knowledge sharing in well-attended events and to provide unique rewards. It was named by teams as the biggest motivator to experiment more! Together, we started living an experimentation culture in A1."
May 23rd 2024 live broadcast
This broadcast has 120+ minutes of keynotes, interviews and winners
The live broadcast starts on May 23rd, 6 pm UTC
TEAM NOMINEE:
Comcast Cable - United States
Digital experience & platforms
Evolution of Comcast Cable's Experimentation Program: Driving Evidence-Based Decision Making One Test at a Time."The Comcast program evolved from ad-hoc testing to a comprehensive approach. A larger digital team reorg disrupted the program 2 years ago. This lead to a complete restructuring, with just 3 of the 12 members staying on. Now at 18 members, we embraced a dual mission of exploration and rigor.We grow our focus from eComm CRO to multi-channel experimentation, achieving balance between improving conversion rate, containment rate, issue resolution & NPS. We embraced a tool-agnostic testing across web, app, chat & IVR. Split.io, FF4J, Quantum Metric, Looker and the warehouse reporting have been added, which already included Adobe Analytics and Target.Experimentation was the domain of a handful employees. This year, nearly everyone one in the 130 digital team will be involved. This helped grow test volumes from 26/year to 140 last year. Notably, we reduced the costs of live prototyping from $100k to $23k per test.Personnel & capacity growth happened despite cuts in funding. Efficiencies generated from streamlining process, removing waste, aligning goals, standardizing and partially automating key artifacts helped buttress the partnership gains. This created confidence & senior leadership advocacy in a test & learn philosophy.We are approaching the limits of what can be achieved through passion, partnership, and process. We have started long-range planning towards holistic system, with goal of embedding and automating innovation and quality across the organization."
May 23rd 2024 live broadcast
This broadcast has 120+ minutes of keynotes, interviews and winners
The live broadcast starts on May 23rd, 6 pm UTC
TEAM NOMINEE:
Enchanting Travels - United States
Web team
From zero to data dynamo: our journey to precision testing and streamlined processes."When I joined ET, testing was more of an afterthought, data wasn't part of the decision making process, velocity wasn't a thing (maybe 3 tests per month, rarely significant) but also impact wasn't visible, with CVR dropping month on month.
Within 3 months from joining, I took over the existing Web team , that focused exclusively on fulfilling tech tickets, and brought in a well defined process that starts and ends with data:1) Funnels are now analyzed weekly, across all different channels;2) Every hypothesis is based off data ( GA + behavioral data) and no test is approved without a clear understanding of what data/report helped deliver it;3) We run monthly workshops involving teams across the business;4) For each test we're also measuring which psychological principles are applied, to improve our prioritization in the long run.I am very lucky to have a team that is in love with data, as it gives them an understanding of the impact they're making.In terms of impact, we've almost doubled CVR from 2.7% to 4.9% by changing all our landing pages from basic copy+forms to a more gamified experience as travel is an emotional experience, we shouldn't’ cage people’s dreams behind boring forms.At current test velocity we expect to run over 200 tests of mixed complexity by the end of the year , aiming to move more and more towards horizontal testing and personalized experiences (including our voice/sales teams) by the end of 2025"
May 23rd 2024 live broadcast
This broadcast has 120+ minutes of keynotes, interviews and winners
The live broadcast starts on May 23rd, 6 pm UTC
TEAM NOMINEE:
Pon Automotive - Netherlands
Digital marketing automation team
Super Tuesday."Super Tuesday is about creating the ultimate digital customer journey. Unlimited experimentation at scale, on every platform and in every phase of the journey.Previously every car brand ran their own experiments and focused on their own silo. Every now and then learnings were shared, but we thought we should optimize this process and learn & experiment together. Starting with Super Tuesday helped us to create a multi-brand optimization community within Pon Automotive. Multi-brand topics, knowledge sharing and fun are the key focus elements of our team.In the community our multidisciplinary team, consisting of Digital Marketers, Analysts, CRO/CX Specialists and Business Stakeholders, comes together every Tuesday. The main business- and CX challenges are our input for the optimization roadmap. From there we follow a structured optimization process: insights, brainstorm, hypothesis creation, prioritization, experimentation and learning & reporting.What makes Super Tuesday truly successful is: we created our own community identity. We have Super Tuesday merch, such as purple sweaters. As a result, everybody in the company recognizes us and asked to help them experiment. Also, we added a competitive element in the form of a Super Tuesday award for the best experiment. Furthermore, we shared responsibility, by making use of rotating team captains. Finally, all brands learn from every experiment. We did 100+ of them last year!That’s how we make impact with Super Tuesday!"
May 23rd 2024 live broadcast
This broadcast has 120+ minutes of keynotes, interviews and winners
The live broadcast starts on May 23rd, 6 pm UTC
TEAM NOMINEE:
TomTom GO Navigation App - Netherlands
D2C team
The Battle of Transformation: From Web to App Testing."TomTom moved from king of hardware to software challenger. Battling Google Maps and Waze, our story is unique: shifting our entire experiment program from Web to App, maintaining speed and quality across teams with the same headcount.Challenges? Plenty:1) App vs. Web meant a surge in development dependency, release cycles, and technical barriers.2) Tool shifts from Sitespect to Purchasely, GA to Mixpanel, Salesforce to Airship.3) Cross-team alignment needed on data strategy, design, and development.4) Limited resources with 20-30% time allocation for lean experimentation.Biggest challenge? Keeping spirits high despite 80% of tests failing. Last year, significant wins had everyone dancing on tables. This year, wins are scarce.Yet, we progressed:- Utilised goal tree mapping to connect revenue targets to testing strategically.- Introduced a 4-step brainstorm framework for focused impact.
- Decentralised processes and documentation across teams.
Created data quality assurance through a CoE analyst.
Boosted cross-team collaboration and collective experimentation through gamification.The outcomes?- MRR boost of 85% from 196 experiments across within a year.- 5x increase in A/B test frequency.- 10x paywall/onboarding screen production boost.- 90% development cost reduction using Purchasely.- Halved time from hypothesis-to-production.- Data-backed experiments up by 35% (66% to 89%)- +150% complete documentation of experiments (33% to 74%)- No. of employee testing doubled (4 to 9)."
May 23rd 2024 live broadcast
This broadcast has 120+ minutes of keynotes, interviews and winners
The live broadcast starts on May 23rd, 6 pm UTC
HONORABLE MENTION - TEAM:
Butlin's - United Kingdom
CRO team
Nurturing a Test & Learn Culture at Butlin's."The Butlin’s CRO programme has come a long way from being a one-member team to now imbibing a Test and Learn culture within the Digital and the wider Marketing team. We’ve been quietly ramping up our CRO activity since the post-Covid lockdown world when we re-opened our resorts and welcomed our guests back.We played a crucial role in enabling the migration of both our websites with the setup of a bespoke server-side testing program that allowed us to validate the impact of the new website. We welcomed Equator over a year ago now, our CRO specialist team, who are integral to our CRO programme and an extension of our team.We’ve consistently delivered some challenging experiments that have continued supporting our trading needs and product roadmap while supercharging the velocity of our CRO programme.In its nascent stages, our CRO programme focussed on the low-hanging fruit and above-the-fold hypotheses. But the goal has stayed the same: Test (everything) & Learn. In the long term, to imbibe the Test & Learn culture within Butlin’s digital team and spread it across the wider Marketing team.Through this process we’ve been able to raise the profile of CRO within our organisation and have truly transformed the way we work as a Digital team. We’ve been able to imbibe the Test and Learn culture whereby our CRO initiatives have brought a new level of credibility with the work that we do. CRO is now well and truly embedded in the business, reflected by the cross-team adoption."
May 23rd 2024 live broadcast
This broadcast has 120+ minutes of keynotes, interviews and winners
The live broadcast starts on May 23rd, 6 pm UTC
HONORABLE MENTION - TEAM:
Quickbooks - United States
Online ecosystem team
Ecosystem experimentation process evolution."1. Context: the QuickBooks Online Ecosystem team at Intuit uses experimentation to engage customers with key product interfaces such as Onboarding and Homepage.2. Problem: QuickBooks' transition from an accounting solution to a suite of ecosystem services began to impact service metrics negatively, leading to:- Slower experimentation pace and fewer experiments due to internal friction.- Disjointed customer journey across different flows.- Inconsistent experiment results impacting business metrics.3. Solution: the team revamped their experimentation practice in three areas:- Process: Introduced an experimentation playbook, weekly reviews, and organization-wide updates via Slack.- Technology: Created a central metrics repository, a reusable framework for code contribution, and a governance forum for cohesive product experiences.- Culture: Incorporated Intuit’s values of 'Stronger Together' and 'Customer Obsession' into the experiment roadmap.4. Impact:- Internal teams: Improved autonomy, velocity, and transparency in running experiments.- Customer experience: Notable improvement in activation, monetization, and engagement metrics.- Business impact: Increased experiment volume and success rate, boosting revenue growth."
May 23rd 2024 live broadcast
This broadcast has 120+ minutes of keynotes, interviews and winners
The live broadcast starts on May 23rd, 6 pm UTC
RISING STAR NOMINEE:
Apoteket - Sweden
Product division
Cultivating confidence: the journey to experimentation success."Experimentation at Apoteket was virtually non-existent 12 months ago,with no dedicated function or process. To address this, the team embarked on a transformative journey, starting with education sessions led by industry experts. A CRO manager was then hired to spearhead experimentation efforts, supported by the timely onboarding of the behavioral insights tool ContentSquare. Collaborative Ways of Working were established with IT to integrate testing seamlessly into the product development roadmap.Internal development focused not only on creating tests but also on building a robust, tool-agnostic technical set up for experimentation. With the sunset of Google Optimize, a new AB testing tool was onboarded, complemented by design workshops that leveraged quantitative insights for early identification of test candidates. The results were transformative: a structured experimentation process emerged, fostering high buy-in from UX designers and product managers. A healthy test backlog (50+ tests planned for 2024) aligned with the product roadmap was established, and a pseudo server-side testing set up led to smooth roll outs of winning variations(80% roll out rate).From its non-existent state, Apoteket's experimentation program has swiftly ascended over the last 12 months to an intermediate level of maturity. Experimentation is now an integral part of Apoteket's digital strategy, with deeper organizational knowledge and confidence in insights gained from testing."
May 23rd 2024 live broadcast
This broadcast has 120+ minutes of keynotes, interviews and winners
The live broadcast starts on May 23rd, 6 pm UTC
RISING STAR NOMINEE:
IU - Germany
CRO team
Empowering Growth: Experimantation journey at IU International University of Applied Sciences."During the past year, our CRO team underwent major changes, including migrating to a new CMS tool, reduction in team size from 10 to 5 CROs and investing in Contentsquare to enhance analysis and hypothesis formulation.Despite a reduction in team size in the latter half of 2023, and a decrease in AB tests, our transition to the new CMS tool in September provided us with increased testing flexibility. To foster an experimentation culture, we introduced monthly meetings with UX/CX/PM members and streamlined communication with executive summary slides for stakeholders.To adapt to fewer CRO managers generating test ideas, we enhanced agility through closer collaboration with CX managers, utilizing Jira Priority Poker for idea prioritization and roadmap planning.Significant Enhancements:• The shift to a new CMS tool enabled cross-domain testing and collaboration among CRO managers, boosting engagement with more CROs involved in one test conception.• Monthly stakeholder meetings improved visibility and engagement across domains.• Executive summaries streamlined analysis, reducing time by approximately 30%. With the executive summary, the content became easier to understand and felt to be viewed by more stakeholders.• Priority Poker in Jira enhanced test idea prioritization and roadmap planning.• The introduction of Contentsquare has enhanced our analytics, broadening stakeholder engagement from just 5 CROs to over 10 individuals, including CX and Content Managers."
May 23rd 2024 live broadcast
This broadcast has 120+ minutes of keynotes, interviews and winners
The live broadcast starts on May 23rd, 6 pm UTC
RISING STAR NOMINEE:
NS - Netherlands
CRO center of excellence
From almost nothing to experimentation by 9 product teams on web and app."NS deserves the Rising Star Award for going from very sporadic experimentation by a central team to over 10+ concurrently on web and app by 9 product teams.NS had a centralized CRO program in 2019, but was not sustainable. When COVID hit, nothing was left of it, and experimentation happened very sporadically for years. To change this, Jeroen and Wouter (NS strategists) had many stakeholder conversations over two years, resulting in an experimentation strategy in 2023 and ~500 employees receiving training.14 UXers and Analysts were appointed experimentation champions in 9 product teams. Most of them did not know each other and came from different teams with no experimentation process. Jeroen and Online Dialogue act as a CoE. In October 2023, Online Dialogue educated the champions and created an environment for them to thrive.The CoE conducted an extensive maturity scan, resulting in an overview of culture, challenges, and a strategy tailored to each team. Challenges included deep-rooted operational history, siloed operations, data issues, and too much stakeholder involvement.CRO is now on the agenda for almost all product team meetings, with more enthusiasm at every level. With an increasing number of experiments (including app testing), the CRO way of working and culture is spreading. There is now a solid foundation from which NS can keep increasing the quality and velocity of all teams."
May 23rd 2024 live broadcast
This broadcast has 120+ minutes of keynotes, interviews and winners
The live broadcast starts on May 23rd, 6 pm UTC
RISING STAR NOMINEE:
Serko - New Zealand
Product delivery team
From Zero to Hero, Experimentation Culture Now at the Heart of Growth Journey."Partnering with Booking.com to bring our enterprise travel tech to small businesses, we recognised success would be driven by CX, so needed to measure and optimise this. Thus began our journey into experimentation. Without an experimentation culture, we had relied on observational data to determine good or bad. A better level of proof for decision making was needed, and we embraced randomised control trials (A/B experiment) as the highest form of evidence.We invested in data infrastructure and tooling to build foundations, and developed our team's ability through training and knowledge sharing. We empowered teams to generate their own hypotheses and test without permission (within boundaries), trusting the ABsmartly tool to give confidence to prove a hypothesis or not.Our first A/B experiment started in May 2023. Since then we launched 150+ unique experiments ranging from very simple to complex feature sets. We prioritised velocity to learn fast, tracking to an OKR for experiment numbers. We championed failure, learning to iterate and re-test. We have a way to go on the maturity curve, but are at the point of no front-end change going live without an experiment.The business outcome has been significant; Millions in incremental revenue identified, millions of potential loss saved through experimentation. What started as a product initiative is now embedded across the organisation. From zero to hero, Serko’s experimentation culture is now at the heart of our growth journey."
May 23rd 2024 live broadcast
This broadcast has 120+ minutes of keynotes, interviews and winners
The live broadcast starts on May 23rd, 6 pm UTC
RISING STAR NOMINEE:
Telenet - Belgium
OptiMice team
Unleashing the powerful beast of experimentation."In early 2023, amidst a complete reorganization, we successfully convinced senior leadership to adopt an experimentation-based decision-making approach, highlighting the success of other companies and the potential for significant improvements on our website.This led to the creation of the OptiMice (optimization, test-mice… you get it) team! A group of 12 dedicated individuals committed to optimizing our digital journeys through experimentation.We initiated our journey by learning from companies with a strong experimentation culture, equipping our team with essential tools like Contentsquare and Airtable, and engaging with CRO agency Online Dialogue for training and inspiration.And results came…fast! Within a year, we conducted 55 experiments across 6 different tribes, yielding 7 breakthrough winners that generated over EUR 1 million in combined cost savings and sales increases.The remaining 48 experiments, while not winners, were far from futile. They provided invaluable insights into less effective strategies and improved our analytical capabilities, reinforcing the essence of experimentation: learning (and sharing these valuable learnings) from every outcome.Looking ahead to 2024, our focus will be enhancing the quality of our experiments (rather than the quantity). We aim to refine the research phase (using a little AI in the process) and emphasize meaningful outcomes over output.Our goal: we will become the best experimentation and CRO team in Belgium!"
May 23rd 2024 live broadcast
This broadcast has 120+ minutes of keynotes, interviews and winners
The live broadcast starts on May 23rd, 6 pm UTC
SUBMIT YOUR 2024 CASE(S)
Case submission deadline was March 31st, 5 pm UTC.The 2024 jury is looking for inspirational stories on (your) growth in experimentation maturity. It's not about the current level of experimentation; we nominate and award for the growth steps that have been taken in the past 12 months.See our about page for more info on our award categories.Submitting is straightforward: all questions in this PDF download. Please scroll down for our submission form.You can submit your case or nominate someone else. When a case from someone else is selected as a finalist, we will reach out to them on your behalf as the case submitter and ask the potential finalist if they accept the nomination and are willing to participate in the interview before we announce anything about their case on our website.
"The Experimentation Culture Awards is not awarding that one experiment. This is about improving the process, structure, trustworthiness, democratization, and motivation for experimentation in organizations."
2024 CASE SUBMISSION FORM
In case the form is not working: use this direct form link.After submitting your case you will receive an email with the confirmation of your entry. We will contact you in the second week of April to let you know if your case entry is selected as one of the final 5 nominees in a specific category.Read on to learn more about the jury members and the process. For question on case submitting: use our contact form.
No time left until the award case submission deadline (March 31, 2024, 5 pm UTC):
ABOUT THIS AWARDS EVENT
This is the 2024 about page.The Experimentation Culture Awards started in 2020 as an event to promote a culture of experimentation globally. Our goal is to help people trying to grow an evidence-based decision-making culture by pushing them in the right direction.We do this by sharing inspirational stories of experimentation maturity growth and nominating individuals, teams and organizations who deserve recognition for their work.In an experimentation culture, you are free to try and fail or succeed, while the direction and the result of work is based on trustworthy gathered evidence. Grow fast, grow forward.
"It's not about that one experiment, it's about improving the process, structure, trustworthiness, democratization and motivation for experimentation and data-driven decision making."
The 2024 jury will be handing out 4 awards:
Rising Star Award 2024: For organizations (teams) that grew from almost no experimentation, to scaling up experimentation quickly.
Team Award 2024: For organizations (teams) that grew from some people applying some CRO / Experimentation to a more solid set-up, and a strong team.
Organization-wide Award 2024: For organizations that grew from a strong experimentation team to experimentation more broadly adopted throughout the organization.
Community Award 2024, sponsored by Convert.com: For an individual, a team, or an organization who really helps the broader experimentation community move forward.
Our goal is to praise every case study that should be praised for growing experimentation culture by nominating this case or at least giving this case an honorable mention.Note: community award entries can not be a self nomination.
THE AWARDS JURY OF 2024
This is our 2024 jury. Read more about our awards or our 2023 jury.
Storm Jarvie (AU) - Rising star award winner 2023 @ Bupa Australia
Luiza de Lange (SE) - Team award winner 2023 @ Tele2 Sweden
Roderik Peeters (NL) - Organization-wide award winner 2023 @ Drukwerkdeal
Daphne Tideman (UK) - Community award winner 2023 @ Women in Experimentation
Shiva Manjunath (US) - Co-Founder and Host @ From A to B podcast
Kelly Wortham (US) - Optimizer and Founder @ Test & Learn Community
Ton Wesseling (NL) - Optimizer and Host @ Experimentation Culture Awards
After the submission deadline has passed every jury member will score each case entry on a 5 star rating scale (besides cases of friends, family, collegues and clients). The best scoring cases will become nominated as finalists and will be interviewed in-depth about their award entry. Based upon the extra information from the interview, the jury will select the winner among the nominees in all categories. The winners will be announced during the broadcast of May 23rd, 6 pm UTC - 8.30 pm UTC.Question on this selection process: please use our contact form.
May 23rd 2024 live broadcast
This broadcast has 120+ minutes of keynotes, interviews and winners
The live broadcast starts on May 23rd, 6 pm UTC
ABOUT THIS AWARDS EVENT
This is the 2023 about page.The Experimentation Culture Awards started in 2020 as an event to promote a culture of experimentation globally. Our goal is to help people trying to grow an evidence-based decision-making culture by pushing them in the right direction.We do this by sharing inspirational stories of experimentation maturity growth and nominating individuals, teams and organizations who deserve recognition for their work.In an experimentation culture, you are free to try and fail or succeed, while the direction and the result of work is based on trustworthy gathered evidence. Grow fast, grow forward.
"It's not about that one experiment, it's about improving the process, structure, trustworthiness, democratization and motivation for experimentation and data-driven decision making."
The 2023 jury will be handing out 4 awards:
Rising Star Award 2023: For organizations (teams) that grew from almost no experimentation, to scaling up experimentation quickly.
Team Award 2023: For organizations (teams) that grew from some people applying some CRO / Experimentation to a more solid set-up, and a strong team.
Organization-wide Award 2023: For organizations that grew from a strong experimentation team to experimentation more broadly adopted throughout the organization.
Community Award 2023: For an individual, a team, or an organization who really helps the broader experimentation community move forward.
Our goal is to praise every case study that should be praised for growing experimentation culture by nominating this case or at least giving this case an honorable mention.This year we stopped with the individual category (experimentation is a team sport, was our reasoning, and we created a separate community award category), we added the Rising Star Award as a full category this year.
THE AWARDS JURY OF 2023
This is our 2023 jury. Read more about our awards or our 2022 jury.
Bjarn Brunenberg (NL) - Individual award Winner 2022 @ TomTom
Erica Chang (CH) - Rising star award Winner 2022 @ SMG, now @ Langster
Tom van den Berg (NL) - Community award Winner 2022 @ CRO Weekly
Erin Weigel (US) - Senior Product Design Manager @ Deliveroo
Flavio De Souza (BR) - Team award Winner 2022 @ Vista
Kelly Wortham (US) - Optimizer and Founder @ Test & Learn Community
Ton Wesseling (NL) - Optimizer and Host @ Experimentation Culture Awards
After the submission deadline has passed every jury member will score each case entry on a 5 star rating scale (besides cases of friends, family, collegues and clients). The best scoring cases will become nominated as finalists and will be interviewed in-depth about their award entry. Based upon the extra information from the interview, the jury will select the winner among the nominees in all categories. The winners will be announced during the broadcast of June 15th, 6.00pm UTC - 8.30pm UTC.Question on this selection process: please use our contact form.
June 15th 2023 recording
This broadcast has 120+ minutes of keynotes, interviews and winners
The live broadcast started on June 15th, 6pm UTC
ABOUT THIS AWARDS EVENT
This is the 2022 about page.The Experimentation Culture Awards started in 2020 as an event to globally promote a culture of experimentation. Our goal is to help people who are trying to grow an evidence-based decision making culture, by giving them a push in the right direction.We do this by sharing inspirational stories of experimentation maturity growth and by nominating individuals, teams and organizations who deserve to be recognized for their work.In an experimentation culture you are free to try and fail or succeed, while the direction and the result of work is based on trustworthy gathered evidence. Grow fast, grow forward.
"It's not about that one experiment, it's about improving the process, structure, trustworthiness, democratization and motivation for experimentation and data-driven decision making."
The 2022 the jury will be handing out at least 3 awards:
Individual award 2022: something 1 person has done in the past 12 months to keep or improve the culture of experimentation in an organization or even broader.
Team award 2022: something a team has done in the past 12 months to keep or improve the culture of experimentation in your organization or even broader.
Organization-wide award 2022: something an organization has done in the past 12 months to keep or improve the culture of experimentation in your organization or even broader.
On top of the three main categories we also have a community award to appreciate efforts that support the experimentation industry as a whole. Last but not least we have the rising star award for the individual, team or organization really going from nothing to something in the past 12 months.Both these extra awards will separately be handed out by the jury to entries in one of the three main categories.
THE AWARDS JURY OF 2022
This is our 2022 jury. Read more about our awards or our 2021 jury.
Aleksander Fabijan (US) - Individual award Winner 2021 @ Microsoft
Rommil Santiago (CA) - Community award Winner 2021 @ Experiment Nation
Denise Visser (NL) - Team award Winner 2021 @ bol.com
Nathan Jorgensen (UK) - Rising star award Winner 2021 @ The Royal Mint
Kevin Anderson (NL) - Organization-wide award Winner 2021 @ ING
Kelly Wortham (US) - Optimizer and Founder @ Test & Learn Community
Ton Wesseling (NL) - Optimizer and Host @ Experimentation Culture Awards
After the submission deadline has passed every jury member will score each case entry on a 5 star rating scale (besides cases of friends, family, collegues and clients). The best scoring cases will become nominated as finalists and will be interviewed in-depth about their award entry. Based upon the extra information from the interview, the jury will select the winner among the nominees in all categories. The winners will be announced during the broadcast of July 7th, 6.00pm UTC - 8.30pm UTC.Question on this selection process: please use our contact form.
July 7th 2022 live broadcast
This live broadcast will have 120+ minutes of keynotes, interviews and winners
The live broadcast starts on July 7th, 6pm UTC
ABOUT THIS AWARDS EVENT
This is the 2021 about & jury page.The Experimentation Culture Awards started in 2020 as an event to globally promote a culture of experimentation. Our goal is to help people who are trying to grow an evidence-based decision making culture, by giving them a push in the right direction.We do this by sharing inspirational stories of experimentation maturity growth and by nominating individuals, teams and organizations who deserve to be recognized for their work.In an experimentation culture you are free to try and fail or succeed, while the direction and the result of work is based on trustworthy gathered evidence. Grow fast, grow forward.
"It's not about that one experiment, it's about improving the process, structure, trustworthiness, democratization and motivation for experimentation and data-driven decision making."
In 2021 we are handing out 3 awards (but we do like surprises):
Individual award 2021: something 1 person has done in the past 12 months to keep or improve the culture of experimentation in an organization or even broader.
Team award 2021: something a team has done in the past 12 months to keep or improve the culture of experimentation in your organization or even broader.
Organization-wide award 2021: something an organization has done in the past 12 months to keep or improve the culture of experimentation in your organization or even broader.
THE AWARDS JURY OF 2021
This is our 2021 jury. Read more about our 2022 awards or our 2022 jury.
Guido Jansen (NL) - Winner of the individual award 2020 @ CRO.Cafe
Annemarie Klaassen (NL) - Winner of the team award 2020 @ VodafoneZiggo
Luis Trindade (PT) - Winner of the organization-wide award 2020 @ Farfetch
Bithika Mehra (US) - Optimizer and Blogger @ Experiment Nation
Manuel da Costa (UK) - Optimizer and Founder @ Effective Experiments
Kelly Wortham (US) - Optimizer and Founder @ Test & Learn Community
Ton Wesseling (NL) - Optimizer and Host @ Experimentation Culture Awards
After the submission deadline passed every jury member scored each case entry on a 5 star rating scale (besides cases of friends, family, collegues and clients). The top 5 in each category became nominated as finalists and was interviewed in-depth about their award entry. Based upon the extra information from the interview, the jury will select the winner among the 5 nominees in all 3 categories. The winners will be announced during the broadcast of July 8th, 6.00pm UTC - 8.30pm UTC.Question on this selection process: please use our contact form.
July 8th 2021 live broadcast
This recording has 150 minutes of keynotes, interviews and winners
ABOUT THIS AWARDS EVENT
This is the 2020 about & jury page.The Experimentation Culture Awards were scheduled as part of the new Outperform Conference in Amsterdam, June 2020. This event got postponed, due to the Covid virus.This outbreak changed a lot, it caused a massive shift for many organizations. We also went through digital transformation and moved the awards show to a virtual event.What have you, your team and your company done the past months to maintain a culture of experimentation, or to even grow it to a next level of maturity? This question is exactly the reason why we wanted to move forward with the Awards online!
On the deadline of July 31st 2020 we had received 65 amazing cases on evidence-based maturity growth in organizations. Stories that will inspire others in these strange times. Stories that inspire us all to keep on moving forward maintaining and improving a culture of experimentation.It's not about that one experiment, it's about improving the process, structure, trustworthiness, democratization and motivation for experimentation and data-driven decision making.
Read on for our awards, the jury and the nominee selection process
Individual award: something 1 person has done in the past months to keep or improve the culture of experimentation in an organization or even broader.
Team award: something a team has done in the past months to keep or improve the culture of experimentation in an organization or even broader.
Organization-wide award: something an organization has done in the past months to keep or improve the culture of experimentation in an organization or even broader.
THE AWARDS JURY OF 2020
This is our 2020 jury. Read more about our awards or our 2021 jury.
Casandra Cambell (US) - Testing & Experimentation Lead @ Shopify
Hazjier Pourkhalkhali (NL) - Global Director, Strategy & Value @ Optimizely
Stephen Pavlovich (UK) - CEO & Founder @ Conversion.com
Marianne Stjernvall (SE) - CRO Lead @ COOP Sweden
Lukas Vermeer (NL) - Director of Experimentation @ Booking.com
Kelly Wortham (US) - Senior Director of Optimization @ Search Discovery
Ton Wesseling (NL) - Chairman & Founder @ Outperform Conference
After the submission deadline of July 31st 2020, 6.00pm UTC, every jury member has scored each entry on a 5 star rating scale. The top 5 in each category has been contacted and interviewed about their award entry. Based upon that extra information the jury has selected the winners among the 5 nominees in every category and those winners were announced in the broadcast of September 24th, 6.00pm UTC.The live show is done, but you can still register your free event ticket to get access to the recording with 6 keynotes, 15 interviews, 3 awards and 2 bonus items. Truly inspirational content on experimentation culture.
REGISTER YOUR FREE TICKET
Register your free ticket For the 2024 experimentation culture awards live broadcast. You'll see nominee case introductions, the winner announcements, and keynotes from last year's winners.The live broadcast starts on May 23rd 2024, 6pm UTC.
(Which is Berlin: 8 pm, London: 7 pm, New York 2 pm, Los Angeles 11 am)
During the live broadcast, we will randomly hand out a $250 gift card, on behalf of Convert.com, among attendees actively using the live chat.
REGISTER YOUR FREE TICKET
Register your free ticket For the 2023 experimentation culture awards recording. You'll see nominee pitches, the winner announcements, and keynotes from last year's winners.The live broadcast started on June 15th, 6pm UTC, and the full recording is now available.
(where you can register for free or log in)
2023 AWARD SHOW SCHEDULE
June 15th 2023 - show starts at 6pm UTC
All times in the schedule are UTC times, which means it starts in/at:- Amsterdam / Berlin / Paris: 8.00pm (+2 hours on UTC)
- London / Dublin: 7.00pm (+1 hour on UTC)
- New York / Boston: 2.00pm (-4 hours on UTC)
- Silicon Valley: 11.00am (-7 hours on UTC)
5.50pm UTC: Livestream starts with a 10 minute countdown - you can start to log in
6.00pm UTC: Introduction of the 2023 Awards by jury chair & host of this show Ton Wesseling
6.16pm UTC: Introduction of all 2023 nominees and mentioning the honorable mentions*
6.22pm UTC: Announcing the 2023 Community Award winner
6.24pm UTC: Rising Star Award nominees case introductions & announcing the winner!
6.45pm UTC: Keynote presentation by Bjarn Brunenberg of TomTom - Winner of the Experimentation Culture Individual award in 2022
7.08pm UTC: Team award nominees case introductions & announcing the winner!
7.29pm UTC: Keynote presentation by Sarah Bütof of Vista - Winner of the Experimentation Culture Team award in 2021
7.50pm UTC: Organization award nominees case introductions & announcing the winner!
8.10pm UTC: Closing remarks by Ton Wesseling
8.15pm UTC: End of the live broadcast
(where you can register for free or log in)
All nominations will be introduced during the event by the case owners. Do not forget to register to get access to the show and/or the recording!
GET ACCESS TO THE RECORDING
Because the show has ended you need to login or create an account directly on our stream website to access the recording.The live broadcast started on July 7th, 2022 - 6pm UTC.
This live broadcast had 120+ minutes of keynotes, interviews and winners
2022 AWARD SHOW SCHEDULE
July 7th 2022
All times in the schedule are UTC times, which means it starts in/at:- Amsterdam / Berlin / Paris: 8.00pm (+2 hours on UTC)
- London / Dublin: 7.00pm (+1 hour on UTC)
- New York / Boston: 2.00pm (-4 hours on UTC)
- Silicon Valley: 11.00am (-7 hours on UTC)
5.50pm UTC: Livestream starts with a 10 minute countdown - you can start to log in
6.00pm UTC: Introduction of the 2022 Awards by jury chair & host of this show Ton Wesseling
6.20pm UTC: Individual award nominees case introductions & announcing the winner!
6.40pm UTC: Keynote presentation by Aleksander Fabijan - Winner of the Experimentation Culture Individual award in 2021
7.00pm UTC: Team award nominees case introductions & announcing the winner!
7.20pm UTC: Keynote presentation by Denise Visser from Team Experimentation at bol.com - Winner of the Experimentation Culture Team award in 2021
7.45pm UTC: Organization award nominees case introductions & announcing the winner!
8.05pm UTC: Announcing the winners of the Rising Star Award and the Community Award!
8.10pm UTC: Closing remarks & Ronny Kohavi master class raffle by Ton Wesseling
8.20pm UTC: End of the live broadcast
All nominations will be introduced during the event by the case owners. Do not forget to register to get access to the show and/or the recording!
GET FREE ACCESS TO THE 2021 LIVE AWARD SHOW RECORDING
Register yourself to be able the recording of the 2021 live show. This event has keynotes, interviews with the 15 finalist nominees and announces the winners. You can check the full schedule here.Clicking on the button below here will take you to our livestream platform, where you also can register a new account directly.
If you registered earlier on this site: check your email for login informationEmails are mainly send by info@experimentationcultureawards.com, but login info also comes from no-reply@editmysite.comUse "reset password" on our live stream platform to set a new pass.
2021 AWARD SHOW SCHEDULE
July 8th 2021
All times in the schedule are UTC times, which means it starts in/at:- Moscow: 9.00pm (+3 hours on UTC)
- Amsterdam / Berlin / Paris: 8.00pm (+2 hours on UTC)
- London / Dublin: 7.00pm (+1 hour on UTC)
- New York / Boston: 2.00pm (-4 hours on UTC)
- Silicon Valley: 11.00am (-7 hours on UTC)
5.50pm UTC: Livestream starts with a 10 minute countdown - you can start to log in
6.00pm UTC: Introduction of the 2021 Awards by jury chair & host of this show Ton Wesseling
6.15pm UTC: Individual award nominees case introductions & announcing the winner!
6.45pm UTC: Keynote presentation by VodafoneZiggo - Winner of the Experimentation Culture Team award in 2020
7.15pm UTC: Team award nominees case introductions & announcing the winner!
7.45pm UTC: Keynote presentation by Farfetch - Winner of the Experimentation Culture Organization-wide award in 2020
8.15pm UTC: Organization award nominees case introductions & announcing the winner!
8.45pm UTC: Closing remarks by Ton Wesseling
All nominations will be introduced during the event by the case owners. Do not forget to register to get access to the show and/or the recording!
GET FREE ACCESS TO THE RECORDING OF THE 2020 SHOW
Register yourself to be able to watch the recording of the live broadcast. The events has 6 keynotes, the 15 final nominees and presents the winners. You can check the full schedule here.
The full recording is accessible through our live stream platform
If you registered earlier: check your email for login information(emails are mainly send by info@experimentationcultureawards.com, but login info also comes from no-reply@editmysite.com - use "Reset password")
2020 AWARD SHOW SCHEDULE
September 24th 2020
All times in the schedule are UTC: Moscow +3hrs, Paris +2hrs, London +1hr, New York -4hrs, Austin -5hrs, San Jose -7hrs
5.50pm - 6.00pm: Livestream starts with a 10 minute countdown - you can start to log in
6.00pm - 6.15pm: Introduction of the 2020 Awards by jury chair & host of this show Ton Wesseling
6.15pm - 6.30pm: Keynote presentation by Stephen Pavlovich - Tactics that could help to create a culture of experimentation
6.30pm - 6.45pm: Keynote presentation by **Marianne Stjernvall **- How to build an experimental organization
6.45pm - 7.05pm: Individual award nominees case introductions & announcing the winner!
7.05pm - 7.20pm: Keynote presentation by Hazjier Pourkhalkhali - Who optimizes the optimizers
7.20pm - 7.35pm: Keynote presentation by Casandra Cambell - Stop just optimizing; start deliberate learning instead
** 7.35pm -7.45pm**: Individual lifetime achievement award given by the 2020 jury to Ronny Kohavi
7.45pm - 8.10pm: Team award nominees case introductions & announcing the winner!
8.10pm - 8.20pm: Keynote presentation by Kelly Wortham - Experimentation Culture From individual to industry
8.20pm - 8.30pm: Keynote presentation by Lukas Vermeer - Experimentation Culture at Booking.com
8.30pm - 8.40pm: Organization-wide lifetime achievement award given by the 2020 jury to Booking.com
8.40pm - 9.02pm: Organization award nominees case introductions & announcing the winner!
9.02pm - 9.05pm: Closing by Ton Wesseling
All keynotes are presentations by the Award jury members.
All nominations will be introduced during the event by the case owners. Do not forget to register to get access to the show and/or the recording!
CONTACT FORM
Use the form below to ask us your question(s) about the awards and the judging process or to submit your help as ambassador or business partner. Our goal is to reply within 2 business days.
Emails are send through info@experimentationcultureawards.comYou will be contacted by a representative of the owner of the Experimentation Culture Awards: Help je Klant Holding BV, registered at the Chamber of Commerce in the Netherlands under number 30251897.
CONTACT FORM COMPLETED
Thank you for asking your question. Our goal is to get back to you within 2 business days. We will get in touch with you through info@experimentationculture.com
TRACKING CONSENT OVERVIEW
TICKET REGISTRATION DONE
Our welcome email should arrive in your email box in 2 minutes. It's coming from info@experimentationcultureawards.com.Keep an eye out for our event announcement emails
and the livestream login link
(which we will email a few hours before the event).