Our thinking

77 experiments in six months: what cross-market testing taught us

On this page
  1. 01The trouble with one-off tests
  2. 02Build once, run everywhere
  3. 03What the numbers showed
  4. 04Speed comes from shared data
  5. 05What we'd tell a team starting out

Most companies run a handful of A/B tests a year, each one built from scratch for one market. For Samsung, working with Cheil, we ran 77 in six months across seven European markets. The difference wasn't more people. It was how the work was set up.

The trouble with one-off tests

Every market has its own pages, its own languages and its own quirks. When a test is built for one market, moving it to the next means building it again. Ideas wait in a queue, and by the time a winner rolls out everywhere, the season has changed.

Build once, run everywhere

We built an experiment framework: a set of reusable, configurable pieces that work on any page, in any market, on every device and browser. A new test became mostly a matter of choosing pieces and setting them up, not writing code from zero.

Every experiment was reviewed for what could be reused, so nothing was built twice. By the last months, the team was shipping twice as many experiments as at the start.

One way of measuring, shared across every market. Photo: Pawel Czerwinski on Unsplash
One way of measuring, shared across every market. Photo: Pawel Czerwinski on Unsplash

What the numbers showed

  • 77 experiments in six months.
  • Seven markets: Germany, France, Italy, Belgium, the Netherlands, Spain and Poland.
  • Up to €1M in peak revenue uplift, within three months.
  • Nearly 29% beyond projections after six months.

Speed comes from shared data

Fast testing only helps if everyone reads the results the same way. One way of measuring, shared across markets, meant a win in one country could be trusted, and tried in the next, without starting the debate again.

What we'd tell a team starting out

  • Build the framework first, not the first test. The second test is where the time is saved.
  • Agree how you'll measure before the first result comes in.
  • Treat every test as a reusable part. Ask what the next market will need before you build.
  • Make rollouts boring. A winning idea should move to the next market with a setting, not a rebuild.

Experiments are where data turns into decisions. Set up this way, they stop being projects and become a habit.

Tell us what you want to build.

A 30-minute call with our CTO to explore what we could build or automate for your business.