Losing tests should save you money

It's easy to get caught up in test "wins" (because they're exciting). But associating the value...

Run only as many variants as the hypothesis requires

If it's possible to test a hypothesis with a single variant, don't test additional variants. Running...

Test quality hypotheses, quickly

The only two elements you can affect are: The quality of your hypotheses, and How quickly...

Talk to your customers

Learn what's unique about your customers: If you don't know what motivates your customers, you have...

Scale wins across the entire organization

Problem - You've conducted research but haven't acted on it

Problem - You don't have access to all customer data and sales KPIs

Without the necessary customer data, you'll struggle to Learn what's unique about your customers and Let...

Problem - Unrelated experiments running in PPC, website, social, email

Organizational silos make it impossible to Scale wins across the entire organization, limit your impact, lead...

Problem - It takes a month or longer to ship an experiment

Don't test everything, but you must test. If you knew exactly what to do you would...

Measure the full sales funnel

Let your customers drive your experimentation roadmap

If you knew exactly what to do you would just do it (and so would your competitors)

If you don't understand your customer's buying process, make that your priority

If you don't know what motivates your customers, you have to find out

Focus on systems, not hacks

It takes robust processes to Scale wins across the entire organization.

Experimentation requires a cross-functional team

A successful experimentation program needs design*, engineering, analytics, and strategy. And if you want to Test...

Don't test everything

Don't guess

If you knew exactly what to do you would just do it (and so would your...

Decouple experimentation from product development

Your product development process is optimized for shipping a reliable, polished experience to users. But experimentation...

Customer research must lead to action

Assume your customers will research your product online

There's a nontrivial audience of visitors coming to your site that are further down the funnel,...

80% of your ideas won't work

Use cookies, not JavaScript, to set custom User IDs

If you're going to create a custom user id to identify visitors then we recommend setting...

UX-focused experiments will only take you so far

User experience matters, and it's worth addressing friction and confusion on your site, but Cosmetic changes...

Take big swings early on

You should be testing drastic, ambitious changes early in your experimentation program. This type of test...

Start with the offer

The Offer, and the way you frame it, is probably the highestleverage element of your site...

Problem - You've run lots of inconclusive tests

This is extremely common! Here's why it might be happening. You're focused on UX UXfocused experiments...

Problem - You can't recruit qualified test subjects for user research

There are a few (nonmutually exclusive) possible reasons: Your standards are too high Anyone who can...

Offer

The Offer is the thing you propose to exchange with visitors to your site typically a...

Offer framing

The copy, images, and information that help a visitor understand the Offer.

Minimum Detectable Effect

Minimum detectable effect (MDE) is a statistical characteristic of each experiment you design. This number indicates...

Mechanism of exchange

The steps by which a visitor obtains the Offer. Usually a form.

Learn what's unique about your customers

Improvements based on best practices and copying competitors will quickly plateau. Cosmetic changes don't matter. Understanding...

Explore before you refine

Experiments can focus on testing wildly different experiences (exploration), or in making smaller improvements to an...

Cosmetic changes don't matter

In ecommerce, minor styling changes can have a measurable impact on shopper behavior. But B2B decisions...