Experimentation

Test quality hypotheses, quickly

The only two elements you can affect are: The quality of your hypotheses, and How quickly...

Scale wins across the entire organization

Problem - Unrelated experiments running in PPC, website, social, email

Organizational silos make it impossible to Scale wins across the entire organization, limit your impact, lead...

Problem - It takes a month or longer to ship an experiment

Don't test everything, but you must test. If you knew exactly what to do you would...

Let your customers drive your experimentation roadmap

Experimentation requires a cross-functional team

A successful experimentation program needs design*, engineering, analytics, and strategy. And if you want to Test...

Don't test everything

Don't guess

If you knew exactly what to do you would just do it (and so would your...

Decouple experimentation from product development

Your product development process is optimized for shipping a reliable, polished experience to users. But experimentation...

80% of your ideas won't work

UX-focused experiments will only take you so far

User experience matters, and it's worth addressing friction and confusion on your site, but Cosmetic changes...

Take big swings early on

You should be testing drastic, ambitious changes early in your experimentation program. This type of test...

Problem - You've run lots of inconclusive tests

This is extremely common! Here's why it might be happening. You're focused on UX UXfocused experiments...

Minimum Detectable Effect

Minimum detectable effect (MDE) is a statistical characteristic of each experiment you design. This number indicates...

Explore before you refine

Experiments can focus on testing wildly different experiences (exploration), or in making smaller improvements to an...

Cosmetic changes don't matter

In ecommerce, minor styling changes can have a measurable impact on shopper behavior. But B2B decisions...