Test quality hypotheses, quickly

The only two elements you can affect are:

  • The quality of your hypotheses, and
  • How quickly you test them and act on the results

Aim to improve these two elements.

To improve hypotheses:

  • Talk to your customersTalk to your customers
    Learn what's unique about your customers:


    If you don't know what motivates your customers, you have to find out
    If you don't understand your customer's buying process, make that you...
  • Remember that Cosmetic changes don't matterCosmetic changes don't matter
    In ecommerce, minor styling changes can have a measurable impact on shopper behavior.

    But B2B decisions are much higher consideration, and this type of change is unlikely to influence your potenti...
  • Don't test everythingDon't test everything

  • Start with the offerStart with the offer
    The Offer, and the way you frame it, is probably the highest-leverage element of your site or funnel.

    Changing the offer itself is one way to Take big swings early on. Once you settle on t...
  • Measure the full sales funnelMeasure the full sales funnel

To improve how quickly you test and act on them:

  • Decouple experimentation from product developmentDecouple experimentation from product development
    Your product development process is optimized for shipping a reliable, polished experience to users.

    But experimentation requires you to Explore before you refine and Take big swings early o...
  • Run only as many variants as the hypothesis requiresRun only as many variants as the hypothesis requires
    If it's possible to test a hypothesis with a single variant, don't test additional variants.

    Running fewer variants lowers your Minimum Detectable Effect and simplifies implementation and QA.
    ...
  • Focus on systems, not hacksFocus on systems, not hacks
    It takes robust processes to Scale wins across the entire organization.