A way to have your team create experiments to run so that you can validate or invalidate risky assumptions that may lead to product failure
Core team, stakeholders (optional)
Lean experiments are useful when your team lacks evidence that a new product or feature will achieve its intended results. Running lean experiments alongside product development helps validate what is truly valuable to your customers/users, without incurring the cost of developing and maintaining production ready software.
During Framing of a D&F, at an Inception or throughout product development. Usually an Assumptions workshop has already been done.
Introduce the team to lean experimentation
Explain what Lean Experiments are, why we do them, and introduce common types of experiments.
“Lean Experiments are based on the Lean Startup approach to creating new products and services under conditions of extreme uncertainty. Lean Experiments are designed to quickly and cheaply gather evidence to validate or invalidate risky assumptions about your product."
Common types of experiments:
A comparison of two versions of a product or feature to see which one performs best. Works best with large sets of users for small incremental optimizations of an experience and business model.
A technique to replace a complex automated technical solution with humans who directly interact with the customer. Helps us validate whether anyone wants our product.
Wizard of Oz Test
A technique to replace the product backend with humans. The customer believes they are interacting with an automated solution. Helps us validate whether anyone wants our product.
Commonly a website that describes the product’s value proposition and asks customers to sign up for the product before it’s available. Helps us validate whether anyone wants our product.
Catalog the team’s “leap-of-faith” assumptions
Add this term to the whiteboard, including the definition so that the team can refer to it throughout the exercise (Assumptions that if we’re wrong about, our product will likely fail).
Ask the team to brainstorm leap-of-faith assumptions on sticky notes for 2 minutes. If a team member comes up with more than two, ask them to self-edit down to those that are most important.
Catalog the product’s unvalidated, leap-of-faith assumptions on the whiteboard.
If you end up with more than a few leap-of-faith assumptions, prioritize one to focus on first. Use the Assumptions practice or consider a 2x2 with “more likely to stop the project” vs. “less likely to stop the project” on the Y-axis and “needs more evidence” vs. “have lots of evidence” on the X-axis. High likelihood of stopping the project assumptions that need more evidence are the target.
Explain that an assumption needs to be converted into a falsifiable hypothesis:
[Specific testable action] will drive [expected measurable outcome]
Introduce the format/template that you will use to track your hypotheses, experiments and measurements.
Describe what each of the following represent:
Distribute a sheet of paper to each team member and instruct them to:
Feel free to copy this Miro board: Validation Research/MVP Experiment on Miro
Prompt everyone to take 5 minutes individually to think of an experiment to test the hypothesis and associated measurable outcomes.
When time is up, have each team member (one by one) stick their sheet up on the wall and share the details of their experiment for group discussion.
Give everyone 5 minutes to share their experiments.
Select an experiment
Explain that the best experiment should require the least amount of effort for the maximum amount of learning. Have the team members silently dot vote (1 or 2 votes depending on group size and # of experiments) on the experiment they think is best to run.
Discuss how to evaluate your experiments
With the experiment selected, talk through how to evaluate it. Typically experiments result in one of three learning outcomes, each with an expected next step:
For any of the outcomes, record the results of the experiment and planned next steps. A good resource to leverage is The Learning Card from Strategyzer.
Plan next steps & action items
Brainstorm and then assign actions for team members to take in order to execute the experiment.
Create an experiment tracker
Keep track of the experiments that were not selected in this session in a backlog. Regularly review these experiments and implement the experiments that are relevant to validate your product direction, especially if your product direction changes.
Success is when the whole team has aligned on a documented Lean Experiment that is ready to be executed on
If there are many assumptions to validate, consider following up with a hypothesis/experiment tracker, as suggested in the last step above. This can help the team visualize what hypotheses should be tested in what order, and their associated experiments.