As explained in Part 1, an effective Product Design Scrum setup (see the diagram below) involves time-boxing the work that the team will do in exploring key research questions. Recall that the Product Design Scrum team consists of a mix of Data Scientists and Software Developers, DevOps and QA, along with a Technical Product Owner, (focused on creating the detailed User Stories), and a Product Manager (focused on client needs and feedback, market analysis and the long-term product roadmap). And yes, there is also a Scrum Master…
So how to decide on what the key research questions are? This is mainly driven by product management who are trying to figure out:
These questions are broken down and then prioritized with input from the Scrum team so that the riskiest questions are at the top of the backlog. Note that this is different from prioritizing in the backlog the key features of an MVP. It is these top User Stories that make it into the Sprint and the team’s goal is to reduce the risk of each Story by the end of the Sprint. Now, some readers may be familiar with this approach as it is known as Riskiest Assumption Test, or RAT for short.
To quote Rik Higham’s excellent article,
A Riskiest Assumption Test is explicit. There is no need to build more than what’s required to test your largest unknown. No expectation of perfect code or design. No danger it will prematurely become a product.
RAT allows quick exploration of product ideas to determine if they are viable or not. Determining viability means answering questions such as: Is the key data available? Is the data of high quality? Can Data Science models be built with (relatively) high accuracy? Do these Data Science models provide useful, actionable insights? During this phase it is much more important to reduce these risks than to have clean, reusable code.
During the Sprint Review it is key to examine the User Stories that have been finished during the Sprint and determine if their risk has been substantially reduced. If after a series of Sprints the biggest risks have been reduced, and the team feels comfortable enough with the remaining unknowns, then a decision can be made to move into the Product Build Phase and start developing the MVP. Importantly, don’t assume that any of the work from the Product Design phase can necessarily be reused in the Product Build Phase (although you shouldn’t preclude this from happening as well). It is also possible that after a number of Sprints you cannot reduce the risk of the key User Stories and a decision is made to kill the potential product. This is RAT at its best.
Do you use RAT in your company? How do you manage key research risks? We would love to hear from you.