Product Analytics in Scrum – As a product nerd I am constantly looking for ways to measure success. That success could be at the level of a single meeting up to the success of a product portfolio in the market. In this great article from Scott Middleton, how approached how to use analytics to measure performance of the scrum.
The highest performance products combine quantitative and qualitative information on a regular if not daily basis to inform decisions and make constant changes. The high-performance teams developing these products succeed because they’ve developed disciplines that make product analytics part of their agile rituals.
Many teams say they’re taking a similar approach but on closer inspection they don’t give enough time or attention to their product analytics and are really acting on instinct rather than data.
This article aims to provide you with some guidelines on how you can incorporate product analytics into your agile rituals through the lens of Scrum.
The guidelines are best understood through these areas:
- How to bring product analytics into your Scrum rituals
- Product Analytics Roles and responsibilities in Scrum
- How to know if its working
These guidelines are focused on Scrum in order to make the guidelines more concrete. That being said, you can take these guidelines and transfer them to Kanban, use them with SAFe or match them to your own flavour of agile.
Product Analytics in Scrum rituals
Attention to data comes through incorporating analytics activities into your regular rituals. Fortunately Scrum has some well-established rituals that you can leverage to get more attention and focus on product analytics.
Making analytics part of your routine will make sure it gets attention but then, over time, will make sure that it is just a regular part of how you develop, what you develop and how your team thinks about their work.
Here is how to make product analytics part of your regular discipline broken down by Scrum ritual (in no particular order):
Sprint Backlog Refinement:
Sprint Backlog Refinement is where you prioritise, add detail and do (rough) estimates for the work you’re aware of. Sometimes it is about adding new work.
You can make product analytics part of this ritual by starting your refinement session with a review of your product analytics. Sometimes it will create immediately obvious work to do, sometimes it will create research tasks to dig into. Sometimes it will highlight glaring oversights in what you are tracking (“what do you mean we put that feature live and have no idea if anyone uses it?” isn’t uncommon to be heard).
By starting your backlog refinement with data, your team can now make informed decisions about prioritisation.
Sprint Planning Meeting:
Sprint Planning is where you decide on what you are going to do in the next sprint, workout who will do it and make concrete plans for getting it done.
Analytics have a limited role to play here if you’ve incorporated product analytics into your Sprint Backlog Refinement. The ways product analytics can play a part in sprint planning are:
- Giving you a last minute sanity check on what you’re going to build. (“Does the data say we still need this?”)
- By ensuring that each task you will work on in a sprint includes details on how analytics will be developed and used for the work. (“How will we track the success of this feature/task?”)
The daily stand-up is a short conversation between team members to get on the same page, uncover blockers and help each other progress.
This is an opportunity to quickly review the key daily metrics for your team or product as well as any focus metrics (such as if you’re focusing on a specific user journey). Things can shift overnight. Additionally, the more regularly you’re talking about analytics the more data driven your team will become.
To make this easier, if often helps if someone on the team (product manager, product owner or analytics lead) has reviewed prior to attending and brings any important insights to the team.
If you don’t have daily analytics then you need to ask whether you need daily or whether daily is overkill for your situation. A consideration here is that daily analytics, with today’s technology, is most often an easy-to-have (usually same effort as weekly) and can only lead to better outcomes.
Sprint Review Meeting
The Sprint Review Meeting is where you review your team’s progress in the completed sprint.
Incorporating product analytics into the Sprint Review Meeting involves:
- Reviewing the initial analytics results of any work that was deployed during the sprint (if you are continuously deploying)
- Ensuring completed features include analytics capabilities so you can measure whether they are successful or not.
- (Optional) reviewing the performance of features/tasks completed in the previous sprint. This activity should be performed during Sprint Backlog Refinement but it might be necessary to include it in the Sprint Review Meeting simply to ensure it happens.
Sprint Retrospective Meeting
The Sprint Retrospective Meeting is where you spend time looking at how to improve.