Backlog refinement sessions; what is this somewhat-neglected SCRUM activity? Why is it sometimes a complete nightmare?

How do you run a refinement session which covers all the necessary points, answers all the questions, evokes team confidence & ownership, and allows estimations to be applied, effectively and efficiently?

In case some explanation is needed, you can read about what a refinement session is here, and I have decided to take some time to explain in my own weird way:

Vio: I need to eat more fruit HIGH-LEVEL REQUIREMENT

Kat: Why do you need to eat more fruit?

Vio: My doctor says I need more vitamins

Kat: O really?? Any particular kind of vitamins?

Vio: Yeah, he said I lack Vitamin C

Kat: What kind of fruit is best for helping with that?

Vio: Oranges and Strawberries REQUIREMENT BECOMING CLEARER

Kat: Cool! Will you go for a check-up again about it?

Vio: In 2 months!

Kat: So I guess per week you should be eating…

Vio: About 150grams a week; a mix of those two fruits

Kat: So 75g oranges per week and 75g strawberries per week, for the next two months?

Vio: Pretty much! REQUIREMENT CLEAR

Kat: How will you do it?

Vio: I usually take fruit with oatmeal for breakfast, so I will make sure I prioritize those fruits and take just over 10g of each every morning GETTING INTO THE SOLUTION

Kat: Great idea!

-

By the end of the conversation, it is clear that Vio needs to eat 75g oranges per week and 75g strawberries per week for two months to increase her Vitamin C levels before her next check-up, and we also know how she intends to do it. At the beginning of the conversation, all we knew was that she needed to eat more fruit.

A refinement session involves understanding and refining high-level requirements to transform them into clear, verifiable requirements that evoke solution conversation and negotiation (the how). Either in the refinement session or the planning session, estimates of some kind will typically also need to be applied. In this story, I have decided to go with story points for my explanation below.

Imagine that Vio’s fruit challenge would cost her 30 points of ‘Vio-effort’ per month, but she also had to manage another task (maybe she needs to drink more milk to deter a calcium deficiency!) which is going to cost her 10 points of effort. If she tries to manage both of these together in one month, she is committing to 40 points of Vio-effort.

Maybe she is not capable of 40 points of Vio-effort in one month; perhaps she can only manage 30. In this case, she shouldn’t try to do both in the same month- she will fail at one or both of her objectives.

We estimate the effort (which is typically a combination of time and complexity) to complete a backlog item so that we can try to plan a healthy scope for an iteration. Understanding effort and velocity also allow us to work out potential release dates for features and manage stakeholder expectations.

Hopefully, the above fruit and calcium analogy helps explain the basics of a refinement session and effort/velocity. I am sure we all hope Vio gets over this dire vitamin deficiency she is currently enduring.

Moving on to the main point of this story:

Refinement sessions can be tough as often a time & speed pressure is surrounding them. Why is this? If a team has a velocity of 60 story points per iteration, then, as a product owner, I need to ensure that backlog items worth 60 story points are refined and ‘ready’ to go for the team by the time the next iteration begins. If not, and the pipeline isn’t well enough prepared, it can cause under-utilized iterations, additional meetings, and a loss of operational rhythm- which in the world of Agile development is an appreciated stability. So, for me, refinement sessions usually have some target to measure their efficiency and effectiveness- like the number of items we discuss as a team and to what extent we make them ‘ready.’

After a few years of managing this challenge, I have experimented and defined some problems (detracting from refinement awesomeness) I have encountered, and some tested solutions for each:

Problem

There are too many people in the session and an impractical amount of feedback to manage (too many chefs in the kitchen basically, and it ends up taking 1 hour to refine a simple story…)

Solutions

Think about who is in a refinement session- if they aren’t in the core team, question why they are there, and if their value isn’t clear, remove them from the invite.

If it is only the core team in the session and you are still sitting with 10+ people, then the core team may be too large, and this should be considered.

It is also OK to cherry-pick who should be there based on specific session content but be careful with this as it can cause problems with collaboration and team dynamics.

Problem

One format does not rule them all and can lead to unnecessary misunderstandings and confusion.

Solutions

People perceive and understand information differently, so try to use a few different formats to explain things (text, wireframes, designs, flow diagrams, etc.). I would especially encourage this for more complex items.

Problem

Unfamiliarity. Some backlog items may not have ever been looked at before, or at least not in a long time: this means that when such items are present in a session, teams need too much time to process what they are seeing.

Solutions

You can reduce this time by sending out an overview of refinement session content to the team in advance- encouraging them to review the content before the session. You could also structure whatever board you may be using to showcase ‘up next’ items and prompt the team to take a look before each session.

Problem

Items are too large. Items with too much content can be overwhelming to discuss; they can cause avoidable confusion, an overload of questions, or people just phasing out.

Solutions

If you remotely sense this in a session, I would advise refocusing the conversation around how the team thinks the item could be broken down and split.

As a general guideline: if I am ever preparing a backlog item and intuitively think ‘this would be more than X story points,’ I often realize it should be decomposed into smaller items. There will be a number that makes sense- for me, it's usually eight or above, but it really depends on the team, iteration size, etc.

Problem

The team isn’t fully engaged. This could be due to many factors, but one I have recently encountered is covid related; trying to run a refinement session remotely, where everyone can turn off their webcam and go on mute, can be tricky. People can be faced with many distractions at home, and there can be too much silence and not enough feedback- which of course, slows things down.

Solutions

One way to tackle this is to assign items to different team members to own in the session. I have always found it a bit weird that it is commonly left to the PO to facilitate refinement sessions fully. Why not encourage shared ownership and delegate some items to different people? It will keep the team on their toes and also serve the dual purpose of team members becoming familiar with the items they need to own in advance.

Problem

Overengineering and fixing problems before they are encountered. For example, programming a lightbox with a fixed message for one, common error which is often received VS programming a lightbox that shows dynamic messages for each specific error that can be received- of which there are many and most of which are never encountered by the end-user.

Solutions

This is a tricky one without a clear-cut solution. In the above example, there is no doubt that the dynamic solution is the better one- it is smarter, more suitable in the long-term, and could avoid duplicate work if a 2nd error starts to occur more often. However, it isn’t what is needed right now, as, at the moment, there is only one error which is often and which, without having a message in place to show to the customer, is degrading the UX of the product.

Agile thinking encourages the avoidance of technical debt and continuous quality built-in throughout the product cycle; however, it also encourages delivering value fast and avoiding gold-plating.

So, what to do in this situation? Well, for me, it always comes down to urgency and effort. What is the effort of the simple solution VS the dynamic solution? When do we want a solution in place? If the simple solution above takes 3 points, and the dynamic solution takes 8, I would go with the simple solution. If the dynamic solution is 5 points, however, and the team has the time to implement it by the desired due date, I might reconsider the dynamic solution.

Hopefully, the above solutions can help with any refinement session efficiency/effectiveness issues you may face. Good luck!