Mitigating the Risk of Story Point Drift > Business Analyst Community & Resources | Modern Analyst > Modern Analyst Blog - A Collaborative Business Analyst Blog

Mitigating the Risk of Story Point Drift

In many Agile projects requirements are not typically written in the form of a formal requirements document. Instead, a collection of concise but effective means of describing what must be built called user stories are often used. User stories describe the behaviour, performance, or interface of a system from a customer’s perspective. A typical user story might look something like this:

As a potential customer I want to be able to view books based on the search criteria I entered.

User stories are not only effective requirements management artefacts, they are also essential to estimate the scope/size of the project and to track the progress of the team. When determining the size of the project, teams estimate the level of effort required to complete each user story and then aggregate their results to come up with their estimate for the scope of the project (for more details on how to estimate level of effort in Agile projects, see Mike Cohn’s excellent book on the subject).

Agile teams will often use a unit-free type of measure called story points. The advantage of using story points is that their inherent value is relative. Instead of trying to come up with absolute values typically related to time (for instance, how many days will it take to complete feature X), story points are only concerned about the relative effort or complexity of a story compared to the other stories that have been or need to be completed. When the total number of story points for the project is combined with a team’s velocity (the number of story points completed per iteration period) project stakeholders are able to get an increasingly accurate picture of the size of the project and the time it will take to complete given the current team.

Initially when the team forms they will estimate story points for some number of baseline stories. For instance, the team may have the following 3 stories as their baseline:

As a user I want to browse the book collection by subject: 8 points

As a user I want to be able to save my payment information: 2 points

As a user I want to be able to recommend my favourite books to people: 5 points

From this baseline other user stories are estimated in terms of their perceived relative amount of effort to accomplish. For larger projects, most user stories are estimated as epics, larger user stories that will be broken down later on in order to actually tackle development.

Over time, new user stories will be added to the product backlog, others removed and some will be altered to reflect changing requirements. All the stories that are added or changed to the backlog will require story point estimation.

One of the potential risks that exists when using story points is what I call ‘story point drift’. Story point drift is when user stories that had a given story point value at the beginning of the project require substantially more or less effort than stories with the same story point value later on in the project for the given amount of effort required to complete a story.

For instance, let’s say I have the following user stories estimated later in the project:

As a user I want to be able to have my bank account linked to my login and set up a monthly withdrawal plan: 8 points

As a user I want to have a theme editor that can customize the look and feel of my online affiliate store: 5 points

While the above user stories may accurately represent their relative size compared to each other, in comparison to the stories that were estimated at the onset of the project it appears as though the latter group’s story points under-represent their complexity and level of effort required. I would suspect that the amount of work to enable automatic bank transactions is substantially larger than the amount of effort required to build a browse by subject feature.

I find the risk of story point drift increases on larger or longer-term Agile projects. Story point drift can occur for several reasons:

The team’s collective memory is short term: when a team begins estimating new stories, they typically draw upon their experiences from recently completed stories. If some of those stories were misclassified (either required materially more or less relative effort than was represented in the story point estimate), then the team can end up believing that these recent stories are the new norm for the value of the story points and use these as references going forward, which skew the story point values from the original baseline references.

The team’s complement has changed: it is not uncommon for project teams to change over time. I’ve noticed that even as teams get larger or smaller that a team’s velocity can appear to remain constant. When investigating this, I’ve found that it’s often because the team begins to estimate stories with fewer points, since they now feel that the story is ‘easier’ since there are more people to work on the project. As a result stories that might have been considered 8 points previously are now being estimated as 5 points. As a result, despite the fact that the team is likely getting more done given their drift in estimates the team’s velocity appears unchanged.

Baselines are not referred to: Agile projects usually thrive by the minimization of overhead that does not lead to customer value and by adapting to non-ideal situations. For example, Agile teams rarely wait for everyone to show up for a meeting – the meeting is time-boxed and starts and ends on time regardless of who is missing. However, at times Agile teams may forget to bring physical objects like their reference stories to meetings. Sometimes teams will try and get through the meeting based on recollection rather than actual reference. Without the actual reference stories and point values in hand, the resulting estimates can be somewhat skewed.

When I’ve seen story point drift it has occurred in small increments over time – you don’t realize that there’s been a dramatic deviation in the estimates until sometimes months later. Story point drift can cause issues with resource planning and progress and time to completion estimation. Here are a couple of strategies I’ve used to help mitigate story point drift:

Bring mixture of original and recent stories as the baseline for new estimation: it never hurts to hold up your original estimates when doing relative comparisons. Having some recent examples are also helpful, in particular since your initial estimates were likely only for a few potential story point values. Having 1 or 2 stories for each possible story point value can also be helpful when discussing what level of estimate to assign to a new story, or when individuals are deadlocked in a session of planning poker.

Know when certain stories should be re-estimated after implementation: once in awhile you will encounter a story that was a lot more (or less) effort than you initially thought. If the level of effort was sufficiently different from another story that was given the same amount of points, you may want to re-estimate the story so that it does not affect your team’s perception of the value of a certain number of story points. I typically would only re-estimate a story once a project has been going for several Sprints – early on you may discover that many of your stories take less or more effort than you thought and you’ll be tempted to resize those stories. However, as long as the stories are taking about the same amount of relative effort, then there’s no need to re-estimate. So if you thought your 3 point stories would take a day to complete but they’re taking the better part of a week, check to see how your 5 and 8 point stories are taking. If those are also taking a fair bit longer than anticipated, then don’t re-estimate.

Spend a little time every few Sprints and analyze relative stories: the ScrumMaster or Project Manager can review some of the completed stories over time to look for possible story point drift. If some drift is detected, bring it up with the team and see what they think. If the team agrees, then re-estimate the stories that appear to be out of what in terms of relative effort.

User stories and story points can be a very good way to manage the requirements and estimation for a project. Keeping an eye on story point drift will ensure that the team has a good handle on the project’s rate of progress and estimated time to completion. With diligence this method of project estimation can be a very accurate and effective tool for Agile teams.

-------

Jarett Hailes
Larimar Consulting Inc.
http://www.larimarconsulting.com

posted @ Sunday, January 17, 2010 8:42 PM by Jarett Hailes

Join the Premier Community for
Business Analysts, Business Process Professionals,
Systems Analysts, Data Analysts, and more…

The Modern Analyst Blog for Business Analysts