What works and what doesn't work in software development? For the first 10 years or so of my career, we followed a strict waterfall development model. Then in 2008 we started switching to an agile development model. In 2011 we added DevOps principles and practices. Our product team has also become increasingly global. This blog is about our successes and failures, and what we've learned along the way.

The postings on this site are my own and don't necessarily represent IBM's positions, strategies or opinions. The comments may not represent my opinions, and certainly do not represent IBM in any way.

Wednesday, April 7, 2010

Struggling with the Product Backlog

One area that our team consistently struggles with is the product backlog. This is not a new problem; even before we adopted agile development processes we had problems with this.

Our legacy system for tracking customer requirements is a Notes database. Notes does some things well, but this particular database is often a black hole. Here are my complaints with it:
  1. The search functionality is terrible. We have literally hundreds of requirements, so a search feature is really important. I can't get a full text search to work at all; maybe it's not enabled.
  2. It doesn't provide an easy way to prioritize requirements to get the most important to bubble up to the top.
  3. Almost anyone who works with customers can open requirements. Some people are not particularly good at writing up requirements. So, they may think that they've done their job by opening a requirement, but the product leadership has no idea what they're really asking for. Some people also don't fill in contact information for the customer requesting the feature, so it's difficult to go back to the source for more details.
  4. There's no good way to find very similar or overlapping requirements. We might have five separate requirements for the same thing, and no way of seeing that.
  5. It's not managed well. Nobody goes through the painful process of looking at all of the requirements to see which ones no longer apply, or which ones have become more important since they were opened.
We have started using Rational Team Concert (RTC or Jazz) for managing requirements. However, we still have this legacy database to deal with, and I'm pretty sure no one has the time or inclination to copy everything from Notes into RTC. Besides, would that even be a good idea? RTC at least has a good search function, and it's easy to prioritize things. But RTC doesn't fix problems 3 through 5.

So, what happens when the product backlog is poorly managed? We start planning a release and ask the product owner what should go into it. Some items are obvious and get added added to the release backlog right away. Then the blank stares begin. How can you even tackle the task of looking through the rest of the requirements to find the best and most important ones?

We end up developing whatever the product leadership has been hearing about the most in the past few months. Sometimes this is good enough, sometimes not. For example, if the support team is consistently hearing the same complaints, but not bringing them forward to product management, those complaints might not be addressed.

Something like an idea cloud would be nice. It could show us the keywords that are coming up most often in the requirements. That would help us find common themes, and overlapping requirements. But I have no idea how we could get that working with Notes or RTC.

Getting people to categorize their requirements when they open them could help too. At least that would help us group similar requirements together. RTC lets you enter keywords on stories, but that's freeform, so people may use different keywords for the same idea.

I would love to get some advice on how to improve this situation. Especially problems 3, 4, and 5.