Is it ok to compare Velocity of two teams?

I attended an event hosted by Rally software in Boston 2 days back, which I honestly thought was very well organized and well run. One of the breakout sessions I attended in this event had close to 25 participants and one of the comments from that session really caught my attention.

One participant mentioned, "Our management is comparing Velocity of 2 teams, one in US (local) and one in Bangalore (offshore) and based on the velocity, they want to make a decision if the local work can be outsourced to a cheaper location". What made this even more interesting was that at least 30%, if not more, from the audience felt that velocity of the teams can be compared. Some mentioned that if team sizes are the same then comparison should be ok. And then there were others like me voicing our opinion, "Velocity between two teams CANNOT and should NOT be compared".

As far as I recall from Mike Cohn's book, and from my practice, my understanding is that  velocity metric is a very 'team' specific metric. Velocity is a measure of how much work (# of Story Points) a team can accomplish in single iteration, and average velocity of a team is used for predicting how long (how many iterations) this team will take to finish their backlog.

Let's take a hypothetical example. There are 5 stories in the product backlog that both team A and team B are asked to work independently on. For this exercise, also consider that both teams are of same size and have equal expertise. Each team were given the numbers they could use to assign story points, "1,2,3,5,7,13,20”. Each team assigned story points to stories based on their understanding (as shown below). Each team also finished 5 stories in single iteration.

Team A calculated story points as

Story 1	7
Story 2	3
Story 3	5
Story 4	3
Story 5	2
Velocity	20

Team B calculated story points as

Story 1	7
Story 2	7
Story 3	5
Story 4	7
Story 5	3
Velocity	29

If one uses merely Velocity as a comparison metric, it gives a false impression that Team B is able to accomplish more than Team A, but that's not the case as both teams accomplished equal amount of work. If the management team used the above comparison to make a decision, they might choose wrongly.

 Thoughts?

How do you measure software quality?

Let me start with a snippet of conversation from one of the meetings I attended…

One of the managers while discussing a certain topic said, "We should release our product with utmost quality". Another manager responded to this statement by asking this question, "How do you measure quality?" The first manager replied, "My definition of quality is a satisfied customer"….

Interesting, "Quality of a product is satisfied customer" - This statement is so true, but at the same time it's so vague. Can you really measure quality of the product based on customer satisfaction? In the Agile world, the customer might not use the product until enough features have been implemented. And by the time customers start using the product, it might be too late to determine the overall quality of the product.

So the question is, how do we measure the software quality of an Agile project?

In my opinion, there are various metrics that can be collectively used to determine the quality of the code and measure customer satisfaction.

1.  All stories should meet the "Done Done" criteria each iteration. "Done Done" means
• Development complete with Unit Tests.
• QA Done with Automated Integration Tests.
• Documentation Done along with updates to Wiki.
• Each story meets Acceptance Test criteria.
• Ensure there are no critical issues at end of iteration.
2. Set organization wide code coverage metrics that all teams adhere to. Ex. Code Coverage should be 85% or more.
3. Monitor number of defects found by the Quality Assurance team and by the Customers.
4. Monitor regression test results for every fix being delivered - this should be almost negligible if unit tests are implemented correctly.

As listed above, I don't think one can pinpoint to a single metric that will determine quality of the product. However, each team can come with their collective list of metrics to measure quality of the product. And if quality is taken care of, it would be hard to find unsatisfied customers.

How do you measure quality of your product?

Mockups and Simulated Data helps with Agile Development

Mockups and simulated data can be a boon in Agile development. Mockups help eliminate dependencies within teams and help with faster development.

Mockup is a practice of using a dummy or placeholder object temporarily that replicates the functionality of what a real object will eventually achieve. Simulated data is hard coded data that can be read from flat files or returned from interfaces until the actual data is available.

Let me take few scenarios and show how Mockups can help in Agile development.

Scenario 1:

Quality Assurance (QA) engineers in a Scrum Team are looking to automate integration tests for the new feature being developed in iteration. However the new feature will only be complete by the end of the iteration and that leaves very little time for QA to develop their tests. The QA and Dev engineers decide at the planning meeting about defining all mockup interfaces in the first few days of the iteration. This will give a head start for the QA engineers to write their integration tests while the actual implementation is still underway.

This is the example of how Mockup interface might look:

Ref: “Ship It, A Practical Guide to Successful Software Projects”

Bool Login(string username, string password){

return true;

}

QA can start using the above interface to write their integration tests. At some point in the iteration, as decided in the planning meeting, development will actually implement the function and it might look something like this

Bool Login(string username, string password){

Bool login = false;

String realpassword = getpassword(username);

If (realpassword.equal(password)){

login = true;

}

return login;

}

For QA the actual implementation of the above interface by development does not cause any changes to the automated test. Thus mockups can help QA and development to do parallel development in Agile development.

Scenario 2:

Consider 5 global scrum teams, working on different layers of the product, trying to build a common end to end use case in a given iteration. This would normally result in various dependencies between teams trying to achieve the final use case. One way to resolve these dependencies are to make these layers talk to each other. Have these teams get together early in the iteration and pre-define the mockup interfaces just as in the previous example. This will allow each team to work on their piece of functionality independently. In this scenario, if for some reason a scrum team is unable to do actual implementation of the mocked interface, it will not cause major disruptions to other teams.

Scenario 3:

A Product Manager is interested in seeing what the final product might look like. Based on the current velocity, the scrum team is at least 5 iterations away before the final UI is ready. Using Mockups and Simulated data the scrum teams can quickly put a UI together, that would help give the Product Manager a bird's eye view of what the product might look in advance. This will also help the scrum teams get some early feedback and ensure that the use case is aligned with the Product Manager's vision.

Our teams use Mockups and Simulated data with Agile development and we were able to reduce intra-team and inter-team dependencies and do fast track development.

I would be interested to hear other strategies….

Managing user stories in enterprise environment

In this blog, I want to take you through a user story example, the challenges with managing user stories in enterprise environment, and an approach to manage user stories.

Managing user stories in an enterprise environment can be challenging. The challenge comes from the fact that Product Manager, Product Owner and Scrum team have different expectations on how stories should be defined. Let's take a simple example - Product Manager writes a story - "As a Bank of America debit card holder, I would like to withdraw money from the ATM, so that I can make cash payments". From the product manager's perspective, who is a customer facing representative, this is a very well written story. This story is a value added feature that is marketable and billable to customer. From the scrum team's perspective, this is an epic, as it is too big to be scheduled in one iteration. Scrum teams like to see stories that can be estimated correctly and scheduled in one iteration.

The epic, sometimes also called themes that product manager writes, needs to be massaged further by decomposing them into smaller stories. The breakdown has to be granular enough with enough details for the scrum team to start planning their iteration. The scrum team gets together with product manager and interviews him for additional information. Some questions that the team might ask are - How does the user interface look like for your story? Will the customer scan or swipe the card? What information does the customer have to enter for authentication? How should the system respond if the customer enters wrong password the first time? How should the system respond if the customer enters incorrect password 3 times in a row? What is the limit on the amount of cash that can be withdrawn? And so on..

The answers to most of the questions will eventually turn into stories. Some of these stories will be granular enough to be scheduled in an iteration and some might turn out to be epics, that would need further breakdown. The point to be noted is, there is a gap between how product manager defines a story and what the scrum team expects, and this needs to be tied together somehow.

The above example just involves one product manager and one scrum team. Let's kick it up a notch and make things a bit more complicated. Now imagine an enterprise group that is building a suite of products with common infrastructure components. This enterprise  group would normally have multiple product owners, over 15 teams split as feature and component scrum teams , and most likely will be geographically distributed. The challenge here is to manage a common backlog, each scrum team's backlog, ensure all backlogs are in prioritized order, manage dependencies between different scrum teams, and tie epics (common backlog) to granular stories (Scrum team's backlog) and vice versa, etc..

So the question is how should user stories be managed in an enterprise environment?  

First, there should be ONE common prioritized product backlog enumerating all epics. These epics should be value-add features that the customer cares for. This list should also give the vision and direction to the scrum teams to align on common deliverables.

Second, A "Work In Progress"(WIP) limit of no more than 2 to 3 epics at a time should be setup (a concept I learnt from my friend Mike Cottmeyer).WIP enforces that the entire org is focused on first few epics. This also makes certain that teams are not writing code, that other teams aren't ready to consume yet.

Third, epics should be decomposed into granular stories and linked together via excel spreadsheet or tools like Rally or VersionOne for tracking purposes e.g. epic1 -- story1, story2, story3,.. . 

Fourth, the decomposed stories should then be transferred to each scrum team's backlog. It should be ensured that each team's backlog is always in prioritized order with enough details for scrum teams to plan their iteration.

The above steps can be tied together through the "Enterprise Scrum - A tiered approach" and "Enterprise scrum - the tiered approach (implementation)" that I discussed in my earlier blog. Remember, User stories are the heartbeat of agile development. The health of the product depends on how well the user stories are written, groomed and managed.