This is the fifth and final part a series of posts on software project management. The series starts with Managing Software Projects Part 1: Tools. Start there to get the whole story.

Project completion is a moving target. The product manager wants to add features (without changing the delivery date, of course) and as you work on something that seemed simple, you realize you didn’t really understand it when you provided an estimate. Once in a while, you get surprised by how easy something turns out to be. And once you finish an early task, you know exactly how long it took. (If you’re doing high-risk/low-confidence tasks early, this transition from low confidence to perfect confidence makes your estimate better and better as the project proceeds.) To know early that you’ll be late, you have to continuously monitor progress.

Monitoring

As we work our Trac tickets, we record the time spent on the tasks. And if we’re bumping up against the estimate, we adjust it. Every night the Trac server runs a job that e-mails me the status of my active projects. It looks something like:

13083: Armstrong
   0.89 weeks work (buffered) will end 2015-06-15, due 2015-06-30 (d:/>)
   https://trac.net/trac/ticket/13083

Performance is in the form (p:tc)
   p is period: d=day, w=week, m=month
   t is trend: \=late, /=early, -=on time
   c is change (since last period): v=worse, >=even, ^=better  

Note that the projected completion is based on a buffered estimate that takes into account confidence. Also, the weeks of work is for the critical path so this isn’t staff time applied to the project; other things are proceeding in parallel and this is time to completion. In this case, we read the performance summary at the end of the line to say that since yesterday we haven’t changed our projection, we’re still ahead of schedule. If since yesterday you’d lost ground, you might see d:/v which says we used a little buffer but we’re still ahead of schedule.

Looking back

Continuous improvement means being better at something the second or third or 47th time you do it than you were during the previous iteration. You can only really accomplish this by looking back and saying, “What went wrong? What went right?” Agile folks call this a “retrospective,” a much more optimistic term than “post mortem!” Again, I have all my ticket data in a database so it’s relatively easy to get the system to tell me what happened. My retrospective report (a jumping off point for a group discussion of the project — or sprint — we just completed) looks something like:

57 tickets processed; 49 (85.96%) have estimates

Per-ticket accuracy
 min: -1  max: 378.6  avg: 15.43  sd: 73.28  median: 0.03
 under: 25  over: 24  exact: 0  u:o ratio: 1.04

Overall accuracy (tickets with estimates):
 est: 685.57  act: 1084.7  overall acc: 0.58

Overall accuracy (with default estimate, 8):
 est: 749.57  act: 1094.78  overall acc: 0.46

Project duration: 28 days.
Ticket open offset from start of project:
   Average: 0.00 (0.00% of dur.),  Weighted: 0.00 (0.00%).

Ticket start offset from start of project:
   Average: 5.3 (19% of dur.),  Weighted: 8.7 (31%).

Let’s go through it one item at a time.

On the first line we see roughly how big the project was. If you see a few hundred tickets, you probably need to think about making smaller, more understandable and manageable groups. Ideally, all of your tickets will have estimates. But small things slip in and sometimes it’s quicker to do a small, well-defined task than to do all the PM overhead. So we strive for a high percentage of tickets with estimates knowing it’ll never be 100%. I have a spreadsheet where I can trend these numbers from project to project so I know if we’re getting lazy and not estimating as much as we used to.

There are many ways to slice and dice estimate accuracy (by code base, by estimator, by ticket type, etc.) but here we just look at it by project. I developed the accuracy metric we live by and find that it provides intuitive insight into accuracy. The formula I use is

(actual time - estimated time) / estimated time

.
This has some useful properties.

• A number less than zero indicates that the task was underestimated (took longer than anticipated); the negative sign has the connotation of “bad.”
• On the other hand, a number greater than zero indicates that the task was overestimated (took less time than expected). While this is good for the delivery date, it’s still “wrong” in the sense that the actual time doesn’t match the estimate.
• The magnitude of the accuracy is how close you were. Frankly, I find -0.1 (it took 10% longer than expected) a “better” estimate than 2 (it took half as long as expected).

The per-ticket accuracy section is the result of computing the accuracy for each ticket then running some statistical functions over the results. Here the median accuracy is pretty close to zero, the ideal. Remember that in CCPM, your goal is to give an estimate that you are as likely to be under as over. If we look at the next line we see that of the 49 tickets estimated they are almost exactly half underestimated and half overestimated. You really can’t do much better than that.

The next section computes the accuracy for the project as a whole (the total time applied vs. the total time originally estimated). Here we see that we ran over (positive accuracy) but it wasn’t horrible (magnitude less than 1).

Next, we consider if we applied our historical average time per ticket to the unestimated tickets, how good was our estimate? Here we get a little closer to zero, which is reassuring.

The next couple of sections relate to how well the project scope was managed and how good the code produced was. In an ideal project all the tickets exist at the start and are scheduled out through the course of the project. But if scope creeps, feature tickets get added during the project. And if the code isn’t very good, tests produce bug tickets late in the project. We compute this offset based on ticket counts (the first number) and hours of work in the tickets (the weighted number). This report illustrates a very successful project with no tickets added after the outset.

We also look at when work on tickets was started relative to the start of the project. In an ideal project, work would be done at at steady pace throughout the project so the median start would be at the half-way point. A project with a long tail of pernicious issues after major development will tend to have that the average start earlier than 50%. A rush of development late in the project will move the median above the mid point. Both scenarios may reflect poor project management and low quality.

Takeaways

So, what can you take away from several weeks of talking about better software project management?

• Use data-based tools; they allow you to analyze your project performance to monitor how you are doing and reflect on how you did.
• Make your tasks small; breaking down big tasks into smaller ones enhances your understanding of scope.
• Schedule high-risk tasks early; that’s the only way you can know early if you’ll be late.
• Bundle uncertainty in a project buffer, not in a pad to every task’s estimate.
• Monitor progress so you are not surprised when you approach your deadline with no chance of meeting it.
• Use experience to fine tune your future estimates and assumptions.

Project management is a bit of an art but a bit of care can make it more predictable.

This is part 4 of a series of posts on software project management. The series starts with Managing Software Projects Part 1: Tools. Start there to get the whole story.

“No battle plan survives contact with the enemy.” — Helmuth von Moltke

The Gantt chart, the work breakdown, and task estimates, especially, are dynamic; they change throughout the life of a project. So, how do you plan for change in a way that allows you to still meet your deadline? Francis Patrick proposes Critical Chain Scheduling and Buffer Management. It’s worth reading the whole paper but I’ll give my take on it here.

What Patrick is trying to do is avoid both Parkinson’s Law (Work expands to fill the time allotted) and Student Syndrome (Everything gets done at the last minute) by pushing the pad — or buffer — to the end of the project.

Task Estimates

When you ask a developer for an estimate, he or she will give you a number but the time it takes to perform a task is really a point on a distribution. You can visualize this as a Weibull distribution like the one below. The area under the curve is 1 unit and you can read the chart as meaning the area to the left of a vertical line is the probability of the task being done by that time on the x axis.

Developers like to meet their deadlines so they are likely to give you a number that they are fairly confident accounts for some things going wrong. Consider the right line on the chart and see that there’s a 90% chance the task would be done in approximately 14 hours. That’s the kind of number you usually plan around. But if you have a bunch of tasks, errors will tend to cancel: some will take more time than expected, some will take less. What if we planned around estimates that we were as likely to be under as over? That is the 50% line in the chart. That’s about 8.5 hours instead of 14, a big improvement.

Project Buffer

When you string together several tasks, using the 50% confidence estimate shortens the schedule considerably. But I mentioned a buffer before. We want to add a little bit of a pad after the estimated completion of the last task and use that padded duration to set our project deadline. We could add a flat 30% or something but since we have a lot of data to work with and a computer is figuring it out for us, we can do better.

• Compute the buffer
  • Find critical path
  • Total estimates
  • Compute weighted confidence
  • Buffer = total estimate * f(weighted confidence)
• Compute project makespan
  • Makespan = estimate + buffer
• Figure out when you’ll finish
  • Project finish = Project start + Makespan

Note that nowhere in this calculation have we assigned due dates for the individual tasks in the project. The tasks are ordered based on the schedule with the intention that the first task should be started now, the next as soon as that’s done, etc. This tends to avoid the procrastination that arises from thinking “This 1-day task isn’t due until next week, so I don’t need to start it now.” If you wait to start and then realize you underestimated, you may not be able to finish on time.

Example

As an example of calculating and using a buffer, let’s look at the project from Part 2.

Patrick has some academic idea of how much buffer you should have. The specifics are hidden behind the f(confidence) in the outline above. I feel it is really specific to your team, experience, business, etc. The function can be linear, exponential, or something more complicated as fits your need.

If we use the binary curve from the figure, we determine the buffer multiplier by going across the x axis to find 2.39, go up to the curve, then go left to find the multiplier on the y axis, in this case around .8. Our buffer is (.8 * 90) = 72, and the project makespan would be (90 + 72) = 162 hours. If you assume your developers get 30 productive hours in each week (allowing for meetings and other distractions), that suggests the UI would be done 5.4 weeks after work on it begins.

Next

In the next post, I’ll discuss monitoring the progress of your project.