RSS

Tag Archives: Software project scheduling

Managing Software Projects Part 3: Scheduling

This is part 3 of a series of posts on software project management. The series starts with Managing Software Projects Part 1: Tools. Start there to get the whole story.


When you are heading out to run errands, it’s not too hard to figure out in what order you will go to the cleaners and the gas station and the grocery store but such intuitive scheduling doesn’t scale well. With dozens of tasks being done by a handful of people, you really want a computer to do the heavy lifting.

As I started working on automatic scheduling of Trac tickets, I did a bit of research and found that this is what academics call the Resource Constrained Project Scheduling Problem. A resource can be anything from a developer’s time to the capacity of a roadway. Your project schedule can’t exceed the constraints imposed by the availability of the resource. Your developer can’t work 25 hours a day and a one-lane dirt road can only accommodate so many cement trucks at a time.

Digging deeper, I found that academics have reference projects and their research is into how to efficiently find the best (usually shortest) schedule for a fixed set of activities and resources. In the end, they are talking about squeezing a few percent out of the time it takes to complete a project. Software project scheduling — at least as I practice it — is very different. The set of tasks is dynamic and the margin of error in the estimates is often 100%. Clearly rigorous academic thinking was not what I was looking for.

My research did turn up a useful technique, however. Several areas of research revolved around a Serial Schedule Generation Scheme. At it’s simplest, SSGS is straightforward and easy to implement; I’ve done it twice in Python and once in Perl. The basic idea is:

  • Find eligible tasks
    • No incomplete predecessors
    • Resource is available
  • Sort eligible tasks
    • Bugs before enhancements
    • High risk before low
    • Fragile code base before stable
  • Schedule one task
  • Find more eligible
  • Repeat at Sort

Like many templates, this one gets its power and flexibility from being able to customize steps.

The scheduler in TracPM gets resource availability from an implementation of IResourceCalendar.

class IResourceCalendar(Interface):
    # Return the number of hours available for the resource
    # on the specified date.
    def hoursAvailable(self, date, resource = None):
        """Called to see how many hours are available on date"""

The IResourceCalendar implementation in Team Calendar returns 0 or 8 based on whether the developer is in or out that day but more flexible calendars could return, for example, 4 hours per day for a developer who is only available half time.

The scheduler sorts tasks with an implementation of ITaskSorter:

class ITaskSorter(Interface):
    # Process task list to precompute keys or otherwise make
    # compareTasks() more efficient.
    def prepareTasks(self, ticketsByID):
        """Called to prepare tasks for sorting."""

    # Provide a compare function for sorting tasks.
    # May be used as cmp argument for sorted(), and list.sort().
    # Returns -1 if t1 < t2.
    def compareTasks(self, t1, t2):
        """Called to compare two tasks"""

TracPM includes a simple sorter but I use a small, in-house plugin that expresses our business rules.

Critical Path

The really interesting work in a project occurs on the critical path. The critical path is interesting because any delay in completing a task on the CP delays the completion of the project. That explanation doesn’t lead to a natural or obvious means of finding those tasks but I did turn up a brute force algorithm that works quite well for dozens or hundreds of tasks:

  • Starting now, compute the ASAP schedule
  • Starting at the end of the ASAP schedule, compute ALAP schedule
  • Find tasks that have the same start and finish time in both schedules

Next Time

In my next post, I’ll explore using the critical path tasks and task estimates to estimate project completion.

 
1 Comment

Posted by on September 14, 2015 in Project Management

 

Tags: , ,

Managing Software Projects Part 2: Techniques

This is part 2 of a series of posts on software project management. The series starts with Managing Software Projects Part 1: Tools. Start there to get the whole story.


Jeff Atwood has noted, “On our project we’re always 90% done.” That phenomenon arises from not having a checklist of remaining work (or from having items on your list that are too big to really track).

Create Small Tasks

Smaller tasks allow for better estimates and being 100% done on 9 of 10 tasks is more accurate than being 90% done on one task. Consider developing a new app. Early in the project, you might have an item like:

Task Estimate
Develop UI 80 hours

And that’s fine for an initial plan, but it’s too big and too vague to implement. So, thinking about it more, you might get to:

Task Estimate
1.0 Develop UI 90 hours
  1.1 Develop API 30 hours
  1.2 Develop Icons 10 hours
  1.3 Develop CLI 15 hours
  1.4 Develop Web UI 15 hours
  1.5 Develop App 20 hours

I’d argue that the 30-hour and 20-hour items need to be broken down more but look what else has happened: by itemizing parts you added 10 hours to the estimate for the UI. You understand it better and you have a more accurate picture of the scope. If you did this immediately after project approval, you’d know early that you had estimated low.

Do High-Risk Tasks ASAP

The risk of delivering all the intended features or meeting your planned schedule is highest at project inception. If you are doing something novel — developing a new algorithm, working with a new widget set — you don’t yet know if it’ll take a day, a week, or a month. That risk can be reflected in assigning a low confidence to the estimate for the novel task.

You might also consider that a task that has not yet been broken down into bite-sized subtasks is risky because you don’t know what you’ll discover as you refine the task. Or you may know that one member of your team is always optimistic. Whatever the source of risk, you want to drive it out of your project as quickly as possible by scheduling high-risk tasks as early as possible. Once they are done, you have a much higher confidence in the remaining, lower-risk plan.

Task Estimate Confidence
1.0 Develop UI 90 hours
  1.1 Develop API 30 hours Medium
  1.2 Develop Icons 10 hours High
  1.3 Develop CLI 15 hours Very High
  1.4 Develop Web UI 15 hours High
  1.5 Develop App 20 hours Low

Confidence Levels

While “high” may be attractive for being intuitive, it’s hard to manipulate. How can we sort the low-confidence tasks before high-confidence ones? What is the overall confidence in the estimate for the UI task? Since we’re storing our project tasks in a database and manipulating them with code, we can assign numeric values behind the scenes.

Confidence Value
Low 1
Medium 2
High 3
Very High 4

It is no accident that there are an even number of choices. When given an odd number of choices, people tend to pick the middle value. By not having a middle choice, users are nudged to think a little harder about how confident they are in their estimate.

For completeness, my system has two more values, one on each end.

  • A task may have no confidence selected and we use 0 to indicate “very low.”
  • A task may be complete, in which case we use 5 to indicate we have perfect confidence that the actual hours worked is how long it took.

Using Confidence Levels

Task Estimate Confidence Value
1.0 Develop UI 90 hours
  1.1 Develop API 30 hours Medium 2
  1.2 Develop Icons 10 hours High 3
  1.3 Develop CLI 15 hours Very High 4
  1.4 Develop Web UI 15 hours High 3
  1.5 Develop App 20 hours Low 1

By taking the average of the confidence values weighted by the number of hours in each task, we can answer the question from above: how much confidence do we have in that 90-hour estimate?

((30*2) + (10*3) + (15*4) + (15*3) + (20*1)) / 90 = 2.39

Looking up 2.39 in the table of values, we can say we have Medium-High confidence in that estimate.

Next Time

In my next post, I’ll talk about scheduling tasks based on your business rules.

 
3 Comments

Posted by on September 8, 2015 in Project Management

 

Tags: , , ,

Managing Software Projects Part 1: Tools

Recently, I had the pleasure of presenting at Sharatoga Tech Talks on “How to deliver software on time (or know early that you’ll be late).” I’ve posted the slides but they lack a lot of narrative so in my next few posts I’ll paraphrase a lot of what I said that night.

Software projects have a reputation for running late, perhaps second only to government projects. (A government software project? Fuggettaboutit!) But if something as complex as the World Trade Center can be completed on schedule, what is my excuse? I’ve never worked on a software system as complex as that must have been! (The construction of One World Trade Center finished two years after the original plan but the start of the project was delayed by two years so once they got going, the project took the ten years they had originally projected. To me that is on schedule.)

I’m not a PMP but I do have three decades’ experience developing software and a reputation at my company for delivering projects on time (or, as I hinted in my presentation title, knowing early when I’ll be late so management can plan around the delay). What I’m saying here isn’t project management gospel, just things that work for me. I hope you see some benefit in them.

Tools

I’m not fan of Microsoft Project — though admittedly, that’s what the big boys use on projects like the World Trade Center — but it does provide tools that are handy whatever software you are using to manage your project.

  • Work Breakdown Structure organizes a project into a hierarchy of progressively smaller subtasks until the lowest level is something you can actually understand and complete
  • Gantt Charts show the relationships between tasks, what has to be done first, what that task completion enables, and so on
  • Project Calendars track the availability of resources (e.g., when your developers are on vacation)

All the cool kids are using Agile methods to develop software and that paradigm introduces other tools to help track projects.

  • Workload charts show how work is divided among team members
  • Burndown charts show how work is progressing

And I find it very helpful to have access to my project data in a database for ad hoc queries and reports.

Open Source Tools

I manage my projects in a souped-up installation of Trac. The basic system provides a ticketing system and a wiki but it has a very flexible and capable plugin architecture that allows you to customize it.

Trac-Hacks is a public repository of plugins including:

  • Timing & Estimation provides fields to record estimates and work
  • Estimation Tools provides workload and burndown charts
  • Master Tickets supports Finish-Start dependencies
  • Subtickets supports Work Breakdown Structure
  • Team Calendar allows team members to indicate when they’ll be out
  • TracJSGantt provides an incredible Gantt chart based only on <div> tags and CSS rules

The TracJSGantt plugin has grown to include support for more project management functions like scheduling in a TracPM subsystem.

Trac Requirements

Trac is written in Python and runs on Linux, OS X, or Windows. It requires an RDBMS to store its data. You can use the SQLite bundled with the distribution but that isn’t really suitable for more than a couple of users. You can use MySQL or PostgreSQL as you prefer. I work on Linux with PostgreSQL and while I try to be platform-agnostic, no doubt some OS- or DB-specific code has crept into my scripts.

Next time

In part 2, I’ll talk about some project management techniques.

 
5 Comments

Posted by on August 31, 2015 in Project Management

 

Tags: , , ,