The Life of a Project - Issue Tracking: Day 3

Read this first, please

This is the third in a series of articles that hopefully will be educational to novices and helpful to more advanced developers alike.  I hope to learn a lot from doing this also.  I am taking the approach of maintaining a journal of the day-to-day process of software development starting from a concept and going to a product.  I hope you enjoy this series of articles.

• Day 1 can be found here.
• Day 2 can be found here.

Note from the author

Just to make it perfectly clear, I want to let everyone know that the days of this project are not necessarily literal, physical days.  For the most part, the days in this project are just logical starting and stopping points.  I am sure you know this already, but at least it is clear now.

Where are we?

We have been handed a project to create an issue tracking system (no longer just an application) targeted at software developers, standalone programmers to be exact.  We have gathered a list of requirements and we even have a vision to drive our project.  Yesterday we analyzed our requirements, did a preliminary risk assessment and did some research to try and reduce the risks we uncovered.  We now have enough information to create a technical specification for our project.

We want to be certain that at each stage of the project we have our risks well understood and under control.  We spent a large part of our time yesterday exploring our known risks, but there will most certainly be more we uncover.  We know from all the available materials as well as personal experience, it is much better to uncover new requirements or flaws in any part of our project (from concept to spec to design to code) before the coding begins and preferably at the earliest possible moment.  The cost of change is extremely high, so we must be alert and take steps to uncover hidden features, requirements and risks.  

It is still far too early to jump into the coding, but prototyping some data input screens, configuration tools and other parts of the system is definitely getting closer to reality.  Before that, though, lets create a big-picture specification for our project as a whole.

Coming to grips with reality

We have landed ourselves a doosey of a project.  Exactly how did we go from a simple(?) monolithic issue tracking application to a plethora of largely independent components?  What the heck have we gotten ourselves into?  There are going to be quite a few independent applications to write and we will most likely have applications written in several languages using several toolkits.  To top it all off, there are quite a few new technologies to master.  

By day 3 of this project, it's clear this isn't going to be a cake-walk.  This is going to take time and effort, effort we don't always like to put forth.  We must deal with this reality and move on.  Here are some of the realities we have learned so far:

• The reality is that when this project was first handed to us we envisioned (assumed) something quite similar to all the other issue tracking systems we have ever seen or used.  We thought (assumed) "this going to be a cake-walk.', right up until we talked to our users.  Damn, pesky users!  
• The reality is that what the user envisioned and what we envisioned were very different things.  
• The reality is that most applications are more complex than our initial estimates of them.  
• The reality is that requirements gathering, risk assessment, specifications and designs are critical to every project.
• The reality is that many developers (myself included) become complacent and comfortable with whatever tools or libraries we use on a regular basis and try to fit every project that comes our way into that comfortable and convenient toolkit.
• The reality is that if we always stay in our nice comfortable little caves (oops, cubicles), eventually someone is going to drop a rock (oops, pink slip) over the entrance and then we're screwed!  
• The reality is that this is an ever changing business with new technologies coming along all the time.
• The reality is that sometimes new isn't better.
• The reality is that sometimes we want to use the new "best" tool without really measuring it up to our requirements.
• The reality is that often, we think all this pre-coding stuff is boring and just skip it all together.
• The reality is that pre-coding can be every bit as fun and challenging as the actual coding will be.

Deliverables

OK, our issue tracking system has many separate sub-projects.  I hesitate to call them components to avoid confusion with a later stage of development when we actually define "codeable" components.  For now, lets think of them as deliverables.  Lets list them here so we can individually dissect them later.

On the client side

• Management application
• Data entry application
• Dynamic form designer
• Journaling service
• Task bar tray-icon application
• Developer studio add-ins (for VC 6 and 7)
• MS Outlook add-in
• MS IE add-in
• Windows Explorer namespace extension

On the server side

• EMail server
• EMail reader agent
• State management and persistence server

Other stuff

• Installation routines
• End-user documentation
• Extension API documentation
• Code documentation

Have we forgotten anything?  Probably.  Lets just hope we find any missing deliverables before we start coding.  NO.  Lets not hope, lets make sure we find them before we start coding.  Lets be sure we cover all our bases and review and revise as needed.  At this point, we are only dealing with paper.  Once our ideas our written in stone (uh, code), things get trickier.

Deliverables review

Client side

We've talked allot about this, but where did "Dynamic form designer" come from?  Isn't this just going to be integrated into the data entry application?  Well, after thinking about it, I have some concerns about integrating the form designer this way.  First, the form designer (and rendering engine) will need to be assessable in the management application also.  Obviously, it needs to be in a library by itself that is loaded by any application that needs it.  This causes a potential problem, though because including the designer and rendering engine together will cause that library to be much larger and we want our data entry application to be as light-weight as possible.  So, we are going to create a separate form designer application and then launch this from our data entry application (and anywhere else) as needed.

Everything else on this list has been reviewed already yesterday or appears to be self explanatory, so I don't think we need to spend much time discussing them here.

Server side

So far in our discussions, we have focused on the client-side applications.  We have done this mostly because these are the components that the users sees and really cares about.  The server side is every bit as important, but less visible (assuming it works right.)  The only time users care about server-side software is when it doesn't work.  

We have discussed the email server and reader agent, but we haven't even touched upon the state and persistence server (SPS, we developers love acronyms don't we?).  What the heck is this?  We haven't discussed in any detail how we are going to get data back and forth from the database.  We talked about run-time customizable data structures, but we didn't discuss how we were going to use these in any detail.  Well, the SPS is key to how this is going to work.  We need a way to represent our issues in memory (when they are being reviewed) and we need a way to get them into and out of our database.  The SPS is going to expose a TCP/IP based interface for requesting information from the database and saving it back.  I haven't decided whether to use SOAP or some proprietary language for this communication, but I am leaning toward using SOAP.

The SPS needs to provide services such as:

1. Request a single issue
2. Request a set of issues
3. Persist an issue
4. Delete an issue
5. Delete a set of issues
6. Modify a set of issues
7. Request a form design document
8. Persist a form design document
9. Request an issue design document
10. Persist an issue design document

It kind of looks like the SPS is shaping up to be a web service, doesn't it?

Other stuff

What the heck!!?  Why does documentation show up 3 times in this list!?  Aren't the code comments enough?  (Oops, what code comments?) 

Items on this list are the things developers rarely think about when making plans for projects.  We like to think about the code and occasionally design, but these things typically fall off the list right up until someone asks us to put together a distribution CD, or someone asks us the same question the 1000th time.  We don't want to make that mistake here.  This is a serious project and when we do create a schedule, we want it to be a reasonable one.  Lets not make the same tired old mistakes again.

User workflow examples

Hey!  Where's that specification you keep promising!  I'm sure that's what you're thinking.  But in-fact, that is exactly what I am writing.  This is not what you may think of as a typical specification, but I have found this to work quite well.  (Actually, to give credit where it's due, my friend and boss Tony Eggers has used this technique and I have seen it work many times.)  

With this type of specification, we start by taking a big picture snapshot of our application, review the deliverables, mock-up some user workflow examples, and mock up some prototype screen shots.  With this information, we can recheck our requirements, analysis, risks and deliverables and make necessary changes.  A workflow example is how we think users want to interact with our application.  (Not the way we want them to.)  

User workflow examples (UWEs, damn! we love acronyms.) serve several purposes.

1. They help us to put our ideas about how the application will be used down on paper.
2. They force us to think specifically about how the user is going to interact with our software.
3. They force us to think about lower level details we often gloss over.
4. They provide a basis for meeting with our users.
5. They provide a guideline for what needs to be prototyped since this is what we are going to talk about with out users.

Example 1: A bug is reported by a user through email 

The user is in Outlook (or the email client of their choice).  They receive an email, open it, read it and identify it as a bug. User clicks FORWARD. User sends the bug to one of the previously created special accounts. Immediately, a dialog box appears asking them to provide additional information.  

Q: What if they haven't created a special account for the particular group that has the bug?
A: We give them a special landing pad for all incoming bugs called bugs@mycompany.com and they send the bug there.

Example 2: User had a brilliant idea at lunch

It doesn't matter what is running or what application they are in. User clicks the tray icon and choose "Record my idea". A window appears from which they can enter various fields. User enters the information. User clicks SAVE.

Example 3: Users fixes a bug

User clicks the "I fixed a bug" icon in their development environment. A window appears from which they can record what they fixed. On this window is a list of projects open, recently changed files (from our journaling system) which the user can put check-marks next to to include with as part of the fix. User enters the information. User clicks the project(s) that were involved in the change. User clicks the file(s) that were involved in the change. (Hmmm... Maybe in addition to files we could show classes/functions?) User clicks SAVE.

Example 4: User wants a list of open issues

User clicks the tray icon and chooses "Open issues" A window appears with all open issues and pre-configured filter options.  (ie. By group, by status, by age, etc) Additional filtering options are provided (date range, etc.) User selects desired options and clicks "PRINT".

Example 5: User has a file he wants to attach to an existing issue or group

User opens Windows Explorer. User locates file(s) and selects them. User right clicks and chooses "Send to -> Mail recipient". Standard email window opens and user sends email to a pre-defined special user.

OR

After user selects files, he drags them to the "My issues" folder on the left tree control. While holding mouse, user finds group or item of interest and lets the mouse go. A dialog appears allowing for additional information to be added if needed.  (May not occur if we are just attaching files, or user has option set.)

Some  notes about our examples:

1. The dialog provides a way to clarify which group(s) to assign the bug to and allows them to create a new group if desired. 
2. When creating the new group, we also allow them to create the special email account and if desired, we automatically add the newly created email to their address book.
3. The automatic dialog display should be optional.  If turned off, there is a landing pad that can be accessed through a shortcut or as part of our management GUI where these things can be quickly reviewed and edited.
4. As always, the dialog is customizable.

That's enough for now.  You may ask why we waited this long to do workflow examples?  The answer is simple.  I forgot about them earlier, but in practice we need to have a pretty good grasp on our project before we try and write them.

What did we forget?

Usually after forcing ourselves to think like users, we realize we have forgotten something somewhere.  What is it?  A good way to approach this question is to ask yourself the question "If I had the final deliverables listed above, could I install the app and demonstrate each of these things right off the bat?"

Opps.  We may need an application for displaying lists of issues.  We could just make this part of our management GUI, though.  After some though, I think we need to treat this as a new application for the same reasons we make the data entry applications separate.  OK, lets add that to the deliverables list.

Oops again.  Since all of this stuff is configurable, there aren't going to be any built-in issue types.  How can we demonstrate to a user how to add an issue when none exist.  We need to include in our Other Stuff section of deliverables templates for issue types.

Deliverables - Take 2

Lets add our newfound items to the list and revise it for the sake of completeness and clarity.

On the client side

• Management application
• Data entry application
• Dynamic form designer
• Issue list viewer
• Journaling service
• Task bar tray-icon application
• Developer studio add-ins (for VC 6 and 7)
• MS Outlook add-in
• MS IE add-in
• Windows Explorer namespace extension

On the server side

• EMail server
• EMail reader agent
• State management and persistence server

Other stuff

• Installation routines
• Sample templates for issue types
• End-user documentation
• Extension API documentation
• Code documentation

I have an idea

As is often the case when going through the process of properly designing a project, we come up with "cool" ideas on how to enhance our application.  Well, I have an idea.  

IDEA : Clearly email is critical and these special email addresses make it all work.  What prevents the user from using an email client to view, edit and delete issues?  Hmmm.......

What's next?

Tomorrow, we are going to assemble some prototypes to demonstrate our UWEs.  We need these so that we can go back to our users ad get their input before we start working on our design document.