The Standalone Programmer: Communicating with Users

Introduction

I am basically an introverted person which is probably why I enjoy software development. When I was younger I thought software development was about coding and probably thought that is was ideal for me since I was an introvert. Had I known then that high quality software development requires so much interaction with non-technical people I might have chosen a different field. Fortunately for me I did choose this field and I have learned that being introverted does not mean that I cannot enjoy interacting with other people. Today I enjoy user interaction at least as much as actual coding.

I have learned (often the hard way) that before I can successfully and effectively deal with the process there are many non-technical skills which must be acquired and mastered first. Skills like setting expectations, understanding user workflow, understanding existing user expectations, understanding business need to name a few. I can sum up all of these skills in one word: Communication. The ability to communicate effectively and accurately is key to success as a standalone programmer.

Communication is the single most important skill for understanding and coping with the whole process not just the code. Communicating with a computer is easy when compared to communicating with executives and users. Where I was once most interested and challenged by writing code (communicating with the computer), I now find myself most challenged and interested in communicating with the users. This change has not been easy and has required a lot of learning and ego-busting but it has paid off tremendously. I have learned that improving the communication between myself and the users is every bit as important as improving the communication between myself and the computer.

Disclaimer

Before I begin I just want to say that I am no expert and the contents of this article are my observations and techniques. You may have better techniques or think that mine are terrible. I would love to hear from you. I am interested in improving my abilities and skills and I am not egotistical about my views in this regard. So, if you have a comment, positive or negative, please leave me feedback and explain what you think. In fact, if there seems to be consensus on better ways of doing things then I will probably update this article to include those better techniques.

Preview

In this article I am focusing on the aspects of direct user communication. Communication is a big subject so I have decided to start with this.

Too much is a bad thing

When working with users it is important to understand their viewpoint and their way of thinking. Having the lines of communication open is critical to long term success. In my attempts to become a better software architect, I have been learning to improve my communication skills. I still have much to learn and I am sure that with more practice I can eventually get a better handle on it.

I have learned the hard way, though, that communication by itself is not enough. Simply being willing and able to talk with users is not enough for success. If I had a dollar for every time someone told me "I told you about that" or "you knew about that" or "I thought you knew" or "that's so obvious" I wouldn't be programming any more. Statements like this prove that communicating with users is not enough. I'm sure that I'm not the only programmer who has been buried in these statements a time or two usually towards what I thought was the end of a project. At one point in my career this was a frighteningly common occurrence. I still dread this statement because it is evidence of how much I still have to learn about communicating with people.

One of the worst mistakes to make with communications is to just dive right into it without a plan. Communication with users needs to be a planned and usually scheduled affair. Planning should include a basic agenda and scope of the communication. Allowing a conversation (whether it be verbal, email, documentation, whatever) to run rampant beyond a focused scope introduces many problems into the process. Unfocused communications divert the users attention from a goal, are difficult to remember because they cover too much, are difficult to file for the same reason and often allows the user to provide too much input.

Now you may be thinking that it's not possible to allow a user to provide too much input. I can tell you from experience that a a user can and will provide too much input if allowed to do so. Earlier this year I finished up a a new software system for a client. Early on in this project I made a special effort to work closely with the users to insure that the system was what they needed. As part of this early effort I spent a couple hours with each of the primary users at the beta site. I asked them to explain their job and workflow to me and listened to them explain their process to me. I tried to ask questions designed to get more accurate information from them.

I thought this process worked well and I proceeded to use this information to build the software. This software has now been installed for 6 months at the beta site with relatively few problems. Until 2 weeks ago that is. A couple weeks ago, a user approached me on Monday afternoon regarding a complex report she needed by Wednesday morning. When I told her that I didn't know anything about the report she told me "you knew about it" and "I told you earlier". These are deadly phrases because they reveal poor planning and follow-up on my part. First, in all likeliness she did tell me about the report months earlier in an early planning meeting, I just forgot about it because at the time it was not what I was working on. Second, when I stated that I didn't know about it I was effectively saying that although she told me about it I didn't consider it important enough to remember.

I have included this little example to display several important concepts which I will endeavor to explain in greater detail.

1. Users always understand their workflow better than I do.
2. When a user tells me something, even in passing, they will expect me to remember it.
3. Failure to document communications with users will usually result in problems later.
4. Users rarely take responsibility for following up with conversations regarding their needs because they perceive this as my responsibility. Also, following up on software projects is not part of their workflow and shouldn't be.
5. People are sensitive to how others treat their importance and the importance of their problems and needs.

This exact scenario has plagued me many times in my career and when I consider it I always find that the cause of the problem is poor communication skills on my part not the user. Taking responsibility for the problem is important because it initiates a desire and a drive to solve the problem and avoid the same situations in the future.

Rule #1: Users understand their workflow better than I do

You're probably thinking this is about as obvious as the nose on my face but somehow this simple fact often alludes me and other software developers. The bottom line is that the software I develop will be used by real people with real needs and expectations. Their needs and expectations are the very reason I am writing the software in the first place. Somehow this fact often gets obscured in the development process.

Most users are more than willing to discuss what they do so long as they are not asked to do it over and over again. One of the greatest lessons a standalone programmer can learn is to listen to his/her users and strive to understand their workflow. Gaining access to the knowledge that the user holds will help to improve the quality and usability of the software by insuring that the users expectations are met.

Incorporating and accommodating the users existing workflow is often quite valuable in improving the users experience. One thing to look out for though, is problems with their existing workflow. The reason for developing this new software (usually, sometimes?) in the first place is to help the user. Just copying their existing workflow is not always the right approach. Often because users have to work with inadequate tools they learn workarounds to the pitfalls of their available tools which become part of their workflow.

Where the developer can assist the user at this stage is in identifying these workarounds and shortcuts. The user will often omit details related to these workarounds because they have come to think of them as just "part of the process". Often these workarounds can be completely removed if identified early.

Addition suggested by Michael P Butler
"By the time a specification is finished, I like to believe I know their system better than they do. Users have a bad habit of telling you what they do (and what the new system should do) rather than telling you why they do something. The ability to ask questions that the user understands but tells you they why rather than the what is a major part of communicating with customers."

After thinking about this comment, I have to agree with Michael. Users often don't fully understand the reasons they do the things they do. Also, users often don't understand how their needs and workflow fit into the bigger picture of the whole organization. As a developer we have the benefit of seeing the big picture of the entire organization and not just the workflow/needs of an individual user. Incorporating each users workflow into the complete organizational system is critical to the outcome of a project.

When reviewing the workflow of all of the users involved in a project it is important to look for overlap in the various aspects of their jobs. Many users will share common needs and processes which can be incorporated into the software as a single feature instead of 2 or more separate features. This can often reduce the complexity of a project and increase its flexibility.

Rule #2: When a user tells me something, even in passing, they will expect me to remember it.

So I have established that listening to the user and allowing the user to communicate their workflow and process is a good thing. Now I will tell you where this will come back to bite you. When I have had my little meetings with users I have encountered 2 basic types of responses. All users are willing to talk about their jobs and requirements but about half will just provide cursory information while the others will deluge me with every little detail. What I want is somewhere in the middle.

What commonly happens with the people who tell me too little is that eventually they will use the line "I thought you knew" or "that's so obvious". Anytime I here this line I remember rule #1. Usually whatever the confusion is about is obvious to the user since it is part of their job and normal workflow. I, however, don't do their job so I am not as familiar with the issues as they are.

The other kind of people will eventually use the line "I told you about that". They probably did tell me, it's just that they told me so much there is no way I could remember it all. Again, this just reinforces rule #1. The user has an intimate understanding of their needs and since I showed interest in them they shared them with me. Since they shared them with me I should know them too.

The solution to these problems, fortunately, is fairly simple. Never assume that a single meeting or conversation can cover all the material. Instead plan on using several small meetings or conversations each targeted at a single specific topic. The first conversation should be used to explain the process you are going to take.

This first conversation is very important because it lays the ground work for the rest. This first conversation is intended to do the following things.

1. Show the user that you are interested in what they do and how they work.
2. Show the user that you intend to solve their problems in the best way for them.
3. Show the user that you have a plan for working with them that will take time but will not take too much of their time.
4. Show the user that you are willing and able to talk about their needs within certain boundaries.
5. Provide the user with time to think about the topics you need to understand and let them know that there will be time in the process to provide more information if they forgot something.
6. Set the users expectations for the rest of the process.

As part of your first conversation you should set out an agenda for what you are going to be talking to them about in later meetings. For me this usually means going from general to specific. I usually want to understand how their role fits into the greater business first. Then I want to know how their daily workflow works. Next, I want to know what kind of data they collect or need to manipulate. Then I want to know what they expect out of the software. Finally I want to understand their expectations for the software.

Rule #3: User communication must be documented

OK, so I have explained how to get what you need from the users. Getting the information out of them, though, is only part of the total picture. I must be sure that I can recall it as needed and I must insure that I actually understood correctly what they told me. The only reasonable way to do this is to document my conversations with the users and verify what I document.

At least for starters a simple hand written notebook will do. For me, though, the problem with hand written notebooks is that they get lost, I can't read my writing and there is no way to easily file the individual pages in a way that makes it easy for me to look up what I wrote. Unfortunately a hand written notebook is often the easiest way to get the notes taken during the actual conversation. Another possibility is to use a digital recorder to record the entire conversation. This is a great idea because it is possible to go back and review the complete conversation in detail whenever needed. (Suggested by Michael P Butler)

I have to admit that this is an area where I still struggle. I have a hard time organizing the notes in a way that works well for retrieval. I have already designed a piece of software to help me with this (basically a documentation library) and I plan to write it sometime this year.

For me the key to documentation is that it has to be usable by me when I need it. Notebooks fail this test because I never know where to look for the information I want. Simple HTML files by themselves don't help much either since there is no easy way to file them under multiple categories as is often the way I will look them up.

Regardless of the problems with the available solutions, any solution is better than no solution at all. For now I am using my companies issue tracking software (DevTrack) to do the job. This solution leaves much to be desired but is the best thing for now. Whatever you do, come up with a solution that is simple, not time consuming and easy to review. If/when I develop my software I will probably post it here on CP.

Rule #4: Users rarely take responsibility for follow-up

Great, so now we have collected the necessary information from the user and documented it. There's only one problem. How do we know that what we documented is actually what the user said or meant. It is easy to misunderstand the user. It is also quite common for the user to omit important details. While this misunderstanding or omission is unintentional its impact on the final product can be significant.

Most of the time software development is not the job of the user. Their job is totally unrelated to mine and therefore they don't think of the issues involved in my job unless I ask them to. Once a conversation is over with the user they will go back to whatever they would usually be doing. All thought of our conversation will be gone. It really isn't their responsibility to follow-up with me and make sure I understood them. If they don't hear from me they can only assume that I fully understood what they told me and merrily wait for their perfect software solution.

Fortunately there is a simple solution to this and if you followed rules 2 and 3 it's already part of the process. Every conversation with the user is targeted and documented. It is a simple matter to send the user via email or mailbox (or whatever) a copy of your notes in electronic or written form and get them to confirm what you have written. Again, since your first conversation set the tone for the rest of your information gathering this should be expected by the user.

I have found that doing this (BTW: I use email) often reveals many little details that the user forgot to mention and often reveals mistakes and misunderstandings on my part. A key thing to remember is that Rule #1 applies to me as well as the user. Users simply don't understand the software development process nor should they. I do and they expect me to do my job well just as they are expected to do their job well.

Addition suggested by José Carlos
"I usually send this documentation:

• Data: what pieces of data will the program use. Sometimes I show the table structure (if the user is capable to understand it) and sometimes I just make a list. This way the user can quickly see if the list lacks any piece of information.
• Processes: what processes the program will perform, how those will affect data (in non technical language) and under what circumstances will those processes be allowed.
• Reports: what repots the program will provide.
• Screen preview: a quick design of how the screen will look like: text boxes, grids, buttons... The users sometimes find something they lack in a screen preview faster than anywhere else."

José makes some excellent points. In particular, I think that the screen preview can be very powerful in helping the user. Also, I would suggest mock ups of reports as well. One idea that I have used in a production system, but not for documenting communications with users is to use a flow chart ala Quick Books, not Visio. If you have QuickBooks 2000, look at the Customer Navigator or Vendor Navigator screens. These are HTML-style screens with pretty icons and arrows. If you don't have QuickBooks, you can see a partial screen shot of one of my applications in my previous article at http://www.codeproject.com/tips/usability_testing2.asp.

Rule #5: People are sensitive to how others treat their importance

Most people take pride in their jobs and expect a certain amount of respect from those around them. Also, with today's economy most people are a little jumpy when it comes to job security and importance. Taking this into consideration early on is important for the standalone programmer because it helps to control the nature and tone of the conversations with users.

All too often I hear developers referring to users as "dumb". I have been guilty of this myself and although I have run into a couple genuinely stupid people the vast majority of my users are quite intelligent. Most users are good at their jobs and often their jobs exist with or without software so software can either be a burden or a blessing. Having a proper appreciation for this fact is very important. Just as I don't want people treating my work as unimportant or calling me "dumb" I should not treat other people this way. (This is also why I consider user interface design one of the most important technical aspects of my job.)

When communicating with users it is inevitable that misunderstandings and omissions will occur. How I deal with these issues when they arise can and does have a significant impact on my future dealings with that user. The act of opening the lines of communication tells the user that you are interested in their roles and respect their importance. Any time an issue arises you have an opportunity to reinforce that belief in the user or show them that your interest was only a cover to get them to tell you what you needed to know.

Treating the user with respect and giving them the benefit of the doubt usually pays off. Most people are reasonable and understand that issues arise and are willing to work with me to solve the problems. The down side is that if I haven't followed my own rules well I probably won't find out about the problem until it is an emergency.

Postmortem

OK, so now I've dissected this problem to the Nth degree lets look at it specifically and see where I screwed up.

Remember I said "As part of this early effort I spent a couple hours with each of the primary users at the beta site". My mistake started very early in the process. I ignored my own rules and paid the price for it. Instead of a series of short and focused conversations I had one long conversation with each user. While this seemed like the thing to to at the time it is very clear to me now that this approach was a bad one.

Now recall that I said "approached me on Monday afternoon regarding a complex report she needed by Wednesday morning". Now I cannot remember whether or not she actually told me about this or not, but this reveals that I failed to follow-up our conversation and get her to confirm my notes. Had I properly documented our conversation and asked her to confirm them this issue would most likely not have occurred. Also you can see here the importance of having multiple small conversations. Had I sent her several pages of notes from our 2 hour meeting it would have been difficult for her to take the time to read them thoroughly and think about them in a focused manner.

Next, recall "When I told her that I didn't know anything about the report...". Oops. Should never have said that. In essence I was telling her that she never told me about it. From her perspective she did her part. She answered my questions and as far as she was concerned I understood everything because I did not ask any follow-up questions or ask her to confirm my notes.

At this point her response "he told me you knew about it and I told you earlier" was inevitable. It was not her fault that the report wasn't where she thought it would be. I failed to use focused conversations with her. I failed to get her to confirm my notes. I failed to follow-up with her to make sure the report was correct. I failed. There is no other explanation for this.

Fortunately, I have a good relationship with this user and was able to get her report done and satisfy her immediate needs. However, now she can rightfully see a pattern in how I work which is not the pattern I want her to see. This will probably come back to haunt me later.

Conclusion

Learning to communicate with non-technical people is a critical skill every standalone programmer needs to survive. Communication is a skill not easily mastered especially when it is not practiced. Communication includes not just what is spoken, but was is not spoken, what is written in emails, formal documents and what is displayed on the screen when the user is using software. Failing to communicate with the users (and other people involved) in a timely, considerate and complete manner will ultimately result in missed requirements, misunderstood needs and expectations, and ultimately poor user reception. Failure to structure the communications can be equally devastating.