Click here to Skip to main content
Click here to Skip to main content

Usability Testing (Chapter 2)

, 22 Jan 2002
Rate this:
Please Sign up or sign in to vote.
Usability: The myth of efficiency.

Introduction

Before I begin, I just want to say that I do not consider myself to be an expert in usability testing, user interface design or just about any other area. I do, however, have some experience with these areas (both positive and negative), and I decided to write this article to share my experience with others. The concepts and ideas I discuss in this article are not earth-shattering and many of you will already know all of this information.

Also, I want to acknowledge my boss (that sounds very odd I know). I have learned from him a great deal concerning usability and "efficiency". Much of the information I cover in this article is knowledge I gained by working with him and watching how he works with users. I have managed to benefit from his help and knowledge and hopefully build on that knowledge.

If you ask just about any software developer how they define efficiency, they will tell you that it is a measure of how fast a function/algorithm/program performs and how few resources it uses. If you ask the average user what it means, they will tell you that it is a measure of how quickly they can do their jobs. These two definitions are not always compatible.

I titled this article "the myth of efficiency" because I feel that as a software developer I have been mislead. Just about all the books, programming journals, web sites, etc. that I have read have focused on efficiency from the programmer's perspective. I am not saying that this perspective is not important or even the right perspective for many problems, just that very often it is counter-productive and should not be the main focus.

Let me just say that I am a big fan of efficient algorithms and I like to see my code perform well. It is difficult to predict how a particular function/algorithm will be used in the future, and making the code efficient often expedites things in the future. Unfortunately, this has often been my primary focus. I have neglected the need the user has to get their job done in the most efficient manner possible. This has resulted in me optimizing the wrong algorithms. The key is not to be concerned with making all or a particular algorithm efficient, the key is to make the right algorithms efficient.

Where to Start

As I said in the introduction, the key to efficiency is making the right algorithms efficient. To do this, we need to understand how an application needs to be used by the user. (I say "needs to be" instead of "will be" because "will be" assumes we are going to decide for the user.) The only way I know to do this is to work with users before development begins and while development is in progress. The key to this is understanding the user's workflow.

User workflow is how the user performs their jobs in the absence of software. Most people have a basic job description which involves a certain number of responsibilities. They also may have a set of requirements related to accounting for their work. They usually have a set of tasks that they perform in repetitive fashion along with a few tasks they perform highly sporadically. In addition, there are typically a set of tasks that they only perform on a very rare occasion.

Many software applications fail to recognize that users already have a defined workflow process before the software was available, and basically try and force the user to alter their workflow to conform to the software. This may work when the software is a critical part of their jobs, but will probably result in decreased workflow efficiency and user dissatisfaction.

To develop a truly "efficient" application, it is necessary to speak with potential users about what they do and how they do it. This discussion is probably best done in the absence of software, prototypes, or screen shots. The user may be confused or misunderstand the software, and fail to provide the most valuable information. One place where software can help, though, is when there is an existing system in place you are going to replace. The user may be able to point out "what they like and dislike."

The goal is not to have the user develop the software for you, just to have them explain how they work. It is the developer's responsibility to convert this into software. With this information, it is possible to better design the application to meet the user's needs. It is also easier to decide what parts of the application need the most optimization and which ones are less important.

Case Studies

The best way I can think of to illustrate the problem and its solution is by using real-world examples.

Workshop management

A recent project I worked on involved replacing an MS Access database with an updated UI and reporting system. The old system had many "features" (I now refer to them as warts). One of the "warts" was a "workshop management system". I hesitate to call it "management system", but that was its intended purpose.

The system had three basic objects. These were workshops, members and registration records. There were basically just three forms in the system. One form was used for member data, one for workshop data, and the other form was used for registration records. Pretty simple and uncomplicated (technically). To navigate among the workshops, the user could use record navigation (i.e., First, Last, Next, Previous, Add, Delete) or a combo box located in the upper right-hand corner. From this screen, they could print a registration list, attendance roster, etc. Navigating members was basically the same, but instead of a combo box, there was a box where the user could enter a member ID and go to that record.

The next step was to ask the user how they did their job before they had the MS Access database. The user informed me that each time a new workshop was created, they would create a document in Word and make multiple copies of it for each date it occurred on. Then they would tack those sheets to a wall. When a person called or came by to register for a workshop, all they had to do was look at the wall to see what workshops were available and what the dates were. Then they just wrote the user's name, phone number and other info on the sheet and re-tacked it to the wall. A week before the workshop, they would take down the sheet and call each registered person to remind them of the workshop. On the day of the workshop, they took down the workshop and gave it to the presenter of that workshop to record attendance.

From this, it was obvious what their "natural workflow" was. It was also obvious that the MS Access database forced them to work differently. With the "sheets on the wall system", all of the needed information was right before their eyes and they did not need to know the person's member ID. The MS Access database forced the user to work in an unnatural workflow. To determine what workshops were available for a certain date/time, they had to navigate through multiple records on the workshop data form. Then once they found the workshop the person wanted to register for, they had to close the workshop form, search for the member ID (usually the person does not know their ID), and finally they had to open the registration form, find the right workshop, and register the user.

I have not attempted to scientifically calculate how much time was lost by using the MS Access database, but I believe it is clear that the cost was high. (In my work, I usually don't get concerned about statistical validity as much as user acceptance.)

My next step was to look around at off-the-shelf applications which were used to perform similar tasks. Many of them worked the same or worse than the one they already had. Some were better but not great. Then I looked at QuickBooks Pro 2000. Although the problems are different, the solutions cross over well. QuickBooks uses an HTML style interface with a few obvious components. (I just want to say that I think QuickBooks implementation is a little overwhelming. There is too much info on many of the screens.)

After playing with the design a little bit, I was able to come up with what I thought was a good start for a workflow solution. My UI solution was to provide the user with an HTML style interface broken down into four areas. (See the screen shot belowSmile | :)

This UI was similar to the old "stick it on the wall" system because all of the information the user had to look for was placed right before their eyes, not hidden in multiple forms, and record navigation. The user can click on any upcoming workshop to access its data form (and from there they can register also). They can click on one of the icons beside the name to either register people for the workshop, cancel the workshop, or print a roster, reminders, etc.

This UI improved on the "stick it on the wall" system because the user could easily filter the information on the screen by the most important criteria (when it occurs). The graphic in the upper-left assists less experienced users by visually representing the workflow process and providing one-click access to the most commonly used functions. (The icons have been whitened out because they do not belong to me, sorry. It looks better with the icons.)

In the end, the users are pleased with this system and feel more productive because of its design. As you can see from this problem and its solution, the efficiency the system needed was not improved performance in the record navigators, improved search criteria, etc. By developing the system to work with the user's workflow, user efficiency is maximized.

Searching

In my first article, I discussed the "quick search" feature I developed for this project. I am now going to elaborate a little on this feature.

In this same database, the user often needed to search for a member in the database. (The database is fairly small, about 3000-4000 members.) Also, there are < 1000 organization names in this database. Users need to search for these also. Sometimes, the user needs to search by name (first, last), phone number, address, member ID, organization name, etc.

In the MS Access database, there were multiple forms for the various "types" of searches users performed. There was a form for searching for a member by name, another form for searching for facilities by name, and another for searching by name/address (1 each for members/facilities). Searching by phone number was not available even though the user often needed to do this.

One common approach to dealing with this problem is to present the user with one search form containing all the possible search fields. There would be two forms, one for members and one for organizations. The user could then enter the name, address, ID and phone number, and perform their search against one or all of the available criteria. From what I have seen, this is a pretty common user interface.

The problem with this approach is that most of the time the user does not have all of the information and is unsure, when first approaching the form, whether all of the fields are required to fulfill the search. Also, usually with this type of form, it is unclear to the user whether pressing ENTER on a search field will activate the search or if they need to click a "Search" button. Users tend to attempt to enter as much information as they know and this usually results in very poor search results due to misspellings, missing, incorrect or out-of-data data, etc. This can quickly result in users adding new members/organizations to the database even though they are already present. This is exactly what had happened in the MS Access database. Users could not find people in the DB so they just added them back into the system.

The other problem with this approach is that it forces the user to make a decision about what type of object they want to search for. Although this seems simple enough, this can reduce efficiency because the user just wants to find some record in the database, not navigate through the application's user interface.

The solution I devised for this problem was to provide a single search edit box located on a navigation bar docked to the left side of the screen. The edit box can also be placed on the toolbar when the navigation bar is not present. This search incorporated the "quick search" methodology I discussed in my first article. Basically, this one search control could be used to search for members or organizations and could search by name, address, phone number, ID #, etc. The control has some basic logic to determine what type of info the user enters (i.e., is this a phone number, ID #, text, etc.).

Then the control searches all reasonable columns and tables in the database for matches. The control also has some basic logic to deal with situations like "Mount Saint Hellens", "Mt. St. Hellens", "MT saint hellens", "MT st helens", etc. It also has logic to improve address searches by expanding or removing directional and numeric portions of addresses, street type indicators, etc.

Because the control is located on the navigation bar (and the app uses almost no modal dialogs), the user can perform a search from anywhere in the application quickly and easily. They do not have to decide what they are searching for, they don't have to worry about providing too much info or too little, and it does not interfere with what they currently have on the screen.

Also to deal with more advanced search requirements, two forms exist which the users can use to search members/facilities as described above. These forms are not used very much.

The result of this implementation is that most searches are fulfilled on the first attempt. This reduces database hits, user confusion, and duplicate data being put into the database. Users can do their jobs more efficiently and with less intrusions forced upon them by the software.

What to Expect

When designing software for user efficiency instead of computer efficiency, your skills of user experience design will be tested. You will need to look at other peoples' attempts at solutions, and you will need to experiment with different methods. You will also need to make a concerted effort to understand the user's natural workflow.

I expect failures and problems. The searching algorithm I described above took many iterations to get to where it is. I spent a tremendous amount of time optimizing this algorithm (not for speed) but for user experience. I played with numerous algorithms for expanding and reducing names, addresses, etc. It took me many iterations to achieve a workable algorithm and even now I am still tweaking it a bit.

The rewards of developing software which produces efficient users instead of efficient algorithms are many. These are just a few.

  1. Efficient users - Efficient users are usually happier users. They make better references and are easier to work with. They provide better feedback to management and this can lead to more work, pay raises, etc.
  2. Better applications - No one cares if the XYZ function of an app works amazingly quickly if the XYZ function only gets used once a year. I have spent countless hours optimizing code that barely ever gets used. This can be very frustrating when you reach the end of a project only to find that the user is struggling with the user-interface instead of waiting on XYZ function.
  3. Fewer bugs - Often, optimizing code introduces subtle bugs. Had a particular search been done as a simple linear search instead of a binary search, a serious bug could have been avoided. I am not saying not to optimize code, just optimize the most important ones the most.
  4. Reduced technical support calls - Because the software works the way the user works, users have an easier time learning how it works and what to do. They also tend to remember how to use it better also.
  5. Pride in your work - I find that making applications that are efficient for the users is just as rewarding as making a particular function/algorithm efficient.
  6. Competitive advantage - If your application is easier to use and works better with user's natural work-flow than your competitors', that is a great selling point. If your competition is already doing this, you are at least as good as they are.

License

This article has no explicit license attached to it but may contain usage terms in the article text or the download files themselves. If in doubt please contact the author via the discussion board below.

A list of licenses authors might use can be found here

Share

About the Author

Matt Gullett
Web Developer
United States United States
No Biography provided

Comments and Discussions

 
GeneralDatabase comparison [modified] PinmemberLokanatha Reddy20-Dec-06 20:48 
GeneralAOM and WrokFlow Pinmemberchris lasater14-Apr-05 9:47 
GeneralThere is basically 3 components of software product: GUI, application logic and data... PinmemberVasek1-Feb-02 8:08 
GeneralRe: There is basically 3 components of software product: GUI, application logic and data... PinmemberMatt Gullett1-Feb-02 9:27 
Thanks for the feedback!
 
I agree with what you are saying for the most part.
 
I have no problem with Access if it does what the user "needs" and works correctly. I have developed access databases before that perform specific tasks and they work well. My specific example was based on an that project because it was recent in my memory. There are other examples I could have chosen. Maybe I should have indicated in my article that Access (in and of itself) was not a problem, just the implementation.
 
"What I really hate is "smarter-than-needed" computer widgets, which tends to make wrong decisions. Soon you can realize the world of your bussiness (or the bussines you are supporting) has changed and your smart component is very difficult to modify, just because it handles such complex task, like a decision whether this number should be automatically interpreted as ZIP number, or telephone routing number."
 
I also hate so called "smart apps" which only interfere (see MS Assistant for ample info.) Again, I should have put a disclaimer in the article. I do beleive, however that there is a place for these type of widgets. Part of User experience design is to determine where to use one method and where not to use it. As developers we should not limit ourselves to "standard" methodoligies when a less "standard" methodolgy solves the particular problem better.
 
"Searching a database more than one time could be a reasonable simplification for the customer, but only if you have very few information to search for. I have already used this approach. The only difference is that I have bound the decision mechanism as close to data as possible. That means, the mentioning decision mechanism does not reside on UI-side. On the bussiness logic side it is accomodated rather. But still you have to decide if the single dropdown box in front of your search textbox would bother the potential user more than possibility of waiting very long time (more than 10 sec is always very long time). The problem could be escalated by adding more than one user: what about 65 client workstations connected to central enterprise system consisting of one, two, or more servers? Would still be applicable your approach of "simple, clever search"? "
 
As described in the article the database is farily small (2000-4000 members and <1000 organizations.). There are about 50 users, probably with about 30 active ones. This system uses a serve-side service to implement 3tier design. The quick search logic is located in the service, not in the UI layer. In the article I was not interested in discussing architectural implementation, only user-level experience, so I left these details out. This back-end system is very effecient and thus allowed for this extra work with no problems. Again, I agree that for large systems where performance might suffer or where the results might confuse users, this approach would not be the right one. Each application warrants its own needs and solutions.
 
"As I mentioned above, there are many simple ways how to ask user without bother him decide which search form is the best. First of all, there is a dropdown listbox choice which can be settled in front of your search box. I will be more satisfied as a user, when knowing the machine is searching for the names of my bussiness partners (people) acctually, rather than trying to browse institutions the sought people are working for.
 
The next possible solution is to show just different forms with different options when the initial dropdown listbox is selected.
 
The third is to use wizard, or wizard-like web pages."
 
Agreed, there are many other ways to allow users to search which are appropriate for a given situation or system.
 
"I am not sure the people tends to fill everything in search form. The more they fill, the shorter will be the result, because the more specific information leads to the more specific search which obviously ends with fewer results. This is very simple mathematic the non-computer people understand."
 
I disagree with this statement because in 4 out of 5 projects I have tested, the average user enters more information when they are presented with the option to do so. This I have seen both by observing users and by tracking search criteria in a db.
 
I also agree that the back-end portion of appliations/systems must be optimized also. If they are not optimized it can be difficult to optimize the user interface adequately.
 
I think we are in agreement about most things. In future articles I will be more careful to add discalimers where appropriate. I greatly appreciate your feedback and look forward to any additional comments you might have.
 
Thanks,
 
Matt Gullett
GeneralRe: There is basically 3 components of software product: GUI, application logic and data... PinmemberDag Oystein Johansen3-Mar-04 7:15 
GeneralRe: There is basically 3 components of software product: GUI, application logic and data... Pinmemberbradw2k28-Apr-04 10:19 
GeneralRe: There is basically 3 components of software product: GUI, application logic and data... Pinmembergfoot21-Dec-04 1:42 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Terms of Use | Mobile
Web02 | 2.8.1411022.1 | Last Updated 23 Jan 2002
Article Copyright 2002 by Matt Gullett
Everything else Copyright © CodeProject, 1999-2014
Layout: fixed | fluid