Recently, I watched a series of people observe informal usability tests.

Two of the observers have recently graduated with Masters degrees in HCI or an adjacent field.

Both recent graduates used a watch to record time-on-task and completion of the task. One actually broke out a stopwatch while the other referred to his wristwatch.

While these stopwatch fixations livened my day, I do wonder about graduate education in the usability field.

I recall that for the first half-dozen website usability tests that I moderated, I also recorded time. Then I realized that timing tasks obscured more important observations, and I haven’t bothered with timing since then. Besides, we can get times off the recordings.

Is the working world really that far off from graduate studies?

So why did these two graduates pull out timers?

Well, I think they were parroting “proper” methods they were taught without understanding when it is useful. If, in grad school, they only practice for ideal research situations, they’re missing out on the realities of the work world.

I’ve worked in an Agile development environment for the last couple years, and for the decade prior to that I worked on fast-moving projects that used whatever SDL I applied to them. The mission: Deliver value, ASAP.

With that charge, decisions are made that don’t allow for insight from in-depth, long-term studies with huge numbers of participants. I’m grateful for even the small, quick sessions of user and design research.

Regardless, I got a chuckle out of seeing these two bring out timers for a completely informal, one-off usability test. As expected, they both missed seeing key interactions because they were watching the clock.

When to be concerned about time, usability-wise

Okay, I’d hate to give the impression that time doesn’t matter. I just find that a long time to complete a task on a website is rarely the issue, instead it may be a symptom of other issues which become apparent during research.

However, I do find response times of a system to a user’s actions to be very important because too much delay in a system’s response can really hurt the user’s experience and even distract people from completing whatever they set out to complete. Still, this class of problem is often noticeable during observation. (Unless you missed it while you were fiddling with your stopwatch.)

With that in mind, I’ll gesture towards Nielsen’s take on response times.