A model for UX design reviews

Design reviews are so important for our work as user experience designers, but they too often fail us. Here is a model for design reviews that overcomes the problems of ego, emotion, and communication that so often get in the way of helpful feedback.

Alaina Kraus, Caitlin Potts, and I presented this process for the Jan 27, 2011 IxDA Lansing meeting.

Roles in the design review process

  1. Designer
  2. Facilitator (may also be a reviewer)
  3. Reviewers (you’ll need at least 2, but 6 may be too many)

Step 1: Designer explains the project and design concerns

At this step the designer shows the design (on paper, on screen, a prototype, etc.) and explains to the reviewers the context of this design. The designer should also point out specific areas that she wants feedback on.

Step 2: Reviewers discuss the design

This is the part that’s going to go against your habits.

The designer steps back and withdraws from the discussion. She should use body language to exclude herself to make it harder for the reviewers to address her. She should instead focus on her listening and note-taking.

Meanwhile, the reviewers are discussing the design amongst themselves. Instead of referring to the designer, they should refer to the design. Instead of talking to the designer, they should talk to each other.

Of course, the reviewers should point out good aspects of the design as well as discuss areas that they’d like to see improved.

By intentionally excluding the designer from this conversation, the dialogue can cover more ground and the feedback can be more honest. Otherwise, the dialogue has a good chance of focusing on a single point or so as the designer begins to explain or defend her decisions. That will derail good feedback, and everyone loses out on good information. People’s feelings are also at risk.

Step 3: The designer rejoins the conversation.

Finally, once the facilitator has determined the design has been discussed enough, he will invite the designer back into the discussion.

The designer can now summarize the notes she has taken in order to give the reviewers the opportunity to catch any misinterpretations. She can also ask follow up questions to clarify feedback she may not have fully understood.

The designer will probably want to defend the work or explain it, but really doesn’t have to. The goal is feedback for her, and at this point, she has it.

Why this method works

This method works because it makes it okay for the designer to simply listen in without feeling a need to defend her work. Likewise, it frees up the reviewers to not have to worry about hurting the designer’s feelings or fear the reaction of feedback taken poorly. While the method may feel a little awkward at first, after a few times it becomes easier.

What kinds of feedback make sense?

The reviewers should provide feedback that matches the fidelity of the design. This is to say, if it’s a rough sketch or task flow diagram, talking about pixel-perfect alignment of the layout is inappropriate. This should be obvious to most of us.

Also, instead of simply presenting your opinion about a design, discuss it in terms of usability heuristics (Nielsen’s list of 10, Tognazzini’s 1st principles document), accessibility concerns, visual design principles (e.g., proximity, alignment, repetition, contrast), and your observations from usability tests.

Where did this method come from?

In the mid-90s I worked at the Michigan State University Writing Center, and we used a similar process that we called the “fishbowl” to teach people to do peer-review writing workshops. In many ways, writing and designing are similar. When I started doing more design work, I recalled this process and adapted it to design reviews. It seems to work great. I credit learning this method from Dr. Sharon Thomas and Dr. Laura Julier of Michigan State University.

Update Feb 26, 2011: Wordcast live on Design Critique

Tim Keirnan over at the Design Critique podcast posted an interview we did on this process. Check it out! And thanks, Tim, for inviting me.

Update July 14, 2011: Michigan UPA workshop

This evening Alaina, Caitlin and I ran this workshop for a Michigan UPA event in Lansing. We had fun and it sounded like the attendees enjoyed themselves while learning this model. Thanks again Second Gear Coworking for letting us use your excellent venue!

Addendum to “Getting More From Analysis” by Lewandoswki and Dilworth

UX Magazine published Getting More From Analysis by Jared Lewandowski and John Dilworth on Dec 16, 2010. The authors provide a  summary of analysis as part of three design activities:

  1. Analysis
  2. Synthesis
  3. Evaluation

Diagramming these activities

They also provide some models for conceiving of these three activities, from linear to finally advising on a traditional equally-weighted 3-circle Venn diagram.

I like the thinking, and the clear reclassifying of design as including all of these activities, as opposed to only synthesis. However, their Venn diagram leaves direction out of the picture. I like this approach better.

Design as Venn diagram of Analysis, Synthesis, Evaluation, with clockwise arrows.

This model simply adds the arrows to what Lewandowski and Dilworth provided. It’s simple, but serves to organize the concepts.

Working together isn’t always simple

In addition, the authors recommend involving analysts in synthesis activities, and designers and developers in analysis activities. This, I agree, is good advice, but they didn’t provide any techniques to help you actually pull this off.

Analysis, synthesis, and evaluation each require different types of thinking, and a person specialized in, say, analysis may end up stepping on the toes of the people specialized in synthesis. The result may be a stunted synthesis activity, shut down because an analyst broke down the problem again before a designer or developer had a real chance to synthesize the ideas into various solutions.

To avoid these types of problems, creating intentional design (be they analysis, synthesis, or evaluation) sessions is important.

Setting up design facilitation sessions

I’ve been listening lately to some lectures by Professor Michael Roberto, and in them he’s advised what he calls the 4-Cs of encouraging real ideas in a debate. Roberto is focused on strategic decision making, but I think we can learn from this advice and apply similar thinking to how to set up design facilitation sessions.

  1. Composition – of group, who should be invited
  2. Context – Atmosphere, shared norms, ground rules, location
  3. Communication – How will we discuss and exchange views? E.g., subgroups, devil’s advocates.
  4. Control the process – Will I as leader be in the room? Do I ask for a packaged solution or do I ask to see a debate.

You might think this is over-thinking it all. Just put the people in the same room and have them discuss the design, and good things should happen right? Maybe. But why gamble? Even 10 minutes of planning a design session based on these 4-Cs could turn a waste of time into a breakthrough design session.

Signing and Encrypting E-mail on Mac OS X 10.6 Using Self-Signed Certificates

A few years ago I wrote about using Thawte’s personal e-mail signing certificates for setting up secure S/MIME encryption with Apple Mail. Well, Thawte, so I understand, is phasing out that service. So, I’ve been wondering how to do self-signing on the Mac to set up S/MIME encrypted e-mails. This evening, I found out.

Credit where it is due: James Walker’s post on how to set up self-signed certificates for e-mail with OS 10.4. His post gave me a few steps to follow that I’m  just updating here to match what is needed for Mac OS 10.6.

Create your certificate

Open up Keychain Access. This is an application in your Applications/Utilities directory. (It is faster to just hit command+spacebar to open Spotlight, then enter keych, and hit the enter key when Keychain Access appears highlighted.)

Click on the Keychain Access menu, hover over the Certificate Assistant option, and then select Create a Certificate….

Create Your Certificate in Apple's Certificate Assistant window

Here are a few details to note about the Create  Your Certificate options.

  • You might want to add an e-mail descriptor to the name field. E.g., Davin Granroth (gmail).
  • Go with Self-Signed Root and S/MIME (Email).
  • By default, the certificate will be valid for a year. If you want to extend that a bit, you need to check the Let me override defaults checkbox. You’ll get to make changes after you click the Continue button.
  • If you need a certificate for your non-primary e-mail account, you’ll need to check the Let me override defaults box for that too.

If you checked the override box, you’ll eventually see a series of Extension windows. Just go with the default values. Apple figures out what you need based on the first screen where you chose the certificate type.

Continue and you’ll see a window with your new certificate information in it. Congratulations!

Certificate Assistant window showing the newly minted cert. It also says: This root certificate is not trusted.

Now if you could only trust that certificate.

Trusting your certificate

If you haven’t already, click the Done button to close that Certificate Assistant window. Now, back in Keychain Access, click on the My Certificates category on the right of the main Keychain Access window.

You’ll see your new certificate listed with a little white X in a red circle on the icon. That indicates the certificate is not trusted. Double-click on the certificate, and a new window will open with details of the certificate.

Certificate window with Always Trust selected.

Near the top of that window you’ll notice the word Trust with a little triangle to the left of it. Click the triangle to twist open the Trust options.

In the When using this certificate select list, select Always Trust. Then close that window. You’ll be prompted for your administrator password. Enter it, and you should be all set. Your new certificate should now be trusted.

Sending signed or encrypted e-mails

At this point, if you restart Apple Mail, you’ll notice a new option available when you compose a message.

Compose message with sign and s/mime options
The check icon indicates that your signed certificate will be included in the message. Once you've exchanged signed certs with your recipient, you'll be able to exchange S/MIME encrypted messages.

For more on exchanging signed or encrypted e-mails, see James Walker’s article. Scroll down to the section on Exchanging Signed or Encrypted E-mail.

Why would you want to send encrypted e-mails?

Hah! “Why wouldn’t you want to,” is the better question. Actually, if you send or receive sensitive information like usernames and passwords, legal information, or confidential business information, you might really want to consider this.

The trick is getting the person you exchange these messages with to also set up S/MIME on their end of the e-mail.

Google Analytics for WordPress (version 4.09) trips HTML validation when tracking outbound links

When I write HTML, running code through the W3C Validator is part of my process. If the code doesn’t pass the validator, there’s a slim chance I’ll consider that code ready for anything.

So after I upgraded this blog to WordPress 3.01’s Twenty Ten theme, I was mildly irritated to find that I had a few validation errors. I viewed the source code to investigate the offending lines of code, and noted that the code the validator cited was not in the source.

This phantom code led me to suspect that DOM scripting from a plugin caused the errors by inserting more code into the markup. That was it, and this evening I finally took the time to track it down. Here’s the scoop.

First, the phantom code that broke validation

W3C validator error message indicating an empty target attribute
The error indicates an empty target attribute. It turned out, this code was inserted by a plugin.

Honestly, the first time I saw this error message, I didn’t notice the error text that pointed to the target attribute. No, I was first startled by a pattern that looked to me like the tail-end of Javascript code used in an anchor’s href or onclick attribute. I was startled because, having already looked through the code, I thought I would’ve noticed that pattern, since it is one that I avoid.

Where was this code? Well, all 9 errors were in the same area of code: links to other websites (blogroll) in the sidebar. I wanted to see it for myself, so I opened the page and viewed the source code. I used the Find command and pasted in some of the code from the validator error message, but the Find came up empty-handed.

While the offending code was not in the served markup, the validator was certainly picking it up, so I went back to the validator to examine that code in more detail.

Code with an onclick javascript snippet inserted for Google Analytics tracking.
Code with an onclick Javascript snippet inserted for Google Analytics tracking. Note the target attribute.

The value of the onclick attribute was the immediate tell for where this code came from. It was the Google Analytics plugin I’m using.

Investigating the plugin that caused the validation error

Google Analytics for WordPress is a fantastic plugin. It makes integrating Google Analytics easy, and it has some excellent advanced settings.

One setting is to track outbound clicks and downloads as events so they are are viewable in Google Analytics.

Checkbox to track outbound links in WordPress.
Checkbox to track outbound links in WordPress.

To test the theory, I unchecked that box, saved the settings, and revalidated the web page.

It passed with flying colors.

Validation error identified, but I’m actually curious about tracking downloads

For the time being, I’m okay with leaving that feature turned off. I’ll let the plugin author know about it and hope the problem will be resolved soon.

Yes, I lose a feature to get back to valid code. Perhaps there is a support group for this kind of behavior.

But in my defense, this is just a personal site. I don’t make any money on it, and my interest in tracking downloads is just curiosity. I only have a few downloads or outbound links I’m interested in, and I have nothing riding on that knowledge beyond satisfied curiosity.

Still, the entirety of this problem can be laid to rest on a simple, meddlesome, and empty target="" attribute. That’s irritating.

Thoughts on Richard Saul Wurman’s Why Design Now talk

Richard Saul Wurman, author of “Information Architects” and founder of the TED conference, spoke at the Cooper-Hewitt National Design Museum‘s “Why Design Now” event. This is that talk.

Wurman emphasizes that so much is designed, even the questions that we ask. To illustrate, at one point he asked Bill Moggridge, who was attending as the new director of the Cooper-Hewitt, how old he is. Bill answered the question, and Wurman pointed out, “Now we all know your age, but you know nothing new.” His point is that the power is in the question, not the answer.

That is worth ruminating on. From a design perspective, coming up with good questions to understand a problem is often difficult. It takes dialogue and exploration and work before you get to the good questions. And then you begin to understand the problem better. Wurman also points out in this talk that organizations too often jump to action before they really understand the problem.

Wurman packs a lot, including some humor, into this short talk. It’s worth watching.

I got to help UXmatters with print styling

UXMatters print pageA few months ago I contacted UXmatters online magazine to let them know of a problem I had printing out articles with Firefox. The first page would print, but that was all. I could switch to Safari and print fine, but that just didn’t seem right.

I found out that they were a little shorthanded on specialized web help, so I  volunteered to assist. Pabini Gabriel-Petit, the publisher and editor-in-chief, graciously accepted my offer.

Creating print CSS is usually simpler than web page layouts, because the goal is typically to have the content of the page print out well on standard size paper. Multiple columns and site navigation are usually uncalled for.

My first try was to revise the existing print CSS file. After half an hour at that, I decided it would be simpler to start from scratch. The original print CSS was quite complex, and it seemed I was searching for one issue: why did the pages stop printing after page 1?

So I rewrote the CSS, simplifying the CSS code greatly. I had the layout in good shape, but oddly still had the 1 page issue.

I went back to the main screen CSS file and begin sifting through it, looking for clues. Then on a hunch, I found the issue.

The global CSS file is thick. There’s a ton going on there to defend against various cross browser issues. One particular rule I saw repeatedly was the use of overflow: hidden. That rule is used for a few reasons, but one of them is to force a floated box (div or what have you) to expand when it wants to collapse, vertically. This allows things like backgrounds to show through properly.

One of the selectors that this rule was used on was actually a wrapper for much of the page content. It made sense to me, in a way, that this could confound a browser into hiding paged content that went beyond a page.

So the real key to the much simpler CSS file was to add in an overflow: visible to a large set of selectors to override that overflow: hidden for printed pages.

I sent Pabini the new code, and over the following weeks she tested it and added some details I had missed. About a month ago she deployed the new print CSS code  to the website.

This little evening project was a fun puzzle, and I’m glad I was able to help make such great content on the UXmatters website print out better for more people.

(Back in 2002 or 2003, I forget when, I wrote an article on print styling for web pages.)

IxDA Lansing kickoff featured speaker Dan Klyn

Last night was the inaugural event for IxDA (Interaction Design Association) Lansing, and information architect Dan Klyn presented “The Nature of Information Architecture.”

Presentation overview

Dan’s presentation was both informative and controversial. He provided some nice background on the naming of the field of information architecture, citing Richard Saul Wurman phrasings at the 1976 AIA convention in Philadelphia and then Wurman’s book “Information Architects.”

Dan also proposed a way of considering what information architecture is through a target diagram, from core to outside as:

  1. Ontology (the study of being)
  2. Taxonomy (the science of order or arrangement)
  3. Choreography (writing/describing circular dance)

I like this description of IA. I do feel it gets to the heart of the work, and I can immediately consider certain areas of my own job in this light. Of course there are plenty of other descriptions of IA from Wurman, Rosenfeld, Morville, and others, but as I bring them to mind, they seem to be more focused on describing IA to outsiders whereas this one speaks to those of us already  in the field.

I wouldn’t, for instance, walk up to a client and say, “I’m concerned with the ontology of your system.” But I can talk with other information architects about questions of ontology, and they will likely bring their own experiences to bear.

After discussing this model for IA and how it circles the concept of understanding, Dan shared two ways to answer the question “How do we know when IA is good?”

  • performance (need to measure change in performance from a benchmark)
  • propriety (how appropriate to the context is the IA solution)

When discussing this question of quality of IA, the point was made that a functional, adequate solution is artless. It is insufficient. We’ve all used a website or service that we end up getting irritated with, and could comment afterward “Well at least it worked.” Good IA goes beyond sheer performance to fulfill propriety as well.

Isn’t naming always controversial?

The controversial part of Dan’s presentation is in the naming of things. The gist of my issue is that I felt Dan was saying that strategic-level IA work—the work that involves not just end-users but is concerned with larger business concerns—is beyond the scope of user experience work. (I really hope I’m not misrepresenting Dan’s meaning.)

My experience as a UX professional (note that I used to refer to myself as an information architect) says that UX begins with understanding both user and business needs, and is best done when exploring the strategic-level in order to frame the tactical work.

That said, I will say that with all the work demanded of UX in my job, I regret that I haven’t had the time to devote to a more traditional strategic IA-based analysis of our systems.

Dan made the point that this strategic work, though done by a smaller group, has greater leverage than choosing which style of form field to use. He is right, of course. I just think that that work is still to be done under the UX umbrella.

Thanks Dan for the talk, and Chris Bachelder for bringing it together

All-in-all, I’m really glad I had this opportunity to take part in Dan Klyn’s presentation. It was well-done and thought-provoking. Dan has shared his slides for “The Nature of Information Architecture.”

IxDA Lansing is the first Michigan group of the Interaction Design Association, and was initiated by Chris Bachelder of Techsmith Corp. I, for one, am grateful to have a Lansing-based group to advance UX events. Thanks Chris!

Stop the stopwatch, UXers!

Stopwatch graphic from Casey Marshall
Stopwatch graphic by Casey Marshall

Recently, I watched a series of people observe informal usability tests.

Two of the observers have recently graduated with Masters degrees in HCI or an adjacent field.

Both recent graduates used a watch to record time-on-task and completion of the task. One actually broke out a stopwatch while the other referred to his wristwatch.

While these stopwatch fixations livened my day, I do wonder about graduate education in the usability field.

I recall that for the first half-dozen website usability tests that I moderated, I also recorded time. Then I realized that timing tasks obscured more important observations, and I haven’t bothered with timing since then. Besides, we can get times off the recordings.

Is the working world really that far off from graduate studies?

So why did these two graduates pull out timers?

Well, I think they were parroting “proper” methods they were taught without understanding when it is useful. If, in grad school, they only practice for ideal research situations, they’re missing out on the realities of the work world.

I’ve worked in an Agile development environment for the last couple years, and for the decade prior to that I worked on fast-moving projects that used whatever SDL I applied to them. The mission: Deliver value, ASAP.

With that charge, decisions are made that don’t allow for insight from in-depth, long-term studies with huge numbers of participants. I’m grateful for even the small, quick sessions of user and design research.

Regardless, I got a chuckle out of seeing these two bring out timers for a completely informal, one-off usability test. As expected, they both missed seeing key interactions because they were watching the clock.

When to be concerned about time, usability-wise

Okay, I’d hate to give the impression that time doesn’t matter. I just find that a long time to complete a task on a website is rarely the issue, instead it may be a symptom of other issues which become apparent during research.

However, I do find response times of a system to a user’s actions to be very important because too much delay in a system’s response can really hurt the user’s experience and even distract people from completing whatever they set out to complete. Still, this class of problem is often noticeable during observation. (Unless you missed it while you were fiddling with your stopwatch.)

With that in mind, I’ll gesture towards Nielsen’s take on response times.

My 2.5 days in San Francisco: MX 2010

Red stone church near green trees, surrounded by skyscrapers.
View from top of Yerbe Buena Gardens, San Francisco, March 2010.

Saturday PM: Sunshine!

I actually began to sweat under my blazer from the warm sun shining brightly through the window.

I had arrived in San Francisco a little early on Saturday, dropped my suitcase off at the Intercontinental Hotel, and walked around the corner to a sandwich shop for a bite to eat and to get online. As I draped my coat over the back of the chair, I decided I really like San Francisco. It’s the sun, I admit it. Oh, and I had already noted that the two billboards I noticed on the taxi from the airport were pure tech: one for an enterprise search system and another for PGP. Billboards talking to me? Amazing.

After settling in at the hotel, I had dinner with my old colleague Chris Burley and his girfriend at a nice Italian restaurant. Chris is awesome. I love talking with him because he has such passion for what he does, which currently is to help lead efforts like urban farming in the Bay area.

Sunday AM: 3 good things

The next morning I woke early due to the time zone difference, and I had three excellent experiences:

  1. In the aching fog of caffeine deprivation, had the best cup of coffee of my life, thanks to the Blue Bottle Café. (I admit, I ordered a second cup to go.)
  2. Paused in the Yerbe Buena Gardens where some elderly practiced tai chi and parents snapped photos as their little children hid behind a waterfall. I stood on a bridge and watched the morning sun ripple on the glass of San Francisco skyscrapers.
  3. Crashed a church service at a music venue called Mezzanine put on by a group that calls itself IKON. I was the oldest person there, amidst a crowd of art school students. We sang, we listened to a teaching from the Word, we had communion. It was good.

Sunday PM: MX day 1

Sunday afternoon saw the start of the 2010 MX Conference.

MX2010 is largely focused on managing user experience and less on the tactical end of UX practice, and there were some thought-provoking presentations from people who have been managing user experience for a number of years, in a number of different types of companies. Off the top of my head, presenters represented firms in financial industries (Vanguard), publishing (Harvard Business Review), retail sporting goods, and online media (Youtube).

The series of talks was fantastic, and was kicked off with a keynote by Jared Spool in which he shared insights like that Gallup’s Customer Engagement (CE11) metric has high correlation to the quality of user experience. Spool’s keynote actually turned out to predict some themes that carried throughout the many presentations. Among them were the importance of establishing a vision for user experience and that experience ultimately must be addressed well across multiple channels (web, mobile, physical space, etc.).

Spool talked about three core attributes necessary for great user experience: Vision, Feedback, and Culture. He posed three questions that UX managers should ask.

  1. VISION: Can everyone on the team describe the experience of using your design 5 years from now?
  2. FEEDBACK: In the last six weeks have you spent more than two hours watching someone use your design or a competitor’s design?
  3. CULTURE: In the last six weeks have you rewarded a team member for creating a major design failure?

After the conference reception, I wound down the evening by taking a walk around a few blocks and ending at a nearby bar. I ate a burger and watched the Academy Awards for a while. Back at the hotel I watched the end of a Clint Eastwood Western flick and fell asleep.

Monday AM+PM: MX day 2

I woke at 4 in the morning. I checked analytics, email, and my usual RSS feeds. I stretched, washed, dressed, and still had time to kill. I read a few chapters in The Shack, a book Adam gave me last week.

I chatted throughout the day with Haakon, a usability specialist attending from the design company Tarantell in Norway, and as he sipped his coffee, I decided to not mention my mere three hour time difference.

The rest of the day was another series of excellent presentations. Themes: customer (more than user) experience, vision that guides the business, new models for working in the network, UX leadership stories from Youtube, customer experience in renovation of thinking at Harvard Business Review Online, understanding the holistic customer, data-driven design decisions (and when not to rely on data for design decisions), experience design as business strategy, and operating as a chief experience officer in your company.

It was great to hear first-hand the stories from these user experience leaders. Now, for what to do with it all when returning to the office.

Tomorrow and then

Tomorrow morning I fly back to Michigan, and need to get my head back into product owner and user experience work. But I also need to hold onto the ideas from this conference, and shift into actively leading user (or is that customer) experience work at Covenant Eyes.