It's amazing to watch the ways that people fall on their face. We've all read about the benefits of usability testing, but until you actually try to sit still through two hours of these videos, it isn't a visceral experience for you. It is exciting, and totally emotionally exhausting. You squirm. And it focuses you like a laser. A really focused laser. Made of razor blades.
For example, we asked a lady to send mail to a friend. Against all odds, she started Evolution (nothing in the menus indicates that it's a mail program; something we hadn't realized before but which was immediately obvious after watching her stalk one-by-one through the menu items muttering to herself along the way).
The correct next step would have been for her to click on the "New" button that's in the upper-left-hand corner of the window. This button didn't even register for her, however. Instead, because she wanted to "send" a mail, she clicked repeatedly on the "Send" part of the "Send / Receive" button just to the right. For about a minute.
This is easy to fix; we just need to change the labels to be more sensible (and then test again on 5-6 people to make sure we changed them appropriately). It was interesting to watch this video and instantly realize that the "Send / Receive" button is all about *how Evolution works* and not about *what the user wants to do*. I've been staring at that button for five years, and never realized it was wrong until I saw that video.
I think this is one of the hard thing about user-facing software: neither you nor anyone you regularly interact with are representative samples of the people who use the software.