How to Create a Prototype – Clever Hans and Photoshop Iteration

How to create a prototype- It’s been over two weeks since we released a bare bones alpha of our site and started letting people in one at a time. Since then we’ve been through approximately:

• 10 iterations of our “view idea” page
• 6 iterations of “browse ideas” page and 3 of “browse people” page
• 10 “profile” pages
• 8 of our “p-home” page (this is the page you see when you log in)
• 3 “search result” versions
• and a whopping 14 basic template revisions

That’s a total of 51 revisions in 18 days, almost all of which are iterations based on feedback from users without writing a single line of code.

Photoshop Development

We’d been very slow to engage in real customer development for the topic: How to Create a Prototype. Despite getting a number of signups to our alpha, we made some early choices in platform which slowed our development down to some degree and made it difficult to get to our minimum viable product (MVP).

To compensate for that, we searched for ways to get customer feedback without actually having a product.

As an interim fix, we created a number of photoshop mockups detailing exactly what information would be shown on what screen with REAL text (none of that “Lorem ipsum dolor sit” stuff). The mockups were heavily layered such that we could demo the “prototype” via screen sharing and demonstrate mouseovers, ajax calls, and other effects by showing/hiding a single group layer.

(For those not photoshop savvy, this just means we could show on screen actions with one click.)

Alpha Hordes

We then took those photoshop mockups and put them in front of real users as often as possible (trying for twice a day) mostly via screen share and let the users look/play with them. In doing so, we got a ton of great feedback that led to so many revisions, but have certain difficulties on how to create a prototype.

Mockups are not clickable and must be controlled by the presenter

This has turned out to be a bit of a blessing in disguise.

Our initial supposition was that not having a working prototype would reduce the quality of feedback. Instead, we’ve found that when the user can’t control the action directly, they’re forced to talk to us in order to get the mouse to move, scroll down, and click on things. This helps encourage the user to speak out loud.

This includes exclamations such as “This page is overwhelming”, “I have no idea what this button is supposed to do”, and “I’m lost.” All of which is useful information for knowing what’s wrong with our site.

The feedback is confusing

As Dr. House says, people lie and they sometimes don’t know, how to create a prototype.

They’ll say they want more information.

Then when you give it to them, they’ll want less information.

Then you’ll revert to the original and they’ll say “perfect!”

I write down every idea a user has, but I’m trying to listen more for what they’re talking about rather than what they’re saying. If they’re talking about the “post idea” button, it means they’re seeing the button.

That’s good if you want them to post an idea and bad if the intended Call-to-Action is elsewhere.

Risk of asking leading questions

As I’ve written before, I have the habit of asking leading questions about items I think should be in the foreground, thus bringing them to the user’s focus. That leads to me feeling perfectly satisfied that the user is talking about the items I think they should be looking at, when they wouldn’t have even noticed them if I hadn’t brought it up.

Screen shares have been helpful here since people seem to ramble on and on when talking on the phone. Awkward silence just encourages them more. Even when they stop, all you have to do is ask “What are you thinking now?” and they’ll keep talking. The lack of physical presence also eliminates the possibility that the user looks to your body language for the “right” answer to a question. (See Clever Hans for a great example of this.)

Live Demos

We’ve also done a number of live demos which present a different type of information.

Body language and eye motion of the user is available

An obvious advantage of the live demo is that we can directly observe where on the screen the user is looking and in what order. This is perfect to observe the “F” pattern and if your intended focal point is accurate. It’s also great to see user body language and if they’re getting uncomfortable or frustrated it’s quickly evident in their seating posture or how they’re holding the mouse.

Users can “click” on things

Even though the photoshop images are not clickable, you can give the users the mouse and watch where they move it. If they’re getting into it, they’ll click on the image a couple times before remembering that it won’t work and then you’ll get the added bonus of seeing what the gut reaction of the person is.

No Substitute

Although I’m very happy with the results and think our designs have improved tremendously, this is no replacement for real development.

It could easily be argued that the time spent on photoshop could have been better spent on writing code. Fortunately, as I have two partners who are far more technically savvy than I am, they’ve been able to develop the site in parallel while I focus on the user interface. This may have slowed us down in the short run, but it’s saved us a dozen coding iterations and allowed us to build up a more robust back end.

Even as a solo developer I think this approach has some benefits, particularly if your site will depend more heavily on user experience than any particular feature.

My only qualm is that I cannot interview enough people to have a statistically relevant sample size and therefore need to take it to the next level in order to tie our results to actionable metrics. So this week and next, we’ll be putting our new knowledge on-line and seeing if our new user experience drives the behavior that we want.