I have been working with a lot of different usability agencies lately and it struck me that very small differences in how a user centred design project are run can have radically different outcomes, not all of which are positive. The main pitfall as far as I can see it occurs as a result of prominence given to either the usability or the development agency in the process and critically on how far the usability agency are asked to go with the process.

Scenario 1 – Usability agency engaged to define the brief to be handed to design/development agency for implementation

This scenario is defined by the size of the remit given to the usability agency, here (in brief) the process runs like so:

1. Engage users and test current website and competitor websites
2. Internal stakeholder engagement
3. Open & closed card sorts etc to define information architecture and functions
4. Produce wireframe prototype of proposed system
5. Design agency apply design to wireframes
6. Clickable jpeg prototypes are tested with users
7. A/B test
8. Launch

Outcome: a detailed and actionable brief effectively written by the target audience to meet their needs.
On the face of it this sounds great yet on more than one occasion the website produced as a result of this process has immediately and intuitively looked wrong to me and the conversion rates have in the end backed up my initial reactions. So what is going wrong? Time to consider a second scenario.

Scenario 2 – design agency led process
In this scenario the usability agency are used by the design agency for insight and verification of ideas prior to implementation, actually a lot of the tasks carried out are the same but this subtle change of focus and instruction has a radically different outcome.
Here the process works like so:

1. Engage users and test current website and competitor websites
2. Contribute to Internal stakeholder engagement with design agency
3. Open & closed card sorts etc to define information architecture and functions
4. Produce functional lists against proposed architecture
5. Brief client and design agency
6. Design agency brainstorm solutions ‘big ideas’ and produce clickable jpeg prototypes
7. Design agency finalise the brief
8. Usability agency test prototype variants and feedback
9. A/B test
10. Launch

In this scenario the design agency are using the usability company within their process taking user insight as just one strand of the briefing process. This has several critical differences from the initial scenario.

1. Before the invention of the car if you had asked a user how to improve their journey times they would have asked for a faster horse. The point here being that it is dangerously limiting to let users define the brief in its entirety, the only outcome will be a better version of what stood before.
2. User insight misses the bigger picture, if the voice of the user is too strong in the brief then the website will be too focussed on what users want (yes there is such a thing) at the cost of the commercial success of the site, sometimes less is more, but this rarely comes out of user feedback. This subtle shift in power is what sets the project free.
3. Wireframes are the death of creativity, there I’ve said it! Wireframes are of course useful in some contexts (lengthy form processes for instance where the design will actually reflect the boxes). In marketing led sites however, a wireframe design results in a web design that looks like the wireframes. At Bloom we prefer functional lists in their suggested positions as simple text on white space, ‘no frames’
4. Freedom! The most important change here is in the user engagement is used to ask questions, it is the creative and technical challenge to the agency to solve these problems and users are used to verify the potential success of ideas prior to the costly build not solve the problems themselves (for the reasons listed above).

For the purposes of this blog I have deliberately simplified the process here but I have tried to capture the essence of my concerns / recommendations. I would like to just take the opportunity to state that I am very pro usability agency and don’t want this to come across as a criticism, however this is still a relatively new and evolving market and we can and are always looking to improve the way we work and the success of our projects for our clients. I’m also aware that many usability agencies wouldnt adhere to the process in scenario 1 anyway so i’m not trying to tarnish everyone with the same brush.

So on that note I’d be particularly interested in comments from usability specialists, agencies, and individuals to help refine this for the greater good. Perhaps some of you think i’ve got it wrong?