Remote user testing 101


Select the right candidates.  Often business units put forward their sales focused contacts.  These are  rarely the users that we want to talk with.  In early communications with potential participants be very clear about what you want to do, most will be happy to refer a colleague if they are not best placed to help this time.

Define your objectives
It sounds obvious I know but make sure that the stakeholders are clear on what you wish to achieve by doing this particular round of testing.

Create a script that is task orientated based on the user journey with plenty of open ended non leading questions too test the structure and layout of the work. Use this as a guide only, keep your conversations natural.

A decent  target test time is 20 minutes.

On the Day

  • Always introduce yourself and make sure that the time is still convenient for the participant
  • Be nice, and friendly but not to familier
  • Tell them what you are going to ask them to do and broadly why you are doing it
  • Make sure that they know that the session is being recorded, in our case both their actions and their voice.
  • They also need to know that this is NOT a test of their ability, there are no rights and wrongs we are just interested in their opinion, so we can ultimately make their experience better.
  • Explain that we want them to think out load.
  • Ask open ended questions that do not lead into an answer.
  • Use ‘silent encouragement’ allowing them to do the talking.

Plan B

Testing presents it’s own challenges anyway and remote international testing with a technological hurdle is right up there in the fly by the seat of your pants ‘will this work stakes’ so always have a Plan B

I had prepared a shorted test route that still gave us good useful information if the subject was pushed for time or the speed of the technology was proving troublesome.

If the tech failed completely the plan was to gain some other  insight that we could feed back in to the process.

I was also prepared for a complete breakdown, here it’s better to admit defeat,  thank the participant for their time and try to keep them onside for another day.

Always, always, always  follow up with a Thank you.

Test early and often

Testing early – gives valuable insight and stops projects from becoming internalised.

One of the challenges is that Bupa International’s clients are truly ‘international’ and activities such as focus groups and one to one testing were unrealistic even if I do fancy the travel.  Additionally ‘clients’ teams themselves are also dispersed between London, Brighton and Copenhagen.  So paper prototyping is was not really an option.

Being an advocate of innovative and collaborative working methods [and  not a fan of pointless meetings] I needed a wire frame and prototyping tool that was capable of remote screen sharing and group collaboration whilst being flexible enough to be a design and delivery mechanism in a semi agile environmemt.  I found this in an application called Operating as a SAAS [software as a service] business it offers a flexible and evolving platform, although it’s not without its own idiosyncrasies,-which includes not being overly happy with Bupa group HQ IT infrastructure.

For formative testing I used a sketch fidelity on the prototype.  Personally I find this useful as it allows participants [as well as stakeholders and project team members] to be a little more divergent in their thinking whist concentrating on structure rather than getting stuck in the details.

Session recording

Pidoco is capable of recording both audio and video in the sessions but this service is currently only available whilst dialling a German landline or via Skype.  Hence we were forced to use Bupas 3 way conferencing facilities to record the audio separately to the visuals, not ideal but still met our objectives

Screen sharing

As the subject uses the prototype remotely we see what they see -the way they see it [note: only the prototype screen is available not the rest of their desktop].  Another interesting function is the ability to add notes to the session for later, place markers and push instant messages to the user.

Change the way you work forever

I’ve been reading ReWork by Jason Fried and David Heinmeier over at 37signals the masters of web based collaboration. It’s mantra is keep it simple, focus, stay agile, and just get on with it and create something. There is some good advice on how to be more productive, getting projects finished and out manoeuvring the bigger players.

If it’s good enough for Seth Godin it’s good enough to for you.

‘Read it quick then get back to work’.