3 Phases of guerrilla user research

Phase 1 – Planning (1 to 2 weeks approx)

Step 1 – Determine the objectives of the research study
● Define which aspects of the value proposition and UX are being examined.
● What is the most important thing I need to learn to determine if this product has any purpose, marketability and viability?
● What are the riskiest assumptions still on table at this point?
○ Drug Treatment example: value proposition (hotels.com for people seeking drug rehabilitation for their loved ones) was still up in the air.
○ UX = matching interface (hotels.com) where users enter a price and are presented matches in the specified price range. Like hotels.com the name of centre withheld until after user had booked. Risk – would reverse auction work for this type of customer segment?
○ Business model tied to this UX so could not move forward until had feedback on the value proposition.
● Needed to decide how many times would need to hear yes before move forward and how many no’s before back to drawing board. What would be our success criteria?

Step 2 – Prepare questions that will validate what’s being tested
● Quality over quantity. Goal is not to place prototype in front of 1000s users instead is to interact with 5-10 handpicked users to gain intensive and pointed insights. Nielsen Norman Group believes testing up to 5 users in a usability study is enough before you start hearing the same things repeated.
● It’s not just about discovering whether the users like or dislike the value proposition – although if 10 disliked it that might be enough. But about discovering exactly why, how and what you can change to make the product better.
● These are not just usability tests (i.e. how user accomplishes the task with the product)
● Rehearse the interview and the prototype demonstration.
● Solution interview: open ended questions, try not to lead the participant.

Setup, recap/verify screener input (3 mins)
● On arrival, warm up participant, verify what you know and maybe get them to expand if it will help you understand their experience.
○ Can you tell us how many times you have booked your loved one into a rehabilitation center?
○ How long was each stay and did the facility offer you discount to prolong the stay?
○ How did you find out about individual rehabilitation centers (word of mouth, internet, list provided by a specialist,..?
○ If you used the internet how did you find the facilities (for instance google) and which sites did you use for finding them and learning about the specific details?

Problem interview (10 mins)
● This goes into more detail and asks more question that earlier problem interview (chapter 3)
● You want deeper insights, how did they solve the problem in the past?
● Try to understand their experience and get them to recite a timeline of problem-solving events in linear format. This prepares them for contextual solution.
● Drug Treatment Center example – focusing on payment
● Who paid cash for a treatment center as opposed to using Medicare?
● For those rehabilitation centers for which you paid yourself, was the process different in terms of your options for making a selection?
● Did the center negotiate the price with you? Do you feel you received fair value for what you paid?
● What do you think makes the cost of the facility worth the price?
● Do you remember how long in terms of time from when you decided upon the treatment center and the patient actually went in? (Prompt: What is urgent? How long did the selection process take?) Was there anything specific you remember
● that was really great or really horrible about the process of finding the treatment center?

In total, we prepared 10 problem questions, got insights on pain points.

Solution demonstration interview (15 mins)

● You may need to demonstrate solutions for more than one key experience.
● Encourage participants to think through your solutions
● Don’t create or ask leading questions or put them on the spot to brainstorm.

● Screen 1: What do you think is going on in this screen?
● Screen 2: What do you think all these pull-down options mean? (Go through each one.)
● Screen 2: What do you think the ratings are based on?
● Screen 3: This is the listing page with information about the rehabilitation center. What do you think is going on here?
● Screen 3: What do you think of the “Apply Now” button? Where do you think it will take you? Screen 3: You might have noticed that we don’t display the treatment centers’ actual names. Would that be an issue for you?
● Screen 3: Would you be comfortable applying to this facility without contacting it first.

Final thoughts (2 mins)
● Share knowledge with participant of what you learned from them.
● Ask if they might be open to a follow up in the future
● Rehearse the interview with a colleague

Step 3 – Scout out venue, map out logistics (who will be there, what they’ll do and when)

Best to have a separate interviewer and note taker (don’t use recording device)
Need event handler to screen and welcome, handle money etc
Go over roles of each in the guerilla user research group before the event
Why a cafe – you want participants to feel relaxed / informal / not feel judged / it’s a familiar environment / cafes are free

Finding cafes
Spend time at exact time and day you plan to run event make sure it’s quiet enough
Test wifi connection, are there table that can sit 3 within range of sockets
Ensure not in line of sight of cafe workers
You’ll need 3 to 4 hours so choose a counter service not table service place

Step 4 – Advertise for participants
• Recommend paying participants as they are helping you.
• Mention in advert that participant will be paid in cash at the interview eg £20 for 30 mins.
• During screening ensure your participant will be able to help you.

Example adverts – Drug Treatment Center
Craigslist – Jaime used this. Advert wording was
Title: Paid Research
Study: Looking for who have experience with Body: Market research firm in is looking for participants to join an upcoming paid research study.
The study is going to be on during the hours of
<# - #> at a café in the area. Please let us know the ideal time that works for you.
The study will last for <#> minutes and the compensation is $<##.00>. (optional) The study will not be recorded on audio or video.
Please respond with your contact information and the best time to reach you.
(optional) Link to survey here:

• Facebook friends / LinkedIn Special Interest Groups (post to specific relevant groups)
• Meetup Group (attend meetup group in your area or post to them)
• Twitter (use hashtags to cast a wide net or @ profiles for hopefully gaining a retweet among a specific group of followers)
• Get referrals of friends of friends who fit you customer segment.
• Canvas an area where there a high concentration of your customer segment eg student at USC canvassed in high traffic areas.

Step 5 – Screen participants and schedule time slots.
• Choose participants who match your hypothesised customer
• Look at provisional persona and think about critical characteristics needed
• You could ask respondents questions which are deal breakers and send them a survey monkey or google document.

Drug Treatment Center example

Screener questions:

● Were you the person seeking treatment or were you seeking treatment for someone else?
● Do you mind telling me how much you paid? Where and when did you go? (To confirm it was residential and for how long.)
● Needed to know if they’d paid for selves or loved ones for treatment. And needed their experience to be within the last 3 years.

Interview Phase (1 day)
● Arrive 30 mins early, get the ideal location in the cafe.
● Participant gets a drink of their choice
● Ideally use a tablet to play the solution prototype as this is easier to pass to participants unlike a laptop
● If you have a note taker they should sit opposite you with participant in the middle facing the wall.
● Designate a team member to pay the participants upfront, pass to them in an unsealed envelope before start, tell them you are paying them for their honest feedback, can even say you are not the designers to put participants at ease.
● Tip the cafe

Conducting the interviews
● Always greet people with a warm smile. I typically stand up to shake hands and immediately thank them for coming.
● Do not begin interviews with small talk. Be professional. You want to quickly build a rapport with them and avoid dwelling on how cool the café is or how difficult it was to find parking. Quickly state the reason why they are there and that you looking to them for their brutal honesty.
● Then, launch into your setup questions.
● Stick to the script. Ask additional follow-up questions if needed to probe deeper into specifics about how they currently solve the problem. Find out how, if, and why your solution would work or not work for them.
● Have the note-taker set his phone as an alarm to keep your pace (typically 15 minutes). The ringer should be on silent so that the phone vibrates to indicate to the researcher at the table that it’s time to move on to the solution portion of the interview, even if there are more questions to go.
● Ensure that you schedule buffer time between participants in case an interview does go longer than planned or a participant shows up late.
● At the end, thank the participants for their time and tell them how helpful their insights were.

Note taking
● Type into a document (eg the strategy toolkit)
● After the interviews colour code the spreadsheet to make analysis quicker eg a major pain point would be coded red

Analysis Phase (2-4 hours)

● Did the guerilla user research validate or invalidate your assumptions?
● Did the experiment fail due to being sloppy?
● Did something unforeseen happen?
● Goal use analysis as a decision point to pivot in a different direction or double down on further experiments that actualise the value proposition.
● Enter validated learnings in the template.

Big picture
● Assess to determine if the correct customer segment was reached, by looking at your provisional personas or initial customer discovery research. If it was not the correct customer segment, begin making new assumptions about the right one.
● Assess if the problem that your product is trying to solve is an actual problem based on the feedback that you heard. Was it a small problem or a big problem?
● Assess if the solution that you showed was on target. If people were not truly excited about the value proposed in the solution prototype, think through possible ways to improve it.
● If the value proposition was validated, congratulations! However, don’t stop there. Determine if there are any easy fixes that can be made to improve the user experience.
● If the value proposition was not validated, assess why immediately. Was it because you had the wrong customer, problem, and or solution? Is it fixable? How can you change the product or user experience?
● Listen to the signal. If the experiment was an abysmal failure, resolve to take a break while you reset your entrepreneurial or intrapreneurial clock.
● If you have a client or stakeholder who does not believe in your research and wants to build the product regardless, you face an existential challenge wherein you are trying to balance your principles and your pocket book. Only you (and your spouse) can answer this one.

What next?
● You invalidated your value proposition. If you are wrong about your original customer segment, go back to Chapter 3 (customer discovery).
● You invalidated your value proposition. If you are wrong about your solution, go back to Chapter 4, Chapter 5, Chapter 6, and Chapter 7.
● You validated that you have product/market fit. Go build a functional MVP and move on to Chapter 9

Leave a Comment

%d bloggers like this: