Purpose

✅ Use Guide To…

Determine if end-users can or cannot do what they should in an existing release.

The specific goals of a given User Test can be:

❌ Don’t Use Guide To…

Validate customer demand or Product Market Fit.

Validate willingness-to-pay at target price points.

Docs

Usability Test Reports

Usability Test Users

Usability Test Feedback

Outline

  1. Prep the test

    1. Confirm logistics
    2. Find 5 Users
    3. Designing the Tasks
    4. Formalize the story into a protocol (w/ sample header)
    5. Run a practice

  2. Run the Test

    1. Warm up the user
    2. Conduct the Pre-Test survey
    3. Run the Test
    4. Debrief

  3. Analyze

    1. Review initial test focus and create broad Categories for feedback
    2. Stack rank feedback by recurrence & severity
    3. Analyze data for insights
    4. Prioritize
    5. Report: Summarize and identify next steps

Detailed Tactics

1. Prep

  1. Confirm logistics
    1. Location - Where will the test take place?
    2. Timeframe - When and for how long will the test take place?
      • Note: Your entire test should stay within 30 to 45 minutes of actual testing.
    3. Moderator / Notetaker assignment - Who will be the moderator and who will be the notetaker?
    4. Recording? - Will the test be recorded?
      • Note: Tools like Hotjar can be used for recording. However, this adds additional cost, complexity and influence on the user behavior. We recommend recording for mature companies, and not for early startups.
    5. Compensation? - Will the test user receive compensation for thei
      • Note: As much as possible, try not to. Rewards create a bias in users’ motivation.If you offer a reward, it should not be too important, otherwise, you’ll end up with testers that are only here for the money. They are not representative of your target (usually) and won’t give you candid feedback. If you do decide to offer comp, the equivalent of $30 to 60 (in gift cards for example) is a fair amount.
  2. Find 5 Users
    1. Screening - Start with the right people to test.
      1. Choose a target persona based on previous understandings of the core user.
      2. Prepare screening questions.
        1. Examples of screening questions:
          • How old are you?
          • What is the highest level of education you've completed?
          • What is your total household income?
          • What is your profession?
          • When was the last time you purchased an item online?
          • Have you ever used [your website]?
    2. Invite Users to participate in the test.
      • Notes on disclaimers to send to confirmed users:
        • It’s better if the user doesn’t try to understand your product before the test. So specifically ask them not to look for information, go to your website or try your product in advance. Best is when they ignore who the test is for or what it’s about.
        • If planned in advance, send a reminder at the beginning of the week to make sure they didn’t forget.
      • Create a custom schedule link with 45-minute windows they can book at their availability.
      • Sample of a message to send after a meeting is booked :