Video summary

Usability Test Demo by Steve Krug

Main summary

Key takeaways

Educational

Main ideas / lessons conveyed

  • Usability testing can be simple and still valuable.
  • The goal is to test the website (not the user)—users can’t “do it wrong.”
  • Get honest, think-aloud feedback while users attempt tasks without assistance.
  • Record the session (with permission) to reduce note-taking and support analysis.
  • Use the test to identify the most serious usability problems, compare findings with an expert’s “top three,” and then fix issues incrementally (tweak first, redesign only if needed).

Methodology / process demonstrated (detailed)

Planning and framing the test

  • Purpose: Determine whether a website works as intended.
  • Duration: Demo is ~20 minutes; a real test would be closer to ~1 hour with more/longer tasks.
  • Testing stance:
    • “Testing the site not you.”
    • Participants can’t fail; it’s safe to make mistakes.
  • Think-aloud instruction: Ask the participant to verbalize:
    • what they’re seeing,
    • what they’re trying to do,
    • what they’re thinking.
  • Help rules:
    • Answer questions only as appropriate, but note that the goal is to observe behavior when they don’t have someone guiding them in real time.
  • Breaks permitted at any point.
  • Recording:
    • With participant permission, record screen and conversation.
    • Used only by the project team.
  • Observers:
    • Web design team observes from another room, mainly via the screen.

Participant briefing / onboarding questions

  • Collect basic context (occupation, Internet usage habits, preferred site types, etc.) to understand user background and expectations.

On-screen interaction (test steps)

  1. Initial page scan (free exploration)

    • Ask participant to narrate:
      • what strikes them,
      • what they think the site is for,
      • what they think they can do.

  2. Task set (structured scenarios)

    • Participant is given printed tasks and asked to complete them while thinking aloud.
    • Tasks in this demo include:
      • calculating monthly cost for different usage needs,
      • finding promises about whether cars will be available when needed.

Post-task wrap-up

  • Ask the participant to list three most serious usability problems they noticed.
  • Then the expert provides their own top three, allowing comparison.

Prioritization and example fixes

  • The expert explains that:
    • definitive answers should come from the people working on the site,
    • the demo is to illustrate the process of finding problems.
  • Fix strategy: Use the “least you can do” approach:
    • Tweak first for usability issues,
    • Redesign only if tweaking doesn’t solve the problem.
  • Two example tweaks described:
    • Remove confusing wording and use tabs to show different plans clearly.
    • Move the word “from” closer to the relevant prices so it’s harder to miss that pricing is minimum-based.

Main usability problems identified in the demo (expert “top three”)

  1. Confusing plan phrase: “rates shown for EVP $50”

    • The participant is “completely stumped” by this phrase.
    • The participant doesn’t understand that it refers to a specific pricing structure/plan context.

  2. Unsatisfying availability reassurance

    • The site’s message makes it sound like cars will always be available (“it’s always gonna be available”).
    • The participant wants more specific, credible details because the generic promise doesn’t address their concern.

  3. Unexpected higher cost after noticing neighborhood availability

    • When checking cars in her area, she finds pricing that is higher/different than expected from the rates page.
    • This discrepancy makes her feel distrustful and potentially distressed about leaving the site.

Additional notes the expert makes (not in the top three)

  • The participant didn’t notice certain extra value plans immediately, though the expert chose not to include it in the top three because it likely would have surfaced with more time or fewer prompts.
  • It took the participant some time to find the availability-related answer, but the expert didn’t rank it as one of the most serious problems because it was eventually found near where expected and didn’t seem to derail task completion.
  • The expert acknowledges other potential user-identified issues may differ; the key goal is practicing identifying problems, not matching exact “definitive” fixes.

Speakers / sources featured

  • Steve Krug (author of Rocket Surgery Made Easy; conducts and explains the demo)
  • Melanie (the test participant; unemployed; volunteer in kindergarten class weekly; prior health care management experience)

Original video