This folder is labeled 'Qualitative Research', but it is actually NNGroup's complete UX research methods toolbox — covering interviews, questionnaires, empathy maps, sample size, task design, moderation, expert evaluation, workshops, and ethics all in one place. The core of qualitative research can be summed up in one sentence:It does not pursue statistical generalizability of 'how many people do this', but seeks to understand the phenomenon and reasons behind 'why this happens'. The content below is organized into nine sections: Method Landscape → Interviews → Turning Research into Deliverables → Sample Size → Task Design → Moderation → Expert Evaluation (without real users) → Workshops → Ethics.
1. Method Landscape: Four Quadrants + When to Use Which Method
NNGroup's classic taxonomy — any research method can be positioned along two axes:
| Axis | Two ends | Key difference |
|---|---|---|
| Nature of data | Attitudinal (what users say) ↔ Behavioral (what users do) | What people say and what they do often diverge — if you want to know real behavior, don't rely only on asking |
| Form of data | Qualitative (discover problems, provide direction) ↔ Quantitative (measure metrics, provide numbers) | Qualitative answers 'why/where things go wrong', quantitative answers 'how severe/how many people' |
| Method | Quadrant | Typical use case |
|---|---|---|
| User interviews / Focus groups | Attitudinal + Qualitative | Uncover needs, motivations, and mental models during the exploration phase |
| Usability testing (qualitative) | Behavioral + Qualitative | Identify interface problems and their causes |
| Usability testing (quantitative) / A/B testing / UX benchmarking | Behavioral + Quantitative | Measure success rate, completion time, conversion rate |
| Survey questionnaires | Attitudinal + (Qualitative or Quantitative) | Collect attitudes/satisfaction from large samples |
| Card sorting / Tree testing | Behavioral + Qualitative/Quantitative | Information architecture (IA) |
| Diary studies / Field studies / Contextual inquiry | Context | Long-term behavior in natural environments |
| Eye tracking / Clickstream analysis | Behavioral + Quantitative | Attention and navigation paths |
There are also two sets of orthogonal labels:Formative(early in design, to find problems for improvement) vs Summative(evaluate overall performance, compare against competitors/previous versions);Remote(online, distributed) vs in-person.
2. Interviews: Funnel-style Questioning, Broad First Then Narrow
User Interviews are the most commonly used attitudinal method in the exploration phase — ask, listen, and probe to uncover experiences, pain points, and mental models. Outputs can feed into empathy maps, need statements, personas, and journey maps.
| Phase | Practice |
|---|---|
| Define goals | Research is not casual chat; first write down 'what do we want to answer' (Why did they come to try this? High points and low points? What makes people abandon it?) |
| Write an interview guide | Open-ended main questions + follow-up probes;Pilot test Calibrate with 1–2 people |
| Warm-up | Start with light, easy questions ('Tell me about your hobbies'), avoid questions that make people feel judged |
| Build rapport | Nod, paraphrase, use verbal and nonverbal cues to build trust, but you don't need to deliberately make friends |
Core technique: The Funnel Technique — for every main question, move from broad to narrow, like guiding a mouse in a maze: first see if it can find the cheese on its own, then gradually close off passages.
- Broad open: 'Tell me about the last time you booked movie tickets' (captures natural behavior, non-leading)
- Open follow-up: 'Can you tell me more about that?' 'Why do you think that?'
- Only then closed questions: 'Did you go alone?' (captures specific facts)
Counterexamples/limitations: Self-report can be distorted (memory errors, social desirability bias to appear better); data quality depends heavily on the researcher not asking leading questions.If you want real behavior → pair with usability testing, don't rely only on interviews.
3. Turning Research into Deliverables: Empathy Maps + Need Statements
Qualitative data scattered across interview transcripts is useless; it must be condensed into artifacts the team can align around:
Empathy Map — a four-quadrant visualization of one user (or user group):
| Quadrant | What to capture |
|---|---|
| Says | Verbatim quotes ('I want something reliable') |
| Thinks | Unspoken inner thoughts ('Am I being stupid?') |
| Does | Actual actions (refreshing repeatedly, comparing prices across stores) |
| Feels | Emotions = adjective + context (impatient: loading too slowly) |
A single-user map is based on one interview;Aggregate multi-user maps synthesize common themes and can serve as the first step to building personas (but do not replace personas). Value: captures characteristics, exposes knowledge gaps (places where more research is needed), educates the team to avoid gut-based assumptions.
User Need Statement(core of the 'Define' stage in design thinking) — compresses a set of insights into one sentence,focusing on verb-based needs rather than noun-based solutions:
Format:
[一个用户(带标签)] 需要 [一个需求(动词、真实、不含方案)] 以实现 [一个目标(价值)]Example:
[Alieda,精通科技、要带两个孩子的妈妈] 需要 [快速自信地比较选项,且不离开舒适区] 以 [把时间花在真正重要的事上]
Key principle: Users don't need a 'dropdown menu' (noun), they need to 'see options and select one' (verb). Locking into a solution too early leads to suboptimal design. Needs can be split into parent-level (long-term vision) and child-level (specific features). Difference from development tasks: need statements provide direction, while development tasks ('the user needs a price comparison table') specify implementation.
4. Sample Size: Qualitative Research Looks for 'Saturation', Not Statistical Formulas
Qualitative research cannot calculate sample size using power analysis — because the goal is not statistical generalizability, but understanding phenomena. It relies on saturation: when new sessions no longer bring new information, you have enough (you start hearing repetition).
| Influencing factors | Need more participants ↔ Can use fewer |
|---|---|
| Population diversity | More participants for more diverse backgrounds ↔ Fewer for a single focused segment |
| Research scope | More for broader goals (e.g., 'users' mental models of taxes') ↔ Fewer for more specific goals (e.g., 'pain points when filling out tax forms') |
| Researcher skill | Novices ask shallow probes and need more people ↔ Experienced researchers get more out of each session, patterns emerge earlier |
Operation: Start with 5–6 people(especially sufficient for initial usability test analysis), after each round ask 'Are we still learning new things?', add 1–2 more people and reassess as needed.Frequent small studies > one large complex study(diminishing marginal returns).
Special case: Card Sorting has a quantitative basis — a Fidelity study (168 participants) found: 5 people gives r=0.75 (insufficient),15 people gives r=0.90 (optimal cost-effectiveness for most projects), 30 people gives r=0.95 (large projects with sufficient budget), 60 people gives r=0.98 (very low cost-effectiveness).
5. Writing Usability Test Tasks: Qualitative Tasks Seek Openness, Quantitative Tasks Seek a Single Correct Answer
All usability studies give participants tasks, but qualitative and quantitative tasks are written in opposite ways:
| Attribute | Qualitative tasks | Quantitative tasks |
|---|---|---|
| Characteristic | Open, exploratory | Specific, focused |
| Level of detail | Moderate, enough to establish motivation | Detailed,ensure a single solution path(otherwise data will be inconsistent) |
| Randomize order | Optional | Required |
| Tasks can be modified mid-session | ✅ | ❌ (all participants must complete the same tasks) |
| Example | 'Use the website to find a recipe you are interested in' | 'Book a double room at the Chicago Hyatt Regency for 1/17–19' |
Shared principles for both: Draw tasks from real user goals (interviews/search logs/customer support/analytics data);don't use interface terminology as clues(❌ 'Sign up for Alertbox' → ✅ 'Subscribe to this website's newsletter'); emotionally neutral (❌ 'Buy a birthday gift for your mom' → ✅ 'Buy a gift for a friend'); use dummy data; pilot test first; tasks are independent and can be randomized; one success criterion per task (don't combine 'find the address and opening hours').
Task ordering also uses a funnel: start with exploratory tasks (observe natural behavior) → then goal-directed tasks → finally targeted tasks (test specific UI elements).
6. Moderating Usability Tests: 10-Step Checklist
| # | Step | Key point |
|---|---|---|
| 1 | Welcome | Don't say 'test', say 'research/research activity' to ease nervousness |
| 2 | Confirm name pronunciation | Build rapport |
| 3 | Inform participants about observers + recording | Even if mentioned during recruitment, remind them again and obtain consent |
| 4 | Sign consent form | Allow enough time to read it |
| 5 | Brief background interview (if needed) | Don't ask questions that could influence results |
| 6 | Set expectations + think-aloud | Emphasize 'we are testing the design, not you'; ask users to think out loud |
| 7 | Give one task at a time | Provide written instructions, ask users to read them aloud to confirm understanding |
| 8 | Post-task follow-up | 'What was particularly easy/difficult?' |
| 9 | Leave 5 minutes for observers to ask questions | |
| 10 | Thank participants, distribute incentives, end recording |
7. Expert Evaluation Methods Without Real Users
When budgets are limited or in early stages, expert methods can be used for an initial sweep — but none can replace user testing, they can only supplement and point directions for testing.
Heuristic Evaluation: Identify problems against a set of principles (Jakob Nielsen's 10 Usability Heuristics are the first choice).
- 3–5 evaluators assess independently(a single evaluator will miss issues, more people have diminishing returns), do not view each other's notes during independent evaluation to avoid mutual influence
- First walk through tasks as a user to familiarize yourself with the interface, then go back to find violations of principles + make recommendations
- Narrow scope and raise detail level (single task/partial interface/specific user group/particular device); keep each session to 1–2 hours
- Synthesize findings with affinity diagram clustering, rank by severity
- Iron rule: Heuristics are not laws — intentional violations are acceptable (e.g., a hamburger menu on small screens violates 'recognition over recall', but is a reasonable compromise due to space constraints), but must be validated with user research
Cognitive Walkthrough: Specifically evaluates learnability(from a new user's perspective), asking 4 questions step by step: ① Will the user try to perform the correct action? ② Will they notice the correct action? ③ Will they connect the action to the goal? ④ Will they perceive progress after acting? All four answers must be 'yes' for that step to count as successful.
- Suitable for: complex/novel/non-standard systems (hospital self-service kiosks, financial software);Not suitable: common patterns (standard e-commerce checkout, walkthroughs are too lengthy)
- Workshop format: 2–6 cross-functional reviewers (product expert/UX/engineer/domain expert) + facilitator + presenter + note-taker; inputs = user personas + core tasks + correct step sequence + interface version; ~90 minutes to evaluate about 2 complete tasks
| Heuristic Evaluation | Cognitive Walkthrough |
|---|---|
| Perspective: Analyst | Perspective: New user |
| Goal: Overall usability | Goal: Learnability |
| Scope: Comprehensive | Scope: Task-specific |
Supporting quantitative tools:
- Severity rating: Sort a long list of problems into 5 levels based on 'frequency × impact × persistence × impact on reputation' (High/Medium/Low/Bug/Good positive feedback). High = blocks critical tasks (cannot pay/register), must fix; also mark 'Good' items to maintain team morale. Use the same scale consistently long-term → builds team trust, speeds communication
- Keep expert reviews objective: Keep Opinions Out: Personal preference does not count in reviews (unless you happen to be the target user). Every problem must have objective evidence (which principle is violated/what efficiency loss occurs); if no objective reason can be found → there may be no problem → when uncertain, it is better not to raise it, instead suggest 'test it'
- Error message scoring rubric: 12 principles × 3 dimensions (Visibility / Communication / Efficiency), each scored 1–4, average score mapped to grades from A (3.3–4.0) to D (≤1.5). Empirical examples: Craigslist 2.08 (poor), Google Flights 3.17 (medium), J.Crew 3.67 (good): 12 principles × 3 dimensions (Visibility / Communication / Efficiency), each scored 1–4, average score mapped to grades from A (3.3–4.0) to D (≤1.5). Empirical examples: Craigslist 2.08 (poor), Google Flights 3.17 (medium), J.Crew 3.67 (good)
VIII. UX Workshops: Solve Problems, Don’t Just Hold Meetings
Meetings vs. Workshops: A meeting = information exchange (shallow and broad); a workshop = collaborative problem-solving for a specific issue (deep and focused, minimum half-day, high preparation cost).Don’t run a workshop if there is no clear problem to solve(it wastes time and damages credibility). Workshops follow the flow of "diverge (sticky notes / brainstorming) → converge (affinity diagram / forced ranking)".
5 types of workshops (cheatsheet): Discovery (align on goals at project kickoff) / Empathy (shift mindset from feature-first to user-first) / Design (rapidly generate diverse ideas, not final deliverables) / Prioritization (cut scope using an impact-effort matrix) / Critique (iterate solutions against criteria). They can be combined (add prioritization into discovery) or compressed (agile teams can condense one into a single morning).
7 foundational activities(all workshops are combinations of these): Post Up / Affinity Mapping / Landscape Mapping / Storyboarding / Forced Ranking / Role Playing / Playback.
Value and pitfalls of including real users: On-site sharing by users can greatly improve outcomes, but — ① do not blindly follow users (their suggestions are perspectives, not instructions,let users leave before the team makes the final evaluation); ② do not disappoint users (clarify in advance that this is brainstorming, not a final design); ③ guard against bias (do not only invite superfans or employees’ relatives; ensure diversity). If real users cannot be recruited → use proxies (researchers / customer service / sales staff).
IX. Research Ethics and Operations
Informed Consent(Historical lessons: the Nuremberg Trials and the Tuskegee Syphilis Study → spurred the creation of ethical norms). A consent form = the researcher discloses information (purpose / format / duration) + the participant understands and participates voluntarily. Content includes: purpose (do not reveal hypotheses that would influence behavior), format and duration, voluntary participation with the right to withdraw at any time, how data will be anonymized / stored / who can access it / whether deletion can be requested, compensation (distributed before tasks to avoid making participants feel their ability is being tested), and guardian signature for minors.Exceptions: A/B testing (routine product use) and intercept studies (using a simple agreement page) may use simplified consent.
Selection Criteria(Recruiting the wrong people invalidates the entire study: for example, recruiting people who do not exercise for a fitness product test).Don’t only look at demographics(Age, gender, and income cannot predict real behavior — "born in 1947, wealthy, married multiple times" could describe either Prince Charles or Ozzy Osbourne). All three categories of criteria are indispensable:
- Inclusion = who you want (specific, directly tied to goals; distinguish "suitable" from "most suitable")
- Exclusion = who must be excluded (UX professionals / developers / industry insiders — they use an expert perspective rather than an ordinary user perspective)
- Diversity = ensure the sample approximates the real distribution (when studying an airline app, you cannot only recruit first-class passengers)
- Use a recruitment matrix to balance segments × age / region; one person can satisfy multiple criteria, and once a category is filled, shift to filling other gaps
Democratize Research (5 steps): Let non-research roles (PMs / designers / PwDR) also conduct research — benefits (drives more research, improves satisfaction, deepens the team’s understanding of research); risks (low-quality research → misleads design, unclear accountability, misallocated resources, hinders UX maturity). 5 steps: ① identify needs → ② categorize resources → ③ meet needs (training + mentorship + assign by difficulty: juniors do simple tasks, seniors do complex ones) → ④ align responsibilities → ⑤ reflect and improve. Target four qualities: Viable / Influential / Sound / Efficient.
Sourced from NNGroup Topics / Qualitative Research (26 articles; the two versions of heuristic evaluation have been merged). For quantitative methods, see NNGroup Quantitative Research (A/B Testing · Sample Size · Confidence Intervals · UX Metrics · Conversion Analysis · Benchmarking · Learning Curves); for usability testing and the ten heuristics, see NNGroup Usability Testing and the Ten Usability Heuristics (Full Table of 10 Heuristics · Complex Application Use · User Pleasure Hierarchy · Testing Methods: Rule of 5 for Users / Recruitment / Remote / Task Scenarios / Step-by-Step Tasks / Attitude vs. Behavior / 4-Step Analysis / Small-Sample Errors / Competitive Evaluation); for a quick method reference, see NNGroup UX Research Method Selection (Quick Reference for 23 Methods / Four-Quadrant Positioning / Selection by Question-Stage-Budget / Five Families: Qualitative Exploration · Quantitative Validation · Expert Evaluation · IA · Behavioral Analysis); for personas produced from interviews, see NNGroup Personas (Nature / Three Types / Scope Hierarchy / Creation / Comparison with JTBD-Prototypes-Segmentation / Application / Maintenance / 5 Major Failures); complementary to Google UX Certificate / UX Research and Empathy – Define. Image/PDF/xlsx templates are retained in sources.
Sources and Related Materials
- NNGroup Quantitative Research (A/B Testing · Sample Size · Confidence Intervals · UX Metrics · Conversion Analysis · Benchmarking · Learning Curves)
- NNGroup Usability Testing and the Ten Usability Heuristics (Full Table of 10 Heuristics · Complex Application Use · User Pleasure Hierarchy · Testing Methods: Rule of 5 for Users / Recruitment / Remote / Task Scenarios / Step-by-Step Tasks / Attitude vs. Behavior / 4-Step Analysis / Small-Sample Errors / Competitive Evaluation)
- NNGroup UX Research Method Selection (Quick Reference for 23 Methods / Four-Quadrant Positioning / Selection by Question-Stage-Budget / Five Families: Qualitative Exploration · Quantitative Validation · Expert Evaluation · IA · Behavioral Analysis)
- NNGroup Personas (Nature / Three Types / Scope Hierarchy / Creation / Comparison with JTBD-Prototypes-Segmentation / Application / Maintenance / 5 Major Failures)
- Google UX Certificate / UX Research and Empathy – Define