NNGroup Analytics & Metrics (Choosing the Right Metrics / Conversion Funnels / Statistical Significance and Confidence Intervals / Four Steps to UX ROI / Sampling and Confounding / Triangulation / Vanity Metrics / Benchmarking)
22 curated pieces from the Nielsen Norman Group analytics and metrics knowledge base. Centered on the core theme of "how to use data to prove the value of UX", it clarifies four key topics: ① Choosing the right metrics (avoid Vanity Metrics — isolated numbers that "only go up and never down" such as total downloads, total page views, and total user count; replace them with ratios/proportions, time frames, user segment breakdowns, and the DAU/MAU Stickiness Ratio. Conversion Rate = the most important metric, but it is affected by external interference from price, seasonality, and marketing. Macro vs Micro Conversions = process milestones plus secondary actions. Success Rate (for usability testing, when user intent is unclear) vs Completion Rate (for analytics tools, when user intent is clear). For time on page/page views, "more ≠ better"; you must distinguish between productive tasks and engaging tasks); ② Division of labor between two quantitative methods (Analytics answers "what users did" — large scale, low cost, no context; Quantitative Usability Testing answers "whether users can complete tasks" — ~40 participants, expensive, supports causal inference; the two are complementary); ③ How to calculate UX ROI (four steps: collect UX metrics → select KPIs → convert to monetary value → report responsibly. Health insurance case: 23,000 → 1,100 tickets × $6 = $131,400 saved per month / $1,576,800 saved per year / $4,655,400 return over three years; design system case: calculating just one video plugin was enough to persuade stakeholders; three major misconceptions = ROI is not only about money / does not have to be exact / does not cover every detail; Don't Overthink); ④ Statistical rigor (Statistical Significance vs Practical Significance, example of 72s vs 75s; Confidence Intervals at 95%, e.g., 4.3min → 3.6–4.9min, with sample size and consistency determining interval width; decision-making when results are not significant = 60% maximum likelihood probability plus qualitative insights; Benchmarking requires ≥20 participants, conducted annually, to quantify business value). Supplemented by content on Confounding Variables (randomization, within-subjects design, age, seasonality, pandemics, competition, experience, prior opinions), Convenience Sampling vs Probability Sampling (quantitative studies ≥40, surveys ≥100, simple random vs Stratified), Triangulation (cross-checking multiple methods/metrics to improve credibility), Statistically-Generated Personas (qualitative foundation + ≥100 participants, ideally 500+, latent class analysis), and Long-Tail Data (power-law distribution complemented by the Pareto principle). Intended audience: product managers.