Tool
How Strong Is Your Research Evidence?
Research Evidence Strength Calculator
Describe one research insight and get a graded assessment of the evidence behind it. Eight questions, about three minutes. Everything is scored in your browser.
What is evidence strength?
Evidence strength is a rating of how confidently a research insight can be acted on, based on how many independent sources support it, how many different kinds of evidence point the same way, whether the people studied represent everyone the decision affects, how recent the data is, and whether anything contradicts it.
Why grade evidence at all
An insight is a claim. The evidence behind it is what makes the claim actionable, and the two separate almost immediately. Six months after a study, someone quotes a finding in a roadmap discussion and nobody remembers whether it came from eight interviews or one offhand remark in a sales call.
Grading evidence at the moment you record it means the claim carries its own confidence level wherever it travels. That matters most when you aren't in the room to qualify it.
The six factors
| Factor | Weight | What it measures |
|---|---|---|
| Method diversity | 25 | How many distinct kinds of evidence point the same way |
| Independent sources | 20 | How many separate evidence streams, not separate participants |
| Segment coverage | 20 | Whether you heard from everyone the decision affects |
| Recency | 15 | How much the product and market have moved since |
| Qual-quant agreement | 15 | Whether your evidence types tell the same story |
| Contradicting evidence | 5 | Whether you looked for disconfirmation, and what you found |
Method diversity (25 points)
Scored by method family, not by method. Interviews, usability tests and field studies are one family; surveys, product analytics, and support or sales conversations are three more.
- One method family6
- Two method families16
- Three or more method families25
Independent sources (20 points)
Separate studies, datasets or evidence streams — not separate participants within one study.
- One4
- Two or three13
- Four or more20
Segment coverage (20 points)
- Fewer than 5 participants, or a single segment5
- 5–15 participants covering the main affected segment12
- Enough participants across every segment the decision affects20
Recency (15 points)
- Under 3 months15
- 3–6 months12
- 6–12 months8
- 12–18 months4
- Over 18 months0
Qual-quant agreement (15 points)
- They agree15
- I only have one type6
- They partly disagree3
- They disagree0
Contradicting evidence (5 points)
- None found, and I looked5
- Found some, but it's explained (different segment, superseded, measurement artefact)4
- I haven't looked0
- Yes, and it's unresolved0
Grades and caps
Strong 75–100. Moderate 55–74. Emerging 35–54. Anecdotal 0–34.
A cap sets a ceiling regardless of score. A single source caps at Moderate. A single method family caps at Moderate. Both together cap at Emerging. Unresolved contradicting evidence caps at Moderate, and so does qual-quant disagreement.
Caps exist because points alone reward volume. An insight built on four recent, well-covered interviews can score highly while still resting entirely on one way of asking. The cap says so.
What this rubric gets wrong
Weights are judgment, not measurement. They reflect how I weigh evidence after years of research practice as a CX/UX researcher in big tech. Someone else would weigh them differently and wouldn't be wrong.
Recency is context-blind. Enterprise B2B findings last longer than consumer ones, and a fixed scale treats them the same.
Sample size is crude. Five participants in a well-designed study can beat fifty in a bad one, and the rubric can't see study quality.
It scores what you can count. The thing that most determines whether research is right — whether you asked the correct question — isn't in here at all.
Common questions
How do you measure the strength of research evidence?
By checking five things: how many independent sources support the finding, how many different research methods point the same way, whether the people studied cover everyone the decision affects, how recent the data is, and whether anything contradicts it. A finding backed by interviews, analytics and support tickets is stronger than one backed by three rounds of interviews, even if the second involved more people.
How many participants do you need for a research finding to be reliable?
Fewer than most people assume for discovering problems, more than most assume for deciding between options. Five to eight participants in one segment will surface most usability problems. But if a decision affects three segments, five participants from one of them tells you about that segment, not about the decision. Coverage matters more than raw count.
What is triangulation in UX research?
Using different kinds of evidence to check the same question. If interviews say users abandon onboarding because it's confusing, analytics should show where they drop out, and support tickets should mention the same step. When those agree, the finding is probably real. When they disagree, the disagreement is itself the finding.
How old is too old for research findings?
It depends on how fast the product and market move, but anything over 18 months deserves a re-check before a decision rests on it. Consumer findings decay faster than enterprise ones. The question isn't the date, it's whether anything has changed that would make people answer differently now.
What's the difference between an insight and a finding?
A finding is what you observed. An insight is what you concluded from it. "Seven of ten participants stopped at the workspace step" is a finding. "Users abandon onboarding because workspace setup asks for decisions they can't make yet" is an insight. Insights are more useful and less certain, which is why they need a confidence rating attached.
How should I record research evidence so it stays useful?
Keep the insight separate from the evidence that supports it, and link them. A document that merges both loses the ability to answer "how much evidence does this rest on" six months later. Record the method, sample, date and confidence level alongside the claim itself.
Where grades go to live
A grade is only useful if it travels with the claim. Research Repository OS stores insights with their evidence strength, linked to the studies and participants behind them, so the confidence level survives the next reorg.
Research Repository OS on Notion Marketplace ↗