Which statistical test does my data need?
Four questions about your outcome and your design. You get a named test, the assumptions worth checking before you run it, and what to switch to if those assumptions do not hold.
Your recommended test
Assumptions to check first
Also consider
Free: the Methodology Selection Decision Tree
The test is the last decision, not the first. This decision tree walks back up to the methodology choice that determines it. We’ll email you the PDF.
Why the test follows from the design, not the other way round
A recurring problem in doctoral analysis chapters is choosing the test after seeing the data. It feels efficient and it quietly invalidates the inference: once you have tried several tests and reported the one that reached significance, your stated p-value no longer means what it claims to mean.
The defensible order is the reverse. Your research question fixes the analysis goal. Your operationalisation fixes the outcome variable type. Your design fixes the number of groups and whether observations are paired. Those four facts are settled at proposal stage — which means your analysis plan can be written, and pre-registered or agreed with your supervisor, before a single participant is recruited.
- Analysis goal — comparing groups, testing a relationship, predicting an outcome, or comparing one sample against a known value.
- Outcome type — numeric, ordinal, categorical, a count, or time-to-event. This constrains the test more than anything else.
- Number of groups — two, or three and more, which moves you from t-tests into ANOVA and its non-parametric equivalents.
- Paired or independent — the same participants measured twice, or different participants in each group. Getting this wrong is the single most common error in submitted analysis chapters.
The recommendation is a well-founded starting point, not statistical advice about your specific dataset. Check the listed assumptions, and for anything high-stakes confirm with your supervisor or a statistician before you write up.
Frequently asked questions
What if my data are not normally distributed?
Use the non-parametric alternative the selector names alongside each recommendation — Mann-Whitney U for an independent two-group comparison, Wilcoxon signed-rank for paired, Kruskal-Wallis for three or more. That said, with reasonably large samples the parametric tests are quite robust to non-normality; it is small samples with clear skew where the switch really matters. Note that these tests answer a slightly different question, about distributions rather than means, and your write-up should reflect that.
How do I know whether my groups are paired?
Ask whether each observation in one group has a specific partner in the other. The same participants measured before and after an intervention are paired. Participants matched one-to-one on age and sex are paired. Two separate cohorts of different people are independent. Analysing paired data as independent throws away the design's main advantage and usually loses you power.
Can I run several tests and report the one that comes out significant?
No — this is the textbook route to an unreproducible thesis. Each additional test inflates the chance of a false positive, so a result selected from many is far more likely to be noise than the reported p-value suggests. Decide your primary analysis in advance, and if you genuinely test multiple outcomes, correct for it and say so.
The recommendation is not what my supervisor suggested.
Take the discrepancy to them rather than around them. Usually one of three things is going on: your supervisor knows a field convention the general logic does not capture, your design has a feature the four questions do not cover such as clustering or repeated measures, or one of you has read the design differently. All three are worth a conversation, and the second is worth a statistician.
Does this tool analyse or store my data?
No. It asks about the structure of your study, not its contents. Nothing is uploaded, and the whole thing runs in your browser.