How to read a data analysis you did not run yourself
You do not need to be a statistician to interview a results section. Five questions will surface most of the problems.
What was measured, exactly
Ask for the definition of every variable in plain language. A surprising share of confusing results comes from a measure that does not mean what the reader assumes it means, such as an active user defined over a window that quietly changed mid-study.
Who is missing from the data
Response rates, dropouts and excluded records shape the answer more than the model does. Ask who was eligible, who responded, and how the excluded differ from the included.
How large is the effect, in units you care about
Statistical significance says an effect is unlikely to be noise. It says nothing about whether the effect matters. Insist on the effect size in real units: pounds, days, percentage points, customers.
What else could explain this
Ask what alternative explanations were considered and what was done about them. A results section that names its own competing explanations is usually more trustworthy than one that does not.
More Reading
How to scope a feasibility study before you spend anything
Most feasibility work fails at the scoping stage, not the analysis stage. Here is the sequence we use to define a question that evidence can actually answer.
18 August 20266 min read
ReadChoosing a research method that fits the question you have
Qualitative or quantitative is the wrong first question. Start with what kind of claim you need to make, and the method usually chooses itself.
29 July 20265 min read
ReadWhat makes a business plan fundable rather than merely complete
Lenders and investors read plans in a particular order and look for a particular kind of honesty. Here is what they check first.
8 July 20267 min read
Read