Sample Size and Common Sense

If you start digging at psychology research, you're very likely gonna be extremely disappointed at how low the sample sizes are.

For example, this video on the differences between reading through books or audio content cites articles with sample sizes which average 30 people.

Regardless of the researchers' reasons, coming up with overarching conclusions about our behavior based on poking at so few people is borderline clownish behavior, being replicated for decades in that specific field as well as others.

Of course, those researchers are budget-constrained. And, statistically, through variance analysis, their sample sizes are valid. But that won't trump common sense. Nobody should change their point of view on any aspect of life based on research with people numbering on the dozens or even hundreds, however insignificant the topic might be.

Some years ago, in the Harvard data scandal with professor Francesca Gino, the most baffling aspect of the whole drama was to me how small the samples were for a researcher from Harvard, in the range of low hundreds. It felt as if she had made quiz for her students and then turned it into a priming behavioral law she could use for her TED talks.

And that's not to mention how psychology research in the West is extremely biased by college students. Are college students exemplary of anything? Are they known for representing the population as a whole? Are they particularly wise or special? No, they are just the human equivalent of a lab rat in the US, though much cheaper actually, they even pay to be there.

Anyways, academic research is not supposed to be overarching when it comes to individual scientific articles, at least not usually. Those articles build on each other; over time, through replication and growing sample sizes, hypotheses become stronger, and reliable conclusions might be drawn out eventually. But that's not how research is often presented on media, be it for the researchers' or the media outlet's benefit.

Unfortunately, we haven't yet been educated to investigate research methods. But I have faith that might change soon, given how open and accessible information has become, with the internet, and especially YouTube. It might only take one viral video on the topic to open the flood gates to a revolution of awareness on this topic.