The other day I came across a very intriguing research paper, bearing a very provocative title: “Why Most Published Research Findings are False”. Published in 2005, this paper was written by John P.A. Ioannidis, a medical professor specializing in epidemiology. His claim is simple (I quote from the abstract of the paper):
The probability that a research claim is true may depend on study power and bias, the number of other studies on the same question, and, importantly, the ratio of true to no relationships among the relationships probed in each scientific field. In this framework, a research finding is less likely to be true when the studies conducted in a field are smaller; when effect sizes are smaller; when there is a greater number and lesser preselection of tested relationships; where there is greater flexibility in designs, definitions, outcomes, and analytical modes; when there is greater financial and other interest and prejudice; and when more teams are involved in a scientific field in chase of statistical significance.
Now, as you might imagine, a scientific paper with such a subject matter would be sure to attract a lot of attention, both positive and negative, and that was indeed the case. But I do not intend to participate in the debate, and this is not the reason I am bringing up this paper here.
I am more interested in the concept of truth, especially in the way it is employed in papers such as Ioannidis’, i.e. in current scientific research.
The probability that a research claim is true may depend on study power and bias, the number of other studies on the same question, and, importantly, the ratio of true to no relationships among the relationships probed in each scientific field. In this framework, a research finding is less likely to be true when the studies conducted in a field are smaller; when effect sizes are smaller; when there is a greater number and lesser preselection of tested relationships; where there is greater flexibility in designs, definitions, outcomes, and analytical modes; when there is greater financial and other interest and prejudice; and when more teams are involved in a scientific field in chase of statistical significance.
Now, as you might imagine, a scientific paper with such a subject matter would be sure to attract a lot of attention, both positive and negative, and that was indeed the case. But I do not intend to participate in the debate, and this is not the reason I am bringing up this paper here.
I am more interested in the concept of truth, especially in the way it is employed in papers such as Ioannidis’, i.e. in current scientific research.
RSS Feed