A Cochrane review has confirmed that some research results are more likely to get published than others, which is one reason why evidence may not be as reliable as some people think.
As the authors noted in their summary, “More than half of results from abstracts, and almost a third of randomized trial results initially presented as abstracts fail to be published in full and this problem does not appear to be decreasing over time. Publication bias is present in that ‘positive’ results were more frequently published than ‘not positive’ results. Reports of methodology research written in English showed that a higher proportion of abstracts had been published in full, as did those from native English-speaking countries, suggesting that studies from non-native English-speaking countries may be underrepresented in the scientific literature.”(Scherer et al 2018).
As Scherer et al (2018) also state, “The consequence is that systematic reviews relying on fully published research may provide inaccurate or biased findings because of an over-reliance on studies with positive results or from English-speaking countries.”
Some of the key results, which are based on analysis of 425 relevant reports, involving 307,028 abstracts, highlight other factors which can mean that research is more or less likely to get published. These results include that:
1. Less than half of all studies, and about two-thirds of randomized trials, initially presented as summaries or abstracts at meetings, are published as journal articles in the 10 years after presentation.
2. Studies with positive results are more likely to be published.
3. Studies with larger sample sizes are more likely to be published.
4. Studies with abstracts presented orally are more likely to be published than those presented as posters.
5. Studies accepted for presentation at a meeting are more likely to be published than those not accepted.
6. Studies describing basic science are more likely to be published that those describing clinical research.
7. Studies describing randomized trials are more likely to be published than those describing other types of studies.
8. Studies that took place in multiple centers are more likely to be published than those at a single center.
9. Studies classified as ‘high quality’ are more likely to be published than ‘low quality’ studies.
10. Studies with authors from an academic setting are more likely to be published than those with authors from other settings.
11. Studies considered by the report authors to have a high impact are more likely to be published than other studies.
12. Studies with funding source reported are more likely to be published than those not reporting funding.
13. Studies originating in North America or Europe are more likely to be published than those originating elsewhere.
14. Studies from English-speaking countries are more likely to be published than studies originating elsewhere. (Scherer et al 2018).
There are a number of reasons that I am sharing this review, and one of the main ones is to highlight the way in which is it important to understand that, while research results are a valuable form of knowledge, they are also subject to a number of different kinds of bias. That doesn’t mean that we should disregard them; it means that we need to consider them within the light of their limitations, within the wider context of knowledge and culture and with an understanding of how knowledge is shaped by other factors and forces. If we do that, we can then use evidence and other kinds of knowledge more effectively to make decisions.