A health food store owner is cryogenically frozen and revived two centuries later. To his surprise, he learns that the steak, cream pies, and hot fudge he once avoided as unhealthy have turned out to be anything but. The film, of course, is Woody Allen’s 1973 “Sleeper,” and his cinematic send-up highlights a serious problem that has long haunted biomedical science and especially nutrition research—namely, that a surprising number of conclusions are based on very thin evidence, and many are not only unreliable but flat-out false.

　　Some of these problems were highlighted in a recent Law and Liberty essay by Theodore Dalrymple entitled “The Fraudulent Laboratory.” Scientific dishonesty does indeed pose a real threat to the credibility of research, but fraud represents only the tip of the iceberg. Fraud and dishonesty imply an intent to deceive, but the rabbit hole of unreliability in research goes far deeper still, to the point that many findings are false despite no deliberate deception on the part of their authors. Many studies are contaminated by biases of design and analysis of which the investigators themselves are unaware.

　　Consider heart disease. In the 1960s, experts began recommending that Americans cut back on dietary saturated fats and cholesterol, which they promoted as the principal culprit behind heart disease, the nation’s number one killer. However, reducing fats and cholesterol did not improve the situation. In fact, there is little robust evidence that low-fat diets improve health, and when the US bought into this approach, it began developing an obesity epidemic, with about 40 percent of adult Americans now qualifying as obese.

　　Fifty years ago, it may have made sense to scientists who found cholesterol-containing plaques choking coronary arteries to place the blame for heart disease on excessive dietary cholesterol consumption. But this reasoning is simplistic in the extreme. For one thing, Ludwig Feuerbach’s dictum that we are what we eat is wrong. We do not, for example, become more bovine when we consume beef. Nor does consuming a low-fat diet appear to lower heart disease risk. In fact, the far greater threat appears to arise from simple sugars, which one nutrition researcher labelled “pure, white, and deadly.”

　　In this case, while there was no attempt to falsify data, the intent to obfuscate seems to have played an important role. An influential 1965 New England Journal of Medicine review article found that fat and cholesterol were the principal dietary culprits in coronary artery disease. Only much later did it become clear that this research had been funded by the Sugar Research Foundation, whose primary aim seems to have been to exonerate sucrose as the culprit. If peer reviewers and readers had known the funding source, they might have subjected the report to greater scrutiny.

　　Yet the deeper problem is not so much a deliberate attempt to mislead but the less-than-robust methods underlying nearly all nutrition research. John Ioannidis, MD DSc, a highly regarded researcher at Stanford University who leveled early criticisms at ventures as diverse as the now-defunct Wall Street darling Theranos and widespread COVID lockdowns, has helped to explain why it is difficult to base nutritional recommendations on truly rigorous research. Perhaps his best-known article, published in 2005, is “Why Most Published Research Findings Are False.” Summarizing his conclusions, he writes:

　　A research finding is less likely to be true when the studies conducted in a field are smaller; when effect sizes are smaller; when there is a greater number and lesser preselection of tested relationships; where there is greater flexibility in designs, definitions, outcomes, and analytical modes; when there is greater financial and other interest and prejudice; and when more teams are involved in a scientific field in chase of statistical significance. Simulations show that for most study designs and settings, it is more likely for a research claim to be false than true. Moreover, for many current scientific fields, claimed research findings may often be simply accurate measures of the prevailing bias.

　　Notwithstanding decades of study, there is still a great deal that we think we know but do not. One difficulty arises from the fact that so many nutritional studies lack randomization, which means that any results are subject to confounding factors. Differences in outcome between two groups that we attribute to diet may in fact be explained by something else—factors such as smoking, alcohol consumption, or exercise. Not surprisingly, people who make an effort to consume only healthy foods may be taking care of themselves in other ways that get credited to diet.

　　This problem stems from the fact that most nutritional studies are observational, not experimental. They rely on self-reported diet and health outcomes, erroneously attempting to conclude that the latter can be attributed to the former. The popular media are part of the problem in the sense that they are happy to report such results, even though their scientific basis is weak. So long as observational studies continue to draw such widespread attention, randomized experimental trials are likely to languish.

　　Another problem is even more fundamental—the difficulty of assessing diet in any rigorous way. Many studies are too dependent on recall, asking people what they have been eating over a long period of time, which is subject to many sorts of bias. When questioned by a researcher, many of us may tend to unconsciously downplay our dietary indiscretions and overplay the soundness of our nutritional choices. This applies not only to what we eat but how much we eat of many different types of food.

　　Like all human endeavors, science is subject to bias, and this very liability constitutes a blind spot for many people, both outside and inside the scientific community.

　　Still another problem concerns the tendency to focus excessively on single nutrients, such as protein, vitamin D-containing foods, or cruciferous vegetables. Ioannidis suggests that the overall role of any single food type or nutrient in accounting for human health is relatively small. It is likely that a person’s overall diet exerts far greater influence, yet researchers often persist in focusing on individual constituents. In many cases, the “noise” from other factors likely overwhelms the “signal” of the dietary ingredient of interest.

　　Ioannidis likens the long-burgeoning numbers of poorly designed, unreliable nutritional studies to a pandemic. Instead of lots of little studies that attempt to answer a panoply of questions, fewer well-targeted studies are needed. While this would diminish the number of nutritional studies and the researchers producing them, it would also likely reduce the overall costs of nutritional research and provide far more reliable conclusions. Nutritional advice should be based on robust science, not competing opinions.

　　Of course, nutrition is not the only area in which findings are questionable or worse. Ioannidis argues that similar problems bedevil other fields such as neuroscience and oncology. Summarizing the full extent of the problem, he has described what he calls the “medical misinformation mess”:

　　First, much published medical research is not reliable or is of uncertain reliability, offers no benefit to patients, or is not useful to decision makers. Second, most healthcare professionals are not aware of this problem. Third, they also lack the skills necessary to evaluate the reliability and usefulness of medical evidence. Finally, patients and families frequently lack relevant, accurate medical evidence and skilled guidance at the time of medical decision-making.

　　Perhaps the most fundamental problem centers on the rationale underlying such studies. In many cases, the intent is not to elucidate the truth but to advance an agenda—for example, to boost profits for a pharmaceutical or medical device company or to advance a researcher’s career. Ioannidis characterizes many successful researchers as “managers absorbing more money.” If a scientifically valid discovery or innovation generates revenue, so much the better, but revenue should not be permitted to bend science.

　　Scientific findings are not necessarily true simply because they are backed by a large data set, have been subjected to complex statistical analysis, or have been published in a peer-reviewed journal. Like all human endeavors, science is subject to bias, and this very liability constitutes a blind spot for many people, both outside and inside the scientific community. We all need to recall that science, at its core, is not a body of irrefutable received facts, but one means among others by which we pursue knowledge. And because it is a human endeavor, it is inevitably subject to human bias.

　　Nutrition research funded by the sugar industry warrants the same scrutiny as studies of the health effects of cigarette smoking underwritten by the tobacco industry. With large sums of money on the line, different and more self-serving questions can be asked, research methods can be tweaked, analyses can be skewed, and results can, where unfavorable, be suppressed or spun in directions deemed to be more advantageous to the funder. One thing is certain—that low-fat yogurt is not so healthful as we have long been led to suppose, especially if it is loaded with sugar.