This is clearly a case of great minds thinking alike. I started writing this post earlier in the week, and just saw that Jack Norris posted an article on the same topic yesterday. If you're at all interested in this topic, I hope you’ll read both; his is a more in depth discussion and this one is a faster overview.
It’s an issue that comes up frequently in the comments section of this blog—how we analyze and use nutrition research. In order to draw conclusions about diet and health, we need to look at all of the research on a certain topic, evaluate the strengths and weaknesses of the studies and then see what most of the research is saying. And even then, conclusions are sometimes not much more than an educated guess.
Two important considerations in all of this are type of study and the issue of bias.
Different types of studies carry different weight
Aside from the ethical considerations, findings from animal research often don’t translate very well to what happens in humans and should always be considered weak evidence. But even among human studies, some types are better than others.
Ecological studies, which compare disease rates among populations (rather than among individuals) are generally used only to generate hypotheses about diet and health, not to make conclusions. This type of research doesn’t control for the many factors that might affect outcome, and makes assumptions about how pooled data from a population translates to effects in individuals (a weakness known as the “ecological fallacy.”) The study that compared rates of hip fractures to protein intake among different populations is one example of ecological research. While the results suggest that high protein intakes are linked to risk for bone fractures, that’s only one of many possible explanations. In fact, in the case of this study, the findings can be explained away by genetics, geography, lifestyle, and other nutrition factors.
It doesn’t mean that ecological studies have no value. But their value lies in raising questions that should be explored by other (better) research, not in proving relationships. Nutritionists never (or at least they shouldn’t ever) make dietary recommendations based on ecological studies.
More reliable information comes from epidemiologic research that compares findings from individuals (as opposed to groups of people) and includes control groups. Two examples are the Adventist Health Study and the EPIC-Oxford Study which have both produced a great deal of information about vegetarian—and sometimes, vegan—diets and health. These epidemiological studies show only associations, though, not cause and effect.
The best types of studies are randomized controlled trials (RCTs) in which researchers feed a particular diet or food or food component to people and compare the effects to a control group. If they are well-designed, these studies do show cause and effect. Ideally, the study is double-blinded which means that the subjects don’t know if they are receiving the “treatment” or the placebo (this isn’t always possible, of course) and the analysis of the data is blinded as well. While RCTs are the gold standard in nutrition research, they’re expensive so they tend to be shorter and smaller than ideal. An example of an RCT that is familiar to most vegans is the research by Dean Ornish in the early 1990s.
Findings need to be published in peer reviewed journals to be meaningful. This means that other professionals scrutinize the research design and analysis. It’s not a perfect system, but it adds an element of oversight that increases the chances that what gets published is meaningful. It’s interesting and thought-provoking when physicians and other health professionals write books or internet articles about their own experiences with their patients, but it’s not research. Without an actual research design, a control group, statistical analysis and peer review, the findings have limited value.
Bias is hard to avoid
Academics scramble for funding wherever they can find it and the food industry has lots of money. This doesn’t affect the data, though; researchers can’t change their findings to accommodate the industry that supported their research. It might affect what the public hears about different topics, though, since the funding industry can press release the findings—or not, if they don’t like them. And even the way researchers talk about their study could reflect bias.
Those of us who interpret research findings for non-nutritionists have to work hard to avoid bias as well—because we all bring some to our work. And, because there is so much research, it’s pretty easy to pick and choose the studies that support one’s particular bias and viewpoint. That’s why a list of citations at the end of an article is not proof that the content is reliable and balanced. Professionals who have built a career on a particular type of diet may gravitate toward research that supports that diet and discount the studies that don’t. Those of us who promote a vegan diet for ethical reasons obviously need to be extremely careful about how that bias affects interpretation of studies.
Evaluating the source of nutrition information
So how do you know whether someone is truly knowledgeable about a nutrition topic or if they are biased in their interpretation of the studies? It’s not all that easy to tell but a few guidelines can help. Generally, those who share nutrition information should be willing to express a little bit of uncertainty regarding recommendations. If somebody tells you that they know the one and only healthful way to eat, that’s something that should raise a red flag or two.
Nutrition experts also have to be willing to go with the flow a little bit and change their perspective and admit to being wrong or to not knowing an answer. My take on vegan nutrition is very different today than it was 20 years ago because new research has made me discard some beliefs. In this field, if you never admit to being wrong, you’re clearly not keeping up with the research!
With vegan nutrition in particular, experts shouldn’t minimize potential concerns. There are some challenges in vegan nutrition and, rather than ignoring them, we have to admit that they exist and find the best ways to deal with them. It may not always present veganism in the best light, but staying true to the science definitely provides the best information to help vegans stay healthy.
Excellent post! This is such great info. Two questions for you: 1) Where would you put the China Study as a nutritional research study? and 2) what beliefs in particular have you discarded due to previous research data?
The China study is ecological research.
A few things that come to mind regarding beliefs I've discared over the years–I used to promote very low fat diets but the evidence no longer supports that recommendation. And I used to think that vegans probably had lower calcium needs than omnivores but newer research casts doubt on that. I know there are others. I'll add a comment if I think of them!
[…] Messina shows how to identify bad nutrition arguments, both for and against veganism. There’s a lot of pro-vegan stuff that gets repeated within our […]
Ah, that's what I figured when I read your description. Thanks for that.
Yea, the latest calcium research is interesting to me, too. Glad to hear your take on it, and thanks fr the fast reply. 🙂
So I take it that you aren't a fan of Esselstyn's no added fats reversal diet? I think that unless you are knocking on death's door, this diet would be most people's last resort. Personally, I'd rather eat cardboard.
LOL Robert! Actually, the only controlled study to show reversal was Ornish, who also had no added fats or higher fat foods in his diet. But, as I've talked about here before, there is nothing in that study to indicate that the low-fat aspect of the diet is the reason for the reversal. And based on what we know about fat and heart disease, it's extremely unlikely that ultra-low-fat is needed for a healthy heart.
I agree with you–those diets are so hard to stick to that I think we end up losing a lot of potential vegans who think this is what vegan food tastes like!
Despite the fact that I worked for almost 20 years as a medical librarian, gathering refereed medical literature in response to clinicians' information needs, I still need reminding about the validity and reliability of nutrition research. I need to lose weight, and am swayed easily, even by anecdotes by people who are slimmer than me. I have tried the extreme low fat diets that are promoted by PCRM, McDougall and Esselstyn, but find I am always hungry. But, all it takes is hearing the latest success story, and I go there again, only to fail again. I look forward to your book, Ginny, since everything I read here is reasonable and evidence-based.
Thanks, Linda. I think it's easy for everyone–no matter how much they know–to be swayed by attractive anecdotal stories.
Oops. I wanted to reply to the thread which Robert started on the palatability of low-fat vegan food. Unfortunately, all I can post is the word 'null.' I agree with Robert, since a little bit of oil can go a long way to carry flavors and enhance textures. McDougall is right when he says drinking from a bottle of oil is disgusting, but take a teaspoon or two to saute onions and garlic or temper a curry–YUM!
I need to get some help with that "null" issue. It comes up for me in firefox, but not explorer. Not sure what it's all about.
Anyway, I agree–food doesn't need to be (and shouldn't be) swimming in oil, but a little bit can help flavors a lot. Some people are happy with very low-fat food, and that's fine. But for those of us who aren't, there's no reason why we can't use higher fat foods like nuts and a little bit of oil.
I love this post. Thank you, Ginny! I suppose you are well-aware of the papers by Ioannidis that have gained a lot of attention in the mainstream press and which claim that it is more likely for a published research claim to be false rather than true. I think the most famous of these papers is the following:
Ioannidis JPA, 2005 Why Most Published Research Findings Are False. PLoS Med 2(8): e124. doi:10.1371/journal.pmed.0020124
I am wondering if you can comment on whether this phenomenon reported by Ioannidis is more pervasive in publications on pharmaceutical drugs or if it is just as observable in published nutrition research as well.
Interesting. No, I haven't seen that paper but will definitely read it. Does it say that most research results are eventually found to be wrong by further research?
I'm looking for a filler base for a swiss chard wrap and would like to use these. I would like to confirm that hummus and baba ganoush are both vegan. Thanks!
Both should be vegan, and I've never seen hummus that wasn't. However, I've seen brands of baba ganoush that have added dairy ingredients. So check the labels!
[…] Comments Ginny Messina shows how to identify bad nutrition arguments, both for and against veganism. There’s a lot of pro-vegan stuff that gets repeated within our […]
Thanks for this post, Ginny. I've been vegan for just over a year, and I've learned a lot from your blog. I'm wondering if you've seen the documentary "Forks Over Knives" and if so, what you think about it. It's coming to a theater in my area this weekend, and my vegan meet-up group has scheduled an outing to go see it.
Carla, I live in the boondocks, so it hasn't been shown anywhere near me. I'm sure I'll eventually see it on DVD. Will be interested to know what you think.
[…] Guide for Vegans Understanding Nutrition Research: A Quick Primer Is It Safe to Eat Soy? Do Ex-Vegans’ Stories Make the Case Against Vegan […]
[…] us fat and he notes that BMI has increased as wheat consumption has increased. But this is an ecological correlation, a type of finding that carries very little weight in evidence-based nutrition. Because ecological […]