“Scientific studies have shown. . .”
“Recent research suggests. . .”
“A study presented recently reported. . .”
These types of phrases are used daily in the popular media to report on medical and scientific studies which produce results the general population may find interesting. They’re also used regularly by those trying to convince you to buy certain products (e.g., supplements), follow certain diet plans, or take prescription drugs. Unfortunately, such references to scientific evidence are frequently misleading.
In medicine, there are many different types of studies and reports that are used as evidence to support a given treatment recommendation. The thing that most people (especially those writing media reports) don’t seem to realize is that some types of evidence are considered much stronger than others. Many times the media doesn’t seem willing or capable of distinguishing very weak evidence from very strong. Those who’re trying to sell you snake oil are much worse, and will often mislead you by using such phrases while really referring to virtually nonexistent or at least extremely limited evidence.
Some general categories of clinical medical evidence, presented from weakest to strongest, are reviewed below.
Anecdotal evidence. This is literally when recommendations are just based on the experiences of a few patients. An example would be if a handful of patients posting on a message board reported that their arthritic pain improved because they took a new herbal supplement that’s just come on the market, or when a doctor reports positive results after using a drug in a few patients for a problem for which it’s not FDA approved. This is very weak evidence.
Animal evidence. This evidence based on experimental findings in animals. While often interesting and many times spurring subsequent human research, obviously the applicability to the clinical care of humans is very limited.
Theoretical evidence.Physicians and scientists have a certain (albeit limited) understanding of human biochemistry and physiology, and a framework for explaining various diseases when something goes wrong. Based on this understanding it is sometimes theorized or assumed that a given drug or treatment shouldwork, based on our comprehension of the pathophysiology and of how the drug or treatment should interrupt that process. Sometimes these theories turn out to be true upon further testing, sometimes not. Again, this is weak evidence.
Observational evidence. Probably most medical studies fall in this category. Large epidemiological studies(such as the famous Framingham Heart Study or Nurses’ Health Study) are of this type. Such studies usually take very large groups of people and via questionnaires, lab studies, periodic physical exams, etc collect vast amounts of data on them over long stretches of time. The data is then analyzed in an attempt to answer specific questions such as “Do smokers have a higher risk of lung cancer than nonsmokers?” (answer - yes, of course) or “Do those with elevated blood pressure have a higher risk of stroke?” (again, yes).
A different research technique in this category is the case-control study. In this type of study, cases of the disease you’re studying are found and matched to “control” patients who do not have the disease, but are very similar in most other ways. You then look back in their medical histories to determine what factors are more strongly associated with those with the disease compared to those without it. These type of studies are often referred to asretrospective studies as well.
Example: say you want to research whether women who’ve taken birth control pills might have a higher risk of breast cancer than those who haven’t. You could design a case-control study to help answer this question by finding a large number of women with a diagnosis of breast cancer, then finding control subjects who don’t have breast cancer but are similar in most other respects (age, weight, smoking history, family history, etc, etc). You’d then just have to look back in the patient’s charts and determine if the women with breast cancer (the “cases”) had a statistically significantly greater chance of having been on birth control pills at some point in the past than the matched women without breast cancer (the “controls”).
It’s extremely important to understand that observational studies suggest correlation, NOT necessarily causation. When you find a correlation, it could mean that one factor causes the other, or it could mean that some other unknown factor you didn’t measure caused the first two to correlate. The classic example would be the observation that the number of ministers in a city correlates with the number of liquor stores. Presumably, this is not because one causes the other but because a third variable (city population) causes the two to correlate. Bigger towns have more ministers and liquor stores, smaller towns fewer. The size of the town is the causative variable causing the first two to correlate.
Observational evidence is also subject to many types of biases and flaws in study design which can produce pretty bizarre and conflicting study results at times. The individual studies have to be closely reviewed to determine their quality and relevance. It usually takes several studies of this type with similar results to convince physicians that the conclusion is valid. Sometimes such multiple similar studies of this type are grouped together and the results pooled to produce a more robust conclusion(this is called a meta-analysis).
For these reasons, this type of evidence would have to be considered only perhaps moderately strong. Many times isolated studies of this type make a splash in the media. Often they’re poor quality studies, without other studies to corroborate their findings, so take these reports with a grain of salt. Such studies often grab a lot of headlines and cause concern among many people unfortunately.
Experimental evidence. These are truly strong studies designed to evaluate a question prospectively. They are similar to case-control studies described above in that there is generally a control group which is compared to a study group. However, these studies are designed by recruiting large groups of people who all meet the study selection criteria who are then randomly assigned to receive the test or treatment being studied, or a placebo such as a sugar pill. In the best such studies, neither the subjects nor the researchers collecting the data know which group is receiving the treatment and which are on placebo until the study is over. This prevents certain types of bias from being introduced which could corrupt the results. These are called double-blinded, placebo-controlled, randomized clinic trials and are really the gold standard for clinical evidence.
An example would be let’s say you wanted to know if diabetic men aged 35-65 with certain cholesterol levels (hint: those would be your selection criteria for potential participants in the study) reduce their rate of heart attack by taking statin-type cholesterol lowering drugs. You’d find a large group of willing patients who fit those criteria, then give them a numbered bottle of pills, and neither you nor the patient would know whether they’re taking the actual drug or a sugar pill. You’d then monitor them for a period of time, and after all the data is collected and the study is over, you’d analyze whether the study group had a statistically significant (meaning unlikely to be due to chance) lower rate of heart attacks than the control group.
This type of study is considered very strong evidence. Unfortunately, not all questions easily lend themselves to being answered by such a study design, and it takes a great deal of money to run such a study. But when such evidence exists for a given treatment, it is considered the strongest available.
In conclusion, I hope that if you bothered to take the time to read this in the future you’ll stop and consider the quality of the evidence being presented when you read “A recent scientific study has shown. . .” in a popular media article. Always try to consider the strength of the “proof” you’re being presented, and discuss anything reported in the media with your physician to get a better perspective before you form a strong opinion or take action.
No comments:
Post a Comment