Kumquats Found to Prevent Cancer!
Exercise + eggplant = longer life!!

Obviously, these "headlines" are fictitious, but they got your attention, didn't they? Reports of new research findings in the area of food and health grab the attention of Americans from all walks of life. After all, food and health are central concerns to each of us. Reporters, professors, and health professionals all want the latest information—as do consumers. A single study and the reports surrounding it can send crowds of people in search of the latest food or supplement that holds promise for good health. Remember how virtually any food containing oats or oat bran practically flew off supermarket shelves in 1990 after an oat bran-cholesterol study was publicized?

Frustrated and confused by the tremendous amount of food and health information being communicated today, Americans look for simple certainties to help them protect their health through diet. The trouble is that single studies rarely provide such certainty, although they often make for great headlines.

The media, health professionals, and educators are the gatekeepers of today's food and health information. They determine, for the most part, what consumers hear, read, and believe about food and health. Along with that comes the responsibility to provide the facts, put them in perspective, and help people determine how the findings may affect their behavior and lives. Fulfilling this responsibility requires that new studies be critically reviewed before being publicized. News releases and study abstracts, although helpful for "previewing" research, do not provide the information necessary to accurately and responsibly report findings to the public.

Happily, with practice, the process of critically reviewing scientific research becomes easier and less time-consuming. This IFIC Review is designed to help members of the media, health professionals, and educators understand how to read and evaluate food and health-related scientific studies. It presents an overview of key information to look for, questions to ask, and other important considerations.

A Process of Discovery and Debate

To consumers, it often seems that contradictory studies about food and health appear in the media almost weekly, leaving many to wonder why researchers can't get it right the first time. The answer is not an easy one to swallow, because to accept it means we must accept uncertainty.

The scientific process is a road of discovery. It is a process of gaining knowledge about the universe through the observation of measurable evidence (l). Contrary to what many people may believe, this "road" is not necessarily a straight one. That is, researchers may take different directions of exploration, causing the "road" to twist, turn, and sometimes even backtrack or come to a dead end before the facts are uncovered. Even then the facts may be only part of a larger, partially understood phenomenon, which requires that more research be conducted before we find the answers.

As a result, the scientific process—how studies are designed, conducted, and reported—frequently generates a great deal of debate. Tracking the debate is often key to putting new research into context. With that in mind, new research studies published in scientific journals should be viewed as discussions among scientists. In these discussions, almost no one gets to have the final word, as it is rare that a study provides a final, complete answer (2). In fact, occasionally even old, accepted research results are revisited and discussed again. With the benefit of new information or technology, scientists sometimes see old results in a new light. The publication of research findings allows researchers to obtain input on their work, which not only confirms or contradicts their results but also adds to the body of literature on a subject and helps shape future research.

The bottom line is that dialogues characterized by cycles of revisions, conjectures, assertions, and contradictions are frequently key to investigating a subject. In addition, although such cycles often frustrate nonscientists and can contribute to increasing public skepticism about advice on food and health, it is important to understand that science is evolutionary, not revolutionary (3). Because scientific research explores the unknown, uncertainty is an unavoidable part of current investigations. Only through repeated research and analyses do certainties emerge.

Types of Research Studies: What They Are & When They Are Used

Broadly, research can be divided into two categories: observational and experimental. Within these categories, there are three basic methods by which scientists investigate questions about food and health (4). It is essential to understand what each method can and cannot do.

Observational Research
Observational research involves examination of specific factors in defined groups of subjects to investigate the relationships between those factors and aspects of health or illness (5). For example, an observational study may focus on the body weight of healthy women aged 50 or older and its relationship to blood pressure in that group. Observational research can only suggest relationships, however. It takes experimental research to determine cause and effect.

Epidemiological research is often observational, but it may also be experimental. It is the study of the distribution and determinants of diseases or other health outcomes in human populations (6). It seeks to expose potential associations between aspects of health (such as cancer and heart disease) and diet, lifestyle, habits, or other factors within populations.

Although epidemiological studies are useful for suggesting relationships between two factors, it is important to remember the basic limitation of epidemiological studies: they do not necessarily indicate cause and effect (5). In fact, the associations that they indicate can actually turn out to be coincidental. A simple example of this would be a study which suggested that driving a Cadillac was associated with increased risk for cardiovascular disease. In this case, the fact that the car was a Cadillac was a coincidence. The association revealed by the study should have been between driver characteristics (gender, age, weight) and the disease.

Observational epidemiological research may be most revealing when considered in the context of what experimental research suggests about a subject (7). For example, to assess whether an association discovered in an epidemiological study is real and not the result of bias or confounding factors, researchers may conduct a randomized clinical trial to confirm a suspected cause-and-effect relationship.

Experimental Research
In experimental research, study subjects (whether human or animal) are selected according to relevant characteristics and are then randomly assigned to either an experimental group or a control group (5). Random assignment ensures that factors (known as variables) that may affect the outcome of the study are distributed equally among the groups and therefore could not lead to differences in the effect of a treatment. The experimental group(s) is then given a treatment (sometimes called an intervention), and the results are compared with those for the control group, which does not receive a treatment (or which receives a placebo, or fake treatment). Any differences in results between the groups can then be attributed to the treatment; that is, the effect can be considered to be caused by the treatment. Controlled experimental research can be fraught with errors, easily becoming uncontrolled. Sometimes these flaws are easily spotted, but in many cases it is worth asking experts—they know what to look for.

Basic research generates data by investigating biochemical substances or biological processes (4). It is often undertaken to confirm observations or discover how a particular process works. For example, an experiment might take place to examine how vitamin E may help prevent oxidation of LDL (low-density lipoprotein) cholesterol, a process believed to play a role in the development of heart disease. This basic research is just part of a larger effort to understand how diet can help reduce risk for heart disease.

Basic research may be conducted in vitro (such as in test tubes) or with animals. Research with animals is an important tool in determining how humans may react when exposed to particular substances. However, it is important to note that, due to differences in physiology and the fact that animals are routinely exposed to levels of compounds far higher than those that human populations typically encounter, one cannot assume that results from studies with animals can be generalized to humans.

Clinical trials deal with the experimental study of human subjects. Trials may attempt to determine whether the findings of basic research are applicable to humans or to confirm the results of epidemiological research. Studies may be small, with a limited number of participants, or they may be large intervention trials that seek to discover the outcome of treatments on entire populations. "Gold standard" clinical trials are double-blind placebo-controlled studies that use random assignment of subjects to experimental and control groups.

Talking with the Experts (sidebar)

Journalists, educators, and health professionals who need to quickly distill the findings of a new study should consider contacting the study authors or other scientists familiar with the body of research on the topic. Experts can answer questions and provide insight that both novice and experienced readers may not be able to unearth by themselves. In addition, interviewing scientists other than the study author can bring valuable insights and contribute to a more balanced perspective of the study.

Questions you may want to ask a study's author or other experts include the following:

• Could the study be interpreted to say something else?

  Scientists often reach different conclusions from the same or similar data, so asking "What's your take on this study" is not unusual. The rationale for different conclusions may be important when putting a study's findings into context.

• Are there any methodological flaws in the study that should be considered when making conclusions?

  The more experts who review a study, the more likely potential flaws—such as confounding variables—will be discovered.

• Are the study's results generalizable to other groups?

  Consumers want to know how research affects them. If study results are applicable only to a narrow group of people, it should be reported as such.

• How does this work fit with the body of research on the subject?

  Even a well-written article may not include a discussion of all relevant research because of space limitations. Yet, it is extremely important to know—and communicate to consumers—whether a study is confirming previous research and therefore adding more weight to scientific beliefs or whether the study's results and conclusions make a wild departure from current thinking on the subject.

What to Look for When Critically Reviewing Scientific Studies

To enhance communication among scientists and to facilitate replication of the study, published research generally follows an established format. This section of this Review highlights important information to look for and questions to ask yourself or to pose to experts. This information should help you to understand each part of the study. Be aware that exactly where the information appears in different articles varies somewhat.

Abstract
The abstract of a published study serves to briefly answer the basic questions about what was studied, how it was done, and the results. Its primary purpose is to allow readers to make an initial evaluation of whether a study is of interest without having to read the complete paper. If only we could just peruse the abstract and consider our review of the study complete! Unfortunately, that is not the case. Abstracts do not provide nearly enough detail to enable readers to assess the validity of a study or put it into context. These can be done only by thoroughly reviewing the rest of the study.

What is a Double-Blind Placebo-Controlled Study? (sidebar)

Considered the "gold standard" of clinical research studies, the double-blind placebo-controlled study provides dependable findings that are free of bias introduced by either the subject or the researcher.

In this type of study, neither the subject nor the researcher conducting the study knows whether the test substance or a placebo has been administered. For the results to be valid and to ensure that the subject cannot violate the "blindness," the placebo and the test substance must be virtually identical (i.e., look, smell, and taste similar).

The "blindness" of the study is crucial. It eliminates the possibility that a participant's personal beliefs will undermine the study's validity. It also prevents the researcher's expectations from influencing the test results.

Introduction
The introduction section "welcomes" the reader to the study. It eases the reader into the research by presenting the question that the researcher seeks to answer or the problem or hypothesis that the study addresses (8). It explains why the study will be conducted, which gives the reader a clue to the potential importance of the research. It also expands a little more on how the research will be conducted. The introduction can be divided into two major parts: the Background section and the Purpose section.

Background: The background information presented in the introduction of a study tells why the researchers think the study is important (8). It should reflect a comprehensive knowledge of the body of research on the subject and should brief the reader on both the previous studies that support the concepts or theories of the current study and those that do not. In essence, it brings the reader up to speed on current thinking and presents the researcher's rationale for pursuing the study.

Purpose: The stated purpose essentially defines the study (9). It dictates how a study will be conducted: the research design, the variables that will be measured, how information will be collected and analyzed, and what conclusions may be drawn.

In some instances you may find that the study does not seem to be appropriately designed or conducted to achieve its purpose. For example, the type of study may not yield the type of information required to answer the stated question or the study population may not fit the purpose. Consultation of experts about these points can be very helpful in determining the validity of the study's conclusions.

Key Questions to Ask:

• What are the inherent limitations of this type of study?
• Does the research design fit the stated purpose of the study?
• Has the author omitted from the Background section important points that could have a meaningful effect on the study design or interpretation of the results?

Methodology
The key question of the methodology section is "How?" This section should enable critical readers to determine whether the research is valid; that is, was it adequately designed to achieve its purpose (9)? Hence, the methodology section warrants careful review. It explains how the research was conducted and should give information in enough detail for the reader to evaluate the study. It should also enable the reader to understand to whom or what the study results apply. Important information featured in the methodology section includes the following (10):

• the setting of a study (in a clinic, laboratory, population, etc.)
• how variables were controlled (how did they adjust for specific subject qualities or outside influences that could affect the results?)
• the sample size
• the number of study groups
• the treatment or variables being observed (e.g., a vitamin supplement or specific diets)
• the length of the study
• how the data were collected
• how and by what statistical procedures the data were analyzed

The methodology section also provides information about the sampling method and whether subjects were randomly assigned or not (in experimental studies). Pay specific attention to these points, because they are among the first steps in conducting research, and flaws present here can render the results invalid.

Randomness in Selection and Assignment: The term "random sample" is familiar to most everyone, but exactly how subjects (the "sample") are selected for the study is of crucial importance. Among other things, the sampling method affects to whom the study results may be relevant.

If the subjects are selected randomly, that is, via a procedure in which all individuals in a population being studied have an equal chance of being selected, then the study results may be generalizable to that population (9). True random selection may be done using a table of random numbers generated by a computer. Calling people picked randomly out of a telephone book between the hours of 1:00 and 3:00 p.m. in the afternoon is not random sampling of the entire population of the United States, for instance. We can all think of a number of reasons why this is not truly random: some people don't have telephones, and some people have unlisted phone numbers. In addition, the sample would likely be light on full-time workers and would be weighted heavily with stay-at-home mothers, elderly and unemployed people, students, people who are ill, and people who work the night shift, for example.

The term "random" also applies to the assignment or the division of subjects into groups. Random assignment ensures that all subjects have an equal chance of being in the experimental and control groups and increases the probability that any unidentified variable will systematically occur in both groups with the same frequency. Randomization is crucial to controlling for variables that researchers may not be aware of or cannot adequately control but that could affect the outcome of an experimental study (9).

To determine the true effect of a treatment, researchers must carefully control for all variables that could affect the outcome of a study. Some of the variables are obvious, such as age, body weight, and gender. To control for these differences, researchers match subjects in experimental and control groups so that they have similar characteristics. Some variables, such as heredity, are more difficult to control for. Still others may be unknown—because knowledge of human biology is still developing, for example. By randomly assigning subjects to study groups, the influence of such variables is minimized and any differences in results between groups can be attributed to the treatment.

A Question of Size—Sample Size, That Is: As you probably know from experience, the primary question about sample size is "Was it big enough to find an effect?" The answer is not always as easy to come by. In fact, it is often a matter of judgment.

For example, when studying the effect of a weight-loss drug, a researcher may decide that a sample size of 100 people is adequate because the effect is easily noted: How many pounds did those who received the drug lose compared with those who did not receive the drug? However, when assessing the average fruit and vegetable consumption among children who participated in a school-based intervention program, several thousand children may be deemed necessary because the increase from such an intervention is likely to be relatively small. That is, the diets of the children in experimental and control groups may not differ much in terms of fruit and vegetable intake, and therefore, the effect of the intervention might not be noticed. It is easier to spot a small effect when you are looking at results for a large sample.

A small sample size, however, does not necessarily mean that the study is flawed. For example, prospective clinical nutrition studies usually have just a small number of subjects because there are so many variables that need to be controlled. When reading a study, be sure to look for the rationale that the researcher used to decide the sample size.

Of Abstracts and News Releases (sidebar)

When deadlines loom near, it may be tempting to rely only on an abstract and a news release for information about a research study instead of taking time to examine the original published study. Resist the temptation! Abstracts and releases are not substitutes for original research. They simply do not provide enough information that can be used to judge the merits of a study or to accurately report the study results.

Medical journals, organizations, and universities regularly issue news releases to stimulate media coverage of research or conferences.

Make news releases and abstracts work for you by using them for ideas, story angles, quotes, potential interview subjects, and a "quick-and-dirty" overview of the research. Once you have a little background on the study, you can thoroughly review the original research article without spending too much additional time. Keep the "Key Questions to Ask" in mind as you review the article, and note specific questions that you have for experts as you go along.

A Word About Methodological Limitations: Often, limitations are placed on researchers—such as finances or the ethics of human testing—and these can severely restrict progress on the study and study results. Aside from these external limitations, there can also be internal limitations, such as those experienced when the current state of knowledge in a field (particularly as it relates to data collection instruments) is known to be limited. Any type of constraint—if it could affect the results of the study—should be openly discussed in the methodology or discussion sections of the study.

Key Questions to Ask:

• Are there any major design flaws in this study?
• Are the data collection measures appropriate to answer the study questions?
• Were methodological limitations acknowledged and discussed?
• What influence might these limitations have had on the results?

Results
Nobody will deny that reading a scientific study up to this point can be difficult and tedious. Now, however, we finally get to the really interesting stuff—the answers. The results section of a study does indeed provide what we might call answers but what scientists would call "data" and the statistical analyses of the data (8). For more precise communication, statistical measures are frequently used to convey the existence and strength of relationships.

The field of statistics is based on the quantification of information. Descriptive statistics present the information in an organized fashion so that it is easier to interpret (9). Some of the more familiar descriptive statistics include percentage, frequency, mean, and standard deviation. Descriptive statistics, however, do not provide information about cause and effect—this is the realm of inferential statistics. As the name implies, inferential statistics often involve making inferences from the results for the sample studied and extrapolating them to a larger population (8).

Understanding Statistical Significance: Without getting too technical, this brief discussion of significance will help the reader understand this common statistical measure.

Researchers generally calculate statistical significance and report it as a "P value." A P value is the probability of obtaining an effect or association in a study sample as or more extreme than the one observed if there was actually no effect in the population. If the results of a study are statistically significant, then the study may have indeed hit upon some real association or effect. The study author will identify what P value he or she has used in the analysis. A P value of less than 5 percent (P < 0.05) is fairly common and would be considered statistically significant (8). This means that the result would occur less than 5 percent of the time if there were no effect. More stringent levels of significance are P < 0.01 and P < 0.001.

If the results of a study are not statistically significant, the author may discuss the statistical power of the study. An in-depth discussion of power is beyond the scope of this Review; however, when present, information about statistical power in a study will help the reader understand whether the study had a chance of finding the answer to the research questions in the first place.

It is easy to get wrapped up in discussions of statistical significance when reading research, but it is important to remember that a statistically significant result does not necessarily mean that the results are important—or relevant to the public. In addition, a statistically significant finding does not guarantee that the research is without biases or confounding factors that could make the statistical value irrelevant (7). Statistical significance is only part of the picture; to get the whole picture, one must consider the context of the study—what other research on the subject reveals.

Communicating Risk: Attention readers: You "risk" misunderstanding and miscommunicating the results of the study if you do not fully comprehend the differences between relative and absolute risk.

Absolute risk refers to the actual risk of an occurrence—the chance that a specific outcome will occur. Relative risk puts risk in comparative terms—the outcome rate for people exposed to the factor in question compared with the outcome rate for those not exposed to the factor. A relative risk of >1 indicates an increased risk of the outcome under investigation; one of <1 indicates a decreased risk of the outcome. Relative risks are the most commonly used measure of morbidity or mortality in the medical literature today. However, in many cases the absolute risk is a far more relevant statistic for the public (11).

For example, suppose that a study shows that a man who brushes his teeth only once a day is 50 percent more likely to have all his teeth fall out in the next 10 years than others who brush their teeth twice per day. This is the relative risk. Yet, the absolute risk that all of the man's teeth will fall out may be only 1 percent. In this case, the relative risk makes the problem—a rare one anyway—seem more important than it really is. However, relative risk can also make a problem appear to be less important than it actually is. Therefore, it is important to consider both relative risk and absolute risk when discussing study results.

Key Questions to Ask:

• What is the real and statistical significance of these results?
• To whom do these results apply?
• How do these results compare to those of other studies on the subject?

Discussion
The discussion section of a study gives the reader some insight into the study subject area and often sheds new light on the results and their meaning. Alternative explanations for the results and the implications of the research may also be presented.

One of the most frequent errors in scientific research is drawing conclusions that are not adequately supported by the data. This may occur for a number of reasons: collection of insufficient or inadequate data, overgeneralization of results, methodological problems, or inherent limitations of the study design. This is why it is important to review the methodology section.

Sometimes, researchers stray from the scientific method by reporting conclusions that are unrelated to the research question that was tested. Although conclusions made in this manner may have merit, it is important to take a second look at whether the study was adequately designed and conducted to support the secondary conclusions (12).

Finally, be wary of absolute conclusions that profess to be the final word on a subject. Good research answers some questions and raises others. A call for more research to investigate particular issues that remain unclear or to replicate the current study findings frequently concludes a journal article.

Key Questions to Ask:

• Are the conclusions supported by the data?
• Are the conclusions of the study related to the stated purpose of the study? If not, do the study design and results support the secondary conclusions?

References
Experts in the subject area can usually tell rather quickly if key research has been omitted from the reference list. If this is the case, the researchers may have failed to adequately review, consider, and evaluate prior work in the field that could have benefited their current study. Also, a reference list that includes both older and newer relevant research can reassure the reader that the author has thoroughly reviewed the entire body of research for background and has not just considered the last few or first few studies conducted on the topic.

Also Consider. . .
Other issues that merit attention in the critical review of studies include the funding sources of a study and the appropriate use of editorials and letters to the editor.

Funding Source: Often, one hears a study being criticized—or its findings dismissed entirely—because it was funded by industry or another interested party. Many scientific journals today require that potential conflicts of interest be disclosed and sources of funding be referenced at the end of a paper. Although it is interesting to note the funding source of a study, it is unfair—and perhaps shortsighted—to simply negate the results solely on the basis of the funding source.

The reason that studies are often funded by organizations that may benefit from the results is obvious. After all, who else but an interested party would allocate the large amounts of money that good research often requires? For example, when a company is seeking approval for a new food ingredient, it is required by law to support adequate studies to demonstrate the ingredient's safety. The government—taxpayers—certainly would not invest millions of dollars to study food ingredients or products that may never come to market!

Ethical researchers do not manipulate data or design studies to support the funder's interests. Indeed, most members of industry do not want a "tell them what they want to hear" researcher; they want to know the real answers to their questions. A critical evaluation of research on its own merit is the best way to assess its validity and importance. If the study is good, its results will stand on their own—regardless of who supported the research.

Editorials and Letters-to-the-Editor: Editorials—or written opinions by experts in a field other than the authors of a study that the editorial addresses—may be one of the most valuable ways for readers to understand a study, its meaning, and its practical implications (13). Editorials often provide perspective on a study, discussing it in the context of other research, as well as identifying potential flaws that may affect the applicability or even veracity of the study results.

Although letters to the editor usually appear in issues following that in which a study is published, if a reader has the time to wait, such letters can be very useful to help identify potential problems with a study. At the least, they can be used as a continuing education tool on what to look for when critically reviewing studies.

Study results that are reported via letters to the editor, however, should not be taken at face value. They cannot substitute for peer-reviewed articles that provide the details necessary for readers to critically review the research.

Meet the Meta-Analysis (sidebar)

A meta-analysis is a statistical method of combining results from separate studies to derive overall conclusions about a question or hypothesis (14). Meta-analyses are conducted in an attempt to reconcile differences among studies in terms of their statistical power or sample sizes or to aggregate relevant findings across studies.

The procedure is most appropriate when examining studies that look at the same question and use similar methods to measure relevant variables. For example, using one type of meta-analysis, scientists examined the relationship between weight reduction and blood lipid levels (15). Although individual studies showed inconsistent results, pooling of data from 70 similar studies showed significant decreases in the levels of total cholesterol and other blood lipids due to weight loss.

The technique of meta-analysis is not without limitations, however. Data from flawed studies may be included, or the analysis may include data from studies that use different methods to measure variables—resulting in a comparison of apples to oranges.

General considerations for judging the validity of a meta-analysis include the following (14):

• Is the objective clearly stated?
• Are the criteria for inclusion or exclusion of studies explicit?
• Is the search mechanism for the determination of suitable studies adequate?
• Is the quality of the trials included assessed?
• Are all of the trials randomized?
• Does the discussion include mention of limitations? Does it put results in context?
• Are the conclusions justified by the data?

Summary

This IFIC Review presents information to help members of the media, health professionals, and educators critically review food and health-related scientific studies. Such critical review is essential to put the results into the context of the body of scientific literature on a subject and to accurately present the relevance of research to the public.

Although the various elements of a study that have been discussed affect whether a piece of research provides valid and relevant answers to a question being investigated, it is important to realize that "perfect" research does not exist (3). Economics, ethics, and the current state of knowledge may limit a study in its ability to find the answers sought.

Given this, it is also essential to remember that the nature of the scientific process is not linear. It is a process that frequently moves in many different directions, generating questions, discussions, and debates along the way.

How does the communicator maneuver through the maze of emerging scientific findings about food and health to deliver accurate, relevant information to the public? First, by reserving judgment about a study until you have sought out other studies and experts to help assess the findings of the study and their importance—or unimportance. In other words, by putting all research into context. Second, by taking a moderate approach to communicating new information. Realize that what may seem to be a revolutionary, life-altering study today may turn out to be just the opposite tomorrow. That is the nature of research and what makes the journey so exciting.

Key Definitions

Bias–Problems in study design that can lead to effects that are not related to the variables being studied (l6). An example is selection bias, which occurs when study subjects are chosen in a way that can misleadingly increase or decrease the strength of an association. Choosing experimental and control group subjects from different populations would result in a selection bias.

Blind, Single or Double–In a single-blind experiment, the subjects do not know whether they are receiving an experimental treatment or a placebo (4). In a double-blind experiment, neither the researchers nor the participants are aware of which subjects receive the treatment until after the study is completed.

Confounding Variable or Confounding Factor–A "hidden" variable that may cause an association that the researcher attributes to other variables (17).

Control Group–The group of subjects in a study to whom a comparison is made to determine whether an observation or treatment has an effect (9). In an experimental study, it is the group that does not receive a treatment. Subjects are as similar as possible to those in the test group.

Correlation–An association, or when one phenomenon is found to be accompanied by another (16). A correlation does not prove cause and effect. Correlation may also be defined statistically.

Experimental Group–The group of subjects in an experimental study that receives a treatment (9).

Generalizability–The extent to which the results of a study are able to be applied to the general population of people that is comparable to the population studied (18).

Incidence–The number of new cases of a disease during a given period of time in a defined population (6).

Meta-analysis–A quantitative technique in which the results of several individual studies are pooled to yield overall conclusions (6).

Outcomes Research–Type of research that is increasingly used by the health industry and that provides information about how a specific procedure or treatment regimen affects the subject (clinical safety and efficacy), the subject's physical functioning and lifestyle, and economic considerations such as saving or prolonging life and avoiding costly complications (19).

Placebo–Sometimes casually referred to as a "sugar pill," a placebo is a "fake" treatment that seems to be identical to the real treatment (4). Placebo treatments are used to eliminate bias that may arise from the expectation that a treatment should produce an effect.

Prevalence–The number of existing cases of a disease in a defined population at a specified time (6).

Prospective Study–Epidemiological research that follows a group of people over a period of time to observe the potential effects of diet, behavior, and other factors on health or the incidence of disease (4). In general, it is considered a more valid research design than retrospective research.

Randomization, or Random Assignment–A process of assigning subjects to experimental or control groups in which the subjects have an equal chance of being assigned to each group (9). Used to control for known, unknown, and difficult-to-control-for variables.

Random Sampling–A method by which subjects are selected to participate in a study in which all individuals in a population have an equal chance of being chosen (9). Helps to ensure the generalizability of the study results.

Reliability–Whether a test or instrument used to collect data, such as a questionnaire, gives the same results if repeated with the same person several times (20). A reliable test gives reproducible results.

Research Design–How a study is set up to collect information or data (10). For valid results, the design must be appropriate to answer the question or hypothesis being studied.

Residual Confounding–The effect that remains after one has attempted to statistically control for variables that cannot be measured perfectly (17). This is a particularly important concept in epidemiological studies because knowledge of human biology is still developing. There may exist unknown variables that could significantly change conclusions made on the basis of epidemiological research.

Retrospective Study–Research that relies on recall of past data or on previously recorded information. Often, this type of research is considered to have limitations because the number of variables cannot be controlled and because memory is not infallible (4).

Risk–A term encompassing a variety of measures of the probability of an outcome. It is usually used in reference to unfavorable outcomes such as illness or death (6). Be certain to distinguish between absolute risk and relative risk.

Risk Factor–Anything statistically shown to have a relationship with the incidence of a disease (4). Does not necessarily infer cause and effect.

Statistical Power–A mathematical quantity that indicates the probability a study has of obtaining a statistically significant effect (16). A high power of 80 percent, or 0.8, indicates that the study—if conducted repeatedly—would produce a statistically significant effect 80 percent of the time. On the other hand, a power of only 0.1 means that there would be a 90 percent chance that the research missed the effect—if one exists at all.

Statistical Significance–The probability of obtaining an effect or association in a study sample as or more extreme than the one observed if there was actually no effect in the population (10). On the basis of the hypothesis that if there truly is no effect, the results of a study are unlikely to have occurred. A P value of less than 5 percent (P < 0.05) means that the result would occur less than 5 percent of the time if there were no effect and is generally be considered evidence of a true treatment effect or a true relationship.

Validity–The extent to which a study or study instrument measures what it is intended to measure (20). Refers to accuracy or truthfulness in regard to a study's conclusion.

Variable–Any characteristic that may vary in study subjects, such as gender, age, body weight, diet, behavior, attitude, or other attribute (10). In an experiment, the treatment is called the independent variable; it is the factor being investigated. The variable that is influenced by the treatment is the dependent variable; it may change as a result of the effect of the independent variable.

References

1. Moore M, ed. Health Risks and the Press: Perspectives on Media Coverage of Risk Assessment and Health. Washington, DC: The Media Institute; 1989.
2. Angell M, Kassirer JP. Clinical research—what should the public believe? N Engl J Med. 1994;331(31):189-190.
3. Williams L. Stalking the Elusive Healthy Diet. The New York Times, October 11, 1995.
4. Why do those #&*?@! "experts" keep changing their minds? UC Berkeley Wellness Letter, February 1996.
5. Blonz E. Scientific studies: how to handle the hype. Vegetarian Times, March 1995.
6. Langseth L. Nutritional Epidemiology: Possibilities and Limitations. ILSI Europe Concise Monograph Series; 1996.
7. Charlton BG. Should epidemiologists be pragmatists, biostatisticians, or clinical scientists? Epidemiology 1996;7(5):552-554.
8. Scandlyn JN. How to read a research article. Orthopaed Nurs. 1987;6(5):21-27.
9. Tait PE. Do-it-yourself evaluation of experimental research. J Vis Impair Blindness October 1984;356-363.
10. Schantz D, Lindeman CA. Reading a research article. J Nurs Admin. March 1982;30-33.
11. Dupont W. Converting relative risks to absolute risks: a graphical approach. Stat Med. 1989;8(6):641-651.
12. Savitz D, Olshan A. Multiple comparisons and related issues in the interpretation of epidemiologic data. Am J Epidemiol. 1995;142(9):904-908.
13. Angell M, Kassirer J. Editorials and conflicts of interest. N Engl J Med. 1996;335(14):1055-1056.
14. Geller NL, Proschan M. Meta-analysis of clinical trials: a consumer's guide. J Biopharm Stat. 1996;6(4):377-394.
15. Dattilo A, Kris-Etherton PM. Effects of weight reduction on blood lipids and lipoproteins: a meta-analysis. Am J Clin Nutr. 1992;56(2):320-328.
16. Moore DS. Statistics Concepts and Controversies. New York: W.H. Freeman and Company; 1996.
17. Beecher H. The concept of residual confounding in regression models and some applications. Stat Med. 1992;11:1747-1758.
18. Berkowitz L, Donnerstein E. External validity is more than skin deep: some answers to criticisms of laboratory experiments. Am Psychologist 1982;37(3):245-257.
19. Gallagher-Allred C, Voss A. Nutrition Intervention and Patient Outcomes: A Self-Study Manual. Ross Products Division, Abbott Laboratories, and The American Dietetic Association; 1995.
20. Sherry B. Descriptive epidemiologic research. Monsen ER, ed. In: Research: Successful Approaches. Chicago: The American Dietetic Association; 1992.