Types of Studies: Strengths and Pitfalls
Four major study designs are described below with their strengths and weaknesses.
Surveys gather data to describe the demographics of a group, such as the Census; the health status of a group of people at a particular time; the utilization of medical services; or the knowledge, beliefs, and attitudes of people regarding health practices, such as cancer screening or HIV prevention. Surveys are a major data collection method in health services research. Researchers may collect data from interviews with patients, the public, clinicians, health care administrators, and others or from reports of hospital discharges, patient charts and medical records, disease registries, and billing documents.
Survey research is extremely complex. Researchers must choose a sample of people to survey. Will the sample include the individual in a household or the entire household? Can a proxy answer questions accurately for a family member? When comparing groups, they must acknowledge the different ways people will identify themselves, especially in terms of race and ethnicity. Racial and ethnic definitions vary widely; many nationalities are collapsed under broad terms, such as "Asian," which may hide differences between people of different cultures.
Whether surveyers want to assess patient preferences, diets, or health care utilization, they must ask questions that are clear and unbiased, a very difficult task! People often give different answers depending on the order of the questions, whether they are asked to agree or disagree with a statement, and how many possible choices they have for answers (Converse and Presser, 1986). Don't forget that interview data depend on people's recall of past events and willingness to share their practices and beliefs with a stranger!
Survey results are often analyzed in terms of rates that describe the number of existing cases (prevalence) of a disease or practice or the number of new cases (incidence) for a particular time. Interpretation of these rates requires some savvy. For example, researchers track the incidence of cancer cases in a population over time. An increase in new cases may mean that more people are developing cancer because there is more exposure to the risk factor involved. Perhaps sun exposure increases in Australia and results in a real increase in melanoma. On the other hand, an increase in breast cancer or prostate cancer cases may also result from changes in screening. When Betty Ford developed breast cancer, there was a dramatic rise in breast cancer rates due to increased mammography use and breast cancer diagnosis.
Because surveys capture people's behaviors and health status at the same time, it is impossible to conclude which factor is the cause or the effect. Heavy coffee drinkers may report more indigestion, but survey data do not allow readers to conclude that coffee causes indigestion (Abramson, 1988).
While survey results often are difficult to interpret and generalize to other groups and time periods, they provide wonderful insights into the practices and health conditions of large groups of people as well as clues for future investigation into the impact of medical practices on health outcomes or the relationship between exposures and disease.
Cohort studies observe groups of individuals before they develop a disease or a particular outcome. Researchers select a group with an exposure or experience and a similar group without that exposure or experience. They follow the groups over time and measure the results.
For example, researchers observe people with asbestos exposure and people without asbestos exposure over a long period of time to detect differences in their rates of lung disease.
There are several advantages to cohort studies. They have the power to detect many different outcomes of an exposure. Because the people are all healthy when the study begins, researchers aren't likely to misclassify them based on knowledge of their outcomes. In the analysis phase, cohort studies allow researchers to calculate a relative risk of developing a disease based on different exposures.
However, several disadvantages exist as well. For many outcomes, it may take many years to detect changes in the groups. Cancer often requires decades to develop. A decline in mortality rates due to mammography will take many years to evaluate. Researchers may be tempted to decrease the length of the study, or participants may drop out of the study as time passes. Because of the time involved and number of participants needed, cohort studies may be very costly. Cohort studies can measure many outcomes but (usually) only one exposure.
Case-control studies differ from cohort studies in terms of time. Whereas cohort studies follow people over time without knowing the results, case-control studies begin with the outcomes. Researchers choose people with a particular result (the cases), such as lung cancer, and people similar to the cases who do not have lung cancer (the controls). The researchers then interview the groups or check their records to ascertain what different experiences they had. They compare the odds of having an experience with the outcome to the odds of having an experience without the outcome.
For example, are people with lung cancer more likely to be smokers than those without lung cancer? Are women with term births more likely to have prenatal care than women with premature births?
Case-control studies offer many benefits to researchers. They are relatively fast and cheap compared to cohort studies and can study multiple exposures. They are very useful for rare diseases that require many years to develop and many people to observe in order to find them.
However, there are many pitfalls. Researchers must be very cautious when they choose the control group. Controls must be very similar to the cases except for the outcome. Cases are more likely to remember past experiences than the healthy controls. It is difficult to provide unbiased measurements of exposure in case-control studies.
Randomized Clinical Trials (RCTs)
RCTs follow two groups of people over time to see who achieves a particular result. In this case, the researchers assign or randomize the people to their groups. If this is done properly, each person has an equal chance of being assigned to either group, reducing the possibility of the groups having different features. Each group receives a different intervention. When the study period ends, the researchers evaluate their different outcomes and calculate the risk of one group developing the result compared to another.
For example, researchers randomize workers to receive indemnity insurance or managed care coverage. They watch people over time and measure their utilization of health services. They compare the groups to evaluate whether the type of insurance coverage influenced their use of medical care.
In clinical experiments, investigators randomize patients with the same disease, such as hypertension, to two different drug groups. One may receive an experimental medication while the other receives a placebo. The researchers measure the effect of the drugs on the hypertension.
Controlled clinical trials hold the advantage in assessing causality. Randomization produces comparable study groups that should only differ in the intervention given. Therefore, differences in outcome can be attributed to that intervention, such as a drug, and the researchers can conclude that the intervention caused the results. Since the people are similar in the beginning of the study, researchers can be confident that the intervention precedes the effect, a major determinant of causality.
However, clinical trials are not perfect. They are expensive and time-consuming. Biases may render their results less reliable. For example, if study participants or researchers know their group assignment, they may expect a particular result. Masking study participants and researchers avoids treating people differently based on the intervention. Losing participants along the way may also jeopardize the results. Did people leave the study because the treatment was too harsh or for reasons unrelated to the study?
While randomized clinical trials can prove causality, ethical considerations usually limit their utilization. Randomizing participants into smoking and non-smoking arms of a study to detect lung cancer would provoke an uproar among scientists and the public. For these reasons, many investigations must be observational in which groups of naturally occurring behaviors are followed and measured.
Remember also that clinical trials occur under very specialized, controlled conditions that do not represent everyday life. Therefore, the results obtained in this environment may not transfer to larger groups of people not engaged in research.
U.S. National Library of Medicine (NLM)
Last updated: 17 June 1998
/databases/freemedl.html/databases/freemedl.htmlhttp://wwwindex.nlm.nih.gov/index/nlmindex.htmlhttp://wwwindex.nlm.nih.gov/index/nlmindex.htmlhttp://wwwindex.nlm.nih.gov/index/medindex.htmlhttp://wwwindex.nlm.nih.gov/index/medindex.html/contacts/contact.html/contacts/contact.html/sitemap.html Types of Studies /sitemap.html