Types of Studies: Strengths and Pitfalls
Four
major study designs are described below with their strengths and weaknesses.
Surveys
Cohort Studies
Case Control Studies
Randomized Clinical Trials
Surveys
Surveys
gather data to describe the demographics of a group, such as the Census; the
health status of a group of people at a particular time; the utilization of
medical services; or the knowledge, beliefs, and attitudes of people regarding
health practices, such as cancer screening or HIV prevention. Surveys are a
major data collection method in health services research. Researchers may
collect data from interviews with
patients, the public, clinicians, health care administrators, and others or
from reports of hospital discharges,
patient charts and medical records, disease registries, and billing documents.
Survey
research is extremely complex. Researchers must choose a sample of people to
survey. Will the sample include the individual in a household or the entire
household? Can a proxy answer questions accurately for a family member? When
comparing groups, they must acknowledge the different ways people will identify
themselves, especially in terms of race and ethnicity. Racial and ethnic
definitions vary widely; many nationalities are collapsed under broad terms,
such as "Asian," which may hide differences between people of
different cultures.
Whether
surveyers want to assess patient preferences, diets, or
health care utilization, they must ask questions that are clear and unbiased, a
very difficult task! People often give different answers depending on the order
of the questions, whether they are asked to agree or disagree with a statement,
and how many possible choices they have for answers (Converse and Presser,
1986). Don't forget that interview data depend on people's recall of past
events and willingness to share their practices and beliefs with a stranger!
Survey
results are often analyzed in terms of rates that describe the number of
existing cases (prevalence) of a disease or practice or the number of new cases
(incidence) for a particular time. Interpretation of these rates requires some
savvy. For example, researchers track the incidence of cancer cases in a
population over time. An increase in new cases may mean that more people are
developing cancer because there is more exposure to the risk factor involved.
Perhaps sun exposure increases in Australia and results in a real increase in
melanoma. On the other hand, an increase in breast cancer or prostate cancer
cases may also result from changes in screening. When Betty Ford developed
breast cancer, there was a dramatic rise in breast cancer rates due to
increased mammography use and breast cancer diagnosis.
Because
surveys capture people's behaviors and health status at the same time, it is
impossible to conclude which factor is the cause or the effect. Heavy coffee
drinkers may report more indigestion, but survey data do not allow readers to
conclude that coffee causes
indigestion (Abramson, 1988).
While
survey results often are difficult to interpret and generalize to other groups
and time periods, they provide wonderful insights into the practices and health
conditions of large groups of people as well as clues for future investigation
into the impact of medical practices on health outcomes or the relationship
between exposures and disease.
Cohort
studies
Cohort
studies observe groups of individuals before they develop a disease or a
particular outcome. Researchers select a group with an exposure or experience
and a similar group without that exposure or experience. They follow the groups
over time and measure the results.
For
example, researchers observe people with asbestos exposure and people without
asbestos exposure over a long period of time to detect differences in their
rates of lung disease.
There
are several advantages to cohort studies. They have the power to detect many
different outcomes of an exposure. Because the people are all healthy when the
study begins, researchers aren't likely to misclassify them based on knowledge
of their outcomes. In the analysis phase, cohort studies allow researchers to
calculate a relative risk of developing a disease based on different exposures.
However,
several disadvantages exist as well. For many outcomes, it may take many years
to detect changes in the groups. Cancer often requires decades to develop. A
decline in mortality rates due to mammography will take many years to evaluate.
Researchers may be tempted to decrease the length of the study, or participants
may drop out of the study as time passes. Because of the time involved and
number of participants needed, cohort studies may be very costly. Cohort
studies can measure many outcomes but (usually) only one exposure.
Case-control
studies
Case-control
studies differ from cohort studies in terms of time. Whereas cohort studies
follow people over time without knowing the results, case-control studies begin
with the outcomes. Researchers choose people with a particular result (the
cases), such as lung cancer, and people similar to the cases who do not have
lung cancer (the controls). The researchers then interview the groups or check
their records to ascertain what different experiences they had. They compare
the odds of having an experience with the outcome to the odds of having an
experience without the outcome.
For
example, are people with lung cancer more likely to be smokers than those
without lung cancer? Are women with term births more likely to have prenatal
care than women with premature births?
Case-control
studies offer many benefits to researchers. They are relatively fast and cheap
compared to cohort studies and can study multiple exposures. They are very
useful for rare diseases that require many years to develop and many people to
observe in order to find them.
However,
there are many pitfalls. Researchers must be very cautious when they choose the
control group. Controls must be very similar to the cases except for the outcome. Cases are more likely to remember past
experiences than the healthy controls. It is difficult to provide unbiased
measurements of exposure in case-control studies.
Randomized
Clinical Trials (RCTs)
RCTs follow two groups of people over time to see who achieves
a particular result. In this case, the researchers assign or randomize the
people to their groups. If this is done properly, each person has an equal
chance of being assigned to either group, reducing the possibility of the
groups having different features. Each group receives a different intervention.
When the study period ends, the researchers evaluate their different outcomes
and calculate the risk of one group developing the result compared to another.
For
example, researchers randomize workers to receive indemnity insurance or
managed care coverage. They watch people over time and measure their
utilization of health services. They compare the groups to evaluate whether the
type of insurance coverage influenced their use of medical care.
In
clinical experiments, investigators randomize patients with the same disease,
such as hypertension, to two different drug groups. One may receive an
experimental medication while the other receives a placebo. The researchers
measure the effect of the drugs on the hypertension.
Controlled
clinical trials hold the advantage in assessing causality. Randomization
produces comparable study groups that should only differ in the intervention
given. Therefore, differences in outcome can be attributed to that intervention,
such as a drug, and the researchers can conclude that the intervention caused
the results. Since the people are similar in the beginning of the study,
researchers can be confident that the intervention precedes the effect, a major
determinant of causality.
However,
clinical trials are not perfect. They are expensive and time-consuming. Biases
may render their results less reliable. For example, if study participants or
researchers know their group assignment, they may expect a particular result.
Masking study participants and researchers avoids treating people differently
based on the intervention. Losing participants along the way may also
jeopardize the results. Did people leave the study because the treatment was
too harsh or for reasons unrelated to the study?
While
randomized clinical trials can prove causality, ethical considerations usually
limit their utilization. Randomizing participants into smoking and non-smoking
arms of a study to detect lung cancer would provoke an uproar among scientists
and the public. For these reasons, many investigations must be observational in
which groups of naturally occurring behaviors are followed and measured.
Remember
also that clinical trials occur under very specialized, controlled conditions
that do not represent everyday life. Therefore, the results obtained in this
environment may not transfer to larger groups of people not engaged in
research.
U.S. National Library of Medicine (NLM)
http://www.nlm.nih.gov/
Last updated: 17 June 1998