Chapter 28: Epidemiology: Measuring Risk

Throughout this text, we have addressed the importance of understanding basic elements of research design and statistics for clinical decision making, especially within the context of evidence-based practice. In this chapter we will present an important perspective in health care research based on principles of epidemiology. The information from epidemiological research can have direct influence on practitioners' day-to-day choices related to diagnosis, prognosis or intervention. The purpose of this chapter is to present statistical methods for measures of disease frequency, estimates of health risks for cohort and case-control studies, and the evaluation of treatment effects in randomized trials.

Classically, the field of epidemiology is concerned with the study of the distribution and determinants of disease, injury, or dysfunction in human populations. Epidemiology literally began as the study of "epidemics," concerned primarily with mortality and morbidity from acute infectious diseases. Many of the health standards we take for granted today, such as clean water supplies, treatment of sewage and food refrigeration, can be credited to discoveries made through epidemiological investigations.

Epidemiologists try to identify those who have a specific disorder, when and where the disorder developed and what exposures are associated with its presence. Epidemiological questions often arise out of clinical experience, laboratory findings or public health concerns about the relationship between societal practices and disease outcomes. Through the analysis of health status indicators and population characteristics, epidemiologists try to identify and explain the causal factors in disease patterns.

As medical cures and treatments have been developed to control many of these problems, and as patterns of disease have changed, the scope of epidemiology has broadened. Today epidemiology includes the study of chronic disease, disability and health status. Because we are often concerned with functional problems as well as disease states, we will use the terms disease, disorder and disability interchangeably to represent health outcomes, including illness, injury and physical, psychological or social dysfunction. This approach fits with the World Health Organization's definition of health, which encompasses social, psychological and physical well-being.¹

Epidemiology is distinguished as a research approach because of its unique concern with the identification of risk factors for disability and disease. Epidemiologic studies are generally distinguished as observational or experimental (or quasi-experimental). In observational studies there is no artificial manipulation of any of the study factors (see Chapter 13). Observational studies are categorized as descriptive or analytic. Descriptive studies are concerned with the distribution and patterns of disease or disability in a population. These are carried out when there is little knowledge about the state of health or frequency of disease. Analytic studies test hypotheses to determine if specific exposures are related to health status or disease occurrence. Case-control and cohort studies are observational analytic approaches (see Chapter 13). Randomized controlled trials (RCT) are experimental analytic studies that are designed to test the effect of interventions on health outcomes (see Chapter 10).

Descriptive epidemiologic studies are done when little is known about the occurrence or determinant of health conditions. They will often provide information that can be used to set priorities for health care planning, and will generate hypotheses that can be studied using analytic methods. Descriptive studies may be presented as case reports, correlational studies, or cross-sectional surveys (see Chapter 14).

Person, Place and Time

The purpose of descriptive epidemiologic studies is to describe patterns of health, disease and disability in terms of person, place and time.

Who experiences this disorder? Relevant characteristics might include age, gender, religion, race, cultural background, education, socioeconomic status, occupation and so on. This is the demography of the disorder. Epidemiologists try to determine if individuals with certain characteristics are more at risk for a particular disorder than others. For example, researchers have studied the increasing prevalence of type 2 diabetes in adolescents,² and the incidence of incontinence in women over age 45.³

Where is the frequency of disorder highest or lowest? Epidemiologists may be concerned with identifying restricted areas within a city or large geographic areas in which disease or exposures are commonly found. They may look at environmental factors such as weather, local industry, water source and lifestyle as potential causative factors. For instance, the early studies in AIDS documented high incidence in San Francisco and New York.⁴ Legionnaire's disease⁵ and severe acute respiratory syndrome (SARS)⁶ are other examples of diseases that had specific geographic origins (see Box 28.1).

When does the disorder occur most or least frequently? The epidemiologist will compare the present frequency of a disorder with that of different time periods. When the frequency of occurrence varies significantly at one point in time, some specific time-related causative factor is sought. Seasonal variations may become obvious, or trends may be related to other historical factors. For example, researchers have found a higher incidence of hip fractures in elderly individuals during winter months,¹² and an increased rate of hospitalization due to adult asthma symptoms in spring months.¹³

Disease Frequency

The statistical measures used to describe epidemiologic outcomes focus on quantification of disease occurrence. The simplest measure of disease frequency would be a count of the number of affected individuals; however, meaningful interpretation and comparisons of such a measure would also require knowing how many people there were in the total population who could have gotten the disease and the length of time over which the occurrence of the disease was monitored. Therefore, measures of disease frequency will always include reference to population size and time period of observation. For example, we might document 35 cases of a disease within 1 year in a population of 3,200 people, or 35/3,200/year. Typically, population size is expressed in terms of thousands, such as 1,000 (10³), 10,000 (10⁴), and 100,000 (10⁵). For instance, the preceding values would be expressed as 10.94 cases per 1,000 per year. To make estimates more useful, such rates are usually calculated in whole numbers, such as 1,094/100,000/year.

The number of cases of a disease that exist in a population reflects the risk of disease for that group. It describes the relative importance of the disease and can provide a basis for comparison with other groups who may have different exposure histories. The two most common measures of disease frequency are prevalence and incidence.

Prevalence

Prevalence is a proportion reflecting the number of existing cases of a disorder relative to the total population at a given point in time. It provides an estimate of the probability that an individual will have a particular disorder at that time. Prevalence (P) is calculated as

For example, we know that obesity has become a national concern. The National Health Interview Survey in 2000 found that the number of adults with self-reported obesity was 7,058 out of a sample of 32,375.¹⁴ The prevalence of obesity in this population is expressed as

Therefore, there is a 22% probability that any randomly selected individual from this population would be obese. Because this value reflects the cross-sectional status of the population at a single point in time, it is also called point prevalence.

Prevalence can also be established for a specified period in time. For example, data obtained from a random sample of 973 newspaper employees found that the number of individuals categorized as having upper limb musculoskeletal complaints after 1 year was 395.¹⁵ The estimate of the prevalence of upper limb musculoskeletal complaints in this population during a 1-year period is, therefore, 41%. This measure, combining existing with new cases of musculoskeletal complaints during the period of one year, is referred to as period prevalence.

Prevalence is most useful as an indicator for planning health services, because it reflects the impact of a disease on the population. Therefore, a measure of prevalence can be used to project requirements such as health care personnel, specialized medical equipment and number of hospital beds. Prevalence should not, however, be used as a basis for examining etiology of a disease because it is influenced by the length of survival of those with the disorder; that is, prevalence is a function of both the number of individuals who develop the disease and the duration or severity of the illness. Because this estimate looks at the total number of individuals who have the disease at a given time, that number will be large if the disease tends to be of long duration.

Incidence

The measure of incidence quantifies the number of new cases of a disorder or disease in the population during a specified time period and, therefore, represents an estimate of the risk of developing the disease during that time. Incidence discounts the effect of duration of illness that is present in prevalence measures. By examining incidence rates for subgroups of the population, such as age groups, ethnic groups and geographic locations, the researcher can identify those groups that demonstrate higher disease rates and target them to investigate specific exposures. Incidence can be expressed as cumulative incidence or incidence rate.

Cumulative incidence (CI) quantifies the number of individuals who become diseased during a specified time period:

For example, in a study of low back pain 196 men who had recently taken up golf were followed over a 1-year period.¹⁶ During that time, 16 new cases of back pain were identified. The 1-year cumulative incidence of first-time back pain for this cohort was 8% (16/196). The specification of the time period of observation is essential to the interpretation of this value. The number of cases would be perceived differently if subjects were followed for 1 or 10 years. Other issues that require consideration in interpreting a measure of cumulative incidence include the possibility that the number of individuals at risk in the cohort will vary over time, and the possibility that the condition under study is caused by other, competing risks.

Person-time. Measuring the total population at risk for cumulative incidence assumes that all subjects were followed for the entire observation period; however, some individuals in the population may enter the study at different times, some may drop out, and others who acquire the disease are no longer at risk. Therefore, the length of the follow-up period is not uniform for all participants. To account for these differences, incidence rate (IR) can be calculated:

As in cumulative incidence, the numerator for this estimate represents the number of new cases of the disorder; however, the denominator is the sum of the time periods of observation for all individuals in the population at risk during the study time frame, or person-time. For example, in the Nurses' Health Study, 121,700 female nurses were enrolled in 1976. During the period of 1976 to 1992, investigators identified 3,603 new cases of breast cancer.¹⁷ Of the women originally enrolled, some left the study as a result of death or loss to follow-up at various times during the period, and some developed breast cancer after different amounts of time, contributing different amounts of time to the denominator. In other words, a woman who died in 1977 in an automobile crash would have contributed 1 person-year to the denominator, whereas two women who developed breast cancer in 1990 would have contributed a total of 28 person-years to the denominator.

Researchers totaled the amount of time each subject was known to be at risk between 1976 and 1992, and obtained the total person-years observed, in this case there were 1,794,565 person-years of observation. The incidence rate was, therefore,

or 2 cases per 1,000 person-years (2 × 10⁻³ years). Incidence rate is often a more efficient measure than cumulative incidence, as it allows for inclusion of all subjects, regardless of the amount of time they were able to participate. Cumulative incidence would only account for those subjects who were available for the entire study period.

The Relationship between Prevalence and Incidence

The relationship between prevalence and incidence is a function of the average duration of the outcome of interest. If the incidence of the disorder is low (few new cases occur) but the duration of the disorder is long, then the prevalence, or proportion of the population that has the disease at a given point in time, may be large. If, however, incidence is high (many new cases of the disease occur) but the disorder is manifest for a short duration (either by quick recovery or death), the prevalence may be low. For example, a chronic disease such as arthritis may have a low incidence but high prevalence. A short-duration curable condition like a common cold may have a high incidence but low prevalence, because lots of people get colds but few actually have colds at any one point in time.

Vital Statistics

Epidemiologists often use incidence measures to describe the health status of populations in terms of birth and death rates that inform us about the consequences of disease. The birth rate is obtained by dividing the number of live births during the year by the total population at midyear. The mortality rate quantifies the incidence of death in a population by dividing the number of deaths during a specific time period by the total population at the midpoint of the time period. These data are generally available through records of state vital statistics reports, census data and birth and death certificates.

The mortality rate can reflect total mortality for the population from all causes of death in the crude mortality rate, in which the total number of deaths during the year is divided by the average midyear population. This value is usually expressed as the number of deaths per 100,000 population; however, when different categories within the population differentially contribute to this rate, it may be more meaningful to look at category-specific rates. A cause-specific rate looks only at the number of deaths from a particular disease or condition within a year divided by the average midyear population. For instance, rates may reflect mortality specifically resulting from diseases such as cancer and heart disease or from motor vehicle accidents. The case-fatality rate is the number of deaths from a disease relative to the number of individuals who had the disease during a given time period.

Other commonly used categories are age, sex and race. Age-specific rates are probably most common because of the differential effect of many diseases across the life span. For example, if one looks at the death rate for cancer across age groups, we would find that mortality was higher for older age categories. Therefore, it may be more meaningful to present age-specific mortality rates for each decade of life, rather than a crude mortality rate; however, this results in a long list of rates that may not be useful for certain comparisons. An overall rate would be more practical, but it would have to account for the variation in rates across age categories. For instance, if we compare the crude cancer mortality rate for today versus the crude rate from 50 years ago, we would have to account for the fact that a larger proportion of the total population now falls in the older age range. Therefore, epidemiologists will often report age-adjusted mortality rates that reflect different weightings for the uneven categories. Methods for calculating adjusted rates are described in most epidemiology texts.

Analytic epidemiology is concerned with testing hypotheses. Measures of association are typically derived for case-control and cohort studies (see Chapter 13), to assess the relationship between specific exposures and disease. These tests will establish if an association exists and the strength of that association. If an association does exist, we say that the specific exposure represents a risk factor for the disease.

The focus on exposures takes a broad view that reflects contemporary concerns including lifestyle practices such as smoking, substance abuse, drinking alcohol or coffee and eating foods high in cholesterol or salt; occupational hazards, such as repetitive tasks or heavy lifting; environmental influences, such as second-hand smoke, toxic waste and sunlight; and specific interventions, such as exercise, medications or treatment modalities. These exposures increase or decrease the likelihood of developing certain disorders or influence the ultimate outcome of a disorder. For example, smoking and sunlight are considered risk factors that increase the chance of developing cancer.^18,19 Stroke patients with comprehension deficits have an increased risk of poor therapeutic outcomes.²⁰ Exercise and higher fitness level in men with diabetes are associated with reduced risk of mortality from cardiovascular disease.²¹

This is a fundamental process in the determination of prognosis, as we attempt to predict outcomes based on patient characteristics. As with all measures of association, risk does not necessarily mean that the exposure causes the outcome.

Relative versus Absolute Effects

Analyses of association are based on a measure of effect that looks at the frequency of disease among those who were and were not exposed to the risk factor. A relative effect is a ratio that describes the risks associated with the exposed group as compared with the unexposed. An absolute effect is the actual difference between the rate of disease in the exposed and unexposed groups, or the difference in the risk of developing the disease between these two groups. To illustrate the concepts of relative and absolute effect, suppose we purchased two books, one costing $3 and the other $6. The absolute difference is $3, whereas the relative difference is that the second book is twice as expensive as the first. Therefore, the relative effect is based on the absolute effect, but takes into account the baseline value. Analogously, we can use measures of incidence of disease in exposed and unexposed groups to determine both relative and absolute effects of particular exposures.

Relative Risk

The most common measure of relative effect is relative risk (RR), which indicates the likelihood that someone who has been exposed to a risk factor will develop the disease, as compared with one who has not been exposed. Relative risk is defined as the ratio of incidence of disease among the exposed subjects to the incidence of disease among the unexposed. Measures of relative risk are appropriate for use with cohort studies.

To determine risk, data are typically organized in a 2 × 2 table, called a contingency table, as shown in Figure 28.1. The vertical columns in the table represent the classification of disease status (the outcome), and the horizontal rows represent exposure status. To facilitate consistency in presentation and calculation, the cells in the table are designated a, b, c, and d, as shown in the figure. Therefore, cell a represents those who have the disease and were exposed, cell b represents those who do not have the disease and were exposed, and so on. The marginal totals for each row and column represent the total numbers of individuals who were exposed (a + b) and were not exposed (c + d), and the total numbers who have the disease (a + c) and do not have the disease (b + d). The sum of all four cells is the total sample size (N).

For a cohort study, we can obtain cumulative incidence estimates for the exposed (CI_E) and unexposed (CI₀) groups. The cumulative incidence for the exposed group is the number of cases of the disease among the total exposed sample, or a/(a + b). The cumulative incidence for the unexposed group is the number of cases of the disease among the total unexposed sample, or c/(c + d).^∗ Therefore,

If the incidence rates of the outcome are the same for the exposed and unexposed groups, the relative risk is 1.0, indicating that the exposure presents no excess risk for the outcome. Therefore, a relative risk greater than 1.0 indicates an increased risk, and a relative risk less than 1.0 means that the exposure decreases the risk of developing the disorder.

Example

To illustrate this application, consider the data shown in Table 28.1 A for a cohort study of the risk of hip fracture associated with leisure time physical activity.²² Data were taken from longitudinal studies over six birth cohorts. For this example, we will look at a subsample of 130 women. The research question is: Does physical activity reduce the risk of hip fracture in elderly women?

Our first step is to determine what proportion of patients who exercised sustained a hip fracture. This is the incidence of hip fracture among exercisers, 48 out of 98, or 49%. Then we determine what proportion of sedentary patients sustained a hip fracture. This is the incidence of hip fracture for those who did not exercise, 20 out of 32, or 63%. Relative risk is the ratio of these two proportions:

This tells us that the risk of hip fracture was decreased among nonsedentary women; that is, those who were active at least 2 hours/week were 0.78 times as likely (less likely) to have a hip fracture as compared with those who were sedentary.

Confidence Intervals for Relative Risk

An important assumption in any research study is that we can draw reasonable inferences about population characteristics based on sample data. This assumption holds true for epidemiologic studies as well. When a risk estimate is derived from a particular set of subjects, the researcher will use that estimate to make generalizations about expected behaviors or outcomes in others who have similar exposure histories. Therefore, it is important to determine a measure of true effect using a confidence interval (see Chapter 18). For example, as shown in Table 28.1B, with an observed relative risk of 0.78 for the association between hip fracture and physical activity, the 95% confidence interval is 0.560 to 1.097. This interval represents a range of values within which the true population effect is expected to fall.

Confidence intervals can also be used to provide information about statistical significance by referring to the null value for relative risk, which is 1.0. We look to see if the null value is included within the 95% confidence interval. If the null value is contained within the confidence interval, and we are 95% confident that the interval contains the true population value, then we cannot rule out 1.0 as the population value. Therefore, the estimate is not considered significant. If the null value is not contained within the interval, the estimate is considered significant; that is, we are 95% sure that the null value is not the true population value. In our example, the 95% confidence interval is 0.56 to 1.097. As this interval contains the null value of 1.0, we would state that the observed association is not statistically significant at the .05 level. Therefore, we must conclude that, although the RR value shows a reduced risk for hip fracture with physical activity, this value could have occurred by chance.

Chi-square can also be used as a test of significance, to determine if the proportions differ across categories in a crosstabulation (see Chapter 25). In this example, the value of chi-square results in p = .184 (see Table 28.1B), which is not significant. This confirms the conclusion drawn from the confidence interval analysis, with a parallel interpretation. Chi-square tells us that the proportion of individuals with and without hip fracture who were in the two physical activity groups was not different from what would be expected just by chance.

Odds Ratio

A case-control study differs from a cohort study in that subjects are purposefully chosen based on the presence or absence of disease (cases or controls) and therefore, we cannot determine the rate of incidence of the disease (see Chapter 13). Relative risk is not an appropriate measure for case-control studies because we cannot calculate cumulative incidence. The relative risk can, however, be estimated using an odds ratio (OR), which is calculated using the formula

The odds ratio is interpreted in the same way as relative risk, with a null value of 1.0.

Example

Consider the data shown in Table 28.2A. These data are from a case-control study which examined the risk for developing plantar fasciitis associated with body mass index (BMI).²³ The researchers assembled a sample of 50 cases and 100 controls, with a 1 : 2 match on age and gender. Among the cases, 29 individuals had a BMI over 30 (considered obese); among the controls, 17 subjects had a BMI over 30. The crude odds ratio for these data is

This means that the odds of developing plantar fasciitis are almost seven times greater for those who are obese than for those who are not. In other words, being obese appears to increase the risk of developing plantar fasciitis.

Confidence Intervals for the Odds Ratio

Confidence intervals can also be generated for the odds ratio to determine the significance of the ratio as an estimate of population values. As shown in Table 28.2B, the confidence interval for the relationship between plantar fasciitis and BMI is 3.13 to 14.51. This interval does not contain the null value of 1.0, and therefore, this represents a significant odds ratio.

Chi-square can also be used to determine if the proportion of individuals varies across categories. This outcome is shown in Table 28.2B, confirming the significant outcome of the confidence interval analysis.

Confounding and Effect Modification

Quite often, in the analysis of the association between a risk factor and disease, researchers seek to infer a potential causal relationship between the two. The researcher also recognizes, however, that in a study of association, cause and effect cannot be readily established (as it can in an experimental study) because other factors may contribute to the observed relationship. Alternatively, we may see no association because other factors actually obscure the relationship between exposure and outcome.

In some cases, these extraneous variables provide information important to understanding how the association varies across different subgroups, such as age or gender. In other situations, such variables create a bias in the interpretation, interfering with the true association being studied. These two complications of analysis are called confounding and effect modification.

The simplest type of analyses are based on crude data. These are data concerning the exposure status and outcome status of all subjects regardless of any other risks or characteristics. Although analyses based on crude data are often reported in the literature, most studies also require more complicated analyses to evaluate the role of other factors in the relationship of exposure and outcome. These analyses are accomplished through stratification or multivariate methods and provide adjusted measures of association. Researchers must consider the potential influence of confounding and effect modification in all analyses and account for them in the design or analysis of data as much as possible.

Confounding

Confounding variables can be thought of as nuisance variables. Confounding is introduced when extraneous variables interfere with the observed association between the exposure and outcome. A confounder is a variable that (1) is associated with the exposure, (2) is a risk factor for the disease independent of the exposure, and (3) is not part of the causal link between the exposure and the disease (Figure 28.2A). In other words, a confounding variable is associated with the predictor variable, but may also be a risk factor for the outcome variable, and therefore must be ruled out. Confounding occurs when the exposure can become confused or distorted by the extraneous variable.

Example. To illustrate this concept, Jackson and co-workers²⁴ examined the association between risk of mortality and receiving influenza vaccine in elders over 65 years. They studied 252 cases who died during an influenza season and 576 age-matched controls. The crude odds ratio for this relationship was 0.76 (95% Cl 0.47, 1.06), indicating that receiving the vaccine decreased the risk of death. The researchers were interested, however, in the potentially confounding effect of limited functional status, which (1) would be associated with not getting the vaccine, (2) would be a risk factor for mortality, and (3) is not a causal link between the vaccine and mortality. When they adjusted for the effect of functional limitations, the odds ratio was lowered to 0.59 (95% Cl 0.41, 0.83). Because older individuals who have functional limitations would be less likely to visit a clinic to get the vaccine, and mortality is also related to functional decline, the crude odds ratio was an underestimate of the protective nature of the vaccine on mortality. When function is taken into account, the actual risk of death is lower. If there were no discrepancy between the crude and unconfounded estimates, there would be no confounding. The degree of discrepancy is indicative of the extent to which function confounded the original data. By eliminating the effect of functional status, a stronger (and significant) relationship was seen between getting the vaccine and decreased mortality.

Adjusting for Confounders. Confounding variables may or may not be present in a study, depending on the source population and how subjects are chosen. When confounding is present, the statistical outcome may not be documenting the true causal factor. A method commonly used to adjust for a potential confounder is stratification, in which the comparison between exposure and disease is done at specific levels of the potential confounder. When a study mentions that researchers "controlled" or "adjusted" for a factor in the analysis, they have tried to remove the effect of that variable as a confounder.

To evaluate the effect of confounding in an analysis, the researcher must collect information on the potentially confounding variable. If the investigator in the vaccine study did not collect data on the subjects' functional status, the analysis of the confounder would not have been possible. Therefore, the researcher must be able to predict what variables are possible confounders. It is conceivable that several confounding factors will be operating in one study. In addition to controlling for confounding in the analysis, researchers can use design strategies, such as matching or homogeneous subjects, to control for these effects (see Chapter 9). For instance, in the study of influenza vaccine, if we were to restrict subjects to only those who were functionally independent, then function would not be a confounding factor.

Age and gender are often considered potential confounders in epidemiologic studies because of their common association with disease and disability, as well as being related to the presence of many exposures. Other common confounders are socioeconomic status, education level, marital status, weight, and cognitive status.

Effect Modification

In contrast to confounding, effect modification occurs when the presence of one variable modifies the association between an exposure and outcome. Where a confounder is a nuisance that needs to be controlled to get an accurate estimate of an association, an effect modifier is a real effect that helps explain the biologic relationship between the exposure and outcome.²⁵ Researchers attempt to cancel the effect of a confounding variable in the design or analysis of a study. Effect modifiers are studied so they can be reported.

An effect modifier will interact with the exposure and disease variables in such a way as to present a constant effect. It is a natural phenomenon that exists independent of the study design and will always be a factor in interpretation of risk. Effect modifiers tend to be biologically related to the variables being studied.

Example. Researchers have studied the association between diabetes and risk of endometrial cancer, with many conflicting results. Friberg et al²⁶ hypothesized that this relationship could be misunderstood because of major modifiers such as physical activity. They studied a cohort of over 36,000 women over 7 years and found a relative risk of endometrial cancer of 2.37, adjusted for age, for women diagnosed with diabetes as compared to women without diabetes.

They then stratified the sample by physical activity (see Table 28.3). For diabetics with high physical activity, they found a RR for endometrial cancer of 1.06 as compared to those without diabetes who were active. Therefore, physically active diabetics were essentially at no increased risk of cancer. For diabetics with low physical activity, however, they found a RR of 2.67. The risk associated with endometrial cancer increased by more than two times for those who did not exercise. The fact that the risk estimates are different for each stratum indicates that physical activity interacts with diabetes as an effect modifier; that is, the association between diabetes and endometrial cancer is significantly modified by physical activity.

Pooled Risk Estimates

When data are stratified, separate risk estimates are calculated for each stratum; however, it is usually more useful to consider a single overall estimate that reflects the association between exposure and disease with the confounding factor taken into account. Several statistical techniques can be used to accomplish this, although the most commonly used procedures belong to a set of estimates proposed by Mantel and Haenszel.²⁷ The Mantel-Haenszel pooled risk estimate provides a weighted summary value that can be used to report the relative risk associated with a specific exposure adjusted for the confounding variable. When the Mantel-Haenszel estimate differs from the crude risk estimate, it is the Mantel-Haenszel estimate that should be reported. It is most appropriately used when the stratum-specific relative risks are uniform, that is, when there is no effect modification. Formulas used to calculate Mantel-Haenszel estimates of relative risk for case-control and cohort studies are given in Table 28.4. The numerators and denominators in these formulas represent the sum of the expressions for each stratum.

When making clinical decisions regarding the effectiveness of interventions, we generally want to know if a treatment will improve the patient's condition, or if it will prevent or decrease the risk of an adverse event. Randomized controlled trials (RCT) are the most effective approach for answering these questions, and we typically use a statistical test to compare groups on means or proportions to determine if they are different from each other after the treatment. Such a conclusion is limited, however, because it does not tell us if the difference is clinically important, nor does it help us estimate the likelihood that our own patient will respond favorably. We know that even with a well-established intervention, patients do not all respond the same way. Some will improve and others will not; some will experience an adverse outcome, and others will be fine. So how can we determine the likelihood that our particular patient will benefit from the intervention?

In addition to specific measures of change, then, we may also consider the outcome of treatment in terms of an "event," which is classified as a success or failure. The success of a treatment is usually determined as a beneficial outcome based on a specific threshold, which may be related to an impairment or functional activity. For instance, back pain is relieved or reduced by a certain percent; a child is able to utter a given number of sentences without stuttering; an obese patient loses a certain amount of weight; a patient with congestive heart failure is able to increase walking tolerance by a given distance. Success may also be indicated by whether an adverse outcome is prevented. For example, a patient may be given a drug to control hypertension to prevent stroke. If the patient subsequently suffers a stroke, he has experienced an adverse outcome. In a RCT, the success of the intervention is reflected by the difference in beneficial or adverse outcomes between treatment and control groups. The concept of relative risk (RR) can be used here in reference to treatment effect, indicating if the risk of a particular outcome for the control group is higher or lower than for the treatment group.

Example

Let's put this in the context of a rehabilitation example. A group of researchers studied the effect of exercise and manipulative therapy for reducing cervicogenic headaches.²⁸ They measured pain intensity and duration as well as the frequency of headaches over a 7-week treatment period. We can appreciate the effect of treatment by setting a threshold that identifies a successful outcome, based on clinically important change. In this study, in addition to the typical comparisons between group means for pain and duration, the authors set a standard of 50% or better reduction in headache frequency as a benchmark for "success" of treatment. If the reduction in recurrence of headache was less than 50%, it was considered an adverse outcome.

Table 28.5A shows the results of this study for the comparison of combined manipulative therapy and therapeutic exercise with a control. We can see that of the 49 patients who received the experimental intervention, 9 had recurring headaches. Of the 48 patients in the control group, 34 had recurring headaches.

Event Rates and Risk Reduction

We can examine differences between group responses in terms of "event rates," or the proportion of subjects in the experimental and control groups that achieved an adverse or successful outcome. We calculate two values: an experimental event rate (EER) for those receiving the intervention and a control event rate (CER) for the control group, as shown in Table 28.5B. For this study, the EER is 18% and the CER is 71%. These values indicate the risk of headaches recurring for each group. But how do they compare?

The ratio of these two values is the relative risk (RR) associated with the intervention:

As shown in Table 28.5B, the RR associated with this intervention is .25. This means that the intervention group was only one-quarter as likely to experience a recurrence of headache as the control group.

Risk Reduction

This effect is better understood, however, as a relative value that reflects the decrease in risk associated with the intervention, called the relative risk reduction (RRR), which is equal to:

As shown in Table 28.5B, RRR = .75 for the headache study. This tells us that there is a 75% reduction in risk for recurring headaches associated with the intervention compared to the control.

As we have discussed before, the disadvantage of RRR is that a relative value does not tell us anything about the actual size of the effect. Figure 28.3 illustrates this limitation. Consider alternative results for a hypothetical comparable study (Study B), where the EER was 8% and the CER was 2%. These effects are certainly not clinically meaningful, and yet their RRR would also be .75.

A more clinically useful measure is the absolute risk reduction (ARR), which indicates the actual difference in risk between the groups, easily calculated by:

For the headache example, then, ARR = .53. There is a 53% absolute reduction in risk of recurring headaches associated with the intervention. If we look at the hypothetical study (B) in Figure 28.3, we can see that the ARR = .06, only a 6% reduction in risk. Therefore, the absolute risk reduction is a better reflection of the true difference in the studies' outcomes.

This example illustrates the importance of considering baseline risk when interpreting risk reduction.²⁹ The control event rate can be used as an estimate of the risk for all subjects at the start of the study, before intervention is applied. Although a RRR of 75% would be considered impressive, we can see that this value is not meaningful when the baseline risk is only 8%. This absolute risk reduction indicates that the treatment will be of minor benefit in Study B. Compare that to a baseline risk of 71% for study A, and we can see that treatment will be of greater benefit.

Number Needed to Treat

The degree of risk reduction should help us decide whether a treatment is worth pursuing, based on the likelihood that the outcome will be successful (avoiding an adverse event). The ARR, however, does not provide the clinician with a clinical value that can be used to estimate the number of patients that would need to be treated before benefit will be observed. The number needed to treat (NNT) was developed as a statistic that provides information about effectiveness in terms of patient numbers.³⁰ The NNT is defined as the number of patients that would need to be treated to prevent one adverse outcome or to achieve one beneficial outcome in a given time period. It can be calculated for any trial that reports a binary outcome.³¹

The NNT is easily calculated as the reciprocal of the ARR:

where ARR is expressed as a decimal. A large treatment effect will translate to a small number needed to treat.

If the NNT is 1.0, this means that we would need to treat 1 patient to avoid 1 adverse outcome; that is, every patient will benefit from treatment. This is, of course, the ideal, although not often the case. The closer to 1.0, the better the NNT. For the headache example, the NNT is 1.9 (shown in Table 28.4B), which is rounded up to 2.0. Therefore, we would need to treat 2 patients with manipulative therapy and exercise to prevent a recurrence of headaches in 1 patient; that is, 1 out of 2 patients will experience a successful outcome. If the absolute risk reduction is zero (no difference between CER and EER), the treatment has had no effect, and the NNT will be infinity. We would need to treat an infinite number of people to see any benefit.

Confidence Intervals for NNT

The NNT is a point estimate, and therefore, as with other estimates, it should be presented with a confidence interval for accurate interpretation.³¹ The confidence limits for NNT are the reciprocals of the confidence limits for ARR, in reverse order. See Table 28.5C for formulas and sample calculations. The null value for ARR is zero, which converts to infinity for the NNT.^† For the headache study, we are 95% confident that the true population ARR is between 36% and 70%. We are 95% confident that the NNT falls between 1.43 and 2.75.

Number Needed to Harm (NNH)

The NNT reflects the prevention of an adverse event, which is generally seen as a successful outcome. For example, treatment for hypertension should prevent adverse events such as stroke, heart attack or death. We can also apply this concept to evaluate serious side effects, risks or complications that occur from treatment, outside of the intended effects. When an intervention poses excess risk to the patient, we would want to assess the absolute risk increase (ARI) associated with it. The reciprocal of this value is called the number needed to harm (NNH), which indicates the number of patients who would need to be treated to cause 1 adverse outcome.^‡ The larger the NNH, the less likely a patient is to experience an adverse outcome. An NNH of 100 would mean that we would need to treat 100 patients to cause one adverse event. An NNH of 1.0 would mean that every patient would experience an adverse event.

The NNH should be considered along with the NNT to evaluate the benefit and harm of an intervention.³² For example, a systematic review of the effects of aspirin for treatment of acute pain analyzed the effectiveness of a 600 mg dose for at least 50% pain relief, with an NNT of 4.4 (95% CI 4.0, 4.9).³³ They also reported an NNH of 38 (95% CI 22,174) for the side effect of gastric irritation. According to this analysis, if 38 patients were treated, 9 would achieve a positive outcome, and 1 patient would be expected to experience gastric irritation.

Cautions in Interpretation of NNT

In terms of clinical decision making, NNT and NNH provide a useful number to help a clinician and patient decide if one or another treatment is worth pursuing, weighing their potential benefit or harm. The NNT is a particularly useful measure for expressing the relative effectiveness of different interventions. It quantifies the effort required to obtain a beneficial outcome. Several factors must be considered, however, when comparing values of NNT across different interventions or studies.

1. The NNT must be interpreted in terms of a time period for treatment and follow-up. The duration of the treatment may make a difference in the frequency of expected positive and negative outcomes. Generally, the NNT will be smaller for an intervention of longer duration.³⁴ The comparison of NNT values for different treatments is only valid when the outcome is measured within the same time period. For instance, for the headache study the report should read: The NNT was 2 over 7 weeks.

2. The interpretation of the NNT will depend on baseline risk. Because some patients will have a greater risk of an adverse outcome before treatment begins, the NNT must be adjusted to account for low and high baseline risks. It may not be reasonable to assume that the same relative risk applies to all patients.³² The patient's age, gender or initial severity may alter the relative risk associated with the intervention. Therefore, when extrapolating NNT measures from the literature, the baseline risk must be taken into account. The control event rate can be used as an indicator of the baseline risk without treatment.

3. To compare NNTs, the outcomes of interest must be the same. For example, in the evaluation of exercise programs, an NNT of 20 for preventing falls in the elderly may be interpreted differently from an NNT of 20 for preventing hip fracture. The NNT for a drug treatment to reduce hypertension will be different if the outcome is stroke, heart attack or death. In the study of cervicogenic headache, the threshold for success was set at 50%. If this threshold were changed, the resulting NNTs would not be comparable.

4. The validity of the clinical trial must also be taken into account. Because NNTs are used to make individual patient decisions and to support reimbursement policies, the degree to which research studies represent actual clinical expectations will influence the application of effective treatments.³⁵ Both internal and external validity must be considered. Replication is important to determine if results stand up to different samples.

5. There are no standard limits for NNT (or NNH) that dictate a decision. Like all research results, NNT should be incorporated into decision making along with the clinician's experience and judgment, the patient's preferences and the nature of the disorder that is being treated.³² For some interventions an NNT of 2 or 3 would be considered good, whereas for others an NNT of 20 or 40 may still be considered clinically effective. Similarly, an acceptable NNH will be determined by the severity of the risks, balanced with the probable benefit. Therefore, the value of NNT and NNH should always be interpreted within the context of a specific disorder, treatment, outcome, disease severity and time period. For example, a review of treatments for hypertension to prevent stroke reported a NNT for moderate hypertensive patients of 13, and a NNT for mildly hypertensive patients of 167.³⁶ The recommendations for clinical management of these patients would obviously be quite different.