Stratified Analysis

Abstract

Both stratum-specific analyses and overall assessment require a point estimate, an interval estimate, and a test of hypothesis. In this lesson, we focus on overall assessment, which is the most conceptually and mathematically complicated of the four steps. For overall assessment, the point estimate is an adjusted estimate that is typically in the form of a weighted average of stratum-specific estimates. The confidence interval is typically a large-sample interval estimate around the adjusted (weighted) estimate. The test of hypothesis is a generalization of the Mantel-Haenszel chi square test.

Appendices

Homework

1.1 ACE-1. Looking at Stratified Data

A case-control study was performed to assess the relationship between alcohol consumption (ALC) and oral cancer (OCa). The results, stratified on smoking status, are displayed below:

	Current Smoker				Former Smoker
	ALC	no ALC			ALC	no ALC
OCa	42	7		OCa	100	2
No OCa	3	4		no OCa	48	5

	Never Smoker
	ALC	no ALC
OCa	158	4
no OCa	125	8

• a. What is the crude odds ratio for these data?

• b. Calculate the stratum-specific ORs. Is there evidence of effect modification by smoking status? Justify your answer.

• c. Which of the following would be appropriate for the analysis of these data? [You may choose MORE than one.]

• i. Calculate an overall summary odds ratio for the relationship between alcohol and oral cancer, adjusted for smoking status.

• ii. Report stratum-specific effects.

• iii. Calculate χ2 tests of association for the alcohol-oral cancer relationship SEPARATELY for the three strata.

• iv. Compare the crude and adjusted estimated ORs to determine whether there is confounding by smoking status.

• v. Calculate a test of heterogeneity to help determine whether there is effect modification by smoking status.

1.2 ACE-2. Overall Assessment

Suppose another case-control study of the same exposure-disease relationship (i.e. alcohol and oral cancer) was performed, but this study was restricted to never smokers. Results of this study, stratified on age, are displayed below:

	Age 40-49				Age 50-59
	ALC	no ALC			ALC	no ALC
OCa	4	25		OCa	12	10
No OCa	22	309		no OCa	37	67

	Age 60+
	ALC	no ALC
OCa	11	12
no OCa	31	67

• a. What is the crude odds ratio for these data?

• b. Calculate the stratum-specific ORs. Is there evidence of effect modification by age? Justify your answer.

• c. Is the observed overall association between ALC and OCa, stratified on age, statistically significant? (Answer this question by carrying out an appropriate χ² test. Be sure to state the null hypothesis and the p-value for the test.)

• d. Calculate a precision-based summary odds ratio. Show your calculations. Which stratum has the smallest variance? Justify your answer.

• e. Calculate a Mantel-Haenszel summary odds ratio. Show your calculations. Based on this summary estimate, is there evidence of confounding by age in these data? Justify your answer.

1.3 ACE-3. Paternal Radiation Exposure and Birth Defects

A case-control study was conducted to assess whether paternal radiation exposure on the job was associated with birth defects. The investigators were concerned with maternal age as a potential confounder, so they stratified the data as follows:

	Maternal age >35				Maternal age <35
	Radiation	No Radiation			Radiation	No Radiation
Birth Defect	21	26		Birth Defect	18	88
Control	17	59		Control	7	95

• a. Calculate a Mantel-Haenszel summary odds ratio. Show your calculations.

• b. Is there evidence of confounding by maternal age in these data? Justify your answer.

• c. Is the observed association between paternal radiation exposure and birth defects, controlling for maternal age, statistically significant? (Answer this question by carrying out an appropriate statistical test. Be sure to state the null and alternative hypotheses and provide a p-value.)

• d. Calculate a test-based 95 % confidence interval for the summary estimate in part a above. (See “Hint” at the end of this entire question)

• e. When possible, information on paternal radiation exposure (for the study described above) was taken from employment records rather than from subject interviews. This was done in an effort to MINIMIZE which of the following? [Choose ONE best answer]:

• i. Detection bias

• ii. Differential misclassification of the outcome

• iii. Recall bias

• iv. Selection bias

Hint: A test-based confidence interval for a measure of effect θ (e.g., θ is an OR or an RR) provides an alternative method for calculating a confidence interval that avoids having to calculate an complex mathematical expression for the estimated variance of ${\rm \hat \theta}$. For a ratio measure of effect (e.g., OR, RR, IDR), the 95% test-based formula is given as follows:

$${\rm \hat \theta}^{( 1{\rm } \pm {\rm 1}{\rm.96/}\chi _{{\rm MHS )}} } {\rm }$$

where ^χ_MHS is the square-root of the Mantel-Haenszel chi-square statistic for a stratified analysis)

1.4 ACE-4. Lead Exposure and Low Birth Weight

The relation between paternal occupational lead exposure and low birth weight was examined in a retrospective cohort study. Men with a blood lead level (BLL) > 50 micrograms per deciliter were considered exposed. Low birth weight (LBW) was defined as a birth weight of less than 2500 grams. Data from the study, stratified on maternal age at child’s birth, are provided below:

	Maternal Age <20				Maternal Age 21+
	High BLL	Low BLL			High BLL	Low BLL
LBW	45	30		LBW	68	48
No LBW	169	214		No LBW	257	354

• a. Is maternal age an independent risk factor for LBW in these data? Show any calculations and justify your answer.

• b. Use the data-based method to determine whether there is confounding by maternal age in this study. Show any calculations and justify your answer.

• c. Is the observed association between paternal blood lead level and low birth weight, controlling for maternal age, statistically significant? Answer this question by carrying out an appropriate statistical test. Be sure to state the null and alternative hypotheses and provide a p-value or p-value range.

• d. Calculate an overall summary risk ratio that gives equal weight to the two strata defined above. (i.e. a simple average)

• e. Calculate a 95% test-based confidence interval for the summary estimate determined for part d above. (See “Hint” at the the end of question 3 for the formula of a test-based confidence interval.)

• f. Suppose that, prior to analyzing the data, the investigators were concerned about the possibility of residual confounding. Which of the following would have been a useful method of addressing this concern? [Choose one best answer]:

__Dividing blood lead level (BLL) into additional, narrower categories

__Dividing low birth weight (LBW) into additional, narrower categories

__Dividing maternal age into additional, narrower categories

__All of the above

• g. Is there evidence of effect modification (by maternal age) on an additive scale in these data? Show any calculations and justify your answer.

• h. Based upon your answer to part 3.G. above, which of the following would be appropriate for the next step in the analysis of these data? [Choose one best answer]:

__Report stratum-specific risk differences

__Calculate an overall adjusted risk difference and associated confidence interval

__Report the crude risk difference

__Perform a statistical test to assess whether there is confounding by maternal age

__None of the above options is appropriate

1.5 ACE-5. Stratified Analysis: Effect Modification

When effect modification is present in a stratified analysis, how should the data be presented? (Choose one best answer.)

_____Provide the crude measure of association

_____Provide the adjusted measure of association

_____Provide the stratum-specific measures of association

_____None of the above

1.6 ACE-6. Stratified Analysis: Confounding

When confounding is present in a stratified analysis, how should the data be presented? [Choose one best answer]:

_____Provide the crude measure of association

_____Provide the adjusted measure of association

_____Provide the stratum-specific measures of association

_____None of the above

1.7 ACE-7. Alcohol Consumption and Bladder Cancer: Race

A case-control study was conducted to assess the potential relationship between alcohol consumption and bladder cancer. Data from the study are summarized below, stratified on three race categories. In answering some of the questions below you may wish to use the Datadesk template Stratified OR/RR.ise.

	White				Black
	ALC	no ALC			ALC	no ALC
Case	72	41		Case	93	54
Control	106	105		Control	113	113

	Asian
	ALC	no ALC
Case	68	33
Control	78	142

• a. Calculate the stratum-specific ORs. Is there evidence of effect modification by race? Justify your answer. (You might use a statistical test here, i.e., the Breslow-Day test, in addition to comparing the point estimates for the three strata.)

• b. Should you do an overall Mantel-Haenszel test for association that controls for race using all three strata? Justify your answer.

• c. Considering only the information of black and white subjects, Is the observed overall association between ALC and Bladder Cancer, stratified on race, statistically significant? (Be sure to state the null hypothesis and the p-value for the test.)

• d. Should you estimate an overall adjusted odds ratio that controls for race using all three strata? Justify your answer.

• e. Considering only the information of black and white subjects, calculate both a precision-based aOR and a mOR. For the aOR, which group, black or white, receives more weight?

• f. Calculate and compare 95% CIs for the aOR and mOR.

• g. What do you conclude about the ALC, bladder cancer relationship?

1.8 ACE-8. Physical Activity and Incidence of Diabetes

A cohort study of physical activity (PA) and incidence of diabetes was conducted over a six-year period among Japanese-American men in Honolulu. Data from that study are summarized below, stratified on body mass index. In answering some of the questions below you may wish to use the Datadesk template Stratified IDR/IDD.ise.

	High BMI				Low BMI
	High PA	Low PA			High PA	Low PA
Diabetes	48	62		Diabetes	54	71
Person-yrs	1050	1067		Person-yrs	1132	1134

• a. Is there evidence of effect modification by BMI? Justify your answer.

• b. Calculate and compare a precision-based aIDR and a mIDR.

• c. Is there evidence of confounding by BMI in these data?

• d. Should you estimate an overall adjusted odds ratio that controls for BMI using all three strata? Justify your answer.

• e. Is the observed association between physical activity and diabetes, controlling for BMI, statistically significant? Be sure to state the null and alternative hypotheses being tested, the test statistic, and the P-value.

• f. Calculate and compare 95% confidence intervals for the aIDR and mIDR.

• g. What do you conclude about the relationship between PA and Diabetes based on these data?

Answers Study Questions and Quizzes

2.1 Q14.1

• 1. For both smokers and non-smokers separately, there appears to be no association between exposure to TXC and the development of lung cancer. Never the less, it may be argued that the RR of 1.3 for smokers indicates a moderate association; however, this estimate is highly non-significant.

• 2. No, the two stratum-specific risk ratio estimates are essentially equal. Again, the RR of 1.3 for smokers indicates a small effect, but is highly non-significant.

• 3. No, even though the crude estimate of effects is 2.1, the correct analysis requires that smoking be controlled, from which the data show no effect of TCX exposure. An adjusted estimate over the two strata would provide an appropriate summary statistic that controls for smoking.

• 4. Since the adjusted point estimate is close to the null value of 1 and the Mantel-Haenszel test statistic is very non-significant, you should conclude that there is no evidence of and E-D relationship from these data

2.2 Q14.2

• 1. Yes, the odds ratio of 11.67 is very high and the MH test is highly significant and, even though the confidence interval is wide, the interval does not include the null value.

• 2. The association may change when one or more variables are controlled. If this happens and the control variables are risk factors, then an adjusted estimate or estimates would be more appropriate.

• 3. Not necessarily. If one or more of these variables are not previously known risk factors for MRSA status, then such variables may not be controlled.

• 4. Some (n = 5) study subjects had to having missing information on either MSRA status or on previous hospitalization information. In fact, it was on the latter variable that 5 observations were missing.

• 5. No, the stratum-specific odds ratios within different age groups are very close (around 11).

• 6. No, the P-value of.95 is very high, indicating no evidence of interaction due to age.

• 7. Yes, overall assessment is appropriate because there is no evidence of interaction due to age.

• 8. No, the crude and adjusted odds ratios are essentially equal.

• 9. Yes, the Mantel-Haenszel test for stratified data is highly significant (P < .).

• 10. The confidence interval is quite wide, indicating that even though the adjusted estimate is both statistically and meaningfully significant, there is little precision in this estimate.

• 11. Yes, overall assessment is appropriate because there is no evidence of interaction due to gender.

• 12. No confounding since, when controlling for gender, the crude and adjusted odds rations are essentially equal.

• 13. Yes, the Mantel-Haenszel test for stratified data is highly significant (P < .).

• 14. The answer to this question is “maybe.” There appears to be interaction because the odds ratio is 8.48 with previous drug use but only 3.66 with no previous drug use. However, both odds ratio estimates are on the same side of 1, so an adjusted estimate will not be the result of opposite effects canceling each other. Moreover, the BD test for interaction is non-significant, which supports doing overall assessment.

• 15. Yes, when controlling for previous drug use, the crude odds ratio of 11.67 is quite different than the much smaller odds ratio of 5.00.

• 16. Yes, the Mantel-Haenszel test for stratified data is highly significant (P < .), and although the confidence interval is wide, it still does not contain the null value.

2.3 Q14.3

• 1. Previous antimicrobial drug use needs to be controlled because it is a confounder.

• 2. Yes, precision is gained from controlling for previous antimicrobial drug use, since the width of the confidence interval for the adjusted estimate is much narrower than the width of the corresponding confidence for the crude data.

• 3. No, neither the adjusted odds ratio nor the confidence interval nor the MH P-value changes either significantly or meaningfully when comparing the results that control for PADMU alone with results that control for additional variables.

• 4. No, all P-values are quite large, indicating that the null hypothesis of no interaction should not be rejected. However, perhaps a comparison of stratum-specific estimates may suggest interaction when more than one variable is controlled.

• 5. Because the estimated odds ratio is undefined in a stratum with a zero cell frequency.

• 6. OR = 4.66 is an appropriate choice because it controls for all three variables. being considered for control. Alternatively, OR = 5.00 is also appropriate because it results from controlling only for previous antimicrobial drug use, which is the only variable that affects confounding and precision.

• 7. Yes, the adjusted odds ratio (close to 5.00) indicates a strong effect that is also statistically significant. The 95% confidence interval indicates a lack of precision, but the results are overall indicative of a strong effect.

• 8. (Note: there is no question 8)

• 9. There are small numbers, including a number of zeros in almost all tables.

• 10. Stratum-specific analyses, even when there are no zero cells, are on the whole unreliable because of small numbers.

• 11. Yes, the odds ratio estimate in table 5 is 24.00 whereas the odds ratio in table 6 is 1.71 and the odds ratio in table 1 is 5.89, all quite different estimates.

• 12. The BD test is not significant, all odds ratio estimates, though different, are all on the same side of the null value, and the strata involve very small numbers.

2.4 Q14.4

• 1. F – Stratification also involves performing an overall assessment when appropriate.

• 2. T

• 3. T

• 4. F – An overall summary estimate may be considered inappropriate if there is considerable evidence of interaction.

• 5. T

• 6. No, Maybe, Yes

2.5 Q14.5

• 1. For a one-sided alternative, the area in the right tail is twice the P-value, so the one-sided P-value is half of.28, or.14.

• 2. Do not reject the null hypothesis of no overall effect. There is no evidence that exposure to TCX is associated with the development of lung cancer when controlling for smoking.

• 3. If there was interaction on opposite sides of the null, then one of the two terms in the sum would be negative and the other would be positive. Consequently, the sum of these terms might be close to zero, yielding a non-significant chi square test, even if the stratum-specific tests were both significant.

2.6 Q14.6

• 1. Most definitely. The numerator is always a sum of quantities that are then squared, in contrast to a sum of quantities that are squared before summing. Consequently, large positive values from some strata may cancel out large negative values of other strata, leading to a non-significant MH test.

• 2. If the stratum-specific effects are all on the same side of the null value, than all quantities in the numerator of the test statistic have the same sign and therefore cannot cancel each other out.

2.7 Q14.7

• 1. For a one-sided alternative, the area in the right tail is twice the P-value so the one-sided P-value is half of.04, or.02.

• 2. The null hypothesis of no overall effect would be rejected at the 5 percent level but not at the 1 percent level.

2.8 Q14.8

• 1. 0.79

• 2. 0.76

• 3. 0.77

• 4. No (To compute the crude IDR, you need to combine the data over both strata).

• 5. No

• 6. increased precision

• 7. fail to reject the null hypothesis

• 8. reject the null hypothesis

2.9 Q14.9

• 1. B

• 2. F: A larger confidence interval means less precision and hence a smaller weight.

• 3. F: The magnitude of the risk ratio is not a factor in determining precision-based weights.

• 4. T

• 5. T

2.10 Q14.10

• 1. Remember that sample size is not as important as a balanced data set in determining precision. The weights correspond to how balanced the data sets are. The more balance, the higher the weight.

2.11 Q14.11

• 1. 4.00

• 2. 4.69

• 3. Black or Hispanic

• 4. 4.16

• 5. debatable – The crude POR of 3.59 and the adjusted POR of 4.16 are different but not that far apart, so deciding whether there is a meaningful difference is debatable. Note, however, that if we require a 10% difference for judging confounding, then 3.59 is below a 10% negative change (3.64) in the adjusted POR. Thus, using a 10% change rule, we would conclude that there is confounding.

2.12 Q14.12

• 1. OR(modified by 0.1) = (5.1x5.1) / (0.1x6.1) = 42.6

• 2. OR(modified by 1.0) = (6x6) / (1x7) = 5.1

• 3. The modified stratum-specific odds ratio may change radically depending on what value is used to adjust all cell frequencies in a stratum with a zero cell frequency. Consequently, such an approach is quite problematic.

• 4. $\begin{array}{l} a\hat OR = \\ \exp \left[ {\frac{{\{.3972 \times \ln (9.31)\} + \{.6618 \times \ln (9.00)\} }}{{.3972 +.6618}}} \right] = \\ \\ \exp \left[ {\frac{{.8862 + 1.4541}}{{1.0590}}} \right] = 9.11 \\ \end{array}$

• 5. $\begin{array}{l} a\hat OR = \\ \exp \left[ {\frac{{\{.0947 \times \ln (42.64)\} + \{.6618 \times \ln (9.00)\} }}{{.0947 +.6618}}} \right] = \\ \\ \exp \left[ {\frac{{.3354 + 1.4541}}{{.7565}}} \right] = 10.65 \\ \end{array}$

• 6. They are different (9.11 using a.5 adjustment vs.10.65 using a.1 adjustment), but not very different.

• 7. It does not seem that the choice of adjustment factor has a great impact on the resulting aOR, even though it can have a great impact on the value of the odds ratio in the stratum being adjusted.

2.13 Q14.13

• 1. The mOR will be undefined because each term in the sum in the denominator will be zero.

• 2. The mOR will be undefined because each term in the sum in the numerator will be zero.

• 3. It is possible, although unlikely, that having zero cell frequencies in every stratum may make the mOR undefined.Nevertheless, if such a situation occurs, the mOR will not work. Instead, you may have to use as an alternative the approach that adds.5 to each cell frequency in any stratum with a zero cell.

• 4. All three adjusted estimates are different, with the mOR being quite separate from the aOR estimates. A key reason for preferring the mOR is that it avoids using an arbitrary modifying value like.5 or.1. Also, the mOR has good statistical properties as described earlier.

2.14 Q14.14

• 1. True

• 2. False – It is the stratum-specific estimates that are most affected by adding a small number to each cell.

• 3. True

2.15 Q14.15

• 1. The null value of 1 is not contained in the confidence interval, but just barely. Although the interval is not very wide, the point estimate of 1.71 is somewhat unreliable since the interval ranges from essentially no association to a moderately strong association.

• 2. 95% CI for aRD:

  $$\begin{array}{l}.0850 \pm \frac{{1.96}}{{\sqrt {490.2243} }} = \\ \\.0850 \pm.0885 = \\ \\ ( -.0035,{\rm }{\rm.1735)} \\ \end{array}$$

• 3. The null value of 0 is just barely contained in the interval. Although the interval is not very wide, the point estimate of.0850 is somewhat unreliable since the interval ranges from essentially no association to a moderately strong association of.17 for a risk difference.

2.16 Q14.16

• 1. 1.57, 12.98

• 2. yes

2.17 Q14.17

• 1. The estimated risk ratios comparing HI-CHL with LO-CHL do not differ very much between the two age groups (2.51 vs. 2.15). Similarly, when comparing MED-CHL with LO-CHL, the estimated risk ratios do not differ very much between the two age groups (2.25 vs. 1.90). Overall, these findings indicate no meaningful interaction of age with cholesterol level.

• 2. Overall assessment is appropriate because there is no evidence of interaction, particularly on opposite sides of the null value.

• 3. No evidence of confounding, since crude and adjusted estimated are essentially equivalent when comparing HI vs. LO Cholesterol groups and when comparing MED vs. LO Cholesterol groups.

2.18 Q14.18

• 1. (1) There is no significant trend in the CHD risk over the three categories. (2) The risks in each strata are the same.

• 2. Since the P-value is 0.0320 (two-sided), conclude that there is a significant trend in the CHD risks over the three categories using the scoring method involving mean CHL values within each CHL category.

• 3. The chi square statistic for trend would likely change somewhat, but would probably lead to the same conclusion obtained for other scoring methods, although no guarantee that this would always occur.

• 4. The chi square statistic and corresponding P-values are slightly different, but both statistics would reject the null hypothesis of no linear trend at the.05 significance level.

• 5. Identical chi square statistics and corresponding P-values. The reason: scores are equally distant. This would also be the case if the scores where 15, 10, and 5.

2.19 Q14.19

• 1. Three dummy variables, because there would be 2x2 = 4 age-by-gender strata.

• 2. D1 = 1 if female > 55 years, else 0; D2 = 1 if female < 55 years, else 0; and D3 = 1 if male > 55 years, else 0.

• 3. logit P = b₀ + b₁E + b₂D1 + b₃D2 + b₄D3 where E takes on the three values 265, 208, and 164 for the three cholesterol strata.

• 4. They are slightly different, as might be expected, but they lead to the same conclusion about the null hypothesis. It is possible, however, that different scoring systems can give different conclusions.

• 5. The logistic regression approach uses a slightly different approach (called maximum likelihood) for obtaining model estimates and the corresponding tests than the earlier approach (summation formula). Although both approaches can lead to slightly different answers, they are equivalent if the sample sizes are large enough. Nevertheless, the logistic regression approach is preferred by most statisticians because of the properties of maximum likelihood estimates.

2.20 Q14.20

• 1. 2.66

• 2. 1.47

• 3. a. 3.60; b.1.20; c. 1.93; d. 1.20

• 4. maybe: There is some evidence of interaction here, but since it is same side interaction, the investigator must decide whether the difference in 3.6 versus the other estimates is meaningful.

• 5. yes

• 6. reject H₀

• 7. fail to reject H₀

• 8. Logit P = b₀ + b₁SMK + b₂SES