Author + information
- John Lawrence, PhD⁎ (, )
- Steve Bai, PhD,
- H.M. James Hung, PhD and
- Robert O'Neill, PhD
- ↵⁎U.S. Food and Drug Administration, 10903 New Hampshire Avenue, Silver Spring, Maryland 20993
To the Editor:
The global treatment effect in a multinational trial design can be difficult to interpret when that global treatment effect does not reflect a consistent finding within the trial regional sites. This global treatment effect may not apply to the populations of each of the countries separately.
The literature on the topic of differences in treatment effects among subsets or subgroups of clinical trials is extensive (1,3–6), and the general consensus is to be cautious in interpreting any such variations because some statistical variability is to be expected and is unlikely to be real.
Because a single multinational study is usually designed to have the statistical power to detect a single global treatment effect of a specified size, it is difficult to get definitive answers about treatment effects in subgroups of the study. If the effect seems roughly equal across countries, it may be less concern to apply the global effect to the specific country. However, if the treatment effect in a specific country were regularly smaller than the global treatment effect, one would be more concerned that the difference is real. That is the motivation for this correspondence.
Twenty-four controlled clinical trials were analyzed. The trials were conducted in the past 15 years, were for cardiovascular (22 studies) or renal disease (2 studies), had country-specific data, and were positive on the primary end point.
For each of the 24 studies, we calculated separately the estimate and standard error of the treatment effect in the United States and the non–United States regions using the primary analysis. We then calculated the difference between the point estimate of the U.S. treatment effect and the non-U.S. treatment effect. We performed 3 analyses: (1) sign test; (2) DerSimonian and Laird (2) random effects meta-analysis; and (3) fixed effects meta-analysis.
Figure 1 shows the point estimates and confidence intervals for the U.S. treatment effect minus the non-U.S. treatment effect for each study. The column on the right is the percentage of patients in that study who were enrolled from the United States.
Figure 1 shows that the majority of the studies had a point estimate to the right of 0. The 2-sided p value from the sign test is 0.023. In Study 9, the treatment effect in the United States seemed to be substantially better than outside the United States, but in Studies 5, 12, 13, 17, and 21, the treatment effect appeared to be substantially worse in the United States than outside of the United States.
In the random effects meta-analysis, the estimate of the between-trial variability was 0. The estimate of the mean log-hazard ratio was 0.103 with a standard error of 0.035. Thus, the approximate confidence interval is (0.031 to 0.175) and the 2-sided p value for the test of mean zero difference is p = 0.007. Because the estimate of the between-trial variability was zero, the point estimate and estimated standard error from the fixed effect model are identical to those from the random effect model described.
It seems that there may be systematic differences between the treatment effects observed in the United States and non-U.S. regions, with the U.S.-specific treatment effect usually being smaller. Some factors that might contribute to differences in treatment effects between regions include differences in compliance, follow-up, and concomitant medications. There are other possible explanations, and in any particular trial the factors that may attenuate the treatment effect may not be anticipated or even measured. In future trials, if there is a concern that there may be a difference in the treatment effect in the United States versus other countries and the U.S.-specific treatment effect is of interest, there are both issues of design and analysis to consider. An analysis could be planned in the protocol to deal with this possible difference. This could include formal tests for interaction or examination of differences in baseline characteristics or background therapy between regions. Planning for a test for qualitative or quantitative interaction is helpful in some cases, but both tests are known to have low power when the differences are moderate, and this situation may not be totally satisfactory for this purpose. In studies in which a goal of the study is to confirm a global treatment effect and a country-specific treatment effect, there should be a plan to obtain a sufficient amount of information in the country or region of interest, and an analysis should be pre-planned to do so.
Please note: This paper reflects the views of the authors and should not be construed to represent the Food and Drug Administration's views or policies. All authors have reported that they have no relationships relevant to the contents of this paper to disclose.
- American College of Cardiology Foundation
- Cui L.,
- Hung H.M.J.,
- Wang S.J.,
- Tsong Y.
- International Conference on Harmonization
- Peto R.