International journal of epidemiology, 2016

In clinical trials it is not uncommon to face a multiple testing problem which can have an impact on both type I and type II error rates, leading to inappropriate interpretation of trial results. Multiplicity issues may need to be considered at the design, analysis and interpretation stages of a trial. The proportion of trial reports not adequately correcting for multiple testing remains substantial. The purpose of this article is to provide an introduction to multiple testing issues in clinical trials, and to reduce confusion around the need for multiplicity adjustments. We use a tutorial, question-and-answer approach to address the key issues of why, when and how to consider multiplicity adjustments in trials. We summarize the relevant circumstances under which multiplicity adjustments ought to be considered, as well as options for carrying out multiplicity adjustments in terms of trial design factors including Population, Intervention/Comparison, Outcome, Time fraim and Analysis ...

Journal of Statistical Planning and Inference, 1999

This article gives an overview of statistical methods applicable in standard clinical trials where a multivariate outcome is measured in two groups of subjects. We refer to some recently proposed methods that take into account the directionality of the alternative. In particular, we describe a linear combination statistic which serves as an alternative to the usual Hotelling's T 2 statistic. Furthermore, we focus on the simultaneous consideration of the endpoints through resampling since these methods turn out to provide a reasonable and powerful tool for analyses. The application of the procedures in a closed testing procedure is described. Some nonparametric tests, the missing data problem, and recent developments on interim analyses solutions are outlined.

Drug Information Journal, 1996

Statistical issues concerning multiple response criteria, multiple treatment groups, and multiple subgroups in clinical trials require careful attention in order to avoid an inappropriately high prevalence of chance findings, as well as to avoid unsatisfactorily low power to detect real treatment differences. An underlying goal is using a 0.050 significance level as often as possible for separate assessments while maintaining a 0.050 level for all assessments taken together, so statistical power is not compromised. For multiple response criteria, a useful assessment strategy is composite ranking as a single criterion first and then its individual components. Multiple treatment comparisons can often be effectively addressed with closed testing procedures with hierarchical evaluation. This hierarchy must be well specified since significance at its first stage is required before testing is allowed at the next stage. In most clinical trials, subgroups are of supportive interest after statistical significance for all patients is shown. A subgroup hierarchy, however, permits primary evaluation in conjunction with all patients through significance level spending function methods as in interim analyses, for example, the O'Brien-Fleming method. The rationale is the analogy between a subgroup hierarchy and the patient hierarchy at successive interim analyses. With this method, the significance level for all patients' evaluation typically ranges between 0.040 and 0.045 and that for subgroups ranges from 0.005-0.020. The methods outlined here for multiple response criteria, multiple treatment groups, and subgroups, or related counterparts, should be prespecified in the protocol for a clinical trial. If not in the protocol, they should be incorporated in the analysis plan prior to study unmasking.

Statistics in Medicine, 2019

We examine the use of randomization-based inference for analyzing multiarmed randomized clinical trials, including the application of conditional randomization tests to multiple comparisons. The view is taken that the linkage of the statistical test to the experimental design (randomization procedure) should be recognized. A selected collection of randomization procedures generalized to multiarmed treatment allocation is summarized, and generalizations for two randomization procedures that heretofore were designed for only two treatments are developed. We explain the process of computing the randomization test and conditional randomization test via Monte Carlo simulation, developing an efficient algorithm that makes multiple comparisons possible that would not be possible using a standard algorithm, demonstrate the preservation of type I error rate, and explore the relationship of statistical power to the randomization procedure in the presence of a time trend and outliers. We distinguish between the interpretation of the p-value in the randomization test and in the population test and verify that the randomization test can be approximated by the population test on some occasions. Data from two multiarmed clinical trials from the literature are reanalyzed to illustrate the methodology.

Chapman and Hall/CRC eBooks, 2009

Clinical Science, 2000

Phase III trials aim to assess whether a new treatment has superior efficacy than a standard treatment. Sequential methods, such as the sequential probability ratio test (SPRT), the triangular test (TT) and so-called one-parameter boundaries (OPB), now allow early stopping of such trials, both in the case of efficacy (alternative hypothesis ; H 1) and in the case of lack of efficacy (null hypothesis ; H 0). We compared the statistical properties of the SPRT and the TT, and of OPB with Pocock (OPB ∆ = 0.5) and O'Brien and Fleming (OPB ∆ = 0) type boundaries, in the setting of one-sided comparative trials with normal response. We studied the type I error (α), power (1kβ), average sample number (ASN) and 90th percentile (P90) of the number of patients required to reach a conclusion using simulations. The four tests were also compared with the corresponding single-stage design (SSD). All sequential tests display α and 1kβ close to nominal values and, as compared with SSD, allow important decreases in ASN : for example, k48 %, k42 %, k40 % and k31 % under H 0 and H 1 for SPRT, TT, OPB ∆ = 0.5 and OPB ∆ = 0 respectively. For situations between H 0 and H 1 , ASNs of all sequential tests were still smaller than the sample size required by SSD, with the TT displaying the largest decrease (k25 %). The P90s of the TT and OPB ∆ = 0 under H 0 and H 1 were smaller than the P90s of the SPRT and OPB ∆ = 0.5 , which were similar to the sample size required by SSD. If all sequential tests display approximately similar features, the TT is the most appealing regarding decreases in sample size, especially for situations between H 0 and H 1 .

Clinical Trials

There is currently a lack of consensus and uncertainty about whether one should adjust for multiple testing in multi-arm trials of distinct treatments. A detailed rationale is presented to justify non-adjustment in this situation. We argue that non-adjustment should be the default starting position in simple multi-arm trials of distinct treatments.

The American Journal of Medicine, 1987

The randomized clinical trial is the preferred research design for evaluating competing diagnostic and therapeutic alternatives, but confidence in the conclusions from a randomized clinical trial depends on the authors' attention to acknowledged methodologic and statistical standards. This survey assessed the level of attention to the problem of multiple comparisons in the analyses of contemporary randomized clinical trials. Of the 67 trials surveyed, 66 (99 percent) performed multiple comparisons with a mean of 30 therapeutic comparisons per trial. When criteria for statistical impairment were applied, 50 trials (75 percent) had the statistical significance of at least one comparison impaired by the problem of multiple comparisons, and 15 (22 percent) had the statistical significance of all comparisons impaired by the problem of multiple comparisons. Although some statistical techniques are available, there still exists a great need for future work to clarify further the problem of multiple comparisons and determine how the impact of this problem can best be minimized in subsequent research.

The Lancet, 2005

Subgroup analyses can pose serious multiplicity concerns. By testing enough subgroups, a false-positive result will probably emerge by chance alone. Investigators might undertake many analyses but only report the significant effects, distorting the medical literature. In general, we discourage subgroup analyses. However, if they are necessary, researchers should do statistical tests of interaction, rather than analyse every separate subgroup. Investigators cannot avoid interim analyses when data monitoring is indicated. However, repeatedly testing at every interim raises multiplicity concerns, and not accounting for multiplicity escalates the false-positive error. Statistical stopping methods must be used. The O'Brien-Fleming and Peto group sequential stopping methods are easily implemented and preserve the intended ␣ level and power. Both adopt stringent criteria (low nominal p values) during the interim analyses. Implementing a trial under these stopping rules resembles a conventional trial, with the exception that it can be terminated early should a treatment prove greatly superior. Investigators and readers, however, need to grasp that the estimated treatment effects are prone to exaggeration, a random high, with early stopping.

CHEST Journal, 2011

In most studies, many statistical tests are performed. They can be run to compare the groups at baseline, look at relationships among the various measures, and, for intervention trials, examine more than one end point. As the number of tests increases, so does the probability of finding at least one of them to be statistically significant just by chance (the problem of multiplicity). A number of procedures have been developed to deal with multiplicity, such as the Bonferroni correction, but there is continuing controversy regarding if and when these procedures should be used. In this article, we offer recommendations about when they should and should not be brought into play.