When dealing with missing data in clinical trials, it is often convenient to work under simplifying assumptions, such as missing at random (MAR), and follow up with sensitivity analyses to address unverifiable missing data assumptions. One such sensitivity analysis, routinely requested by regulatory agencies, is the so-called tipping point analysis, in which the treatment effect is re-evaluated after adding a successively more extreme shift parameter to the predicted values among subjects with missing data. If the shift parameter needed to overturn the conclusion is so extreme that it is considered clinically implausible, then this indicates robustness to missing data assumptions. Tipping point analyses are frequently used in the context of continuous outcome data under multiple imputation. While simple to implement, computation can be cumbersome in the two-way setting where both comparator and active arms are shifted, essentially requiring the evaluation of a two-dimensional grid of models. We describe a computationally efficient approach to performing two-way tipping point analysis in the setting of continuous outcome data with multiple imputation. We show how geometric properties can lead to further simplification when exploring the impact of missing data. Lastly, we propose a novel extension to a multi-way setting which yields simple and general sufficient conditions for robustness to missing data assumptions.

Additional file 1 The STATA DO-File (.do) provides example code for using multiple imputation with interval regression.

Missing data are commonly encountered in clinical trials due to dropout or nonadherence to study procedures. In trials in which recurrent events are of interest, the observed count can be an undercount of the events if a patient drops out before the end of the study. In many applications, the data are not necessarily missing at random and it is often not possible to test the missing at random assumption. Consequently, it is critical to conduct sensitivity analysis. We develop a control-based multiple imputation method for recurrent events data, where patients who drop out of the study are assumed to have a similar response profile to those in the control group after dropping out. Specifically, we consider the copy reference approach and the jump to reference approach. We model the recurrent event data using a semiparametric proportional intensity frailty model with the baseline hazard function completely unspecified. We develop nonparametric maximum likelihood estimation and inference procedures. We then impute the missing data based on the large sample distribution of the resulting estimators. The variance estimation is corrected by a bootstrap procedure. Simulation studies demonstrate the proposed method performs well in practical settings. We provide applications to two clinical trials.

OBJECTIVES: Where trial recruitment is staggered over time, patients may have different lengths of follow-up, meaning that the dataset is an unbalanced panel with a considerable amount of missing data. This study presents a method for estimating the difference in total costs and total Quality – Adjusted Life Years (QALY) over a given time horizon using a repeated measure mixed model (RMMM). To the authors’ knowledge this is the first time this method has been exploited in the context of economic evaluation within clinical trials. METHODS: An example (EVLA trial, NIHR HTA project 11/129/197) is used where patients have between 1 and 5 years of follow up. Early treatment is compared with delayed treatment. Coefficients at each time point from the repeated measures mixed model were aggregated to estimate total mean cost and total mean QALY over 3 years. Results were compared with other methods for handling missing data: Complete-Case-Analysis (CCA), multiple imputation using linear regression (MILR) and using predictive mean matching (MIPMM), and Bayesian parametric approach (BPA). RESULTS: Mean differences in costs obtained varied among the different approach, CCA, MIPMM and MILRM recorded greater mean costs in delayed treatment, £216 (95% CI -£1413 to £1845), £36 (95% CI to £-581 to 652£), £30(95% CI to -£617 to 679£), respectively. While RMM and BPA showed greater costs in early intervention, -£67 (95% CI -£1069 to £855), -£162 (95% CI -£728-£402), respectively. Early intervention was associated with greater QALY among all methods at year 3, RMM show the highest QALYs, 0.073 (95% CI -0.06 to 0.2). CONCLUSION: MIPMM show most efficient results in our cost-effectiveness analysis. By contrast when the percentage of missing is high RMM shows similar results than MIPMM. Hence, we conclude that RMM is a flexible way and robust alternative for modelling continuous outcomes data that can be considered missing-at-random.

Stata do-files and data to support tutorial "Sensitivity Analysis for Not-at-Random Missing Data in Trial-Based Cost-Effectiveness Analysis" (Leurent, B. et al. PharmacoEconomics (2018) 36: 889).Do-files should be similar to the code provided in the article's supplementary material.Dataset based on 10 Top Tips trial, but modified to preserve confidentiality. Results will differ from those published.

Currently, various methods have been proposed to handle missing data in clinical trials. Some methods assume that the missing data are missing at random (MAR), which means that it is assumed that subjects who stopped treatment would still maintain the treatment effect. In many cases, however, researchers often assume that the missing data are missing not at random (MNAR) to conduct additional sensitivity analyses. Under the MNAR assumption, whether using some conservative imputation methods such as RTB (return to baseline) method, J2R (jump to reference) method, and CR (copy reference) method, or optimistic imputation methods like multiple imputation (MI) and its derivative RD (retrieved dropout) method, biases compared to the true treatment effect can occur in some scenarios. This paper aims to propose a method that can impute results while considering the occurrence of intercurrent events, thereby reducing the bias compared to the true treatment effect. This method combines the RD method with the RTB formula, reducing the biases and standard errors associated with using either method alone. Considering the differing treatment effects between RD subjects and non-RD subjects, our imputation results often align more closely with the true drug efficacy.

This survey is a component of the Robert Wood Johnson Foundation's Health Tracking Initiative, a program designed to monitor changes within the health care system and their effects on people. Focusing on care and treatment for alcohol, drug, and mental health conditions, the survey reinterviewed respondents to the 1996-1997 CTS Household Survey (COMMUNITY TRACKING STUDY HOUSEHOLD SURVEY, 1996-1997, AND FOLLOWBACK SURVEY, 1997-1998: [UNITED STATES] [ICPSR 2524]). Topics covered by the questionnaire include (1) demographics, (2) health and daily activities, (3) mental health, (4) alcohol and illicit drug use, (5) use of medications, (6) health insurance coverage including coverage for mental health, (7) access, utilization, and quality of behavioral health care, (8) work, income, and wealth, and (9) life difficulties. Five imputed versions of the data are included in the collection for analysis with multiple imputation techniques.

Importance: While multiple imputation models for missing data and use of mixed effects models generally provide better outcome estimates than using only observed data or last observation carried forward in clinical trials, such approaches usually cannot be applied to visual outcomes from retrospective analyses of clinical practice settings, so-called real-world outcomes.

Objective: To explore potential utility of survival analysis techniques for retrospective clinical practice visual outcomes.

Design: Retrospective cohort study with 12-year observation period.

Setting: A tertiary eye center.

Participants: Of 10,744 eyes with neovascular AMD receiving anti-VEGF therapy between October 2008 and February 2020, 7802 eyes met study criteria (treatment-naïve, first-treated eyes starting anti-VEGF therapy).

Main outcome measures: Kaplan-Meier estimates and Cox proportional hazards modelling were used to consider: VA reaching ETDRS (Early Treatment Diabetic Retinopathy Study) letter score 70 (Snellen equivalent 20/40) or better; duration VA sustained at or better than 70 (20/40); and VA declining to 35 (20/200) or worse.

Results: The median time to attaining VA letter score ≥ 70 (20/40) was 2.0 years (95% CI 1.87 - 2.32) after the first anti-VEGF injection. Predictive features were baseline VA (HR 1.43 per 5 ETDRS letter score or 1 line [95% CI 1.40 - 1.46]), baseline age (HR 0.88 per 5 years [95% CI 0.86 - 0.90]), and injection number (HR 1.123 [95% CI 1.101 - 1.15]). Of the 56% attaining this outcome, median time sustained at 70 (20/40) or better was 1.1 years (95% CI 1.1 - 1.2).

Conclusions and relevance: Survival analysis potentially addresses key limitations of retrospective clinical practice (real-world) data by accounting for variable observation time-points and follow-up durations. Modelling with multiple covariates reveals factors that may help inform the likely visual trajectory of an individual. We demonstrate the utility of our proposed analyses by showing that patients with neovascular AMD beginning anti-VEGF therapy are more likely to experience positive visual outcomes within the first 2.9 years, typically maintaining this for 1.1 years, but then deteriorating to poor vision within 8.7 years. We believe that this dataset, combined with our statistical approach for retrospective analyses, may provide long-term prognostic information for patients newly diagnosed with this condition.

BackgroundUrinary tract infections (UTIs) are common and result in an enormous economic burden. The increasing prevalence of antibiotic-resistant microorganisms has stimulated interest in non-antibiotic agents to prevent UTIs.ObjectiveTo evaluate the cost-effectiveness of cranberry prophylaxis compared to antibiotic prophylaxis with trimethoprim-sulfamethoxazole (TMP-SMX) over a 12 month period in premenopausal women with recurrent UTIs.Materials and MethodsAn economic evaluation was performed alongside a randomized trial. Primary outcome was the number of UTIs during 12 months. Secondary outcomes included satisfaction and quality of life. Healthcare utilization was measured using questionnaires. Missing data were imputed using multiple imputation. Bootstrapping was used to evaluate the cost-effectiveness of the treatments.ResultsCranberry prophylaxis was less effective than TMP-SMX prophylaxis, but the differences in clinical outcomes were not statistically significant. Costs after 12 months in the cranberry group were statistically significantly higher than in the TMP-SMX group (mean difference €249, 95% confidence interval 70 to 516). Cost-effectiveness planes and cost-effectiveness acceptability curves showed that cranberry prophylaxis to prevent UTIs is less effective and more expensive than (dominated by) TMP-SMX prophylaxis.ConclusionIn premenopausal women with recurrent UTIs, cranberry prophylaxis is not cost-effective compared to TMP-SMX prophylaxis. However, it was not possible to take into account costs attributed to increased antibiotic resistance within the framework of this randomized trial; modeling studies are recommended to investigate these costs. Moreover, although we based the dosage of cranberry extract on available evidence, this may not be the optimal dosage. Results may change when this optimal dosage is identified.Trial RegistrationISRCTN.org ISRCTN50717094

*Values are summary estimates of the 5 data sets obtained by multiple imputation, combined using Rubin's rules.†Transformed using the power transformation 1−(1-VAS/100)1.61[23].

Five algorithms are described for imputing partially observed recurrent events modeled by a negative binomial process, or more generally by a mixed Poisson process when the mean function for the recurrent events is continuous over time. We also discuss how to perform the imputation when the mean function of the event process has jump discontinuities. The validity of these algorithms is assessed by simulations. These imputation algorithms are potentially very useful in the implementation of pattern mixture models, which have been popularly used as sensitivity analysis under the non-ignorability assumption in clinical trials. A chronic granulomatous disease trial is analyzed for illustrative purposes.

Differences between study arms adjusted for baseline scores (multiple imputations).

Predictive values are measures of the clinical accuracy of a binary diagnostic test, and depend on the sensitivity and the specificity of the diagnostic test and on the disease prevalence among the population being studied. This article studies hypothesis tests to simultaneously compare the predictive values of two binary diagnostic tests in the presence of missing data. The hypothesis tests were solved applying two computational methods: the expectation maximization and the supplemented expectation maximization algorithms, and multiple imputation. Simulation experiments were carried out to study the sizes and the powers of the hypothesis tests, giving some general rules of application. Two R programmes were written to apply each method, and they are available as supplementary material for the manuscript. The results were applied to the diagnosis of Alzheimer’s disease.

BackgroundOral pre-exposure prophylaxis (PrEP) using co-formulated emtricitabine (FTC) and tenofovir disoproxil fumarate (TDF) is a potent HIV prevention method for men and women, with its efficacy highly dependent on adherence. A pivotal HIV efficacy study combined with a directly observed pharmacological study defined the thresholds for HIV protection in men who have sex with men (MSM), which are the keys to PrEP promotion and development of new PrEP agents. For African women at risk for HIV and belonging to a priority group considered due to disproportionately high incident HIV infections, the variable adherence in PrEP clinical trials and the limited pharmacologic data have resulted in a lack of clarity about the PrEP adherence required for HIV protection. We propose a study to quantify the adherence–concentration–efficacy thresholds of TDF/FTC PrEP among African cisgender women to inform decisions about optimal PrEP dosing and adherence for HIV protection.MethodsWe randomized 45 low-risk HIV-uninfected African women, aged 18–30 years old, to directly observe the TDF/FTC PrEP of two, four, or seven doses per week for 8 weeks. A complementary age-matched pregnant women cohort at high risk of HIV, who will receive seven doses per week, was recruited (N = 15) with the primary aim of establishing benchmark concentrations in dried blood spots and peripheral blood mononuclear cells. Plasma, whole blood (WB), urine, hair, vaginal fluid, and vaginal tissue (non-pregnant women only) were archived for future testing. Drug concentrations were measured using methods validated for each biological matrix. Pharmacokinetic models were fitted to drug concentrations to quantify concentration–adherence thresholds. To define the drug concentrations associated with HIV protection, we applied the newly defined thresholds from the primary pharmacologic trial to the subset of women randomized to TDF/FTC or TDF in the Partners PrEP Study with the drug concentration assessed in plasma and WB samples. Multiple imputation was used to construct a data set with drug concentrations at each visit when an HIV test was performed for the entire cohort, replicating the work for MSM.DiscussionThe proposed study generated the first African women-specific TDF–PrEP adherence–concentration–efficacy thresholds essential for guiding the accurate interpretation of TDF/FTC PrEP programs and clinical trials of novel HIV prevention products using TDF/FTC as an active control. Clinical Trial RegistrationClinicalTrials.gov, identifier (NCT05057858).

Note. All data within table is pooled data from 20 generated sets of imputed data using the Multiple Imputation procedure.

Inbreeding coefficients, confidence intervals, p-values and missing data statistics (relative increase in variance (), and fraction of missing information ()) for multiple imputation with different multinomial logit models, and for single imputation with IMPUTE2.

Percentage of missing original data (prior to multiple imputation) with respect to the prediction models’ variables, and according to data set.

IntroductionThe UK Government developed the Change4Life Food Scanner app to provide families with engaging feedback on the nutritional content of packaged foods. There is a lack of research exploring the cost-effectiveness of dietary health promotion apps.MethodsThrough stakeholder engagement, a conceptual model was developed, outlining the pathway by which the Food Scanner app leads to proximal and distal outcomes. The conceptual model informed the development of a pilot randomized controlled trial which investigated the feasibility and acceptability of evaluating clinical outcomes in children and economic effectiveness of the Food Scanner app through a cost-consequence analysis. Parents of 4–11 years-olds (n = 126) were randomized into an app exposure condition (n = 62), or no intervention control (n = 64). Parent-reported Child Health Utility 9 Dimension (CHU9D) outcomes were collected alongside child healthcare resource use and associated costs, school absenteeism and parent productivity losses at baseline and 3 months follow up. Results for the CHU9D were converted into utility scores based on UK adult preference weights. Sensitivity analysis accounted for outliers and multiple imputation methods were adopted for the handling of missing data.Results64 participants (51%) completed the study (intervention: n = 29; control: n = 35). There was a mean reduction in quality adjusted life years between groups over the trial period of –0.004 (SD = 0.024, 95% CI: –0.005; 0.012). There was a mean reduction in healthcare costs of –£30.77 (SD = 230.97; 95% CI: –£113.80; £52.26) and a mean reduction in workplace productivity losses of –£64.24 (SD = 241.66, 95% CI: –£147.54; £19.07) within the intervention arm, compared to the control arm, over the data collection period. Similar findings were apparent after multiple imputation.DiscussionModest mean differences between study arms may have been due to the exploration of distal outcomes over a short follow-up period. The study was also disrupted due to the coronavirus pandemic, which may have confounded healthcare resource data. Although measures adopted were deemed feasible, the study highlighted difficulties in obtaining data on app development and maintenance costs, as well as the importance of economic modeling to predict long-term outcomes that may not be reliably captured over the short-term.Clinical trial registrationhttps://osf.io/, identifier 62hzt.

Demographic and clinical characteristics of patients previously diagnosed with T1D (overall and stratified by COVID-19 diagnosis).

Results of external validation according to data set.