Ordinary least squares and stepwise selection are widespread in behavioral science research; however, these methods are well known to encounter overfitting problems such that R2 and regression coefficients may be inflated while standard errors and p values may be deflated, ultimately reducing both the parsimony of the model and the generalizability of conclusions. More optimal methods for selecting predictors and estimating regression coefficients such as regularization methods (e.g., Lasso) have existed for decades, are widely implemented in other disciplines, and are available in mainstream software, yet, these methods are essentially invisible in the behavioral science literature while the use of sub optimal methods continues to proliferate. This paper discusses potential issues with standard statistical models, provides an introduction to regularization with specific details on both Lasso and its related predecessor ridge regression, provides an example analysis and code for running a Lasso analysis in R and SAS, and discusses limitations and related methods.

Modern high-dimensional statistical inference often faces the problem of missing data. In recent decades, many studies have focused on this topic and provided strategies including complete-sample analysis and imputation procedures. However, complete-sample analysis discards information of incomplete samples, while imputation procedures have accumulative errors from each single imputation. In this paper, we propose a new method, Sample-wise COmbined missing effect Model with penalization (SCOM), to deal with missing data occurring in predictors. Instead of imputing the predictors, SCOM estimates the combined effect caused by all missing data for each incomplete sample. SCOM makes full use of all available data. It is robust with respect to various missing mechanisms. Theoretical studies show the oracle inequality for the proposed estimator, and the consistency of variable selection and combined missing effect selection. Simulation studies and an application to the Residential Building Data also illustrate the effectiveness of the proposed SCOM.

PurposeThis study aims to investigate the relationship between the total muscle-to-fat ratio (tMFR) and female urinary incontinence (UI), determine whether tMFR can serve as a useful index for predicting UI, and identify factors that may influence this relationship.MethodsWe retrospectively analyzed data from 4391 adult women participating in the National Health and Nutrition Examination Survey (NHANES) conducted between 2011 and 2018. The correlation between tMFR and UI was examined using a dose-response curve generated through a restricted cubic spline (RCS) function, LASSO and multivariate logistic regression. Furthermore, predictive models were constructed incorporating factors such as age, race, hypertension, diabetes, cotinine levels, and tMFR. The performance of these predictive models was evaluated using training and test datasets, employing calibration curves, receiver operating characteristic curves, and clinical decision curves. Mediation effects were also analyzed to explore potential relationships between tMFR and female UI.ResultsIn a sample of 4391 adult women, 1073 (24.4%) self-reported experiencing UI, while 3318 (75.6%) reported not having UI. Based on the analyses involving LASSO regression and multivariate logistic regression, it was found that tMFR exhibited a negative association with UI (OR = 0.599, 95% CI: 0.497-0.719, P < 0.001). The results from the restricted cubic spline chart indicated a decreasing risk of UI in women as tMFR increased. Furthermore, the model constructed based on logistic regression analysis demonstrated a certain level of accuracy (in the training dataset: area under the curve (AUC) = 0.663; in the test dataset: AUC = 0.662) and clinical applicability. The mediation analysis revealed that the influence of tMFR on the occurrence of UI in women might potentially occur through the blood index lymphocyte count (P = 0.040).ConclusionA high tMFR serves as a protective factor against UI in women. Furthermore, lymphocyte might be involved in the relationship between tMFR and female UI.

The equality-constrained lasso problem augments the standard lasso by imposing additional structure on regression coefficients. Despite the broad utilities of the equality-constrained lasso, existing algorithms are typically computationally inefficient and only applicable to linear and logistic models. In this article, we devise a fast solution to the equality-constrained lasso problem with a two-stage algorithm: first obtaining candidate covariate subsets of increasing size from unconstrained lasso problems and then leveraging an efficient combined alternating direction method of multipliers/Newton-Raphson algorithm. Our proposed algorithm leads to substantial speedups in getting the solution path of the constrained lasso and can be easily adapted to generalized linear models and Cox proportional hazards models. We conduct extensive simulation studies to demonstrate the computational advantage of the proposed method over existing solvers. To further show the unique utility of our method, we consider two real-world data examples: a microbiome regression analysis and a myeloma survival analysis; neither example could be solved by naively fitting the constrained lasso problem on the full predictor set. Supplementary materials for this article are available online.

Dispersal, the movement of individuals between populations, is crucial in many ecological and genetic processes. However, direct identification of dispersing individuals is difficult or impossible in natural populations. By using genetic assignment methods, individuals with unknown genetic origin can be assigned to source populations. This knowledge is necessary in studying many key questions in ecology, evolution and conservation.

We introduce a network-based tool BONE (Baseline Oriented Network Estimation) for genetic population assignment, which borrows concepts from undirected graph inference. In particular, we use sparse multinomial Least Absolute Shrinkage and Selection Operator (LASSO) regression to estimate probability of the origin of all mixture individuals and their mixture proportions without tedious selection of the LASSO tuning parameter. We compare BONE with three genetic assignment methods implemented in R packages radmixture, assignPOP and RUBIAS.

Probability of the origin and mixture proportion estimates of both simulated and real data (an insular house sparrow metapopulation and Chinook salmon populations) given by BONE are competitive or superior compared to other assignment methods. Our examples illustrate how the network estimation method adapts to population assignment, combining the efficiency and attractive properties of sparse network representation and model selection properties of the L₁ regularization. As far as we know, this is the first approach showing how one can use network tools for genetic identification of individuals' source populations.

BONE is aimed at any researcher performing genetic assignment and trying to infer the genetic population structure. Compared to other methods, our approach also identifies outlying mixture individuals that could originate outside of the baseline populations. BONE is a freely available R package under the GPL license and can be downloaded at GitHub. In addition to the R package, a tutorial for BONE is available at https://github.com/markkukuismin/BONE/.

Objective: The predictors of in-hospital mortality for intensive care units (ICU)-admitted HF patients remain poorly characterized.We aimed to develop and validate a prediction model for all-cause in-hospital mortality among ICU-admitted HF patients.

Design: A retrospective cohort study.

Setting and Participants: Data were extracted from the MIMIC-III database. Data on 1,177 heart failure patients were analysed.

Methods: Patients meeting the inclusion criteria were identified from the MIMIC-III database and randomly divided into derivation and validation groups. Independent risk factors for in-hospital mortality were screened using XGBoost and LASSO regression models in the derivation sample. Multivariable logistic regression analysis was used to build prediction models. Discrimination, calibration, and clinical usefulness of the predicting model were assessed using the C-index, calibration plot, and decision curve analysis. After pairwise comparison, the best performing model ...

Simulation results for the location level of contamination in the model(1 − δ)N(0, 1) + δN(−10, 1) in high-dimensional data set.

Permutation tests for equality of the group distributions using distance components analysis (lines 2 to 6), and permutation F-tests for the presence of 2-by-2 interactions (lines 7 to 16). Results based on 999 permutations.Lasso.

The adaptive lasso is a method for performing simultaneous parameter estimation and variable selection. The adaptive weights used in its penalty term mean that the adaptive lasso achieves the oracle property. In this work, we propose an extension of the adaptive lasso named the Tukey-lasso. By using Tukey's biweight criterion, instead of squared loss, the Tukey-lasso is resistant to outliers in both the response and covariates. Importantly, we demonstrate that the Tukey-lasso also enjoys the oracle property. A fast accelerated proximal gradient (APG) algorithm is proposed and implemented for computing the Tukey-lasso. Our extensive simulations show that the Tukey-lasso, implemented with the APG algorithm, achieves very reliable results, including for high-dimensional data where p > n. In the presence of outliers, the Tukey-lasso is shown to offer substantial improvements in performance compared to the adaptive lasso and other robust implementations of the lasso. Real-data examples further demonstrate the utility of the Tukey-lasso. Supplementary materials for this article are available online.

People's networks are considered key in explaining fertility outcomes—whether people want and have children. Existing research on social influences on fertility is limited because data often come from small networks or highly selective samples, only few network variables are considered, and the strength of network effects is not properly assessed. We use data from a representative sample of Dutch women reporting on over 18 000 relationships. A data-driven approach including many network characteristics accounted for 0 to 40% of the out-of-sample variation in different outcomes related to fertility preferences. Individual characteristics were more important for all outcomes than network variables. Network composition was also important, particularly those people in the network desiring children or those choosing to be childfree. Structural network characteristics, which feature prominently in social influence theories and are based on the relations between people in the networks, hardly mattered. We discuss to what extent our results provide support for different mechanisms of social influence, and the advantages and disadvantages of our data-driven approach in comparison to traditional approaches.

In the field of polymer informatics, utilizing machine learning (ML) techniques to evaluate the glass transition temperature Tg and other properties of polymers has attracted extensive attention. This data-centric approach is much more efficient and practical than the laborious experimental measurements when encountered a daunting number of polymer structures. Various ML models are demonstrated to perform well for Tg prediction. Nevertheless, they are trained on different data sets, using different structure representations, and based on different feature engineering methods. Thus, the critical question arises on selecting a proper ML model to better handle the Tg prediction with generalization ability. To provide a fair comparison of different ML techniques and examine the key factors that affect the model performance, we carry out a systematic benchmark study by compiling 79 different ML models and training them on a large and diverse data set. The three major components in setting up an ML model are structure representations, feature representations, and ML algorithms. In terms of polymer structure representation, we consider the polymer monomer, repeat unit, and oligomer with longer chain structure. Based on that feature, representation is calculated, including Morgan fingerprinting with or without substructure frequency, RDKit descriptors, molecular embedding, molecular graph, etc. Afterward, the obtained feature input is trained using different ML algorithms, such as deep neural networks, convolutional neural networks, random forest, support vector machine, LASSO regression, and Gaussian process regression. We evaluate the performance of these ML models using a holdout test set and an extra unlabeled data set from high-throughput molecular dynamics simulation. The ML model’s generalization ability on an unlabeled data set is especially focused, and the model’s sensitivity to topology and the molecular weight of polymers is also taken into consideration. This benchmark study provides not only a guideline for the Tg prediction task but also a useful reference for other polymer informatics tasks.

In this article, we consider estimation of common structural breaks in panel data models with unobservable interactive fixed effects. We introduce a penalized principal component (PPC) estimation procedure with an adaptive group fused LASSO to detect the multiple structural breaks in the models. Under some mild conditions, we show that with probability approaching one the proposed method can correctly determine the unknown number of breaks and consistently estimate the common break dates. Furthermore, we estimate the regression coefficients through the post-LASSO method and establish the asymptotic distribution theory for the resulting estimators. The developed methodology and theory are applicable to the case of dynamic panel data models. Simulation results demonstrate that the proposed method works well in finite samples with low false detection probability when there is no structural break and high probability of correctly estimating the break numbers when the structural breaks exist. We finally apply our method to study the environmental Kuznets curve for 74 countries over 40 years and detect two breaks in the data. Supplementary materials for this article are available online.

Identifying the latent cluster structure based on model heterogeneity is a fundamental but challenging task arises in many machine learning applications. In this article, we study the clustered coefficient regression problem in the distributed network systems, where the data are locally collected and held by nodes. Our work aims to improve the regression estimation efficiency by aggregating the neighbors’ information while also identifying the cluster membership for nodes. To achieve efficient estimation and clustering, we develop a distributed spanning-tree-based fused-lasso regression (DTFLR) approach. In particular, we propose an adaptive spanning-tree-based fusion penalty for the low-complexity clustered coefficient regression. We show that our proposed estimator satisfies statistical oracle properties. Additionally, to solve the problem parallelly, we design a distributed generalized alternating direction method of multiplier algorithm, which has a simple node-based implementation scheme and enjoys a linear convergence rate. Collectively, our results in this article contribute to the theories of low-complexity clustered coefficient regression and distributed optimization over networks. Thorough numerical experiments and real-world data analysis are conducted to verify our theoretical results, which show that our approach outperforms existing works in terms of estimation accuracy, computation speed, and communication costs. Supplementary materials for this article are available online.

Permutation tests for equality of the group distributions using distance components analysis (lines 2 to 6), and permutation F-tests for the presence of 2-by-2 interactions (lines 7 to 16), in the comparison of ridge-regression vs lasso. Results based on 999 permutations.Ridge-regression vs lasso.

Additional file 9: Table S3. Prediction performance of all reported models on mystery data.

ObjectiveThis study investigates predictors of weight loss among individuals with serious mental illness participating in an 18-month behavioral weight loss intervention, using Lasso regression to select the most powerful predictors.MethodsData were analyzed from the intervention group of the ACHIEVE trial, an 18-month behavioral weight loss intervention in adults with serious mental illness. Lasso regression was employed to identify predictors of at least five-pound weight loss across the intervention time span. Once predictors were identified, classification trees were created to show examples of how to classify participants into having likely outcomes based on characteristics at baseline and during the intervention.ResultsThe analyzed sample contained 137 participants. Seventy-one (51.8%) individuals had a net weight loss of at least five pounds from baseline to 18 months. The Lasso regression selected weight loss from baseline to 6 months as a primary predictor of at least five pound 18-month weight loss, with a standardized coefficient of 0.51 (95% CI: −0.37, 1.40). Three other variables were also selected in the regression but added minimal predictive ability.ConclusionsThe analyses in this paper demonstrate the importance of tracking weight loss incrementally during an intervention as an indicator for overall weight loss, as well as the challenges in predicting long-term weight loss with other variables commonly available in clinical trials. The methods used in this paper also exemplify how to effectively analyze a clinical trial dataset containing many variables and identify factors related to desired outcomes.

Additional file 7: Table S1. Sequencing data information.

In this study, we investigate the bias and variance properties of the debiased Lasso in linear regression when the tuning parameter of the node-wise Lasso is selected to be smaller than in previous studies. We consider the case where the number of covariates p is bounded by a constant multiple of the sample size n. First, we show that the bias of the debiased Lasso can be reduced without diverging the asymptotic variance by setting the order of the tuning parameter to 1/n. This implies that the debiased Lasso has asymptotic normality provided that the number of nonzero coefficients s0 satisfies s0=o(n/ log p), whereas previous studies require s0=o(n/ log p) if no sparsity assumption is imposed on the inverse of the second moment matrix of covariates. Second, we propose a data-driven tuning parameter selection procedure for the node-wise Lasso that is consistent with our theoretical results. Simulation studies show that our procedure yields confidence intervals with good coverage properties in various settings. We also present a real economic data example to demonstrate the efficacy of our selection procedure.

New algorithms are continuously proposed in computational biology. Performance evaluation of novel methods is important in practice. Nonetheless, the field experiences a lack of rigorous methodology aimed to systematically and objectively evaluate competing approaches. Simulation studies are frequently used to show that a particular method outperforms another. Often times, however, simulation studies are not well designed, and it is hard to characterize the particular conditions under which different methods perform better. In this paper we propose the adoption of well established techniques in the design of computer and physical experiments for developing effective simulation studies. By following best practices in planning of experiments we are better able to understand the strengths and weaknesses of competing algorithms leading to more informed decisions about which method to use for a particular task. We illustrate the application of our proposed simulation framework with a detailed comparison of the ridge-regression, lasso and elastic-net algorithms in a large scale study investigating the effects on predictive performance of sample size, number of features, true model sparsity, signal-to-noise ratio, and feature correlation, in situations where the number of covariates is usually much larger than sample size. Analysis of data sets containing tens of thousands of features but only a few hundred samples is nowadays routine in computational biology, where “omics” features such as gene expression, copy number variation and sequence data are frequently used in the predictive modeling of complex phenotypes such as anticancer drug response. The penalized regression approaches investigated in this study are popular choices in this setting and our simulations corroborate well established results concerning the conditions under which each one of these methods is expected to perform best while providing several novel insights.

The dashed entries correspond to predictors that are estimated “0”.