Some variable selection and regularization methodological approaches in quantile regression with applications

Abstract

The importance of robust variable selection and regularization as solutions to the collinearity influential high leverage points’ adverse effects in a quantile regression (QR) setting cannot be overemphasized, just as the diagnostic tools that identify these high leverage points. In the literature, researchers have dealt with variable selection and regularization quite extensively for penalized QR that generalizes the well-known least absolute deviation (LAD) procedure to all quantile levels. Unlike the least squares (LS) procedures, which are unreliable when deviations from the Gaussian assumptions (outliers) exist, the QR procedure is robust to Y-space outliers. Although QR is robust to response variable outliers, it is vulnerable to predictor space data aberrations (high leverage points and collinearity adverse effects), which may alter the eigen-structure of the predictor matrix. Therefore, in the literature, it is recommended that the problems of collinearity and high leverage points be dealt with simultaneously. In this thesis, we propose applying the ridge regression procedure (RIDGE), LASSO, elastic net (E-NET), adaptive LASSO, and adaptive elastic net (AE-NET) penalties to weighted QR (WQR) to mitigate the effects of collinearity and collinearity influential points in the QR setting. The new procedures are the penalized WQR procedures i.e., the RIDGE penalizedWQR (WQR-RIDGE), the LASSO penalizedWQR (WQR-LASSO), the E-NET penalized WQR (WQR-E-NET) and the adaptive penalized QR procedures (the adaptive LASSO penalized QR (QR-ALASSO) and adaptive E-NET penalized QR (QR-AE-NET procedures and their weighted versions). The penalized WQR procedures are based on the computationally intensive high-breakdown minimum covariance determinant (MCD) determined weights and the adaptive penalized QR procedures are based on the RIDGE penalized WQR (WQR-RIDGE) estimator based adaptive weights. Under regularity conditions, the adaptive penalized procedures satisfy oracle properties. Although adaptive weights are commonly based on the RIDGE regression (RR) estimator in the LS setting when regressors are collinear, this estimator may be plausible for the symmetrical distributions at the ℓ1-estimator (RQ at τ = 0.50) rather than at extreme quantile levels. We carried out simulations and applications to well-known data sets from the literature to assess the finite sample performance of these procedures in variable selection and regularization due to the robust weighting formulation and adaptive weighting construction. In the collinearityenhancing point scenario under the t-distribution, the WQR penalized versions outperformed the unweighted procedures with respect to average shrunken zero coefficients and correctly fitted models. Under the Gaussian and t-distributions, at predictor matrices with collinearity-reducing points, the weighted regularized procedures dominate the prediction performance (WQR-LASSO forms best). In the collinearity-inducing and collinearity-reducing points scenarios under the Gaussian distribution, the adaptive penalized procedures outperformed the non-adaptive versions in prediction. Under the t-distribution, a similar performance pattern is depicted as in the Gaussian scenario, although the performance of all models is adversely affected by outliers. Under the t-distribution, the QR-ALASSO andWQR-ALASSO procedures performed better in their respective categories.