Bayesian variable selection and shrinkage using Lasso methods

Abstract

Least squares method is the usual way of treating a multiple regression problem. But not all available predictors are meaningful for the response variable. Poor performance in terms of prediction accuracy and interpretation are problems arising when overfitting the data. Variable selection methods improve interpretation and prediction by producing models of lower dimension, while shrinkage techniques reduce the variance of predicted values by shrinking predictors’ coefficients towards zero.LASSO performs both shrinkage and variable selection by shrinking some coefficients towards zero and setting others exactly equal to zero. A tuning parameter is involved, which controls the shrinkage procedure while k-fold Cross Validation is used to specify its optimal value. Additionally, the lasso estimates can be defined as a Bayesian posterior mode when regression coefficients are placed under independent double-exponential (Laplace) priors.