Enhancing Predictive Accuracy in Seemingly Unrelated Regression Models: A Comparative Study of Shrinkage and Ensemble Estimators.

Abstract:

In econometric models, a system of equations is designed to increase the efficiency of estimation. However, with an increase in data, even in large data sets, multicollinearity compromises the estimation efficiency of the models, particularly Seemingly Unrelated Regression (SUR) models. Shrinkage estimators are effective in overcoming the problem of multicollinearity but pose challenges of over shrinkage bias in large datasets, whereas ensemble learning improves prediction by aggregating multiple models. However, there is dearth of study in the combine application shrinkage estimators and ensemble learning methods. The objective of the study is to assess the performance of shrinkage and ensemble shrinkage-based estimators in the sense of alleviating the multicollinearity problem in SUR models under different sample size scenarios. The Monte Carlo method simulates the covariance matrix with collinearity level of 0.5, 0.6, 0.7, 0.8 and 0.9 among variables the error variance-covariance matrix with non-diagonal elements of 0.8. The estimators used in the study are Ridge, Lasso, Elastic Net, Boosting Ridge, Boosting Lasso, Boosting Elastic Net, Stacking Ridge, Stacking Lasso, and Stacking Elastic Net. Regularization parameters of 0.01 and 0.8, along with sample sizes of 30, 100, 1000, 5000, 10,000, 50,000, 100,000 and 200,000 were considered. Based on the results obtained in relation to the findings, it is clear that an improvement in SUR coefficients can be achieved by using ensemble shrinkage-based estimators instead of traditional shrinkage when dealing with a large sample size and high levels of multicollinearity. Of all the proposed estimators, Stacking Elastic Net performed best in all situations since it had the least value of RMSE.