Comparison of the Effects of Ridge Biasing Constant in Remedying Multicollinearity on Gamma and Exponentially Distributed Data

Abstract:

Ridge regression as a solution to multicollinearity depends on the value of k, the ridge biasing constant. Since no optimum value can be found for k, as k is generally bounded between 0 and 1, i.e. 0≤ k ≤1 and it varies from one application to another. This has posed a major limitation of ridge regression in that ordinary inference procedures are not applicable and exact distributional properties are not known; and the choice of the biasing constant, k, is a judgmental one. This work examined the effect of ridge biasing constant, k, on different sample sizes using data combination from gamma and exponential distributions when multicollinearity exists. The sample sizes of 140, 100, 80, 50, 30, 20 and 10 and ridge constants, k=0.01, 0.02, . . . ., 0.1 respectively were used in the study. The Anderson Darling Test was used to check for the distribution of the data which were found to follow gamma and exponential distributions. The findings lay credence to how the ridge regression drastically remedies the effect of multicollinearity among independent variables. The study also revealed that the VIF consistently decreased as the ridge constant increased. While the ridge regression has a slight effect on the R-squared, sample sizes were found not to have any significant change or pattern on the VIFs. Since the study has shown that the VIF reduced drastically as the ridge constant increases, it is recommended to use a VIF that reduces multicollinearity to an acceptable minimum while maximizing the R-squared. This study recommends a ridge constant of 0.1 as all multicollinearity issues have been remedied at more than 90% if not completely. The study recommends using a large sample size to help stabilize the R2 values while remedying multicollinearity.