Linear Discriminant Analysis and Multinomial Logistic Regression in Classification and Predictive Modeling: A Comparative Approach

Abstract:

The goal of this study was to compare two different methods of classification; Linear Discriminant Analysis and Multinomial Logistic Regression to make the choice between the two, depending on the characteristics of the data. Since both are appropriate for the development of linear classification models, Linear Discriminant Analysis makes more assumptions like normality and equal covariance among the explanatory variables on the underlying data, but when violated it is assumed that the Multinomial Logistic Regression is a more flexible and more robust method of analysis. In this work, some guidelines for proper choice were set up which was based on some predictive accuracy. The performance of the methods was studied by a real dataset and a simulated dataset. We started with the real dataset where all the assumptions failed, also, we performed an appropriate transformation on the real dataset and Linear Discriminant Analysis was performed on it. Next we compare with simulated data where all the assumptions of Linear Discriminant Analysis are satisfied. From the result where the assumptions were violated, Multinomial Logistic Regression performs better than Linear Discriminant Analysis, also the result from the analysis performed on the transformed data shows that the Multinomial Logistic Regression also performed better, and whenever the assumptions hold as in the case of the simulated data, Linear Discriminant Analysis slightly performs better. Hence Multinomial Logistic Regression serves as an alternative whenever the assumptions of discriminant analysis fail instead of transforming the data.