Ordinal Regression by a Generalized Force-based Model

Ordinal regression aims to determine the ranking of a certain pattern on a discrete ranking scale. It is employed in many real applications. For example, student satisfaction surveys usually involves rating teachers based on an ordinal scale poor, average, good, very good, excellent. Hence, the class labels present a natural ordering, i.e. a pattern associated with class label average has a higher rating (or better) than another having class poor, but having class good is better than both labels. When dealing with this kind of problems, two facts are decisive: the misclassification costs are not the same for the different errors (it is clear that classifying an excellent teacher as poor should be more penalized than classifying it as very good) and the ordering information can be used to construct better quality models.

Data Gravitational Models (DGMs) have been proposed for nominal classification in recent years [1][2][3][4]. They are instance-based models developed around the idea of a data-driven gravitational law. The underlying ideas of the DGMs are:

• there exists a force between any two patterns
• the force follows the Newton's law of universal gravitation where the body masses are substituted by set of data points
• the class value of a test pattern is determined by comparing the force of attraction between the pattern and the different classes.

A new instance-based algorithm for ordinal classification problems have been studied. The proposed algorithm extends the state-of-the-art gravitational models by generalizing the scaling behavior of the class-pattern interaction force. Like the other gravitational models, our algorithm classifies new patterns by comparing the magnitude of the force that each class exerts on a given pattern. To address ordinal problems, the algorithm assumes that, given a pattern, the forces associated to each class follow an unimodal distribution, i.e. present only one maximum which is absolute. Furthermore, a weight matrix is employed to define the importance of each attribute in each class. The error formulation considered allows us to estimate the parameters of the model using global and local optimization procedures. One of the strengths of our model is its high grade of interpretability which is a requisite in most of real applications. The algorithm proposed has been compared to other well-known ordinal regression algorithms on discretized regression datasets and real ordinal regression datasets. Experimental results demonstrate that the proposed algorithm can achieve competitive generalization performance and it is validated using nonparametric statistical tests.

References

1. C. Wang and Y. Chen, “Improving nearest neighbor classification with simulated gravitational collapse,” in Advances in Natural Computation, ser. Lecture Notes in Computer Science, L. Wang, K. Chen, and Y. Ong, Eds. Springer Berlin Heidelberg, 2005, vol. 3612, pp. 845–854.
2. Y. Zong-chang, “A vector gravitational force model for classification,” Pattern Analysis and Applications, vol. 11, pp. 169–177, 2008.
3. L. Peng, B. Yang, Y. Chen, and A. Abraham, “Data gravitation based classification,” Information Sciences, vol. 179, no. 6, pp. 809–819, Mar. 2009.
4. A. Cano, A. Zafra, and S. Ventura, “Weighted data gravitation classification for standard and imbalanced data,” Cybernetics, IEEE Transactions 41 on, vol. PP, no. 99, pp. 1–16, 2013.
5. http://www.uco.es/grupos/ayrna/orreview