Abstract
This paper introduces a new cost function for evaluating the multi-class classifier. The new cost function facilitates both a way to visualize the performance (expected cost) of the multi-class classifier and a summary of the misclassification costs. This function overcomes the limitations of ROC in not being able to represent the classifier performance graphically when there are more than two classes. Here we present a new scalable method for producing a scalar measurement that is used to compare the performance of the multi-class classifier. We mathematically demonstrate that our technique can capture small variations in classifier performance.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Flach, P., Blockeel, H., Ferri, C., Hernández-Orallo, J.S.: Decision support for data mining; introduction to roc analysis and its applications. Data Mining and Decision Support: Integration and Collaboration, 81–90 (2003)
Hand, D.J., Till, R.J.: A simple generalisation of the area under the roc curve for multiple class classification problems. Machine Learning 45, 171–186 (2001)
Swets, J., Dawes, R., Monahan, J.: Better decisions through science. Scientific American, 82–87 (2000)
Ferri, C., Hernández-Orallo, J., Salido, M.: Volume under the roc surface for multi-class problems. In: Proceedings of ECML (2003)
Srinivasan, A.: Note on the location of optimal classifiers in roc space. Technical report, Oxford University Technical Report PRG-TR-2-99 (1999)
Li, Y.: A generalization of AUC to an ordered multi-class diagnosis and application to longitudinal data analysis on intellectual outcome in pediatric brain-tumor patient. PhD thesis, Georgia University (2009)
Mossman, D.: Three-way roc’s. Medical Decision Making 19, 78–89 (1999)
Asuncion, A., Newman, D.: UCI machine learning repository (2007)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Hassan, M.R., Ramamohanarao, K., Karmakar, C., Hossain, M.M., Bailey, J. (2010). A Novel Scalable Multi-class ROC for Effective Visualization and Computation. In: Zaki, M.J., Yu, J.X., Ravindran, B., Pudi, V. (eds) Advances in Knowledge Discovery and Data Mining. PAKDD 2010. Lecture Notes in Computer Science(), vol 6118. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-13657-3_14
Download citation
DOI: https://doi.org/10.1007/978-3-642-13657-3_14
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-13656-6
Online ISBN: 978-3-642-13657-3
eBook Packages: Computer ScienceComputer Science (R0)