Abstract
Nonsmooth optimization provides efficient algorithms for solving many machine learning problems. In particular, nonsmooth optimization approaches to supervised data classification problems lead to the design of very efficient algorithms for their solution. In this chapter, we demonstrate how nonsmooth optimization algorithms can be applied to design efficient piecewise linear classifiers for supervised data classification problems. Such classifiers are developed using a max–min and a polyhedral conic separabilities as well as an incremental approach. We report results of numerical experiments and compare the piecewise linear classifiers with a number of other mainstream classifiers.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Astorino, A., Fuduli, A.: Nonsmooth optimization techniques for semisupervised classification. IEEE Trans. Pattern Anal. Mach. Intell. 29(12), 2135–2142 (2007)
Astorino, A., Gaudioso, M.: Polyhedral separability through successive LP. J. Optim. Theory Appl. 112(2), 265–293 (2002)
Astorino, A., Fuduli, A., Gorgone, E.: Non-smoothness in classification problems. Optim. Methods Softw. 23(5), 675–688 (2008)
Astorino, A., Fuduli, A., Gaudioso, M.: DC models for spherical separation. J. Glob. Optim. 48(4), 657–669 (2010)
Asuncion, A., Newman, D.J.: UCI Machine Learning Repository. University of California, School of Information and Computer Science, Irvine. http://www.ics.uci.edu/~mlearn/MLRepository.html (2007)
Azimov, A.Y., Gasimov, R.N.: On weak conjugacy, weak subdifferentials and duality with zero gap in nonconvex optimization. Int. J. Appl. Math. 1, 171–192 (1999)
Azimov, A.Y., Gasimov, R.N.: Stability and duality of nonconvex problems via augmented Lagrangian. Cybern. Syst. Anal. 3, 120–130 (2002)
Bagirov, A.M.: Minimization methods for one class of nonsmooth functions and calculation of semi-equilibrium prices. In: Eberhard, A., et al. (eds.) Progress in Optimization: Contribution from Australasia, pp. 147–175. Kluwer, Boston (1999)
Bagirov, A.M.: A method for minimization of quasidifferentiable functions. Optim. Methods Softw. 17(1), 31–60 (2002)
Bagirov, A.M.: Max-min separability. Optim. Methods Softw. 20(2–3), 271–290 (2005)
Bagirov, A.M., Ugon, J.: Supervised data classification via max-min separability. In: Jeyakumar, V., Rubinov, A.M. (eds.) Continuous Optimisation: Current Trends and Modern Applications, Chap. 6, pp. 175–208. Springer, Berlin (2005)
Bagirov, A.M., Ugon, J.: Piecewise partially separable functions and a derivative-free algorithm for large scale nonsmooth optimization. J. Glob. Optim. 35(2), 163–195 (2006)
Bagirov, A.M., Yearwood, J.: A new nonsmooth optimization algorithm for minimum sum-of-squares clustering problems. Eur. J. Oper. Res. 170(2), 578–596 (2006)
Bagirov, A.M., Ugon, J., Webb, D.: An efficient algorithm for the incremental construction of a piecewise linear classifier. Inf. Syst. 36(4), 782–790 (2011)
Bagirov, A.M., Ugon, J., Webb, D., Karasozen, B.: Classification through incremental max-min separability. Pattern Anal. Appl. 14, 165–174 (2011)
Bagirov, A.M., Ugon, J., Webb, D., Ozturk, G., Kasimbeyli, R.: A novel piecewise linear classifier based on polyhedral conic and max-min separabilities. TOP 21(1), 3–24 (2013)
Bennett, K.P., Mangasarian, O.L.: Bilinear separation of two sets in n-space. Comput. Optim. Appl. 2, 207–227 (1993)
Bobrowski, L.: Design of piecewise linear classifiers from formal neurons by a basis exchange technique. Pattern Recognit. 24(9), 863–870 (1991)
Chai, B., Huang, T., Zhuang, X., Zhao, Y., Sklansky, J.: Piecewise linear classifiers using binary tree structure and genetic algorithm. Pattern Recognit. 29(11), 1905–1917 (1996)
Gasimov, R.N.: Characterization of the Benson proper efficiency and scalarization in nonconvex vector optimization. Multiple Criteria Decision Making in the New Millennium. In: Lecture Notes in Economics and Mathematical Systems, vol. 507, pp. 189–198. Springer, New York (2001)
Gasimov, R.N.: Augmented Lagrangian duality and nondifferentiable optimization methods in nonconvex programming. J. Glob. Optim. 24, 187–203 (2002)
Gasimov, R.N., Ozturk, G.: Separation via polyhedral conic functions. Optim. Methods Softw. 21(4), pp. 527–540 (2006)
Gasimov, R.N., Rubinov, A.M.: On augmented Lagrangians for optimization problems with a single constraint. J. Glob. Optim. 28(2), 153–173 (2004)
Herman, G.T., Yeung, K.T.D.: On piecewise-linear classification. IEEE Trans. Pattern Anal. Mach. Intell. 14(7), 782–786 (1992)
Kasimbeyli, R.: A nonlinear cone separation theorem and scalarization in nonconvex vector optimization. SIAM J. Optim. 20(3), 1591–1619 (2010)
Kasimbeyli, R., Mammadov, M.: On weak subdifferentials, directional derivatives and radial epiderivatives for nonconvex functions. SIAM J. Optim. 20(2), 841–855 (2009)
Kostin, A.: A simple and fast multi-class piecewise linear pattern classifier. Pattern Recognit. 39, 1949–1962 (2006)
Lo, Z.-P., Bavarian, B.: Comparison of a neural network and a piecewise linear classifier. Pattern Recognit. Lett. 12(11), 649–655 (1991)
Michie, D., Spiegelhalter, D.J., Taylor, C.C. (eds.) Machine Learning, Neural and Statistical Classification. Ellis Horwood, London (1994)
Pallaschke, D., Rolewicz, S.: Foundations of Mathematical Optimization (Convex Analysis Without Linearity). Kluwer, Dordrecht (1997)
Palm, H.C.: A new piecewise linear classifier. In: Pattern Recognition, Proceedings of the 10th International Conference on Machine Learning, vol. 1. pp. 742–744 (1990)
Park, Y., Sklansky, J.: Automated design of multiple-class piecewise linear classifiers. J. Classif. 6, 195–222 (1989)
Rubinov, A.M.: Abstract Convexity and Global Optimization. Kluwer, Dordrecht (2000)
Rubinov, A.M., Gasimov, R.N.: The nonlinear and augmented Lagrangians for nonconvex optimization problems with a single constraint. Appl. Comput. Math. 1(2), 142–157 (2002)
Rubinov, A.M., Gasimov, R.N.: Strictly increasing positively homogeneous functions with application to exact penalization. Optimization 52(1), 1–28 (2003)
Rubinov, A.M., Yang, X.Q., Bagirov, A.M., Gasimov, R.N.: Lagrange-type functions in constrained optimization. J. Math. Sci. 115(4), 2437–2505 (2003)
Schulmeister, B., Wysotzki, F.: The piecewise linear classifier DIPOL92. In: Bergadano, F., De Raedt, L. (eds.) Proceedings of the European Conference on Machine Learning (Catania, Italy), pp. 411–414. Springer, New York/Secaucus (1994)
Singer, I.: Abstract Convex Analysis. Wiley, New York (1997)
Sklansky, J., Michelotti, L.: Locally trained piecewise linear classifiers. IEEE Trans. Pattern Anal. Mach. Intell. 2(2), 101–111 (1980)
Sklansky, J., Wassel, G.S.: Pattern Classifiers and Trainable Machines. Springer, Berlin (1981)
Tenmoto, H., Kudo, M., Shimbo, M.: Piecewise linear classifiers with an appropriate number of hyperplanes. Pattern Recognit. 31(11), 1627–1634 (1998)
Witten, I.H., Frank, E.: Data Mining: Practical Machine Learning Tools and Techniques, 2nd edn. Morgan Kaufmann, San Francisco (2005)
Acknowledgements
R. Kasimbeyli and G. Özturk are the recipients of a grant of the Scientific and Technological Research Council of Turkey—TUBITAK (Project number:107M472).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer Science+Business Media New York
About this chapter
Cite this chapter
Bagirov, A.M., Kasimbeyli, R., Öztürk, G., Ugon, J. (2014). Piecewise Linear Classifiers Based on Nonsmooth Optimization Approaches. In: Rassias, T., Floudas, C., Butenko, S. (eds) Optimization in Science and Engineering. Springer, New York, NY. https://doi.org/10.1007/978-1-4939-0808-0_1
Download citation
DOI: https://doi.org/10.1007/978-1-4939-0808-0_1
Published:
Publisher Name: Springer, New York, NY
Print ISBN: 978-1-4939-0807-3
Online ISBN: 978-1-4939-0808-0
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)