Skip to main content
Log in

Classification through incremental max–min separability

  • Theoretical Advances
  • Published:
Pattern Analysis and Applications Aims and scope Submit manuscript

Abstract

Piecewise linear functions can be used to approximate non-linear decision boundaries between pattern classes. Piecewise linear boundaries are known to provide efficient real-time classifiers. However, they require a long training time. Finding piecewise linear boundaries between sets is a difficult optimization problem. Most approaches use heuristics to avoid solving this problem, which may lead to suboptimal piecewise linear boundaries. In this paper, we propose an algorithm for globally training hyperplanes using an incremental approach. Such an approach allows one to find a near global minimizer of the classification error function and to compute as few hyperplanes as needed for separating sets. We apply this algorithm for solving supervised data classification problems and report the results of numerical experiments on real-world data sets. These results demonstrate that the new algorithm requires a reasonable training time and its test set accuracy is consistently good on most data sets compared with mainstream classifiers.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Astorino A, Gaudioso M (2002) Polyhedral separability through successive LP. J Optim Theory Appl 112(2):265–293

    Article  MathSciNet  MATH  Google Scholar 

  2. Bobrowski L (1991) Design of piecewise linear classifiers from formal neurons by a basis exchange technique. Pattern Recognit 24:863–870

    Article  Google Scholar 

  3. Kostin A (2006) A simple and fast multi-class piecewise linear pattern classifier. Pattern Recognit 39:1949–1962

    Article  MATH  Google Scholar 

  4. Palm HC (1990) A new piecewise linear classifier. Pattern recognition. In: Proceedings 10th international conference on 16–21 June, vol 1, pp 742–744

  5. Chai B, Huang T, Zhuang X, Zhao Y, Sklansky J (1996) Piecewise linear classifiers using binary tree structure and genetic algorithm. Pattern Recognit 29:1905–1917

    Article  Google Scholar 

  6. Bagirov AM (2005) Max–min separability. Optim Methods Softw 20(2–3):271–290

    MathSciNet  Google Scholar 

  7. Bagirov AM, Ugon J (2005) Supervised data classification via max–min separability. In: Jeyakumar V, Rubinov AM (eds). Continuous optimisation: current trends and modern applications. Springer, Berlin, pp 175–208

    Google Scholar 

  8. Sklansky J, Wassel GS (1981) Pattern classifiers and trainable machines. Springer, Berlin

    MATH  Google Scholar 

  9. Sklansky J, Michelotti L (1998) Locally trained piecewise linear classifiers. IEEE Trans Pattern Anal Mach Intell 2(2):101–111

    Article  Google Scholar 

  10. Tenmoto H, Kudo M, Shimbo M (1998) Piecewise linear classifiers with an appropriate number of hyperplanes. Pattern Recognit 31(11):1627–1634

    Article  Google Scholar 

  11. Herman GT, Yeung KTD (1992) On piecewise-linear classification. IEEE Trans Pattern Anal Mach Intell 14(7):782–786

    Article  Google Scholar 

  12. Sanchez NS, Triantaphyllou E, Chen J, Liao TW (2002) An incremental learning algorithm for constructing Boolean functions from positive and negative examples. Comput Oper Res 29:1677–1700

    Article  MathSciNet  Google Scholar 

  13. Kuncheva L (2000) Clustering and selection model of classifier combination. In: Proceedings of knowledge-based engineering systems and allied technologies. Brighton, UK

  14. Jackowski K, Wozniak M (2009) Algorithm of designing compound recognition system on the basis of combining classifiers with simultaneous splitting feature space into competence areas. Pattern Anal Appl. doi:10.1007/s10044-008-0137-7

  15. Raducanu B, Vitra J (2008) Online nonparametric discriminant analysis for incremental subspace learning and recognition. Pattern Anal Appl 11(3):259–268

    Article  MathSciNet  Google Scholar 

  16. Leski JM (2003) Neuro-fuzzy system with learning tolerant to imprecision. Fuzzy Sets Syst 138(2):427–439

    Article  MathSciNet  Google Scholar 

  17. Leski JM (2004) Epsilon-insensitive fuzzy c-regression models: Introduction to epsilon-insensitive fuzzy modeling. IEEE Trans Syst Man Cybern Part B Cybern 34(1):4–15

    Article  Google Scholar 

  18. Park Y, Sklansky J (1989) Automated design of multiple-class piecewise linear classifiers. J Classif 6:195–222

    Article  MathSciNet  MATH  Google Scholar 

  19. Lo Z-P, Bavarian B (1991) Comparison of a neural network and a piecewise linear classifier. Pattern Recognit Lett 12(11):649–655

    Article  Google Scholar 

  20. Schulmeister B, Wysotzki F (1994) The piecewise linear classifier DIPOL92. In: Bergadano F, De Raedt L (eds) Proceedings of the European conference on machine learning on machine learning (Catania, Italy). Springer, New York, pp 411–414

  21. Michie D, Spiegelhalter DJ, Taylor CC (eds) (1994) Machine learning, neural and statistical classification. Ellis Horwood, London

  22. Bagirov AM (1999) Minimization methods for one class of nonsmooth functions and calculation of semi-equilibrium prices. In: Eberhard A et al (eds) Progress in optimization: contribution from Australasia. Kluwer, Dordrecht, pp 147–175

  23. Bagirov AM (2002) A method for minimization of quasidifferentiable functions. Optim Methods Softw 17(1):31–60

    Article  MathSciNet  MATH  Google Scholar 

  24. Asuncion A, Newman DJ (2007) UCI machine learning repository. University of California, School of Information and Computer Science, Irvine, CA. http://www.ics.uci.edu/mlearn/MLRepository.html

  25. Witten IH, Frank E (2005) Data Mining: Practical machine learning tools and techniques, 2nd edn. Morgan Kaufmann, San Francisco

    MATH  Google Scholar 

Download references

Acknowledgements

Dr. Adil Bagirov is the recipient of an Australian Research Council Australian Research Fellowship (Project number: DP 0666061). Dr. Adil Bagirov and Prof. Bülent Karasözen are thankful for the support of TUBITAK (Turkish Scientific and Technical Research Council) and the Australian Mathematical Sciences Institute which initiated this current work by their supporting mutual visits. We would like to thank two anonymous referees for their useful suggestions that improved the quality of the paper.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Adil M. Bagirov.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Bagirov, A.M., Ugon, J., Webb, D. et al. Classification through incremental max–min separability. Pattern Anal Applic 14, 165–174 (2011). https://doi.org/10.1007/s10044-010-0191-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10044-010-0191-9

Keywords

Navigation