Skip to main content
SpringerLink
Log in

Attribute weighted Naive Bayes classifier using a local optimization

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

The Naive Bayes classifier is a popular classification technique for data mining and machine learning. It has been shown to be very effective on a variety of data classification problems. However, the strong assumption that all attributes are conditionally independent given the class is often violated in real-world applications. Numerous methods have been proposed in order to improve the performance of the Naive Bayes classifier by alleviating the attribute independence assumption. However, violation of the independence assumption can increase the expected error. Another alternative is assigning the weights for attributes. In this paper, we propose a novel attribute weighted Naive Bayes classifier by considering weights to the conditional probabilities. An objective function is modeled and taken into account, which is based on the structure of the Naive Bayes classifier and the attribute weights. The optimal weights are determined by a local optimization method using the quasisecant method. In the proposed approach, the Naive Bayes classifier is taken as a starting point. We report the results of numerical experiments on several real-world data sets in binary classification, which show the efficiency of the proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Asuncion A, Newman D (2007) UCI machine learning repository. School of Information and Computer Science, University of California http://www.ics.uci.edu/mlearn/MLRepository.html

  2. Bagirov AM, Nazari Ganjehlou A (2010) A quasisecant method for minimizing nonsmooth functions. Optim Methods Soft 25(1):3–18

    Article  MATH  MathSciNet  Google Scholar 

  3. Chang C, Lin C (2001) LIBSVM: A library for support vector machines Software available at http://www.csie.ntu.edu.tw/cjlin/libsvm

  4. Chickering DM (1996) Learning Bayesian networks is NP-complete. In: Fisher D, Lenz H (eds) Learning from data: artificial intelligence and statistics V. Springer, Berlin, pp 121–130

    Chapter  Google Scholar 

  5. Domingos P, Pazzani M (1997) On the optimality of the simple Bayesian classifier under zero-one loss. Mach Learn 29:103–130

    Article  MATH  Google Scholar 

  6. Dougherty J, Kohavi R, Sahami M (1995) Supervised and unsupervised discretization of continuous features. In: Proceedings of the 12th international conference on machine learning, pp 194–202

  7. Fayyad UM, Irani KB (1993) On the handling of continuous-valued attributes in decision tree generation. Mach Learn 8:87–102

    Google Scholar 

  8. Friedman N, Geiger D, Goldszmidt M (1997) Bayesian network classifierss. Mach Learn 29:131–163

    Article  MATH  Google Scholar 

  9. Hall M (2007) A decision tree-based attribute weighting filter for Naive Bayes. Knowl Based Syst 20:120–126

    Article  Google Scholar 

  10. Heckerman D, Chickering DM, Meek C (2004) Large-sample learning of Bayesian networks is NP-Hard. J Mach Learn Res 5:1287–1330

    Google Scholar 

  11. Jiang L, Zhang H (2006) Weightily averaged one-dependence estimators. In: Proceedings of the 9th biennial pacific rim international conference on artificial intelligence, Guilin, China, pp 970–974

  12. Jiang L, Wang D, Cai Z, Yan X (2007) Survey of improving Naive Bayes for classification. In: Proceedings of the 3rd international conference on advanced data mining and applications, 4632, Springer, Berlin, pp 134–145

  13. Keogh EJ, Pazzani MJ (1999) Learning augmented Bayesian classifiers: A comparison of distribution-based and classification-based approaches. In: Proceedings of international workshop on artificial intelligence and statistics, pp 225–230

  14. Kohavi R (1996) Scaling up the accuracy of naive-Bayes classifiers: a decision-tree hybrid. In: Proceedings of 2nd ACM SIGKDD International, conference on knowledge discovery and data mining, pp 202–207

  15. Langley P, Iba W, Thompson K (1992) An analysis of Bayesian classifiers. In: 10th international conference artificial intelligence, AAAI Press, pp 223–228

  16. Langley P, Saga S (1994) Induction of selective Bayesian classifiers. In: Proceedings of tenth conference, uncertainty in artificial intelligence, Morgan Kaufmann, pp 399–406

  17. Lu J, Yang Y, Webb GI (2006) Incremental discretization for naive-Bayes classifier. Springer, Heidelberg, 4093, pp 223–238

  18. Ozsen S, Gunecs S (2009) Attribute weighting via genetic algorithms for attribute weighted artificial immune system (AWAIS) and its application to heart disease and liver disorders problems. Expert Syst Appl 36:386–392

    Article  Google Scholar 

  19. Pazzani MJ (1996) Constructive induction of cartesian product attributes, ISIS: In-formation, Stat Induction Sci 66–77

  20. Pearl J (1988) Probabilistic reasoning in Intelligent systems: networks of plausible inference. Morgan Kaufmann

  21. Sun W, Yuan YX (2006) Optimization theory and methods: nonlinear programming. Springer, Berlin

    Google Scholar 

  22. Taheri S, Mammadov M, Bagirov AM (2011) Improving Naive Bayes classifier using conditional probabilities, In the proceedings of ninth Australasian data mining conference (AusDM 2011), vol 125. Ballarat, Australia

  23. Taheri S, Mammadov M (2011) Tree augmented Naive Bayes based on optimization. In: Proceedings of 42nd annual Iranian mathematics conference, Vali-e-Asr University of Rafsanjan, Iran

  24. Wang S, Min F, Wang Z, Cao T (2009) OFFD: Optimal Flexible Frequency Discretization for Naive Bayes Classification. Springer, Heidelberg, pp 704–712

  25. Webb GI, Boughton J, Wang Z (2005) Not so Naive Bayes: aggregating one dependence estimators. Mach Learn 58:5–24

    Article  MATH  Google Scholar 

  26. Wu J, Cai Z (2011) Attribute weighting via differential evolution algorithm for attribute weighted Naive Bayes (WNB). J Comput Inf Syst 7(5):1672–1679

    Google Scholar 

  27. Xindong W et al (2008) Top 10 algorithms in data mining. Knowl Inf Syst 14:1–37

    Article  Google Scholar 

  28. Yatsko A, Bagirov AM, Stranieri A (2010) On the discretization of continuous features for classification, School of Information Technology and Mathematical Sciences, University of Ballarat Conference. (http://researchonline.ballarat.edu.au:8080/vital/access/manager/Repository)

  29. Ying Y, Geoffrey I (2009) Discretization for naive-Bayes learning: managing discretization bias and variance. Mach Learn 74(1):39–74

    Article  Google Scholar 

  30. Ying Y (2003) Discretization for naive-Bayes learning, PhD thesis, School of Computer Science and Software Engineering of Monash University

  31. Zhang H, Sheng S (2005) Learning weighted Naive Bayes with accurate ranking. In: Proceedings of the 4th IEEE international conference on data mining 567–570

  32. Zhou Y, Huang TS (2006) Weighted Bayesian network for visual tracking. In: Proceedings of the 18th international conference on pattern recognition (ICPR’O6), 0-7695-2521-0106

Download references

Author information

Authors and Affiliations

  1. School of Science, Information Technology and Engineering, University of Ballarat, Ballarat, VIC, 3353, Australia

    Sona Taheri, John Yearwood & Musa Mammadov

  2. National ICT Australia, VRL, Melbourne, Australia

    Musa Mammadov

  3. School of Civil, Environmental and Mining Engineering, University of Adelaide, Adelaide, SA, 5005, Australia

    Sattar Seifollahi

Authors
  1. Sona Taheri
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. John Yearwood
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Musa Mammadov
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sattar Seifollahi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sattar Seifollahi.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Taheri, S., Yearwood, J., Mammadov, M. et al. Attribute weighted Naive Bayes classifier using a local optimization. Neural Comput & Applic 24, 995–1002 (2014). https://doi.org/10.1007/s00521-012-1329-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-012-1329-z

Keywords

Search

Navigation