Elsevier

Pattern Recognition Letters

Volume 132, April 2020, Pages 123-131
Pattern Recognition Letters

A new nested ensemble technique for automated diagnosis of breast cancer

https://doi.org/10.1016/j.patrec.2018.11.004Get rights and content

Highlights

  • A new nested ensemble model is proposed for improving breast cancer diagnosis.

  • Both voting and stacking techniques are used to build nested ensemble (NE) model.

  • Two-layer nested ensemble models are tested on Wisconsin diagnostic (WDBC) dataset.

  • Experimental results show that the two-layer nested ensemble system outperforms the single classifiers and previous works.

Abstract

Nowadays, breast cancer is reported as one of most common cancers amongst women. Early detection of this cancer is an essential to aid in informing subsequent treatments. This study investigates automated breast cancer prediction using machine learning and data mining techniques. We proposed the nested ensemble approach which used the Stacking and Vote (Voting) as the classifiers combination techniques in our ensemble methods for detecting the benign breast tumors from malignant cancers. Each nested ensemble classifier contains “Classifiers” and “MetaClassifiers”. MetaClassifiers can have more than two different classification algorithms. In this research, we developed the two-layer nested ensemble classifiers. In our two-layer nested ensemble classifiers the MetaClassifiers have two or three different classification algorithms. We conducted the experiments on Wisconsin Diagnostic Breast Cancer (WDBC) dataset and K-fold Cross Validation technique are used for the model evaluation. We compared the proposed two-layer nested ensemble classifiers with single classifiers (i.e., BayesNet and Naïve Bayes) in terms of the classification accuracy, precision, recall, F1 measure, ROC and computational times of training single and nested ensemble classifiers. We also compared our best model with previous works reported in the literatures in terms of accuracy. The results demonstrate that the proposed two-layer nested ensemble models outperformance the single classifiers and most of the previous works. Both SV-BayesNet-3-MetaClassifier and SV-Naïve Bayes-3-MetaClassifier achieved accuracy 98.07% (K=10). However, SV-Naïve Bayes-3-MetaClassifier is more efficiency as it needs less time to build the model.

Introduction

Globally, breast cancer comprises approximately 15% percent of all cancers affecting females [1]. Approximately 1 in 37 breast cancer patients will die as a result of the disease and it has been cited as the second most common cause of cancer-related death amongst females [2]. Breast cancer can occur in females of any age, but most commonly tends to affect females between the ages of 15 and 54 years old [3]. Preventative screening is key to the early detection and treatment of breast cancers, and many countries around the world have successfully initiated screening programs that have resulted in an almost one-third reduction in the burden of disease [4].

There are several techniques that can be used to distinguish benign breast cancers from malignant tumors that will go on to infiltrate other organs. Fine-needle aspiration cytology (FNAC) and mammography are two well-known and extremely common procedures that are used to diagnose breast cancers, but both of these suffer from a lack of satisfying diagnostic performance. For example, in using mammography technique, the doctors to look for the symptom of breast cancer uses an X-ray image of the breast. However, when interpreting the mammography the doctors’ decision may vary and the mammography also suffers from limitations such as false-negative results, false-positive results, etc. [5]. For the FNAC, a pathologist, radiologist and oncologist together to make a final decision in breast cancer diagnosis. It is possible to make errors due to fatigue or inexperience and it is also time consuming. Therefore, developing techniques to allow for intelligent automated prediction of breast cancer disease pathways would be of great benefit to the medical field. Data a mining and machine learning based intelligent automated prediction systems could improve the cancer diagnosis capability and reduce the diagnosis errors. Moreover, these systems can provides decision support for the doctors for an opportunity of early identification of breast cancer.

Data mining is a process which utilizes available data to find hidden, useful information that may not be directly recognizable [6]. It is a technique that has successfully been implemented in predicting outcomes related to liver disease [7], [8], heart disease [9], [10], [11], Parkinson disease [12], Cerebral palsy [13], Epileptic seizure [14], as well as other types of cancers, including lung [15], oropharyngeal and thyroid cancers [16], [17]. For breast cancer in particular, a number of data mining and machine learning techniques have already been applied in order to develop automated diagnosis models. Naïve Bayes, BayesNet, logistic regression, decision tree, K-Nearest Neighbor, neural networks, AdaBoost algorithms, Support Vector Machine (SVM) [18], [19], [20] are widely applied for breast cancer detection by finding patterns in input data according to given classes. These models are limited in that they have a fixed loop, which does not allow for further shaping and accuracy of the algorithm. In this study, we are proposing a new technique of data mining and machine learning that will allow for increased accuracy and therefore more accurate prediction of outcomes.

In this study, we have proposed a new nested ensemble (NE) technique and used this new approach to create an accurate, automatic prediction model which can detect the benign breast tumors from malignant ones. The objective of these predictions is to classify patients into benign and malignant categories, thereby allowing those with benign breast cancers to avoid or minimize the extent of invasive procedures they will have to undergo. The proposed approach allows us to apply several ensemble methods in same time to improve the performance of the prediction system.

The rest of our work is organized as follows: in Section 2, we briefly introduce some related work. In Section 3, we present our proposed nested ensemble methods. We will then introduce our experiments on WDBC dataset in Section 4. Section 5 shows the experimental results and discussion about the obtained outcomes. Finally, in Section 6 we conclude the work and provide some future works.

Section snippets

Data mining applications in breast cancer

The classification is one of most important supervised data analysis techniques. A number of classification algorithms such as neural networks, AdaBoost algorithms, Support Vector Machine (SVM), KNN and K*Tree and feature selection methods [21], [22], [23], [24] have been applied in variety of research fields [25], [26], [27], [28], [29]. In this section, we briefly reviewed some breast cancer applications by using a wide variety of data mining algorithms which can be effectively used to

Proposed nested ensemble model

This research introduces a new approach to ensemble classifiers and this new method is called nested ensemble (NE) method. In this section, we first discuss the formal definition of the research problem and then we present nested ensemble (NE) algorithm.

Let D={D1,D2,D3,,Dn} be a set of datasets, A={A1,A2,A3,,An} be a set of different algorithms and E={E1,E2,E3,,En} be a set of different ensemble learning techniques. We propose a new model that can combine two or more ensemble learning

Dataset

The breast cancer Wisconsin diagnostic (WDBC) dataset was used in our experiments. This dataset is obtained from University of California, Irvine (UCI) machine learning repository [55]. It includes 32 tumor features of 569 subjects. The 32 features are comprised by 30 actual tumor features, a subject ID number and a class label that indicates each subject has benign or malignant tumor. In this data set, 10 real-valued factors are evaluated for each cell nucleus which are displayed in Table 2.

Evaluation matrix

A

Results without ensemble technique

In this section, the experiment concentrates on evaluating the prediction performance of BayesNet and Naïve Bayes classifiers. The experimental outcomes obtained for the WBCD dataset are presented in Table 3. It can be seen from Table 3 that the BayesNet algorithm had a better performance compared with the Naïve Bayes algorithm. The highest accuracy for BayesNet algorithm is 95.25% when K=104 whereas the best accuracy for Naïve Bayes algorithm is 93.32% when K=3.

Nested ensemble with 2-MetaClassifier

We reported the performances of

Conclusion

Timely and accurate diagnosis of different diseases is a main challenge in the healthcare research area. Breast cancer is one of the major causes of female deaths all over the world, leading to significant health interest in this domain. Significantly, this paper has introduced a new hybrid ensemble technique in order to improve the classification algorithms for early diagnosis of breast cancer. In this regard, the study evaluated the performance of hybrid ensemble methods using various K-fold

Acknowledgments

This paper is partially supported by the Commonwealth Innovation Connections Grant, Australia (no. RC54960).

References (69)

  • I. Gadaras et al.

    An interpretable fuzzy rule-based classification methodology for medical diagnosis

    Artif. Intell. Med.

    (2009)
  • D.E. Goodman et al.

    Artificial immune system classification of multiple-class problems

    In Proc. of Intelligent Engineering Systems

    (2002)
  • H. Cevikalp et al.

    Large margin classifiers based on affine hulls

    Neurocomputing

    (2010)
  • L. Rokach

    Ensemble-based classifiers

    Artif. Intell. Rev.

    (2010)
  • E. Bauer et al.

    An empirical comparison of voting classification algorithms: bagging, boosting, and variants

    Mach. Learn.

    (1999)
  • T. Zhang

    Solving large scale linear prediction problems using stochastic gradient descent algorithms

    Proceedings of the Twenty-first International Conference on Machine Learning

    (2004)
  • N. Landwehr et al.

    Logistic model trees

    Mach. Learn.

    (2005)
  • D. Koloseni et al.

    Differential evolution based nearest prototype classifier with optimized distance measures for the features in the data sets

    Expert Syst. Appl.

    (2013)
  • J.A. Sáez et al.

    Statistical computation of feature weighting schemes through data estimation for nearest neighbor classifiers

    Pattern Recognit.

    (2014)
  • C.K. Lim et al.

    A weighted inference engine based on interval-valued fuzzy relational theory

    Expert Syst. Appl.

    (2015)
  • A. Sanchez et al.

    Scaled radial axes for interactive visual feature selection: a case study for analyzing chronic conditions

    Expert Syst. Appl.

    (2018)
  • D. Wang et al.

    Context-based probability neural network classifiers realized by genetic optimization for medical decision making

    Multimedia Tools Appl.

    (2018)
  • X. Zhu et al.

    Local and global structure preservation for robust unsupervised spectral feature selection

    IEEE Trans. Knowl. Data Eng.

    (2018)
  • E. Aličković et al.

    Breast cancer diagnosis using ga feature selection and rotation forest

    Neural Comput. Appl.

    (2017)
  • S. Jeyasingh et al.

    Modified bat algorithm for feature selection with the wisconsin diagnosis breast cancer (WDBC) dataset

    Asian Pacific J. Cancer Prev.

    (2017)
  • Cancer, 2017, http://www.who.int/mediacentre/factsheets/fs297/en/,(accessed on December...
  • J.G. Elmore et al.

    Variability in radiologists’ interpretations of mammograms

    N. Engl. J. Med.

    (1994)
  • M. Kantardzic

    Data Mining: Concepts, Models, Methods, and Algorithms

    (2011)
  • M. Hassoon et al.

    Rule optimization of boosted c5. 0 classification using genetic algorithm for liver disease prediction

    Computer and Applications (ICCA), 2017 International Conference on

    (2017)
  • M. Abdar et al.

    Improving the diagnosis of liver disease using multilayer perceptron neural network and boosted decision trees

    J. Med. Biol. Eng.

    (2017)
  • A.H. Alkeshuosh et al.

    Using PSO algorithm for producing best rules in diagnosis of heart disease

    Computer and Applications (ICCA), 2017 International Conference on

    (2017)
  • M. Abdar et al.

    Impact of patients gender on parkinsons disease using classification algorithms

    J. AI Data Mining

    (2018)
  • E. Kabir et al.

    A computer aided analysis scheme for detecting epileptic seizure from eeg data

    Int. J. Comput. Intell. Syst.

    (2018)
  • T. Choudhury et al.

    Intelligent classification of lung & oral cancer through diverse data mining algorithms

    Micro-Electronics and Telecommunication Engineering (ICMETE), 2016 International Conference on

    (2016)
  • Cited by (140)

    • A novel enhanced hybrid clinical decision support system for accurate breast cancer prediction

      2023, Measurement: Journal of the International Measurement Confederation
    View all citing articles on Scopus

    I, Xujuan, hereby confirm on behalf of all authors that there are no known conflicts of interest associated with this publication and there has been no significant financial support for this work that could have influenced its outcome.

    View full text