Optimal design of adaptive type-2 neuro-fuzzy systems: A review

Highlights

• Learning algorithms of T2FLS are reviewed.

• Hybrid learning of parameters are reviewed particularly.

• The learning algorithms for T2FLS are divided into three categories.

• Comparison of the three categories is discussed at the end.

Abstract

Type-2 fuzzy logic systems have extensively been applied to various engineering problems, e.g. identification, prediction, control, pattern recognition, etc. in the past two decades, and the results were promising especially in the presence of significant uncertainties in the system. In the design of type-2 fuzzy logic systems, the early applications were realized in a way that both the antecedent and consequent parameters were chosen by the designer with perhaps some inputs from some experts. Since 2000s, a huge number of papers have been published which are based on the adaptation of the parameters of type-2 fuzzy logic systems using the training data either online or offline. Consequently, the major challenge was to design these systems in an optimal way in terms of their optimal structure and their corresponding optimal parameter update rules. In this review, the state of the art of the three major classes of optimization methods are investigated: derivative-based (computational approaches), derivative-free (heuristic methods) and hybrid methods which are the fusion of both the derivative-free and derivative-based methods.

Introduction

Since the inception of the fuzzy set theory in 1965, the mathematical advancements have progressed to exceptionally high standards. A plethora of research has been conducted on fuzzy systems and its implementations in many disciplines. A demanding analysis is required to collect the information on fuzzy systems i.e., the theoretical and real-time applications of fuzzy sets available in literature. In this paper, a literature review is conducted through searching of bibliographic databases. The search is limited to four databases, IEEE Xplore, SpringerLink, ScienceDirect and Wiley online library, and to the years 2000–2014, respectively. These four databases are the major publishers in the field of fuzzy logic theory.

In this survey, the term “fuzzy system” was searched initially. The search for this term identified papers in every aspect of the field such as control system, modeling, design, theorem, expert system, knowledge, regression and classification. A total of 98,702 conference publications and 55,715 journal publications are searched using the above term. The search is then refined by the term “type-2 fuzzy”, using the query of 〈fuzzy system〉 AND 〈type-2 fuzzy〉, leaving away the publications in type-1 fuzzy logic theory. The annual number of publications for type-1 and type-2 fuzzy logic theory can be seen in Fig. 1a. The large number of publications reported for type-1 fuzzy logic theory is due to the fact that the early introduced type-1 fuzzy logic systems (FLSs) have several software packages that simplify the task of researchers. However, a continuous increase in publication of type-2 FLSs (T2FLSs) can be seen in Fig. 1b. The search is again refined by the query 〈fuzzy system〉 AND 〈fuzzy learning〉 in order to pick the publications on fuzzy learning models only, which is also the main research focus of this review. Fig. 2 shows the trend in the number of publications on fuzzy learning, which shows the wider interest in adaptive fuzzy logic systems rather than conventional fuzzy logic systems in which the parameters are fixed.

The learning and tuning can be used interchangeably in the design of a FLS. However, the difference between the two is that the former is a process in FLSs where the search does not depend on predefined parameters and automatic design of a FLS starts from the scratch, whereas the later starts the optimization of a FLS with a set of predefined parameters and focuses to find the best set. Different approaches of soft computing can be applied here to enhance the computational and predictive performance of fuzzy systems. Indeed, research has demonstrated that formalizing an issue pertaining human expert knowledge is a difficult and time consuming job. More often than not, it does not even prompt completely fulfilling results. For that reason, a sort of data-driven approach of fuzzy systems is usually beneficial [1].

Generally, a fuzzy system with learning ability allows its different parameters to be tuned. The dashed arrow crossing the blocks of a T2FLS in Fig. 3 shows the possible components that can be tuned. During the design of a non-adaptive fuzzy system, experts assign linguistic labels to problem variables by using fuzzy membership functions (MFs). However they cannot give the precise MFs defining the semantics of these labels. Normally, these values are defined by partitioning the domain of interest. Through discretization, the variables in the domain are partitioned into the equivalent number of intervals that of linguistic labels considered. The process needs to define a uniform fuzzy partition with symmetric and identical shape fuzzy sets. However, this approach generally ends up with a sub-optimal performance of the fuzzy system [1]. In order to address this as a specific end goal, different learning techniques have been reported in literature for the generation of fuzzy set automatically. These techniques include decision tree [2], [3], [4], clustering [5], [6], hybrid models [7], [8], [9] and evolutionary algorithms [10], [11], [12]. The presence of 3D-MF in T2FLS necessitates the adjustment of more parameters than T1FLS, which makes the learning process more complicated [13]. The footprint of uncertainty (FOU) in interval T2FLSs (IT2FLS) can also be tuned to improve the performance in the presence of noise [14].

In general, fuzzy modeling is a system modeling with fuzzy rule based systems (FRBS) that represents a local model which is effectively interpretable and analyzable [15]. When the expert is not available or does not have sufficient information to stipulate the fuzzy rules, then numerical information is utilized to determine these rules. Two distinguished fusions of fuzzy with neural networks (NNs) also known as neuro-fuzzy models [16] and with genetic algorithm known as genetic fuzzy systems [15] have been used to automatically generate the fuzzy rules. FRBS is a universal approximator as it can approximate any function to the desired degree of accuracy [17], [18]. FRBS is a preferable choice over NNs, as the parameters involved have a real world meaning and consequently, the initial guess parameters can substantially enhance the training algorithm.

The optimization methods for FLSs can be broadly categorized into three methods as shown in Fig. 4. In this review, the main motivation is to present the state of the art of the three major classes of optimization methods:

• Derivative-based (computational approaches),

• Derivative-free (heuristic methods),

• Hybrid methods which are the fusion of both the derivative-free and derivative-based methods.

To the best of our knowledge, there is no paper in literature which focuses on the comparison of all the different learning algorithms listed in this survey paper specifically on interval type-2 fuzzy neural networks (T2FNNs). Recently, a book has been published in Elsevier in which some of the methods are compared [19]. We think that the learning control by using T2FNNs will be a hot topic in the near future too. This survey paper, which collects all of the learning methods in literature, will help researchers a lot to see their pros and cons in one paper.

The rest of the paper is organized as follows: The derivative-based optimization algorithms for T2FLS are described in Section 2. The derivative-free optimization algorithms are given in Section 3. Section 4 discusses the hybrid learning algorithms of T2FLS. Some comparison and discussions are given in Section 5.