An ensemble of intelligent water drop algorithms and its application to optimization problems
Introduction
Optimization is a process that is concerned with finding the best solution of a given problem from among a range of possible solutions, within an affordable time and cost [66]. Optimization can be applied to many real-world problems, in a large variety of domains. As an example, mathematicians apply optimization methods to identify the best outcome pertaining to some mathematical functions within a range of variables [63]. In the presence of conflicting criteria, engineers often use optimization methods to find the best performance of a model subject to certain criteria, e.g. cost, profit, and quality [37]. Numerous methods have been developed and used to solve many NP-hard (i.e. problems that have no known solutions in polynomial time) [33] optimization problems [39], [64], [69]. A number of recent survey papers that provide comprehensive information on optimization methods and their associated categorizations are also available in the literature [11], [30], [33], [35]. In this study, we focus on the Swarm Intelligence (SI) methodology for undertaking optimization problems.
Among a variety of optimization methods, SI constitutes an innovative family of nature inspired models that has attracted much interest from researchers [8]. SI models stem from different natural phenomena pertaining to different swarms, e.g. ant colony optimization (ACO) is inspired by the foraging behavior of ants [13], [14], while particle swarm optimization (PSO) is inspired by the social behaviors of bird flocking or fish schooling [56]. In this paper, we investigate a relatively recent swarm-based model known as the intelligent water drop (IWD) algorithm [50]. IWD is inspired by the natural phenomenon of water drops flowing with soil and velocity along a river. It imitates the natural phenomena of water drops flowing through an easier path, i.e., a path with less barriers and obstacles, from upstream to downstream. Specifically, IWD is a constructive-based, meta-heuristic algorithm, comprising a set of cooperative computational agents (water drops), that iteratively constructs the solution pertaining to a problem. The solution is formulated by water drops that traverse a path with a finite set of discrete movements. A water drop begins its journey with an initial state. It iteratively moves step-by-step passing through several intermediate states (partial solutions), until a final state (complete solution) is reached. A probabilistic method is used to control the movements of the water drops. Specifically, each water drop in the IWD algorithm has two key attributes: soil and velocity. They are used to control the probability distribution of selecting the movement of the water drop, and to find the partial solution. The soil represents an indirect communication mechanism, and enables the water drop to cooperate with other nearby water drops. The soil level indicates the cumulative proficiency of a particular movement. Contrary to the ant colony algorithm [15], in which the pheromone level is constantly updated, the soil level is dynamically updated with respect to the velocity of the water drop. In other words, the velocity influences the dynamics of updating the soil level, which is used to compute the probability of the movement of the water drop from the current state to the next. In addition, the velocity is related to heuristic information pertaining to the problem under scrutiny. This information is used to guide the water drop to move from one state to another.
The IWD algorithm is useful for tackling combinatorial optimization problems [53]. IWD initially was applied for solving the travelling salesman problem (TSP) [50]. Over the past few years, it has been successfully adopted to solve different NP-hard optimization problems [57]. Table 1 summarizes a number of applications that have been successfully solved using the IWD algorithm. The success of the IWD algorithm stems from two salient properties [4], [52], [53]: (i) its cooperative learning mechanism allows water drops to exchange their search knowledge and (ii) the algorithm is able to memorize the search history.
As can be seen in Table 1, most of the reported IWD investigations in the literature focus on solving optimization problems in different application domains. Only a small number of studies pertaining to the theoretical aspects of the IWD algorithm to improve its performance are available in the literature. As an example, an Enhanced IWD (EIWD) algorithm to solve job-shop scheduling problems was proposed by Niu et al. [42]. The following schemes have been introduced to increase diversity of the search space and enhance the original IWD performance: (i) varying the initial soil and velocity values, (ii) employing the conditional probability in the selection probability, (iii) bounding the soil level, (iv) using the elite mechanism to update the soil and (v) combining the IWD algorithm with a local search method.
In Alijla et al., [4], the modified IWD algorithm was introduced to address the limitations of fitness proportionate selection method in the original IWD algorithm. Two ranking-based methods i.e. linear and exponential were introduced. The extendibility of the IWD algorithm was investigated by Kayvanfar and Teymourian [25]. In particular, a hybrid IWD and local search algorithm was introduced. The variable neighborhood structure (VNS) algorithm was used to tackle scheduling problems of unrelated parallel machines. The aforementioned modifications focus on the soil level and the algorithm exploitation capability to enhance its performance.
In this paper, two enhancements pertaining to the modified IWD algorithm introduced in [4] are proposed to achieve a balance between exploration (navigating through new regions of the search space) and exploitation (searching a specific region of the search space thoroughly). The proposed enhancements include: (i) an ensemble of the modified IWD algorithms in a Master-River, Multiple-Creek (MRMC) model, whereby a divide-and-conquer strategy is utilized to improve the search process and (ii) a hybrid MRMC-IWD model, which improves the exploitation capability of MRMC-IWD with a local improvement method to constraint the search to the local optimal solution, rather than the entire search space. To evaluate the proposed models (i.e. MRMC-IWD and hybrid MRMC-IWD) and to facilitate a performance comparison study with other state-of-the-art methods, two case studies related to optimization problems that have been widely used in the literature [24], [31], [38], [58], [67], namely TSP and rough set features subset selection (RSFS), are conducted. These problems are selected because they are NP-hard, and have different level of difficulties. The complexity of the problem (i.e., the number of alternatives) grows exponentially with the size of the problem [19]. Since TSPs have known bounds, they are useful to ascertain the effectiveness of the solutions produced by the proposed models. On the other hand, RSFS is crucial in pattern recognition applications. Contrary to TSP, RSFS presents strong inter-dependency among the decision variables (i.e., features). The feature sequences within the subset are not important, and the optimal solutions are normally unknown [70]. Therefore, both TSP and RSFS problems are selected as case studies to evaluate the usefulness of the proposed models and to benchmark the results against those published in the literature. As a result, the effectiveness of the proposed models for undertaking general optimization problems can be validated.
The rest of this paper is organized as follows. In Section 2, the background of the IWD and modified IWD algorithms is described. In Section 3, the proposed MRMC-IWD and hybrid MRMC-IWD models are explained. In Section 4, the case studies, i.e. RSFS and TSP, are explained in details. The results are analyzed and discussed in Section 5. Conclusions and suggestions for future research are presented in Section 6.
Section snippets
Background of the IWD algorithm
The IWD algorithm is a constructive-based, nature-inspired model introduced by Shah-Hosseini [50]. It is motivated by the dynamic of water flowing in a river, e.g. water follows an easier path which has fewer barriers and obstacles, water flows at a particular speed, water stream changes the environmental properties of the river, which subsequently changes the direction of water flow. The IWD algorithm computationally realizes some of these natural phenomena, and uses them as a computational
The proposed models
Two modifications are proposed to enhance the performance of the modified IWD algorithm. Firstly, an ensemble of the modified IWD algorithms, in a Master-River, Multiple-Creek IWD (MRMC-IWD) model is proposed. Secondly, the MRMC-IWD model is integrated with a local search method to enhance the performance by empowering the exploitation capability. These modifications are explained in detail in the following sub-sections.
Case studies
Two optimization problems, namely TSP and RSFS, were considered to validate the effectiveness of the proposed models, and to facilitate the comparison against other state-of-the-art methods in the literature. They are NP-hard combinatorial optimization problems, and have different levels of difficulty (as explained in Section 1). A detailed description of each problem is as follows.
Experiments and results
A series of experiments pertaining to the TSP and RSFS optimization problems using benchmark data sets was conducted to evaluate both MRMC-IWD and hybrid MRMC-IWD models. The details are as follows.
Conclusions
In this paper, two new models, i.e., MRMC-IWD and hybrid MRMC-IWD, have been proposed as extensions of the modified IWD algorithm. MRMC-IWD comprises an ensemble of the modified IWD algorithms. It consists of a master river and multiple independent creeks, in order to exploit the exploration capability of the modified IWD algorithm. The hybrid MRMC-IWD model integrates a local search method (i.e., the IILS method) into MRMC-IWD, in order to improve the exploitation capability by focusing the
Acknowledgment
The first author acknowledges the Islamic Development Bank (IDB) for the scholarship to study the Ph.D. degree at the Universiti Sains Malaysia (USM). The authors would like to thank the anonymous referees for their careful reading of the paper and their valuable comments and suggestions, which greatly improved the paper.
References (72)
- et al.
A modified intelligent water drops algorithm and its application to optimization problems
Expert Syst. Appl.
(2014) - et al.
Feature selection with intelligent dynamic swarm and rough set
Expert Syst. Appl.
(2010) - et al.
Hybrid metaheuristics in combinatorial optimization: a survey
Appl. Soft Comput.
(2011) - et al.
A survey on optimization metaheuristics
Inf. Sci.
(2013) - et al.
Feature selection for classification
Intell. Data Anal.
(1997) - et al.
Ant colony optimization theory: a survey
Theor. Comput. Sci.
(2005) - et al.
Feature subset selection by gravitational search algorithm optimization
Inf. Sci.
(2014) An effective implementation of the Lin–Kernighan traveling salesman heuristic
Eur. J. Oper. Res.
(2000)- et al.
Neural-intelligent water drops algorithm to select relevant textural features for developing precision irrigation system using machine vision
Comput. Electron. Agric.
(2011) - et al.
Finding rough and fuzzy-rough set reducts with SAT
Inf. Sci.
(2014)