Elsevier

Neurocomputing

Volume 175, Part A, 29 January 2016, Pages 450-458
Neurocomputing

Brief Papers
Stochastic stability of nonlinear discrete-time Markovian jump systems with time-varying delay and partially unknown transition rates

https://doi.org/10.1016/j.neucom.2015.10.081Get rights and content

Abstract

This paper is concerned with stochastic stability of a class of nonlinear discrete-time Markovian jump systems with interval time-varying delay and partially unknown transition probabilities. A new weighted summation inequality is first derived. We then employ the newly derived inequality to establish delay-dependent conditions which guarantee the stochastic stability of the system. These conditions are derived in terms of tractable matrix inequalities which can be computationally solved by various convex optimized algorithms. Numerical examples are provided to illustrate the effectiveness of the obtained results.

Introduction

In recent years, Markovian jump systems (MJSs), an important class of stochastic hybrid systems in which the switching between subsystems are governed by a finite-state Markov chain, have received an extensive attention from researchers due to their flexibility in modeling real-world problems (see, for example [1], [2], [3], [4], [5], [6], [7], [8], [9], [10], [11] and the references therein). They have been used in many practical systems, which are subjected to random abrupt changes in the inputs, internal variables and other system parameters caused by the occurrence of some inner discrete events in the system such as random failures/repairs of the components, change of the subsystems interconnections and so on [1], [12], [13].

On the other hand, time-delay, an inherent characteristic of many practical systems, is ubiquitous in dynamical systems and usually is a source of poor performance, oscillations or instability [14]. Thus, the problem of stability analysis and control of time-delay systems is essential and of great importance for both theoretical and practical reasons. This problem has attracted considerable attention from researchers in the field of systems and control theory [15], [16], [17], [18], [19], [20], [21], [22]. Recently, MJSs with time-delay have drawn much attention in the literature. Some important problems in systems and control theory have been intensively developed for continuous-time/discrete-time MJSs with delays. For instance, we refer the readers to [23], [24], [25] for stochastic and mean square stability analysis of linear Markovian jump systems (LMJSs) with delay, and [13], [26], [27], [28], [29], [30], [31] for the problems of state feedback/output feedback stabilization and H control of LMJs with mode-dependent/independent delays. Some other important control issues of MJSs such as guaranteed cost control, sliding-mode control, fault detection and so on were also investigated, for example, in [32], [33], [34], [35], [36], [37], [38], [39]. Particularly, in recent work [40], based on the Lyapunov–Krasovskii functional method, delay-independent conditions were derived in terms of linear matrix inequalities to ensure the stochastic stability of a class of nonlinear discrete-time impulsive MJSs with constant delay and partly unknown transition probabilities. It was shown that, when the interval of consecutive impulsive times is appropriately large, a stable discrete-time Markovian jump delay system can retain its stochastic stability with destabilizing impulses. Also using the framework of the Lyapunov–Krasovskii functional method, the problem of state bounding for a class of discrete-time LMJSs with interval time-varying delay and bounded disturbance input was considered in [41] for the first time. By constructing a set of improved Lyapunov–Krasovskii functionals in combination with delay-decomposition technique, delay-range-dependent conditions expressed in terms of matrix inequalities were derived which ensure that all state trajectories of the system are mean square bounded.

However, as discussed in [24], for MJSs, the transition probabilities of the jumping process are critical factors which determine the behavior of the system. In most studies of MJSs found in the literature, these probabilities are usually assumed to be completely accessible [2], [3], [4], [6], [35], [25], [26], [27], [28], [37], [38], [39]. In practice, incomplete transition probabilities of the corresponding Markov chain are often encountered especially when adequate samples of the transitions are costly or time-consuming to obtain. To this point, such a study of MJSs with partially unknown transition probabilities is very relevant and therefore it should be receiving a greater focus [24], [31], [32], [40], [42], [43], [44], [45], [46], [47].

Besides that, discrete-time systems with delays have strong background in engineering applications. Firstly, with the rapid development of computer-based computational techniques, discrete-time systems are more suitable for computer simulation, experiment and computation. Secondly, many practical systems are in the form of nonlinear and/or non-autonomous continuous-time systems with time-varying delays. A discretization from continuous-time systems leads to discrete-time systems described by difference equations which inherit the similar dynamical behavior of the continuous ones [48]. In addition, Markovian jumps are likely to exist in discrete-time delay systems [40]. Thus, it is of interest and necessary to study stability of discrete-time MJSs with delay, especially time-varying delay. Due to the fact that most practical systems are only stable with delays in a certain range, delay-dependent stability conditions, which utilize information on the size of time delays, are reasonable and less conservative than delay-independent conditions. Therefore, much attention from researchers has been devoted to derive less conservative delay-dependent stability conditions for time-delay systems with the key objective in achieving a maximum allowable delay bound [18], [19], [20], [49], [50], [51]. To reduce the conservatism of stability conditions, it is important to improve fundamental inequalities to be used in establishing such stability criteria [20]. For continuous-time systems with delays, some new integral-based inequalities were proposed in [19], [20]. Most recently, novel summation inequalities were derived in [52], [53] by extending the Wirtinger-based integral inequality. These summation inequalities provide a powerful tool to derive less conservative stability conditions for discrete-time systems with interval time-varying delay in the framework of tractable linear matrix inequalities.

Motivated by the aforementioned discussion, in this paper, we study the stochastic stability of a class of nonlinear discrete-time MJSs with interval time-varying delay and partially unknown transition probabilities. A new weighted summation inequality, which is reduced to the summation inequalities proposed in [52], [53] in critical case, is first derived. We then employ the newly derived inequality to establish delay-dependent conditions which guarantee the stochastic stability of the system.

The remainder of this paper is organized as follows. In Section 2, some preliminary results are presented. New summation inequalities and their applications to stochastic stability analysis of a class of nonlinear discrete-time MJSs with interval time-varying delay and partially unknown transition rates are presented in 3 A new summation inequality, 4 Stability conditions, respectively. Numerical examples are provided in Section 5 to illustrate the effectiveness of the obtained results.

Notations: Throughout this paper, we denote Z and Z+ as the set of integers and positive integers, respectively. For a,bZ, ab, Z[a,b] denotes the set of integers between a and b and Za={kZ:ka}. We denote Rn the n-dimensional Euclidean space with vector norm ·, Rn×m the set of n×m real matrices and Sn+ the set of symmetric positive definite matrices. For matrices A,BRn×m, col{A,B} and diag{A,B} denote the block matrix (AB) and (A00B), respectively.

Section snippets

Preliminaries

Let (Ω,F,Pr) be a complete probability space and {rk,kZ0} be a discrete-time Markovian jump process specifying the system mode which takes value in a finite set M={1,2,,q} with transition probabilities Pr{rk+1=j|rk=i}=pij,i,jM,where pij0, i,jM, and j=1qpij=1 for all iM. We denote Π=(pij) the state transition probability matrix and p=(p1,p2,,pq) the initial probability distribution, where pi=Pr{r0=i}, iM.

Consider a class of nonlinear discrete-time Markovian jump systems (DMJSs) in the

A new summation inequality

The objective of this section is to establish a new lower bound for the gap of (7) SRg(u)=i=kτ2kτ1αkiuT(i)Ru(i)cαs(i=kτ2kτ1u(i))TR(i=kτ2kτ1u(i)).For given τ1,τ2Z+, τ1τ2, we denote =τ2τ1+1 the length of interval Z[τ1,τ2].

Lemma 3

The following equalities hold for all α(0,1)sαi=kτ2kτ1αik(i+1k+τ2)=ατ2δατ1+1(1α)2,dαi=kτ2kτ1αik(i+1k+τ2)2=(1+α)ατ2ατ1+1(δ2+α)(1α)3,where δ=1+(1α).

Proof

The proof is straightforward and thus is omitted here. □

We also denote ηα=1αα1α, γα=dαcαs

Stability conditions

In this section, we employ the summation inequality proposed in the preceding section to derive delay-dependent stability conditions for system (1).

Let us denote ei=[0n×(i1)nIn0n×(7i)n], Ai=Aie1+Adie3, Di=(AiIn)e1+Adie3, i=1,,7, and the following notationsζ0(k)=col{(x(k)x(kτ1)x(kτ)x(kτ2)),(x1a(k)x2a(k)x3a(k))},x1a(k)=1σ(τ1)s=kτ1kx(s),x2a(k)=1σ(ττ1)s=kτkτ1x(s),x3a(k)=1σ(τ2τ)s=kτ2kτx(s),F1=col{e1e2,e1e2+(1+τ1)/η1(e2e5)},F2=col{e2e3,e2+e32e6},F3=col{e3e4,e3+e42e7},Π0i=AiTP˜i

Examples

In this section, we give some examples to demonstrate the effectiveness of the obtained results.

Example 1

Consider a three-modes system (1) with the system matrices taken from [41]A1=(0.100.0500.090.10.100.12),A2=(0.10000.10.120.1500.16),A3=(0.10.10.1500.10.10.10.150.1),Ad1=(0.100.10.10.1000.10.16),Ad2=(0.100.10.10.09000.10.12),Ad3=(0.120.10.10.10.100.10.10.11).Nonlinear perturbation is given byF(rk,x(k),x(kτ(k)))=C(rk)x(k)1+x(kτ(k))2+ex(k)D(rk)x(kτ(k)),where C1=C2=(0.10.0500.050.10000.01

Conclusion

In this paper, a new weighted summation inequality has been proposed. Then, by employing the newly derived inequality in combination with improved reciprocally convex inequality, new delay-dependent stochastic stability conditions have been derived for a class of nonlinear discrete-time Markovian jump systems with interval time-varying delay and partially unknown transition probabilities. Numerical examples have been provided to illustrate the effectiveness of the theoretical results obtained

Acknowledgments

The authors would like to thank the Editor-in-Chief, the Associate Editor and the Anonymous Reviewers for their helpful comments and suggestions. This work was partialy supported by the ARC Discovery (Grant DP130101532), the NAFOSTED of Vietnam (Grant 101.01-2014.35) and the Research Fund of Hanoi Pedagogical University No.2 (Grant C.2015.01).

L.V. Hien received the B.Sc, M.Sc., and Ph.D. degrees, all in mathematics from the Faculty of Mathematics and Informatics, Hanoi National University of Education, Vietnam, in 2001, 2004 and 2011, respectively. He began with Hanoi National University in 2001, where he is currently an Associate Professor in mathematics. Prof. Hien is a reviewer of more than 20 international journals and he is the author or co-author of more than 30 journal papers. His research interests include the qualitative

References (54)

  • W.H. Chen et al.

    Delay-dependent stability and H control of uncertain discrete-time Markovian jump systems with mode-dependent time delays

    Syst. Control Lett.

    (2004)
  • J. Xiong et al.

    Stabilization of discrete-time Markovian jump linear systems via time-delayed controllers

    Automatica

    (2006)
  • W. Qi et al.

    State feedback controller design for singular positive Markovian jump systems with partly known transition rates

    Appl. Math. Lett.

    (2015)
  • L. Wu et al.

    Sliding mode control with bounded L2 gain performance of Markovian jump singular time-delay systems

    Automatica

    (2012)
  • Y. Zhang

    Stability of discrete-time Markovian jump delay systems with delayed impulses and partly unknown transition probabilities

    Nonlinear Dyn.

    (2014)
  • L. Zhang et al.

    Stability and stabilization of Markovian jump linear systems with partially unknown transition probability

    Automatica

    (2009)
  • Q. Ma et al.

    Stability and synchronization for Markovian jump neural networks with partly unknown transition probabilities

    Neurocomputing

    (2011)
  • X. Xing et al.

    Finite-time stability of Markovian jump neural networks with partly unknown transition probabilities

    Neurocomputing

    (2015)
  • L.V. Hien

    A novel approach to exponential stability of nonlinear non-autonomous difference equations with variable delays

    Appl. Math. Lett.

    (2014)
  • Y. Li et al.

    Estimating stable delay intervals with a discretized Lyapunov-Krasovskii functional formulation

    Automatica

    (2014)
  • A. Seuret et al.

    Hierarchy of LMI conditions for the stability analysis of time-delay systems

    Syst. Control Lett.

    (2015)
  • P.T. Nam et al.

    Discrete Wirtinger-based inequality and its application

    J. Frankl. Inst.

    (2015)
  • O.L.V. Costa et al.

    Discrete-time Markov Jump Linear Systems

    (2005)
  • R. Elliott et al.

    State and mode estimation for discrete-time jump Markov systems

    SIAM J. Control Optim.

    (2005)
  • C.Z. Souza

    Robust stability and stabilization of uncertain discrete-time Markovian jump linear systems

    IEEE Trans. Autom. Control

    (2006)
  • J.C. Geromel et al.

    Dynamic output feedback control of discrete-time Markov jump linear systems through linear matrix inequalities

    SIAM J. Control Optim.

    (2009)
  • M.S. Ali et al.

    Delay-dependent stability criteria of uncertain Markovian jump neural networks with discrete interval and distributed time-varying delays

    Neurocomputing

    (2015)
  • Cited by (14)

    • Online reinforcement learning multiplayer non-zero sum games of continuous-time Markov jump linear systems

      2022, Applied Mathematics and Computation
      Citation Excerpt :

      Blair [3] proposed a discrete-time MJLSs optimal control problem and Wonham [4] derived another way to solve the relevant quadratic in optimal control problem. Zhang [5] studied the stochastic stability with a partial unknown transition probability [6–8] of MJLSs. On the other hand, adaptive dynamic programming has been shown to be a framework to sovle the optimal control problems of dynamic systems.

    • Robust stabilization of non-stationary Markov jump 2-D systems with multiplicative noises

      2021, Journal of the Franklin Institute
      Citation Excerpt :

      Many results involving performance analysis and controller/filter/observer design for various classes of Markov jump systems, both linear and nonlinear models, have been published. We refer the reader to [7–13] for a few references. The theory for two-dimensional (2-D) Markov jump systems has also received considerably growing attention in the past few years [14–20].

    • Observer-based H<inf>∞</inf> control for discrete-time one-sided Lipschitz Markovian jump delayed systems under partially unknown transition probabilities

      2020, Journal of the Franklin Institute
      Citation Excerpt :

      Now, in the light of the literature survey presented in the Introduction section and the discussion here, we summarize some points. Firstly, different from the relevant discrete-time study available in [33], our work considers OSL condition, an appropriately chosen LKF, the improved summation inequality, the external disturbance, and discusses the analysis, synthesis, and observation problems. Secondly, in comparison to the results (OSL MJSs with continuous dynamics) available in the literature [34–36], this work considers OSL MJSs with discrete dynamics, and the practical factors of the state delays and external disturbances for MJSs, which to the best of our knowledge, have not been taken into account, till date [34–36].

    • Stability analysis of two-dimensional Markovian jump state-delayed systems in the Roesser model with uncertain transition probabilities

      2016, Information Sciences
      Citation Excerpt :

      Some important problems in systems and control theory have been extensively developed for DMJSs. Particularly, stochastic and mean square stability of MJSs with time-varying delay was investigated in [6,17,23], based on extended LKFs and LMI techniques. The problems of state feedback/output feedback stabilization and H∞ control of MJSs with delays were addressed in [2,4,24].

    • New inequality-based approach to passivity analysis of neural networks with interval time-varying delay

      2016, Neurocomputing
      Citation Excerpt :

      Although it was theoretically and numerically shown that the integral inequalities proposed in [40] give tighter lower bound than the ones based on Jensen׳s inequality and its variants, particular challenging remains on how to construct a suitable and effective LKF as well as how to cope with the relationship between possibly fast time-varying delay and its interval. In the proof of Theorem 1, we have constructed an improved LKF, which fully utilizes information of the neuron activation functions and the time-delay in single, double and triple integral terms, meanwhile the technique of reciprocally convex combination [42] has been used to deal with interval time-varying delay. This approach allows us to obtain less conservative conditions without using additional free-weighting matrices.

    View all citing articles on Scopus

    L.V. Hien received the B.Sc, M.Sc., and Ph.D. degrees, all in mathematics from the Faculty of Mathematics and Informatics, Hanoi National University of Education, Vietnam, in 2001, 2004 and 2011, respectively. He began with Hanoi National University in 2001, where he is currently an Associate Professor in mathematics. Prof. Hien is a reviewer of more than 20 international journals and he is the author or co-author of more than 30 journal papers. His research interests include the qualitative and asymptotic behavior of differential-difference equations, stability analysis and control of complex dynamical systems and time-delay systems.

    N.T. Dzung received the B.Sc. and M.Sc. degrees in mathematics from the Faculty of Mathematics, Mechanics and Informatics, VNU University of Science, Hanoi, Vietnam, in 2001 and 2009, respectively. He is a Lecturer in mathematics at Hanoi Pedagogical University No. 2, Vietnam, where he is currently pursuing the Ph.D. degree in mathematics. His research interests include Markov jump systems, stability analysis and control of time-delay systems.

    H. Trinh received the B.Eng. (Hons.), M.Eng.Sc., and Ph.D. degrees from the University of Melbourne, Melbourne, VIC, Australia, in 1990, 1992, and 1996, respectively, all in electrical and electronic engineering. He began with Deakin University, Geelong, VIC, Australia, in 2001, where he is currently an Associate Professor. His current research interests include systems and control theory, fault diagnosis and fault tolerant control, time-delay systems, and application of control theory to industrial systems and power systems. He has published a research book entitled Functional Observers for Dynamical Systems (Springer, 2012), and over 100 refereed journal papers on control engineering. He is very passionate about transferring/instilling his knowledge of control engineering to the next generation of engineers and Ph.D. researchers at Deakin University. Prof. Trinh was a recipient of some national competitive research grants, such as Australian Research Council Grants, which allowed him to conduct his own research interests. He was a recipient of some recognitions for excellence in teaching from the offices of the Dean and Deputy Vice-Chancellor at Deakin University.

    View full text