Skip to main content

Advertisement

SpringerLink
Log in

Using genetic algorithms to generate test sequences for complex timed systems

  • Focus
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

The generation of test data for state-based specifications is a computationally expensive process. This problem is magnified if we consider that time constraints have to be taken into account to govern the transitions of the studied system. The main goal of this paper is to introduce a complete methodology, supported by tools, that addresses this issue by representing the test data generation problem as an optimization problem. We use heuristics to generate test cases. In order to assess the suitability of our approach we consider two different case studies: a communication protocol and the scientific application BIPS3D. We give details concerning how the test case generation problem can be presented as a search problem and automated. Genetic algorithms (GAs) and random search are used to generate test data and evaluate the approach. GAs outperform random search and seem to scale well as the problem size increases. It is worth to mention that we use a very simple fitness function that can be easily adapted to be used with other evolutionary search techniques.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  • Akcelik V, Bielak J, Biros G, Epanomeritakis I, Fernandez A, Ghattas O, Kim EJ, Lopez J, O’Hallaron D, Tu T, Urbanic J (2003) High resolution forward and inverse earthquake modeling on terascale computers. In: 16th ACM/IEEE conference on supercomputing, SC’03. ACM Press, New York

  • Ammann P, Offutt J (2008) Introduction to software testing. Cambridge University Press, Cambridge

    Book  MATH  Google Scholar 

  • Blanquart JP, Armengaud E, Baufreton P, Bourrouilh Q, Griessnig G, Krammer M, Laurent O, Machrouh J, Peikenkamp T, Schindler C, Wien T (2011) Towards cross-domains model-based safety process, methods and tools for critical embedded systems: the CESAR approach. In: 30th international conference on computer safety, reliability, and security, SAFECOMP’11, LNCS, vol 6894. Springer, Berlin, pp 57–70

  • Bosik BS, Uyar MÜ (1991) Finite state machine based formal methods in protocol conformance testing. Comput Netw ISDN Syst 22:7–33

    Article  Google Scholar 

  • Chow T (1978) Testing software design modeled by finite state machines. IEEE Trans Softw Eng 4:178–187

    Article  MATH  Google Scholar 

  • Derderian K (2006) Automated test sequence generation for finite state machines using genetic algorithms. PhD. thesis, Brunel University

  • Derderian K, Hierons RM, Harman M, Guo Q (2004) Input sequence generation for testing of communicating finite state machines (CFSMs). In: 6th annual conference on genetic and evolutionary computation, GECCO’04, LNCS, vol 3103. Springer, Berlin, pp 1429–1430

  • Derderian K, Hierons RM, Harman M, Guo Q (2006) Automated unique input output sequence generation for conformance testing of FSMs. Comput J 49(3):331–344

    Article  Google Scholar 

  • Derderian K, Merayo MG, Hierons RM, Núñez M (2009) Aiding test case generation in temporally constrained state based systems using genetic algorithms. In: 10th international conference on artificial neural networks, IWANN’09, LNCS, vol 5517. Springer, Berlin, pp 327–334

  • Derderian K, Hierons RM, Harman M, Guo Q (2010) Estimating the feasibility of transition paths in extended finite state machines. Autom Softw Eng 17(1):33–56

    Article  Google Scholar 

  • Derderian K, Merayo MG, Hierons RM, Núñez M (2011) A case study on the use of genetic algorithms to generate test cases for temporal systems. In: 11th international conference on artificial neural networks, IWANN’11, LNCS, vol 6692. Springer, Berlin, pp 396–403

  • Duale AY, Uyar MÜ (2004) A method enabling feasible conformance test sequence generation for EFSM models. IEEE Trans Comput 53(5):614–627

    Article  Google Scholar 

  • Goldberg D (1989) Genetic algorithms in search. Optimisation and machine learning. Addison-Wesley, Reading

    Google Scholar 

  • Grieskamp W, Kicillof N, Stobie K, Braberman V (2011) Model-based quality assurance of protocol documentation: tools and methodology. Softw Test Verif Reliab 21(1):55–71

    Article  Google Scholar 

  • Guo Q, Hierons RM, Harman M, Derderian K (2003) Computing unique input/output sequences using genetic algorithms. In: 3rd international workshop on formal approaches to testing of software, FATES’03, LNCS, vol 2931. Springer, Berlin, pp 164–177

  • Harman M, McMinn P (2010) A theoretical and empirical study of search-based testing: local, global, and hybrid search. IEEE Trans Softw Eng 36(2):226–247

    Article  Google Scholar 

  • Hennie F (1964) Fault-detecting experiments for sequential circuits. In: 5th annual symposium on switching circuit theory and logical design, pp 95–110

  • Hierons RM, Ural H (2002) Reduced length checking sequences. IEEE Trans Comput 51(9):1111–1117

    Article  MathSciNet  Google Scholar 

  • Hierons RM, Ural H (2006) Optimizing the length of checking sequences. IEEE Trans Comput 55(5):618–629

    Article  MathSciNet  Google Scholar 

  • Hierons RM, Bowen J, Harman M (eds) (2008) Formal methods and testing, LNCS, vol 4949. Springer, Berlin

  • Hierons RM, Bogdanov K, Bowen J, Cleaveland R, Derrick J, Dick J, Gheorghe M, Harman M, Kapoor K, Krause P, Luettgen G, Simons A, Vilkomir S, Woodward M, Zedan H (2009a) Using formal methods to support testing. ACM Comput Surv 41(2):9:1–9:76

  • Hierons RM, Merayo MG, Núñez M (2009b) Testing from a stochastic timed system with a fault model. J Logic Algebr Progr 78(2):98–115

    Article  MATH  Google Scholar 

  • Jones BF, Eyres DE, Sthamer HH (1998) A strategy for using genetic algorithms to automate branch and fault-based testing. Comput J 41(2):98–107

    Article  Google Scholar 

  • Kalaji AS, Hierons RM, Swift S (2009) Generating feasible transition paths for testing from an extended finite state machine (EFSM). In: 2nd international conference on software testing verification and validation, ICST’09. IEEE Computer Society, pp 230–239

  • Kalaji AS, Hierons RM, Swift S (2011) An integrated search-based approach for automatic testing from extended finite state machine (EFSM) models. Inf Softw Technol 53(12):1297–1318

    Article  Google Scholar 

  • Karypis G (2011) METIS: A software package for partitioning unstructured graphs, partitioning meshes and computing fill-reducing orderings of sparse matrices. version 5.0. Tech. rep., Department of Computer Science & Engineering, University of Minnesota.

  • Laurent O (2010) Using formal methods and testability concepts in the avionics systems validation and verification (V&V) process. In: 3rd international conference software testing, verification, and validation, ICST’10. IEEE Computer Society, pp 1–10

  • Lee D, Yannakakis M (1994) Testing finite state machines: state identification and verification. IEEE Trans Comput 43:306–320

    Article  MathSciNet  Google Scholar 

  • Lee D, Yannakakis M (1996) Principles and methods of testing finite state machines: a survey. Proc IEEE 84(8):1090–1123

    Article  Google Scholar 

  • Loureiro A, González J, Pena TF (2003) A parallel 3D semiconductor device simulator for gradual heterojunction bipolar transistors. J Numer Modell Electr Netw Devices Fields 16(1):53–66

    Article  MATH  Google Scholar 

  • Mairhofer S, Feldt R, Torkar (2011) R Search-based software testing and test data generation for a dynamic programming language. In: 13th annual conference on genetic and evolutionary computation, GECCO’11. ACM Press, New York, pp 1859–1866

  • McMinn P (2004) Search-based software test data generation: a survey. Softw Test Verif Reliab 14(2):105–156

    Article  Google Scholar 

  • Merayo MG, Núñez M, Rodríguez I (2007) Generation of optimal finite test suites for timed systems. In: 1st IEEE & IFIP international symposium on theoretical aspects of software engineering, TASE’07. IEEE Computer Society, pp 149–158

  • Merayo MG, Núñez M, Rodríguez I (2008a) Extending EFSMs to specify and test timed systems with action durations and timeouts. IEEE Trans Comput 57(6):835–848

    Article  MathSciNet  Google Scholar 

  • Merayo MG, Núñez M, Rodríguez I (2008b) Formal testing from timed finite state machines. Comput Netw 52(2):432–460

    Article  MATH  Google Scholar 

  • Merayo MG, Núñez M, Hierons RM (2011) Testing timed systems modeled by stream X-machines. Softw Syst Model 10(2):201–217

    Article  Google Scholar 

  • Merayo MG, Núñez M, Rodríguez I (2012) A formal framework to test soft and hard deadlines in timed systems. Softw Test Verif Reliab. doi:10.1002/stvr.448

  • Michael CC, McGraw G, Schatz MA (2001) Generating software test data by evolution. IEEE Trans Softw Eng 27(12):1085–1110

    Article  Google Scholar 

  • Michalewicz Z (1996) Genetic algorithms + data structures = evolution programs, 3rd ed. Revised and extended edn. Springer, Berlin

  • Molinero C, Núñez M, Andrés C (2009) Combining genetic algorithms and mutation testing to generate test sequences. In: 10th international conference artificial neural networks, IWANN’09, LNCS, vol 5517. Springer, Berlin, pp 343–350

  • Molinero C, Núñez M, Hierons RM (2011a) Creating adaptive sequences with genetic algorithms to reach a certain state in a non-deterministic FSM. In: IEEE symposium on artificial life, ALIFE’11. IEEE Computer Society, pp 22–29

  • Molinero C, Núñez M, Hierons RM (2011b) Experimental comparison of different techniques to generate adaptive sequences. In: 11th international conference on artificial neural networks, IWANN’11, LNCS, vol 6692. Springer, Berlin, pp 404–411

  • Myers G (2004) The art of software testing, 2nd edn. Wiley, New York

    Google Scholar 

  • Núñez A, Fernández J, García JD, Carretero J (2008) Analyzing scalable high-performance I/O architectures. In: 3rd international conference on parallel and distributed processing techniques and applications, PDPTA’08, pp 631–637

  • Núñez A, Fernández J, García JD, García F, Carretero J (2010) New techniques for simulating high performance MPI applications on large storage networks. J Supercomput 51(1):40–57

    Article  Google Scholar 

  • Núñez A, Fernández J, Filgueira R, García F, Carretero J (2012) SIMCAN: a flexible, scalable and expandable simulation platform for modelling and simulating distributed architectures and applications. Simul Modell Practice Theory 20(1):12–32

    Article  Google Scholar 

  • Oh J, Harman M, Yoo S (2011) Transition coverage testing for simulink/stateflow models using messy genetic algorithms. In: 13th annual conference on genetic and evolutionary computation, GECCO’11. ACM Press, New York, pp 1851–1858

  • Petrenko A (2001) Fault model-driven test derivation from finite state models: annotated bibliography. In: 4th Summer school on modeling and verification of parallel processes, MOVEP’00, LNCS, vol 2067. Springer Berlin, pp 196–205

  • Petrenko A, Boroday S, Groz R (2004) Confirming configurations in EFSM testing. IEEE Trans Softw Eng 30(1):29–42

    Article  Google Scholar 

  • Ramalingom T, Thulasiraman K, Das A (2003) Context independent unique state identification sequences for testing communication protocols modelled as extended finite state machines. Comput Commun 26(14):1622–1633

    Article  Google Scholar 

  • Srinivas M, Patnaik LM (1994) Genetic algorithms: a survey. IEEE Comput 27:17–27

    Article  Google Scholar 

  • Stone SS, Haldar JP, Tsao SC, Hwu WW, Liang ZP, Sutton BP (2008) Accelerating advanced MRI reconstructions on GPUs. In: 5th conference on computing frontiers, CF’08. ACM Press, New York, pp 261–272

  • Szalay AS, Kunszt PZ, Thakar A, Gray J, Slutz D, Brunner RJ (2000) Designing and mining multi-terabyte astronomy archives: the sloan digital sky survey. In: 26th ACM SIGMOD international conference on management of data. ACM Press, New York, pp 451–462

  • Utting M, Legeard B (2007) Practical model-based testing: a tools approach. Morgan-Kaufmann, Menlo Park

    Google Scholar 

  • Whitley D (1999) A free lunch proof for gray versus binary encodings. In: 1st genetic and evolutionary computation conference, GECCO’99. Morgan Kaufmann, Menlo Park, pp 726–733

Download references

Acknowledgments

This research was partially supported by the Spanish MEC project TESIS (TIN2009-14312-C02-01) and the UK EPSRC project Testing of Probabilistic and Stochastic Systems (EP/G032572/1). We would like to thank Karnig Derderian for his participation in the previous stages of this research. We would also like to thank the anonymous reviewers of the paper for the careful reading and useful comments that have helped to improve the quality of the paper.

Author information

Authors and Affiliations

  1. Departamento de Sistemas Informáticos y Computación, Universidad Complutense de Madrid, Madrid, Spain

    Alberto Núñez, Mercedes G. Merayo & Manuel Núñez

  2. Department of Information Systems and Computing, Brunel University Uxbridge, Middlesex, UB8 3PH, UK

    Robert M. Hierons

Authors
  1. Alberto Núñez
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mercedes G. Merayo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Robert M. Hierons
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Manuel Núñez
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alberto Núñez.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Núñez, A., Merayo, M.G., Hierons, R.M. et al. Using genetic algorithms to generate test sequences for complex timed systems. Soft Comput 17, 301–315 (2013). https://doi.org/10.1007/s00500-012-0894-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-012-0894-5

Keywords

Search

Navigation