Skip to main content

Advertisement

SpringerLink
Log in

A market-oriented hierarchical scheduling strategy in cloud workflow systems

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

A cloud workflow system is a type of platform service which facilitates the automation of distributed applications based on the novel cloud infrastructure. One of the most important aspects which differentiate a cloud workflow system from its other counterparts is the market-oriented business model. This is a significant innovation which brings many challenges to conventional workflow scheduling strategies. To investigate such an issue, this paper proposes a market-oriented hierarchical scheduling strategy in cloud workflow systems. Specifically, the service-level scheduling deals with the Task-to-Service assignment where tasks of individual workflow instances are mapped to cloud services in the global cloud markets based on their functional and non-functional QoS requirements; the task-level scheduling deals with the optimisation of the Task-to-VM (virtual machine) assignment in local cloud data centres where the overall running cost of cloud workflow systems will be minimised given the satisfaction of QoS constraints for individual tasks. Based on our hierarchical scheduling strategy, a package based random scheduling algorithm is presented as the candidate service-level scheduling algorithm and three representative metaheuristic based scheduling algorithms including genetic algorithm (GA), ant colony optimisation (ACO), and particle swarm optimisation (PSO) are adapted, implemented and analysed as the candidate task-level scheduling algorithms. The hierarchical scheduling strategy is being implemented in our SwinDeW-C cloud workflow system and demonstrating satisfactory performance. Meanwhile, the experimental results show that the overall performance of ACO based scheduling algorithm is better than others on three basic measurements: the optimisation rate on makespan, the optimisation rate on cost and the CPU time.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Ardagna D, Pernici B (2007) Adaptive service composition in flexible processes. IEEE Trans Softw Eng 33(6):369–384

    Article  Google Scholar 

  2. Buyya R, Yeo CS, Venugopal S, Broberg J, Brandic I (2009) Cloud computing and emerging IT platforms: vision, hype, and reality for delivering computing as the 5th utility. Future Gener Comput Syst 25(6):599–616

    Article  Google Scholar 

  3. Cao H, Jin H, Wu X, Wu S (2009) ServiceFlow: QoS-based hybrid service-oriented grid workflow system. J Supercomput 53(3):371–393

    Article  Google Scholar 

  4. Cao H, Jin H, Wu X, Wu S, Shi X (2009) DAGMap: efficient and dependable scheduling of DAG workflow job in grid. J Supercomput 51(2):201–223

    Article  Google Scholar 

  5. Chang R, Chang J, Lin P (2009) An ant algorithm for balanced job scheduling in grids. Future Gener Comput Syst 25(1):20–27

    Article  Google Scholar 

  6. Chen J, Yang Y (2007) Adaptive selection of necessary and sufficient checkpoints for dynamic verification of temporal constraints in grid workflow systems. ACM Trans Auton Adaptive Syst 2(6)

  7. Chen J, Yang Y (2007) Multiple states based temporal consistency for dynamic verification of fixed-time constraints in grid workflow systems. Concurr Comput, Pract Exp 19(7):965–982

    Article  Google Scholar 

  8. Chen J, Yang Y (2010) Temporal dependency based checkpoint selection for dynamic verification of temporal constraints in scientific workflow systems. In: ACM trans on software engineering and methodology, in press. http://www.ict.swin.edu.au/personal/yyang/papers/ACM-TOSEM-checkpoint.pdf. Accessed on 1st August 2010

  9. Chen W, Zhang J, Chung HSH, Zhong W, Wu W, Shi Y (2009) A novel set-based particle swarm optimization method for discrete optimization problems. IEEE Trans Evol Comput 14(2):278–300

    Article  Google Scholar 

  10. Chen W, Zhang J (2009) An ant colony optimization approach to a grid workflow scheduling problem with various QoS requirements. IEEE Trans Syst Man Cybern, Part C, Appl Rev 39(1):29–43

    Article  Google Scholar 

  11. Chen W, Zhang J, Yu Y (2007) Workflow scheduling in grids: an ant colony optimization approach. In: Proc 2007 IEEE congress on evolutionary computation, pp 3308–3315

    Chapter  Google Scholar 

  12. Chin S, Suh T, Yu H (2009) Adaptive service scheduling for workflow applications in service-oriented grid. J Supercomput 52(3):253–283

    Article  Google Scholar 

  13. Deelman E, Gannon D, Shields M, Taylor I (2008) Workflows and e-science: an overview of workflow system features and capabilities. Future Gener Comput Syst 25(6):528–540

    Google Scholar 

  14. Dou WC, Chen JJ, Liu JX, Cheung SC, Chen GH, Fan SK (2008) A workflow engine-driven SOA-based cooperative computing paradigm in grid environments. Int J High Perform Comput Appl 22(3):284–300

    Article  Google Scholar 

  15. Thomas E (2008) SOA: principles of service design. Prentice Hall, New York

    Google Scholar 

  16. Foster I, Kesselman C (2004) The grid: blueprint for a new computing infrastructure, 2nd edn. Morgan Kaufmann, San Mateo

    Google Scholar 

  17. Foster I, Yong Z, Raicu I, Lu S (2008) Cloud computing and grid computing 360-degree compared. In: Proc 2008 grid computing environments workshop, pp 1–10

    Chapter  Google Scholar 

  18. Hadoop. http://hadoop.apache.org/. Accessed on 1st August 2010

  19. Han JW, Kamber M (2006) Data mining: concepts and techniques, 2nd edn. Elsevier, Amsterdam

    MATH  Google Scholar 

  20. Kao B, Molina HG (1997) Deadline assignment in a distributed soft real-time system. IEEE Trans Parallel Distrib Syst 8(12):1268–1274

    Article  Google Scholar 

  21. Kuo I, Horng S, Kao T, Lin T, Lee C, Terano T, Pan Y (2009) An efficient flow-shop scheduling algorithm based on a hybrid particle swarm optimization model. Expert Syst Appl 36(3):7027–7032, Part 2

    Article  Google Scholar 

  22. Kwok Y, Ahmad I (1999) Static scheduling algorithms for allocating directed task graphs to multiprocessors. ACM Comput Surv 31(4):406–471

    Article  Google Scholar 

  23. Law AM, Kelton WD (2007) Simulation modelling and analysis, 4th edn. McGraw-Hill, New York

    Google Scholar 

  24. Liu K, Chen J, Yang Y, Jin H (2008) A throughput maximization strategy for scheduling transaction-intensive workflows on SwinDeW-G. Concurr Comput, Pract Exp 20(15):1807–1820

    Article  Google Scholar 

  25. Liu X, Chen J, Wu Z, Ni Z, Yuan D, Yang Y (2010) Handling recoverable temporal violations in scientific workflow systems: a workflow rescheduling based strategy. In: Proc 10th IEEE/ACM international symposium on cluster, cloud and grid computing (CCGrid10), Melbourne, Australia, pp 534–537

    Chapter  Google Scholar 

  26. Liu X, Chen J, Yang Y (2008) A probabilistic strategy for setting temporal constraints in scientific workflows. In: Proc 6th international conference on business process management (BPM2008), Milan, Italy, vol 5204, pp 180–195

    Google Scholar 

  27. Liu X, Yang Y, Chen J, Wang Q, Li M (2009) Achieving on-time delivery: a two-stage probabilistic scheduling strategy for software projects. In: Proc the 2009 international conference on software process: trustworthy software development processes, Vancouver, BC, Canada. Springer, Berlin

    Google Scholar 

  28. Liu X, Yang Y, Jiang Y, Chen J (2010) Preventing temporal violations in scientific workflows: where and how. IEEE Trans Softw Eng. doi:10.1109/TSE.2010.99. http://www.ict.swin.edu.au/personal/xliu/papers/TSE-2010-01-0018.pdf. Accessed on 1st August 2010

    Google Scholar 

  29. Martinez A, Alfaro FJ, Sanchez JL, Quiles FJ, Duato J (2007) A new cost-effective technique for QoS support in clusters. IEEE Trans Parallel Distrib Syst 18(12):1714–1726

    Article  Google Scholar 

  30. Moore M (2004) An accurate parallel genetic algorithm to schedule tasks on a cluster. Parallel Comput 30(1):567–583

    Article  MathSciNet  Google Scholar 

  31. Mullen RJ, Monekosso D, Barman S, Remagnino P (2009) A review of ant algorithms. Expert Syst Appl 36(6):9608–9617

    Article  Google Scholar 

  32. Oh J, Wu C (2004) Genetic-algorithm-based real-time task scheduling with multiple goals. J Syst Softw 71:245–258

    Article  Google Scholar 

  33. Raghavan B, Ramabhadran S, Yocum K, Snoeren AC (2007) Cloud control with distributed rate limiting. In: Proc 2007 ACM SIGCOMM, Kyoto, Japan, pp 337–348

    Google Scholar 

  34. Buyya R, Bubendorfer K (2009) Market oriented grid and utility computing. Wiley, New York

    Book  Google Scholar 

  35. Russell N, van der Aalst WMP, ter Hofstede AHM (2006) Exception handling patterns in process-aware information systems. Technical Report BPM-06-04, BPMcenter.org

  36. Sahai A, Durante A, Machiraju V (2002) Towards automated SLA management for web services. Technical Report HPL-2001-310 (R.1), HP Laboratories Palo Alto

  37. SECES (2008) Proc first international workshop on software engineering for computational science and engineering, in conjunction with the 30th international conference on software engineering (ICSE2008), Leipzig, Germany, May 2008

  38. Taylor IJ, Deelman E, Gannon DB, Shields M (2007) Workflows for e-science: scientific workflows for grids. Springer, Berlin

    Google Scholar 

  39. Tracy DB, Howard JS, Noah B (2001) Comparison of eleven static heuristics for mapping a class of independent tasks onto heterogeneous distributed computing systems. J Parallel Distrib Comput 61(6):810–837

    Article  Google Scholar 

  40. van der Aalst WMP, Hee KMV (2002) Workflow management: models, methods, and systems. MIT Press, Cambridge

    Google Scholar 

  41. VMware. http://www.vmware.com/. Accessed on 1st August 2010

  42. Wang M, Kotagiri R, Chen J (2009) Trust-based robust scheduling and runtime adaptation of scientific workflow. Concurr Comput, Pract Exp 21(16):1982–1998

    Article  Google Scholar 

  43. Wieczorek M, Prodan R, Hoheisel A (2007) Taxonomies of the multi-criteria gridworkflow scheduling problem. CoreGRID Technical Report Number TR-0106, August 30

  44. Yan J, Yang Y, Raikundalia GK (2006) SwinDeW—a peer-to-peer based decentralized workflow management system. IEEE Trans Syst Man Cybern, Part A, Syst Hum 36(5):922–935

    Article  Google Scholar 

  45. Yang Y, Liu K, Chen J, Lignier J, Jin H (2007) Peer-to-peer based grid workflow runtime environment of SwinDeW-G. In: Proc 3rd international conference on e-science and grid computing (e-Science07), Bangalore, India, pp 51–58

    Google Scholar 

  46. Yang Y, Liu K, Chen J, Liu X, Yuan D, Jin H (2008) An algorithm in SwinDeW-C for scheduling transaction-intensive cost-constrained cloud workflows. In: Proc 4th IEEE international conference on e-science (e-Science08), Indianapolis, USA, pp 374–375

    Google Scholar 

  47. Yang Z, Koelbel C, Cooper K (2009) Hybrid Re-scheduling mechanisms for workflow applications on multi-cluster grid. In: Proc 9th IEEE/ACM international symposium on cluster computing and the grid, pp 116–123

    Google Scholar 

  48. Yu J, Buyya R (2005) A taxonomy of workflow management systems for grid computing. J Grid Comput 3:171–200

    Article  Google Scholar 

  49. Yu J, Buyya R (2006) Scheduling scientific workflow applications with deadline and budget constraints using genetic algorithms. Sci Program 14(3–4):217–230

    Google Scholar 

  50. Yu J, Buyya R (2008) Workflow scheduling algorithms for grid computing. In: Xhafa F, Abraham A (eds) Metaheuristics for scheduling in distributed computing environments. ISBN:978-3-540-69260-7. Springer, Berlin

    Google Scholar 

  51. Zhang L, Chen Y, Yang B (2006) Task scheduling based on PSO algorithm in computational grid. In: Proceedings—ISDA 2006: Sixth international conference on intelligent systems design and applications, Jinan, vol 2, pp 696–701

    Chapter  Google Scholar 

  52. Zhang XD, Li XP, Wang Q, Yuan YC (2008) Hybrid particle swarm optimization algorithm for cost minimization in service-workflows with due dates. Tongxin Xuebao/Commun 29(8)

Download references

Author information

Authors and Affiliations

  1. Institute of Intelligent Management, School of Management, Hefei University of Technology, Hefei, Anhui, 230009, China

    Zhangjun Wu & Zhiwei Ni

  2. Faculty of Information and Communication Technologies, Swinburne University of Technology, Hawthorn, Melbourne, 3122, Australia

    Zhangjun Wu, Xiao Liu, Dong Yuan & Yun Yang

Authors
  1. Zhangjun Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiao Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhiwei Ni
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Dong Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yun Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiao Liu.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Wu, Z., Liu, X., Ni, Z. et al. A market-oriented hierarchical scheduling strategy in cloud workflow systems. J Supercomput 63, 256–293 (2013). https://doi.org/10.1007/s11227-011-0578-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-011-0578-4

Keywords

Search

Navigation