Skip to main content
SpringerLink
Log in
Book cover

International Workshop on Process-Aware Systems

PAS 2014: Process-Aware Systems pp 59–74Cite as

Temporal Verification for Scientific Cloud Workflows: State-of-the-Art and Research Challenges

  • Conference paper

  • 392 Accesses

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 495))

Abstract

Cloud computing is establishing itself as the latest computing paradigm in recent years. As doing science in the cloud is becoming a reality, scientists are now able to access public cloud centers and employ high-performance computing resources to run scientific applications. However, due to the dynamic nature of the cloud environment, the usability of scientific cloud workflow systems can be significantly deteriorated if without effective service quality assurance strategies. Specifically, workflow temporal verification as the major approach for workflow temporal QoS (Quality of Service) assurance plays a critical role in the on-time completion of large-scale scientific workflows. Great efforts have been dedicated to the area of workflow temporal verification in recent years and it is high time that we should define the key research issues for scientific cloud workflows in order to keep our research on the right track. In this paper, we systematically investigate this problem and present four key research issues based on the introduction of a generic temporal verification framework. Meanwhile, state-of-the-art solutions for each research issue and open challenges are also presented. Finally, SwinDeW-V, an ongoing research project on temporal verification as part of our SwinDeW-C cloud workflow system, is also demonstrated.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Amazon EC2, http://aws.amazon.com/ec2/ (accessed on September 1, 2014)

  2. Akioka, S., Muraoka, Y.: Extended Forecast of CPU and Network Load on Computational Grid. In: Proc. 2004 IEEE International Symposium on Cluster Computing and the Grid, pp. 765–772 (2004)

    Google Scholar 

  3. Barga, R., Gannon, D.: Scientific versus Business Workflows. In: Workflows for e-Science (2007)

    Google Scholar 

  4. Buyya, R., Yeo, C.S., Venugopal, S.: Market-Oriented Cloud Computing: Vision, Hype, and Reality for Delivering IT Services as Computing Utilities. In: Proc. 10th IEEE International Conference on High Performance Computing and Communications (2008)

    Google Scholar 

  5. Chen, J., Yang, Y.: Multiple States based Temporal Consistency for Dynamic Verification of Fixed-time Constraints in Grid Workflow Systems. Concurrency and Computation: Practice and Experience 19(7), 965–982 (2007)

    Article  Google Scholar 

  6. Chen, J., Yang, Y.: Adaptive Selection of Necessary and Sufficient Checkpoints for Dynamic Verification of Temporal Constraints in Grid Workflow Systems. ACM Trans. on Auto. and Adapt. Sys. 2(2) (2007)

    Google Scholar 

  7. Chen, J., Yang, Y.: Temporal Dependency based Checkpoint Selection for Dynamic Verification of Temporal Constraints in Scientific Workflow Systems. ACM Transactions on Software Engineering and Methodology 20(3), Article 9 (2011)

    Google Scholar 

  8. Chen, J., Yang, Y.: Localising Temporal Constraints in Scientific Workflows. Journal of Computer and System Sciences 76(6), 464–474 (2010)

    Article  MathSciNet  Google Scholar 

  9. Chen, J., Yang, Y., Chen, T.Y.: Dynamic Verification of Temporal Constraints on-the-fly for Workflow Systems. In: Proc. the 11th Asia-Pacific Software Engineering Conference, pp. 30–37 (2004)

    Google Scholar 

  10. Chen, J., Yang, Y.: Activity Completion Duration Based Checkpoint Selection for Dynamic Verification of Temporal Constraints in Grid Workflow Systems. Int. J. High Perform. Comput. Appl. 22(3), 319–329 (2008)

    Article  Google Scholar 

  11. Chen, W., Zhang, J., Yu, Y.: Workflow Scheduling in Grids: An Ant Colony Optimization Approach. In: Proc. 2007 IEEE Congress on Evolutionary Computation, pp. 3308–3315 (2007)

    Google Scholar 

  12. Chen, W., Zhang, J.: An Ant Colony Optimization Approach to a Grid Workflow Scheduling Problem With Various QoS Requirements. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews 39(1), 29–43 (2009)

    Article  Google Scholar 

  13. Cooper, K., Dasgupta, A., Kennedy, K., Koelbel, C., Mandal, A.: New Grid Scheduling and Rescheduling Methods in the GrADS Project. In: Proc. 18th International Parallel and Distributed Processing Symposium, pp. 199–206 (2004)

    Google Scholar 

  14. Dean, J., Ghemawat, S.: Mapreduce: Simplified Data Processing on Large Clusters. Communications of the ACM 51(1), 107–113 (2008)

    Article  Google Scholar 

  15. Deelman, E., Gannon, D., Shields, M., Taylor, I.: Workflows and e-Science: An Overview of Workflow System Features and Capabilities. Fut. Gene. Comp. Syst. 25(5), 528–540 (2009)

    Article  Google Scholar 

  16. Dou, W., Zhao, J., Fan, S.: A Collaborative Scheduling Approach for Service-Driven Scientific Workflow Execution. Journal of Computer and System Sciences 76(6), 416–427 (2010)

    Article  MATH  MathSciNet  Google Scholar 

  17. Eder, J., Panagos, E., Rabinovich, M.: Time Constraints in Workflow Systems. In: Jarke, M., Oberweis, A. (eds.) CAiSE 1999. LNCS, vol. 1626, pp. 286–300. Springer, Heidelberg (1999)

    Chapter  Google Scholar 

  18. Foster, I., Yong, Z., Raicu, I., Lu, S.: Cloud Computing and Grid Computing 360-Degree Compared. In: Proc. 2008 Grid Computing Environments Workshop, pp. 1–10 (2008)

    Google Scholar 

  19. Hamzeh, K.: Performance Analysis of Cloud Computing Centers Using M/G/m/m+r Queuing Systems. IEEE Transactions on Parallel and Distributed Systems 23(5), 936–943 (2012)

    Article  Google Scholar 

  20. Cloud Harmony, http://cloudharmony.com/ (accessed on September 1, 2014)

  21. Kao, B., Garcia-Molina, H.: Deadline Assignment in a Distributed Soft Real-Time System. IEEE Trans. Parallel Distrib. Syst. 8(12), 1268–1274 (1997)

    Article  Google Scholar 

  22. Liu, X., Chen, J., Yang, Y.: A Probabilistic Strategy for Setting Temporal Constraints in Scientific Workflows. In: Dumas, M., Reichert, M., Shan, M.-C. (eds.) BPM 2008. LNCS, vol. 5240, pp. 180–195. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  23. Liu, X., Chen, J., Wu, Z., Ni, Z., Yuan, D., Yang, Y.: Handling Recoverable Temporal Violations in Scientific Workflow Systems: A Workflow Rescheduling Based Strategy. In: Proc. 10th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, pp. 534–537 (2010)

    Google Scholar 

  24. Liu, X., Ni, Z., Chen, J., Yang, Y.: A Probabilistic Strategy for Temporal Constraint Management in Scientific Workflow Systems. Concurrency and Computation: Practice and Experience 23(16), 1893–1919 (2011)

    Article  Google Scholar 

  25. Liu, X., Yang, Y., Yuan, D., Zhang, G., Li, W., Cao, D.: A Generic QoS Framework for Cloud Workflow Systems. In: Proc. International Conference on Cloud and Green Computing, pp. 713–720 (2011)

    Google Scholar 

  26. Liu, X., Chen, J., Yang, Y.: Temporal QoS Management in Scientific Cloud Workflow Systems. Elsevier (2012)

    Google Scholar 

  27. Liu, X., Yuan, D., Zhang, G., Li, W., Cao, D., He, Q., Chen, J., Yang, Y.: The Design of Cloud Workflow Systems. Springer (2012)

    Google Scholar 

  28. Liu, X., Yang, Y., Yuan, D., Chen, J.: Do We Need to Handle Every Temporal Violation in Scientific Workflow Systems? ACM Trans. on Soft. Eng. and Method. 23(1), Article 5 (2014)

    Google Scholar 

  29. Liu, X., Yang, Y., Cao, D., Yuan, D.: Selecting Checkpoints along the Time Line: A Novel Temporal Checkpoint Selection Strategy for Monitoring a Batch of Parallel Business Processes. In: Proc. 35th International Conference on Software Engineering (NIER Track), pp. 1281–1284 (2013)

    Google Scholar 

  30. Liu, X., Ni, Z., Wu, Z., Yuan, D., Chen, J., Yang, Y.: An Effective Framework of Light-Weight Handling for Three-Level Fine-Grained Recoverable Temporal Violations in Scientific Workflows. In: Proc. 16th IEEE International Conference on Parallel and Distributed Systems, pp. 43–50 (2010)

    Google Scholar 

  31. Liu, X., Ni, Z., Wu, Z., Yuan, D., Chen, J., Yang, Y.: A Novel General Framework for Automatic and Cost-Effective Handling of Recoverable Temporal Violations in Scientific Workflow Systems. Journal of Systems and Software 84(3), 492–509 (2011)

    Article  Google Scholar 

  32. Liu, X., Ni, Z., Yuan, D., Jiang, Y., Wu, Z., Chen, J., Yang, Y.: A Novel Statistical Time-Series Pattern based Interval Forecasting Strategy for Activity Durations in Workflow Systems. Journal of Systems and Software 84(3), 354–376 (2011)

    Article  Google Scholar 

  33. Liu, X., Yang, Y., Jiang, Y., Chen, J.: Preventing Temporal Violations in Scientific Workflows: Where and How. IEEE Transactions on Software Engineering 37(6), 805–825 (2011)

    Article  Google Scholar 

  34. Liu, X., Yuan, D., Zhang, G., Chen, J., Yang, Y.: SwinDeW-C: A Peer-to-Peer Based Cloud Workflow System. In: Furht, B., Escalante, A. (eds.) Handbook of Cloud Computing. Springer (2010)

    Google Scholar 

  35. Marjanovic, O., Orlowska, M.E.: On Modelling and Verification of Temporal Constraints in Production Workflows. Knowledge and Information Systems 1(2), 157–192 (1999)

    Article  Google Scholar 

  36. Matsunaga, A., Tsugawa, M., Fortes, J.: CloudBLAST: Combining MapReduce and Virtualization on Distributed Resources for Bioinformatics Applications. In: Proc. 4th IEEE International Conference on e-Science, pp. 222–229 (2008)

    Google Scholar 

  37. Windows Azure, http://www.microsoft.com/windowsazure/ (accessed on September 1, 2014)

  38. Ming, M., Humphrey, M.: A Performance Study on the VM Startup Time in the Cloud. In: Proc. 5th IEEE International Conference on Cloud Computing, pp. 423–430 (2012)

    Google Scholar 

  39. Moldovan, D., Copil, G., Hong-Linh, T., Dustdar, S.: MELA: Monitoring and Analyzing Elasticity of Cloud Services. In: Proc. 5th IEEE International Conference on Cloud Computing Technology and Science, vol. 1, pp. 80–87 (2013)

    Google Scholar 

  40. Netto, M.A.S., Bubendorfer, K., Buyya, R.: SLA-Based Advance Reservations with Flexible and Adaptive Time QoS Parameters. In: Krämer, B.J., Lin, K.-J., Narasimhan, P. (eds.) ICSOC 2007. LNCS, vol. 4749, pp. 119–131. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  41. Hoffa, C., Mehta, G., Freeman, T., Deelman, E., Keahey, K., Berriman, B., Good, J.: On the Use of Cloud Computing for Scientific Workflows. In: Proc. 4th IEEE International Conference on e-Science, pp. 640–645 (2008)

    Google Scholar 

  42. Prodan, R., Fahringer, T.: Overhead Analysis of Scientific Workflows in Grid Environments. IEEE Transactions on Parallel and Distributed Systems 19(3), 378–393 (2008)

    Article  Google Scholar 

  43. Askalon Project, http://www.dps.uibk.ac.at/projects/askalon (accessed on September 1, 2014)

  44. CloudBus Project, http://www.cloudbus.org/ (accessed on September 1, 2014)

  45. GridAnt Project, http://www.globus.org/cog/projects/gridant/ (accessed on September 1, 2014)

  46. GridBus Project, http://www.gridbus.org (accessed on September 1, 2014)

  47. Kepler Project, http://kepler-project.org/ (accessed on September 1, 2014)

  48. Pegasus Project, http://pegasus.isi.edu/ (accessed on September 1, 2014)

  49. Taverna Project, http://www.mygrid.org.uk/tools/taverna/ (accessed on September 1, 2014)

  50. Triana Project, http://www.trianacode.org/ (accessed on September 1, 2014)

  51. UNICORE Project, http://www.unicore.eu/ (accessed on September 1, 2014)

  52. Russell, N., van der Aalst, W.M.P., ter Hofstede, A.H.M.: Exception Handling Patterns in Process-Aware Information Systems. Technical Report BPM-06-04, BPMcen-ter.org (2006)

    Google Scholar 

  53. Russell, N., van der Aalst, W.M.P., ter Hofstede, A.H.M.: Workflow Exception Patterns. In: Martinez, F.H., Pohl, K. (eds.) CAiSE 2006. LNCS, vol. 4001, pp. 288–302. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  54. Scientific Workflows Survey, http://www.extreme.indiana.edu/swf-survey/ (accessed on September 1, 2014)

  55. Taylor, I.J., Deelman, E., Gannon, D.B., Shields, M.: Workflows for e-Science: Scientific Workflows for Grids (2007)

    Google Scholar 

  56. VMware, http://www.vmware.com/ (accessed on September 1, 2014)

  57. Workflow Coalition Management, The Workflow Reference Model. Technical Report WFMC-TC-1003 (1995)

    Google Scholar 

  58. Liu, X., Chen, J., Liu, K., Yang, Y.: Forecasting Duration Intervals of Scientific Workflow Activities Based on Time-Series Patterns. In: Proc. 2008 IEEE Fourth International Conference on eScience, pp. 23–30 (2008)

    Google Scholar 

  59. Yu, J., Buyya, R.: A Taxonomy of Workflow Management Systems for Grid Computing. Journal of Grid Computing (3), 171–200 (2005)

    Google Scholar 

  60. Yu, J., Buyya, R.: Workflow Scheduling Algorithms for Grid Computing. Technical Report GRIDS-TR-2007-10, The University of Melbourne, Australia (2007)

    Google Scholar 

  61. Yu, Z., Shi, W.: An Adaptive Rescheduling Strategy for Grid Workflow Applications. In: Proc. 2007 IEEE International Symposium on Parallel and Distributed Processing (IPDPS 2007), pp. 115–122 (2007)

    Google Scholar 

  62. Yuan, D., Yang, Y., Liu, X., Chen, J.: A Data Placement Strategy in Scientific Cloud Workflows. Future Generation Computer Systems 26(6), 1200–1214 (2010)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Shanghai Key Laboratory of Trustworthy Computing, East China Normal University, China

    Qiudan Wang, Xiao Liu & Zhou Zhao

  2. School of Computer Science and Technology, Anhui University, China

    Futian Wang

Authors
  1. Qiudan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiao Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhou Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Futian Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Computer Science and Engineering, Shanghai Jiao Tong University, China

    Jian Cao

  2. School of Software, Tsinghua University, 100084, Beijing, P.R. China

    Lijie Wen

  3. East China Normal University, 200062, Shanghai, China

    Xiao Liu

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Wang, Q., Liu, X., Zhao, Z., Wang, F. (2015). Temporal Verification for Scientific Cloud Workflows: State-of-the-Art and Research Challenges. In: Cao, J., Wen, L., Liu, X. (eds) Process-Aware Systems. PAS 2014. Communications in Computer and Information Science, vol 495. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-46170-9_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-662-46170-9_6

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-662-46169-3

  • Online ISBN: 978-3-662-46170-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation