Abstract
Most distributed system specifications have performance benchmark requirements, for example the number of particular kinds of transactions per second required to be supported by the system. However, determining the likely eventual performance of complex distributed system architectures during their development is very challenging. We describe SoftArch/MTE, a software tool that allows software architects to sketch an outline of their proposed system architecture at a high level of abstraction. These descriptions include client requests, servers, server objects and object services, database servers and tables, and particular choices of middleware and database technologies. A fully-working implementation of this system is then automatically generated from this high-level architectural description. This implementation is deployed on multiple client and server machines and performance tests are then automatically run for this generated code. Performance test results are recorded, sent back to the SoftArch/MTE environment and are then displayed to the architect using graphs or by annotating the original high-level architectural diagrams. Architects may change performance parameters and architecture characteristics, comparing multiple test run results to determine the most suitable abstractions to refine to detailed designs for actual system implementation. Further tests may be run on refined architecture descriptions at any stage during system development. We demonstrate the utility of our approach and prototype tool, and the accuracy of our generated performance test-beds, for validating architectural choices during early system development.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Balsamo, S., Simeoni, M., and Bernado, M. 2002. Combining stochastic process algebras and queueing networks for software architecture analysis. In Proceedings of the 3rd International Workshop on Software and Performance, Rome, Italy, ACM Press.
Bass, L., Clements, P., and Kazman, R. 1998. Software Architecture in Practice. Addison-Wesley.
Beaumont, M. and Jackson, D. 1998. Visualising complex control flow. In 1998 IEEE Symposium on Visual Languages, Halifax, Canada. IEEE.
Chen, M., Tang, M., and Wang, W. Software architecture analysis—A case study. In Proceedings of COMPSAC'99.
CSIRO, Middleware Technology Evaluation project, www.cmis.csiro.au/mte.
Dai, X. and Grundy, J.C. Architecture for a component-based, plug-in micro-payment system. In Proceedings of 5th Asia-Pacific Web Conference, September, Xi'an, China, Springer LNCS.
ECPerf Performance Benchmarks, August 2002. ecperf.theserverside.com/ecperf.
Egyed, A. and Kruchten, P. 1999. Rose/Architect: A tool to visualize architecture, In Hawaii International Conference on System Sciences. IEEE.
Gomaa, H., Menascé, D., and Kerschberg, L. 1996. A software architectural design method for large-scale distributed information systems. Distributed Systems Engineering Journal, IEE/BCS.
Gorton, I. and Liu, A. 2000. Evaluating enterprise java bean technology. In Proceedings of Software—Methods and Tools, Wollongong, Australia, No. 6–9 IEEE.
Grundy, J.C. and Hosking, J.G. 1995. ViTABaL: A visual language supporting design by tool abstraction. In Proceedings of the 1995 IEEE Symposium on Visual Languages. Darmsdart, Germany, IEEE CS Press, pp. 53–60.
Grundy, J.C. and Hosking, J.G. 2000. High-level static and dynamic visualisation of software architectures, In 2000 IEEE Symposium on Visual Languages, IEEE.
Grundy, J.C. and Liu, A. 2000. Directions in engineering non-functional requirement compliant middleware applications. In Proceedings of the 3rd Australasian Workshop on Software and Systems Architectures, Sydney, Australia, Monash University.
Grundy, J.C., Mugridge, W.B., and Hosking, J.G. 2000. Constructing component-based software engineering environments: Issues and experiences. Information and Software Technology: Special Issue on Constructing Software Engineering Tools, 42(2):117–128.
Grundy, J.C., Cai, Y., and Liu, A. 2001. Generation of distributed system test-beds from high-level software architecture descriptions. In Proceedings of the 2001 IEEE International Conference on Automated Software Engineering, San Diego, CA.
Hill, T. and Noble, J. 1999. Visualizing implicit structure in java object graphs. In Proceedings of SoftVis'99, Sydney, Australia.
Hu, L. and Gorton, I. 1997. Aperformance prototyping approach to designing concurrent software architectures. In Proceedings of the 2nd International Workshop on Software Engineering for Parallel and Distributed Systems, IEEE, pp. 270–276.
Juiz, C. and Puigjaner, R. 2002. Performance modelling of pools in soft real-time design architectures. Simulation Practice & Theory, 9(3–5):215–240, Elsevier.
Jurie, M.R., Rozman, I., and Nash, S. 2000. Java 2 distributed object middleware performance analysis and optimization. SIGPLAN Notices, 35(8):31–40, ACM.
Kazman, R. 1996. Tool support for architecture analysis and design. In Proceedings of the Second International Workshop on Software Architectures, ACM Press, pp. 94–97.
Khare, R., Guntersdorfer, M., Oreizy, P., Medvivovic, N., and Taylor, R.N. 2001. xADL: Enabling architecture-centric tool integration with XML. In Proceedings of the 34th Hawaii International Conference on System Sciences, Maui, Hawaii, IEEE.
Lee, S.C. and Offutt, J. 2001. Generating test cases for XML-based Web component interactions using mutation analysis. In Proceedings of the 12th International Symposium on Software Reliability Engineering, Hong Kong, China, IEEE CS Press.
Leo, J. 1999. OO enterprise architecture approach using UML. In Proceedings of the 2nd Australasian Workshop on Software Architectures, Melbourne, Monash University Press, pp. 25–40.
Liu, A. 2000. Dynamic distributed software architecture design with PARSE-DAT. In Proceedings of Software—Methods and Tools, Wollongong, Australia, IEEE.
Luckham, D.C., Augustin, L.M., Kenney, J.J., Veera, J., Bryan, D., and Mann, W. 1995. Specification and analysis of system architecture using Rapide. IEEE Transactions on Software Engineering, 21(4):336–355.
Ma, Y.-S., Oh, S.-U., Bae, D.-H., and Kwon, K.-R. 2001. Framework for third party testing of component software. In Proceedings of the Eighth Asia-Pacific Software Engineering Conference, IEEE CS Press, pp. 431–434.
McCann, J.A. and Manning, K.J. 1998. Tool to evaluate performance in distributed heterogeneous processing. In Proceedings of the Sixth Euromicro Workshop on Parallel and Distributed Processing, IEEE, pp. 180–185.
Microsoft Corp. 2002. Using.NET to implement Sun Microsystem's Java Pet Store J2EE BluePrint application. http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnbda/html/psimp.asp.
Nimmagadda, S., Liyanaarachchi, C., Gopinath, A., Niehaus, D., and Kaushal, A. 1999. Performance patterns: Automated scenario based ORB performance evaluation. In Proceedings of the Fifth USENIX Conference on Object-Oriented Technologies and Systems, USENIX, pp. 15–28.
Petriu, D., Amer, H., Majumdar, S., and Abdull-Fatah, I. 2000. Using analytic models for predicting middleware performance. In Proceedings of the Second International Workshop on Software and Performance, ACM, pp. 189–94.
Robbins, J., Hilbert, D.M., and Redmiles, D.F. 1998. Extending design environments to software architecture design. Automated Software Engineering, 5(3):261–390.
Shannon, B. 2000. Java 2 Platform, Enterprise Edition: Platform and Component Specifications. Addison-Wesley.
Shaw, M. and Garlan, D. 1996. Software Architecture. Prentice Hall.
Topol, B., Stasko, J., and Sunderam, V. 1998. PVaniM: Atool for visualization in network computing environments. Concurrency: Practice & Experience, 10(14):1197–1222.
Vogal, A. 1998. CORBA and enterprise Java beans-based electronic commerce. In International Workshop on Component-Based Electronic Commerce, Fisher Center for Management & IT, UC Berkeley.
Woodside, C. 2001. Software resource architecture and performance evaluation of software architectures. In Proceedings of the 34th Hawaii International Conference on System Sciences, IEEE, Maui, HA.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Grundy, J., Cai, Y. & Liu, A. SoftArch/MTE: Generating Distributed System Test-Beds from High-Level Software Architecture Descriptions. Automated Software Engineering 12, 5–39 (2005). https://doi.org/10.1023/B:AUSE.0000049207.62380.74
Issue Date:
DOI: https://doi.org/10.1023/B:AUSE.0000049207.62380.74