Project 01-05

SYSTEST - Project Status

Summary

SYSTEST is a three-year project that started in March 2002. The project successfully completed its second of three years in March 2004. The project is on schedule and has been successful in developing usable methods and tools for system verification, validation, and testing (VVT). The primary products of SYSTEST are:

VVT Methodology Guidelines (VVTMG) - a 400+ page document containing general and specific guidelines for VVT.
VVT Process Model (VVTPM) - a software set that allows evaluating a VVT strategy in terms of its impact on cost, schedule, and risk.
Dissemination papers - a series of professional papers and presentations describing the SYSTEST results.

The VVTMG and VVTPM will be released for public use following the completion of the project. They are currently being evaluated in pilot projects as part of Work Package 6 (WP6) within each of the commercial partners. The figure below shows the project plan and status. All project tasks are on schedule.

Milestones and Accomplishments

Major milestones and accomplishments are:

Date Due	Ident	Task or Deliverable	Status
Mar 02	M0.1	Project Kickoff	Completed
Aug 02	D1.1	Report on VVT state of the art	Completed
Aug 02	D1.2	SYSTEST Project Technical Requirements	Completed
Mar 03	M4.1	Evaluation of COTS tools	Completed
Mar 03	D2.1	Methodology Assessment Criteria	Completed
Mar 03	D2.2a	VVT Methodology Guidelines (First Draft)	Completed
Mar 03	M5.1	VVTMG Assimilation Kit (for pilot projects)	Completed
May 03	D3.1	VVT Process Model	Completed
Aug 03	M6.2	Start use of VVTMG and VVTPM on pilot projects	Completed
Sep 03	M4.2	Integrated platform for VVT Process Model	Completed
Sep 03	M6.1	First Internal SYSTEST Users Group Meeting	Completed
Mar 04	M5.2	Assessment/Evaluation of the Assimilation Process (on pilot projects)	Completed
Jan 04	D2.2b	VVT Methodology Guidelines (Second Draft)	Completed
Jun 04	M8.3	European SYSTEST Users Group Meeting (at INCOSE symposium, Toulouse)	Completed
Dec 04	M6.2	Complete pilot projects evaluation
Mar 05	D2.2c	VVT Methodology Guidelines (final)
Mar 05	D3.1a	VVT Process Model (final)

Papers and Presentations

More technical information on SYSTEST and its results can be found in the following extensive list of papers and presentations that have resulted from the project:

SYSTEST Project was presented at the 3rd European Systems Engineering Conference in Toulouse, France on 22-24, May 2002. Same presentation has been given at the INCOSE 2002, the 12th International Symposium in Las Vegas, Nevada, USA, on 28 Jul - 1 Aug. 2002.
VOLLERTHUN A., ENGEL A., HASKINS C. “Methodology Development for Systems Verification, Validation & Testing”, Conference paper presented at the 12th Annual International Symposium of INCOSE, Las Vegas, USA, 2002.
ENGEL A. "A Methodology for Modeling VVT Risk & Cost”. Conference paper presented at the 13th Annual International Symposium of INCOSE in Washington, DC, USA. June 29 - July 3, 2003.
HOPPE M., LÉVÁRDY V., VOLLERTHUN A., WENZEL S. "Interfacing the Verification, Validation, and Testing Process Model with Product Development Methods". Conference paper presented at the 13th Annual International Symposium of INCOSE in Washington, USA. July 1-3, 2003.
BUEHLER O., WEGENER J. “Evolutionary Functional Testing of an Automated Parking System”. International Conference on Computer, Communication and Control Technologies: CCCT '03 and The 9th. International Conference on Information Systems Analysis and Synthesis: ISAS '03, Orlando, Florida, USA. July 31- August 2, 2003.
HOPPE M., LÉVÁRDY V., IGENBERGS, E., "Das VVT Process Model für die Verifikations- und Validierungsplanung". Paper published in the Conference proceedings of the 15th Annual Spring Academy of the TU Munich and TU Budapest Balatonfüred, Hungary. May 17-21, 2003.
BARESEL A., POHLHEIM H., SADEGHIPOUR S., “Structural and Functional Sequence Test of Dynamic and State-Based Software with Evolutionary Algorithms”. Proceedings of the Genetic and Evolutionary Computation Conference - GECCO 2003, Chicago, Illinois, USA; pp. 2428 - 2441. July, 2003.
BARESEL A., STHAMER H. “Evolutionary Testing of Flag Conditions”. Proceedings of the Genetic and Evolutionary Computation Conference - GECCO 2003, Chicago, Illinois, USA; pp. 2442 - 2454. July, 2003.
ENGEL A., BARAD M. "A Methodology for Modeling VVT Risks and Costs”. Systems Engineering, Volume 6, issue 3, pp. 135 – 151, Wiley InterScience, Online ISSN: 1520-6858, Print ISSN: 1098-1241. July, 2003.
LÉVÁRDY V., HOPPE M., WENZEL S., VOLLERTHUN A. “Process Modeling Procedure for Verification, Validation and Testing Planning”. Conference paper was presented at the ASME 2003 Design Engineering Technical Conferences and Computers and Information in Engineering Conference (DETC2003/DTM-48682) Chicago. September 2-6, 2003.
LÉVÁRDY V., HOPPE M., “Systemtechnische Methodik zur Verifikations- und Validierungsplanung - Zwischenergebnisse des EU-Projekts SysTest”. Presented at the 3rd German Day of Systems Engineering, Munich, Germany. November 28, 2003.
BARESEL A., CONRAD M., SADEGHIPOUR S., WEGENER J. “The Interplay between Model Coverage and Code Coverage”. Paper to appear in the proceedings of the International European Conference on Software Testing, Analysis and Review. EuroSTAR 03.
SYSTEST project was presented by Avner Engel from IAI at the 4th Israeli System Engineering Conference in Haifa, Israel on 7 January 2004, as part of “The Modeling Systems’ VVT Risks and Costs” presentation.
BUEHLER, J., WEGENER, J. “Automatic Testing of an Autonomous Parking System using Evolutionary Computation”. SAE World Congress, Detroit, USA, March 8 – 11, 2004.
WEGENER, J.: “Test Case design by means of the Classification-Tree Method and the Classification-Tree Editor”. Proceedings of Test and Analysis of Component-based Systems (TACoS 2004), Barcelona, Spain, March 27 – April 04, 2004.
LÉVÁRDY V., HOPPE M. “Verification, Validation and Testing Strategy Planning Supported by a Process Model”. Conference paper published in the proceedings of the Tools and Methods for Competitive Engineering (TMCE) 2004 conference, Lausanne, Switzerland. April 12-16, 2004.
ENGEL A., BOGOMOLNI I., SHACHAR S., GRINMAN A. “Using historical cost and risk data to optimize the VVT process” (The 14th Annual International Symposium of INCOSE, Toulouse, France. June 20-24, 2004).
HOPPE M., LÉVÁRDY V., LEARDI C., MENDIKOA I., DE ABAJO N., GONZALEZ J., LOBATO V., PEREGRINA S. ”Application Experiences of the VVT Process Modeling Procedure at the Verification and Validation Planning”. Conference published in the proceedings of the 14th Annual International Symposium of INCOSE, Toulouse, France. June 20-24, 2004.
HOPPE M., LÉVÁRDY V., HASKINS C., BOGOMOLNI I., IGENBERGS E. “Generic Verification, Validation and Testing Methodology”. Tutorial presented at the 14th Annual International Symposium of INCOSE in Toulouse, France. June 20-24, 2004.
LÉVÁRDY V., HOPPE M., HONOUR E. ”Verification, Validation & Testing Strategy and Planning Procedure”. Conference paper published in the proceedings of the 14th Annual International Symposium of INCOSE, Toulouse, France. June 20-24, 2004.
LAMMERMANN F., BARESEL A., WEGENER J. “Evaluating Evolutionary Testability with Software Measurements”. Proceedings of the Genetic and Evolutionary Computation Conference, Part 2 - GECCO 2004, Seattle, Washington, USA, June 26-30, 2004.
WEGENER J., BÜHLER O. “Evaluation of Different Fitness Functions for the Evolutionary Testing of an Autonomous Parking System”. Proceedings of the Genetic and Evolutionary Computation Conference, Part 2 - GECCO 2004, Seattle, Washington, USA, June 26-30, 2004.
BARESEL A., STHAMER H., WEGENER J. “Applying Evolutionary Testing to Search for Critical Defects“. Proceedings of the Genetic and Evolutionary Computation Conference, Part 2 - GECCO 2004, Seattle, Washington, USA, June 26-30, 2004.

Page last modified 25 Sep 04