Guidelines for Cross-Vendor DICOM TestingTitle: Application Entities Interconnectivity Level 3 Test Plan Example Master Test Plan Document ID: 1.2.840.113702.1.3.1.2 Version Date: 1/20/96
Application Entities Interconnectivity Level_3 Test Plan Example Master Test Plan
The purpose of this Test Plan document is to prescribe the scope, approach, resources, and schedule of the testing activities for interconnectivity testing between the Picker PQ200 CT scanner and any other DICOM conformant device that is characterized by a complementary conformance statement. The goal of the Test Plan is to identify the items being tested, the features to be tested, the testing tasks to be performed, the personnel responsible for each task, and the risks associated with the test plan.
This document is the Level_3 Test Plan example document for the combined Association Services and CT Storage Service Class. Association Services and Storage are operationally inseperable for the Picker CT SCU and thererfore the Test Plan, Test Design and Test Case documents have been written to reflect this operational reality. Interconnectivity testing of the Picker CT does not require multiple separable Test Plans, thererfore this Master Test Plan will suffice.
A test plan document shall have the following structure:
1. Test-plan identifier
3. Test items
4. Features to be tested
5. Features not to be tested
7. Item pass/fail criteria
8. Suspension criteria and resumption requirements
9. Test deliverables
10. Testing tasks
11. Environmental needs
13. Staffing and training needs
The Test Plan Identifier shall be a unique as specified in the DICOM Guidelines for Cross-Vendor Testing. This example Master Test Plan has the ID: 1.2.840.113702.1.3.1.2
2.1 Objectives. This test plan for testing the interconnectivity between two Application Entities supports the following objectives.
(1) To detail the activities required to prepare for and support the test.
(2) To communicate to all responsible parties the tasks which they are to perform and the schedule to be followed in performing the tasks.
(3) To define the sources of information used to prepare the test plan.
(4) To define the test tools and environment needed to conduct the test.
2.2 Background. This Master Test Plan was created to provide a concrete example of a mapping from the DICOM Interconnectivity template documents (Levels 1 and 2) to an executable Test Suite. Two DICOM conforming medical imaging systems currently in operation at the Milton S. Hershey Medical Center, Penn State College of Medicine were chosen as the target systems. Such a concrete mapping is essential to assure completeness of the testing Guidelines document set, and to assist users of the Guidelines to define appropriate test document sets for their own application environment.
2.3 Scope. A test for interconnectivity between two Application Entities has the objective to show that the Application Entities are able to exchange messages in accordance with the DICOM Standard, ACR/NEMA Standards Publication No. PS 3. This Master Test Plan presents a schema for organizing the testing of two Application Entities for interconnectivity. The plan partitions the testing process in exactly the same way that the DICOM Standard organizes the Service Classes and the Association Services. A complete testing cycle has been developed for the Association Services and the CT Image Storage SOP Class, with the accompanying set of documents required to record the design and execution of the testing process. The Association Services are distinguished for testing because they are required to be operational before any of the Services Classes can be tested.
This Master Test Plan and any other testing documents at this level serve only to present the schema for organizing testing of the CT Storage Service Class and the Association Services. The testing documentation at this level does not attempt to present detailed testing information. That degree of detail is left to the testing documentation associated with the Service Class and the Association Services. The test plan for each category includes functionality testing of the Application Entities, the operator interfaces for invoking message transfer, and indications of successful and failed operations.
NEMA MEDPACS Contract with Information and Control Technology Inc. - To Develop Guidelines for Cross-Vendor DICOM Testing, August 2, 1995.
Digital Imaging and Communications in Medicine (DICOM), ACR/NEMA Standards Publications Nos. PS 3.2- 1992 to PS 3.8-1992 (inclusive)
IEEE Standard for Software Test Documentation, ANSI/IEEE Std. 829-1983.
Application Specific Documentation:
DICOM Conformance Statement - Picker PQ 2000, PQ CT, and PQS CT Scanners, Rev 1.2, Picker International, November, 1994.
Computed Tomography Networking Operator's Guide, Part No. T55B-1109, Rev. -2 Picker International, 1994.
The item to be tested for interconnectivity will be the HANI Application Entity as implemented in Software Revisions 4.2 and 4.3A of the Picker PQ 2000 CT Scanner. As specified by the DICOM Standard, the Upper Layer Association Services are included in the Application Entities and are therefore included in the testing process.
The following features will be tested. They represent the only services which are available on the PQ 2000 CT scanner at the specified software revision level:
CT Image Information Object Storage SOP Class (UID: 1.2.840.10008.5.1.4.1.1.2).
The HANI AE serves as the SCU for CT Image Information Object Storage.
AE functionality will not be analyzed down to the software debug level. The functionality and interoperability of application software in either of the systems under test will not be evaluated. Clinical utility of application software, particularly as it pertains to the utilization of previously transmitted DICOM Information Objects will not be tested.
The test personnel use the DICOM Standard, the respective Conformance Statement and documentation of the Application Entities to prepare all test design, and test case specifications. The approach in this Master Test Plan is to organize the testing schema for the CT Information Object Storage Service Class and the Association Services. Detailed expansion of testing details will be developed within the Test Design and Test Case documents.
The overall test plan and the design of tests depends on the amount of information and the control that the designers and implementors have made available to the system operators. In testing for interconnectivity of Application Entities, there are three levels of information access that may be available to the user:
The Picker PQ2000 CT scanner supports Event level access, i.e. the user is informed that a connection has been established, a transfer is in progress, and that the transfer completed successfully or was terminated due to an error. Such access is available at the operator's interface.
Command Data Access and Data Set Access are not readily available.
The Pass/Fail criterion for interconnection of two Application Entities in each of the Service Classes and the Association Services is that they should exhibit the behavior expressed in the DICOM Standard. For each test in each category, the operational events should take place in the proper sequence and each of the Application Entities should be left in the appropriate state following the operation.
8.1 Suspension Criteria. The Association Services are a prerequisite to the operation of all of the Services Classes and must operate successfully in order for testing of any of the Service Classes to proceed. Should any of the Association Services primitives not perform as predicted, then the tests for interconnectivity shall be suspended.
The CT Storage Services Class will be tested. Should the testing within the Service Class not perform as predicted, then the testing shall be suspended.
8.2 Resumption Requirements. Testing of the Association will resume from the beginning when the reasons for suspension of testing have been determined, have been corrected, and new versions of the Application Entities have been submitted to the testing group.
Testing within the Service Class will resume from the beginning of testing for that Service Class when the reasons for suspension of testing have been determined and new versions of the Application Entity in question have been submitted to the testing group. .
The following documents shall be included:
Master Test Plan (This document)
Test Design Specifications
Test Case Specifications
Test Incident Report Logs
Test Summary Reports
Test Input Data and Test Output Data
(1) Preparation of the test plan
(2) Preparation of the test design specification
(3) Preparation of eight test-case specifications
(4) Prepare the hardware test environment
(5) Prepare the software test environment (install latest rev, DICOM configuration)
(6) Perform the test procedures
(7) Resolve test incident reports
(8) Repeat tasks (6) - (9) until all test procedures are successful
(9) Prepare the test summary report
11.1 Environmental Needs
The following represent the essential hardware and software needs:
11.2 Description of Actual Testing Environment
A Picker PQ200 CT scanner which is in routine clinical use was utilized for testing after normal working hours. The Picker CT scanner was connected via 10Base5 coaxial cable to a Cabletron Mini-MMAC configured as a bridge. The Cabletron unit was connected via multi-mode optical fiber to a port on a Cisco router. This router is part of the campus backbone of the Milton S. Hershey Medical Center and Penn State College of Medicine. The network connecting the devices under test experienced representative (albeit off hours) medical center network traffic during the testing period.
The section of Radiologic Computing and Imaging Science of the Penn State College of Medicine was responsible for managing, designing, preparing, and executing the example test suite. Information and Control Technologies, inc. was responsible for witnessing, checking, and resolving all aspects of the testing process. The test items were provided by their respective manufacturers and incorporated into clinical systems owned and operated by the Department of Radiology of the Milton S. Hershey Medical Center.
Test designs assume a thorough and complete knowledge of the operation of both systems under test as well as knowledge of the primary service tools and their appropriate operation.
Testing was performed by Dr. Fred Prior, Penn State, and Dr. Franklyn Bradshaw, ICT with the assistance of Dr. David Channin of Penn State. Technical assistance was also provided by Mr. James Daley and the technical support staff of General Electric Medical Systems.
Testing was conducted at the Milton S. Hershey Medical Center, November 3 and 4, 1995.