Common use of Delivery Testing and Acceptance Clause in Contracts

Delivery Testing and Acceptance. 11.01 OMT shall deliver Corrective Releases, ISE Requested Releases, and New Releases in the manner set forth herein. 11.01.1 The Parties intend that ISE will order ISE Requested Releases from OMT on a fixed time and fixed price basis. 11.01.2 OMT acknowledges that software and documentation quality and time to production start are of critical importance to ISE. 11.01.3 ISE acknowledges that OMT requires a defined scope before committing to deliveries on a fixed time and price basis. Changes to the scope may cause delays in OMT’s ability to adhere to delivery schedules. ISE further acknowledges that OMT requires timely deliveries of specifications and clarifications on changed and new requirements on functional and non-functional features in the software, and that delays in providing such specifications and clarifications may cause delays in OMT’s ability to adhere to delivery schedules. 11.01.4 The Parties acknowledge that they have increased their efforts under this Agreement to monitor and maintain software and documentation quality, and both Parties shall work together on these efforts for the duration of the Agreement. In addition, the Parties have agreed to implement a Software and Documentation Quality Penalty and Bonus Program, as set out below, to create incentives to improve software and documentation quality. 11.01.5 OMT shall attend semiannual executive quality reviews, which are separate from the support meetings and capacity planning meetings. The Parties shall mutually agree on the frequency and location of such quality reviews, and the Parties shall confirm the agenda for the quality review at least [***] in advance of any such meeting. The Parties acknowledge that the Acceptance Testing procedures in this Agreement are more comprehensive than the procedures in the previous support agreement, and that such procedures may affect schedules for providing testing and Acceptance. 11.02 The Parties shall plan and administer Corrective Releases, ISE Requested Releases, and New Releases in the manner set forth herein. 11.02.1 OMT shall, twice per year, provide ISE a one-year “look-ahead” product schedule of OMT releases and related Third Party Standard Software. The Parties recognize that these product schedules are forecasts that are subject to change. The Third Party Standard Software schedule will include, but will not be limited to, VMS, MS Windows NT, RTR, BMC Patrol, to the degree that accurate release information is available from the suppliers of Third Party Standard Software. 11.02.2 The Parties agree that Corrective Releases, New Releases and ISE Requested Releases can be delivered as separate deliveries or combined into one delivery. *** Certain information on this page has been omitted and filed separately with the Commission. Confidential treatment has been requested with respect to the omitted portions. 11.02.3 If ISE requests, OMT shall provide, telephone and on-site support for the installation and testing of releases. 11.02.3.1 OMT will provide telephone support during Operational Support Hours free of charge, and OMT will provide telephone support during non-Operational Support Hours or on-site support on a time and material basis. ISE will request telephone support during non-Operational Support Hours in respect of release administration at least [***] in advance, and ISE will request on-site support [***] in advance. 11.02.3.2 OMT shall, upon ISE’s request, provide temporary on-site assistance (up to five (5) working days) on a time and material basis, with four (4) weeks’ written notice. Long-term on-site assistance shall, if necessary, be provided subject to a separate agreement between OMT and ISE. In the event on-site support is requested by ISE, OMT undertakes to send duly qualified OMT personnel. 11.02.4 OMT undertakes, if requested by ISE, to provide a reasonable level, up to a maximum of [***], of training in connection with each release. Such training is provided to ISE for such number of ISE personnel as ISE may nominate, and will be provided in Stockholm, or, if agreed between the Parties, at an ISE site. The extent of the training to be provided is to be agreed between the Parties in respect of each release. Release training will, subject to one month’s notice, be provided by OMT on a time and material basis. 11.02.5 ISE acknowledges that assigning qualified OMT staff to training, on-site support and other such activities may have an impact on release and project scheduling. 11.02.6 Both Parties shall appoint a project manager for each release. 11.03 OMT shall test and deliver Corrective Releases, ISE Requested Releases, and New Releases in the manner set forth herein. 11.03.1 OMT undertakes to conduct functional testing, regression testing, capacity testing and non-functional testing for each release in the OMT Test System, a multi-node test environment similar to the Production System, the specifications for which are set forth in Schedule 9. 11.03.2 The Parties agree that, during the development phase, OMT may deliver the release to ISE prior to final delivery in order for ISE to complete a preliminary assessment of whether the release operates as designed. Such interim deliveries shall not be subject to the Software and Documentation Quality Penalty and Bonus Program. 11.03.3 OMT acknowledges that final delivery of a release will be Production Ready. A release is Production Ready when: 11.03.3.1 The release is delivered in accordance with the supporting documentation (i.e., a functional description, an enhancement request, or a service request); 11.03.3.2 All existing functionality remains at the same level of quality (functionally and technically) and at materially the same performance levels specified in the performance and capacity metrics determined in accordance with Schedule 8; 11.03.3.3 The release is delivered with the applicable Documentation; and 11.03.3.4 OMT is confident that the release will successfully pass Acceptance Testing, and, once implemented in the Production System, will not impair Operating Capabilities. *** Certain information on this page has been omitted and filed separately with the Commission. Confidential treatment has been requested with respect to the omitted portions. 11.03.4 ISE shall take final delivery once OMT declares that the release is Production Ready. 11.03.5 OMT shall deliver test reports, release notes, and other information, as specified in Article 10, with each Corrective Release, New Release or ISE Requested Release. OMT shall deliver updated Documentation when such release has changed the Documentation previously provided to ISE. 11.03.6 The release notes that accompany a New Release or an ISE Requested Release shall specify, inter alia, the necessary releases of the Software Product and the necessary Third Party Standard Software and revisions. The release notes that accompany a Corrective Release shall not require a change of the specification of the Production System, unless otherwise is explicitly agreed by the Parties. 11.04 Unless otherwise agreed to by both Parties in writing or pursuant to this Agreement, ISE shall conduct Acceptance Testing on Corrective Releases, ISE Requested Releases, and New Releases to determine whether the release is Accepted or Rejected in the manner set forth herein. The purpose of Acceptance Testing is for ISE to validate that the release contains the desired functionality, meets the performance and capacity criteria, and will not impair Operating Capabilities when implemented on the Production System. The framework for Acceptance Testing is set forth in this Article; the mechanics of Acceptance Testing are set forth in the Acceptance Testing Regulations in Schedule 10. Acceptance Testing shall consist of two phases, as follows: 11.04.1.1 The first phase of Acceptance Testing shall consist of a unit test. ISE shall perform the unit test immediately after final delivery by OMT. ISE undertakes to complete the unit test within two (2) weeks of final delivery. The purpose of the unit test is to ensure that the code is stable enough to proceed with Acceptance Testing, under the criteria agreed between the parties from time to time. If the unit test results are satisfactory, ISE shall acknowledge delivery and shall commence Acceptance Testing. 11.04.1.1.1 ISE shall, at an early phase of the project, provide OMT with the unit test scripts, as further described in Schedule 10. OMT shall review the unit test scripts, and ISE and OMT shall agree upon the unit test scripts to be used in the unit test. 11.04.1.2 The second phase of Acceptance Testing shall consist of an acceptance test. 11.04.1.2.1 ISE shall prepare a sufficient number of acceptance test scripts to fully test the delivered software. ISE shall, simultaneous with approval of the functional descriptions for the project, provide OMT with the acceptance test scripts. OMT shall review the acceptance test scripts, and ISE and OMT shall agree upon the acceptance test scripts to be used in the acceptance test. 11.04.1.2.2 The acceptance test shall consist of four types of testing: functional testing, regression testing, capacity testing and non-functional testing. ISE shall perform functional testing and non-functional testing for all new functionality included by OMT in the delivery. ISE shall perform regression testing for the existing functionality.

Delivery Testing and Acceptance. 11.01 OMT shall deliver Corrective Releases, ISE Requested Releases, and New Releases in the manner set forth herein. 11.01.1 The Parties intend that ISE will order ISE Requested Releases from OMT on a fixed time and fixed price basis. 11.01.2 OMT acknowledges that software and documentation quality and time to production start are of critical importance to ISE. 11.01.3 ISE acknowledges that OMT requires a defined scope before committing to deliveries on a fixed time and price basis. Changes to the scope may cause delays in OMT’s ability to adhere to delivery schedules. ISE further acknowledges that OMT requires timely deliveries of specifications and clarifications on changed and new requirements on functional and non-functional features in the software, and that delays in providing such specifications and clarifications may cause delays in OMT’s ability to adhere to delivery schedules. 11.01.4 The Parties acknowledge that they have increased their efforts under this Agreement to monitor and maintain software and documentation quality, and both Parties shall work together on these efforts for the duration of the Agreement. In addition, the Parties have agreed to implement a Software and Documentation Quality Penalty and Bonus Program, [***] as set out below, to create incentives to improve software and documentation quality. 11.01.5 OMT shall attend semiannual executive quality reviews, which are separate from the support meetings and capacity planning meetings[***]. The Parties shall mutually agree on the frequency and location of such quality reviews, and the Parties shall confirm the agenda for the quality review at least [***] in advance of any such meeting]. The Parties acknowledge that the Acceptance Testing procedures in this Agreement are more comprehensive than the procedures in the previous support agreement, and that such procedures may affect schedules for providing testing and Acceptance[***]. 11.02 The Parties shall plan and administer Corrective Releases, ISE Requested Releases, and New Releases in the manner set forth herein. 11.02.1 OMT shall, twice per year, provide ISE a one-year “look-ahead” product schedule of OMT releases and related Third Party Standard Software. The Parties recognize that these product schedules are forecasts that are subject to change. The Third Party Standard Software schedule will include, but will not be limited to, VMS, MS Windows NT, RTR, BMC Patrol, to the degree that accurate release information is available from the suppliers of Third Party Standard Software. 11.02.2 The Parties agree that Corrective Releases, New Releases and ISE Requested Releases can be delivered as separate deliveries or combined into one delivery. *** Certain information on this page has been omitted and filed separately with the Commission. Confidential treatment has been requested with respect to the omitted portions. 11.02.3 If ISE requests, OMT shall provide, telephone and on-site support for the installation and testing of releases. 11.02.3.1 OMT will provide telephone support during Operational Support Hours free of charge, and OMT will provide telephone support during non-Operational Support Hours or on-site support on a time and material basis. ISE will request telephone support during non-Operational Support Hours in respect of release administration at least [***] in advance, and ISE will request on-site support [***] in advance]. 11.02.3.2 OMT shall, upon ISE’s request, provide temporary on-site assistance (up to five (5) working days) on a time and material basis, with four (4) weeks’ written notice. Long-term on-site assistance shall, if necessary, be provided subject to a separate agreement between OMT and ISE. In the event on-site support is requested by ISE, OMT undertakes to send duly qualified OMT personnel[***]. 11.02.4 OMT undertakes, if requested by ISE, to provide a reasonable level, up to a maximum of [***], of training in connection with each release. Such training is provided to ISE for such number of ISE personnel as ISE may nominate, and will be provided in Stockholm, or, if agreed between the Parties, at an ISE site. The extent of the training to be provided is to be agreed between the Parties in respect of each release. Release training will, subject to one month’s notice, be provided by OMT on a time and material basis. 11.02.5 ISE acknowledges that assigning qualified OMT staff to training, on-site support and other such activities may have an impact on release and project scheduling. 11.02.6 Both Parties shall appoint a project manager for each release. 11.03 OMT shall test and deliver Corrective Releases, ISE Requested Releases, and New Releases in the manner set forth herein. 11.03.1 OMT undertakes to conduct functional testing, regression testing, capacity testing and non-functional testing for each release in the OMT Test System, a multi-node test environment similar to the Production System, the specifications for which are set forth in Schedule 9[***]. 11.03.2 The Parties agree that, during the development phase, OMT may deliver the release to ISE prior to final delivery in order for ISE to complete a preliminary assessment of whether the release operates as designed. Such interim deliveries shall not be subject to the Software and Documentation Quality Penalty and Bonus Program[***]. 11.03.3 OMT acknowledges that final delivery of a release will be Production Ready. A release is Production Ready when: 11.03.3.1 The release is delivered in accordance with the supporting documentation (i.e., a functional description, an enhancement request, or a service request)[***]; 11.03.3.2 All existing functionality remains at the same level of quality (functionally and technically) and at materially the same performance levels specified in the performance and capacity metrics determined in accordance with Schedule 8[***]; 11.03.3.3 The release is delivered with the applicable Documentation; and[***] 11.03.3.4 OMT is confident that the release will successfully pass Acceptance Testing, and, once implemented in the Production System, will not impair Operating Capabilities[***]. *** Certain information on this page has been omitted and filed separately with the Commission. Confidential treatment has been requested with respect to the omitted portions. 11.03.4 ISE shall take final delivery once OMT declares that the release is Production Ready. 11.03.5 OMT shall deliver test reports, release notes, and other information, as specified in Article 10, with each Corrective Release, New Release or ISE Requested Release. OMT shall deliver updated Documentation when such release has changed the Documentation previously provided to ISE. 11.03.6 The release notes that accompany a New Release or an ISE Requested Release shall specify, inter alia, the necessary releases of the Software Product and the necessary Third Party Standard Software and revisions. The release notes that accompany a Corrective Release shall not require a change of the specification of the Production System, unless otherwise is explicitly agreed by the Parties. 11.04 Unless otherwise agreed to by both Parties in writing or pursuant to this Agreement, ISE shall conduct Acceptance Testing on Corrective Releases, ISE Requested Releases, and New Releases to determine whether the release is Accepted or Rejected in the manner set forth herein. The purpose of Acceptance Testing is for ISE to validate that the release contains the desired functionality, meets the performance and capacity criteria, and will not impair Operating Capabilities when implemented on the Production System. The framework for Acceptance Testing is set forth in this Article; the mechanics of Acceptance Testing are set forth in the Acceptance Testing Regulations in Schedule 10. Acceptance Testing shall consist of two phases, as follows: 11.04.1.1 The first phase of Acceptance Testing shall consist of a unit test. ISE shall perform the unit test immediately after final delivery by OMT. ISE undertakes to complete the unit test within two (2) weeks of final delivery. The purpose of the unit test is to ensure that the code is stable enough to proceed with Acceptance Testing, under the criteria agreed between the parties from time to time. If the unit test results are satisfactory, ISE shall acknowledge delivery and shall commence Acceptance Testing. 11.04.1.1.1 ISE shall, at an early phase of the project, provide OMT with the unit test scripts, as further described in Schedule 10. OMT shall review the unit test scripts, and ISE and OMT shall agree upon the unit test scripts to be used in the unit test. 11.04.1.2 The second phase of Acceptance Testing shall consist of an acceptance test. 11.04.1.2.1 ISE shall prepare a sufficient number of acceptance test scripts to fully test the delivered software. ISE shall, simultaneous with approval of the functional descriptions for the project, provide OMT with the acceptance test scripts. OMT shall review the acceptance test scripts, and ISE and OMT shall agree upon the acceptance test scripts to be used in the acceptance test. 11.04.1.2.2 The acceptance test shall consist of four types of testing: functional testing, regression testing, capacity testing and non-functional testing. ISE shall perform functional testing and non-functional testing for all new functionality included by OMT in the delivery. ISE shall perform regression testing for the existing functionality[***].