TESTING ENVIRONMENT. For each project phase the following tests are performed (in given order): 1. System tests • During the system test phase the separate system components are tested; the components are regarded as stand-alone, independent components. The following system components are distinguished: o Vehicle (IVS modem) o GSM / UMTS network o Stand-alone PSAP modem o Alarmcentrale 1-1-2 (PSAP1), with an integrated PSAP modem o Interface Alarmcentrale 1-1-2 / Rijkswaterstaat o Interface Alarmcentrale 1-1-2 / EUCARIS o Meldkamercentrale (PSAP2) • Testing of the component ‘Vehicle (IVS modem)’ is entirely on responsibility of the IVS modem supplier. Testing of the component ‘GSM / UMTS network’ is entirely on responsibility of the network providers (MNO’s). • System tests are performed in the lab environment (i.e. fully controlled environment), meaning that no operational users are involved. Tests are strictly related to the technical working of the equipment. 2. Integration tests • During the integration tests phase focus is placed on the end-to-end working of the connected system components and the ‘functionality’. • Integration tests are performed in the lab environment, meaning that no operational users are involved. Tests are strictly related to the technical working of the equipment. 3. Performance tests • During the performance tests phase focus is placed on the process of handling an eCall. Attention is paid to following aspects: usability and performance criteria. • During the performance tests the eCall test fleet is involved in drive testing. Performance tests are performed by operational users. • The key performance criteria that are used during the performance tests phase are: o Percentage of success (also known as ‘Success rate’) with variation in signal strength, caused by variation in clutter density o Duration as defined by the time table Tests are always based on a predefined set of test scenarios. Each set of test scenarios is defined before a particular test cycle is started. Next to the system tests, integration tests and performance tests in a laboratory environment testing by driving a pre-defined route will be done. The emphasis is on testing realistic combinations of eCalls under different circumstances. The eCalls will be generated in the field, all at a time. Emphasis is also on the processes in PSAP and TMC with the information generated by the eCall. In every scenario all the KPIs will be taken into account. D4.1 Draft KPIs, test specification and methodology The following different test scenarios will be tested: Environment Highway Type of eCall 2 automatic eCalls Dangerous goods involved No dangerous goods Environment Rural Type of eCall 1 automatic eCalls, 4 manual eCalls Dangerous goods involved No dangerous goods Environment Highway Type of eCall 2 automatic eCalls Dangerous goods involved Dangerous goods present in pick-up truck Environment Rural Type of eCall 1 automatic eCall Dangerous goods involved No dangerous goods Environment Urban Type of eCall 1 manual eCall Dangerous goods involved No dangerous goods Environment Highway D4.1 Draft KPIs, test specification and methodology
Appears in 2 contracts
Sources: Draft Kpis, Test Specification and Methodology, Draft Kpis, Test Specification and Methodology
TESTING ENVIRONMENT. For each project phase the following tests are performed (in given order):
1. System tests • During the system test phase the separate system components are tested; the components are regarded as stand-alone, independent components. The following system components are distinguished: o Vehicle (IVS modem) o GSM / UMTS network o Stand-alone PSAP modem o Alarmcentrale 1-1-2 (PSAP1), with an integrated PSAP modem o Interface Alarmcentrale 1-1-2 / Rijkswaterstaat o Interface Alarmcentrale 1-1-2 / EUCARIS o Meldkamercentrale (PSAP2) • Testing of the component ‘Vehicle (IVS modem)’ is entirely on responsibility of the IVS modem supplier. Testing of the component ‘GSM / UMTS network’ is entirely on responsibility of the network providers (MNO’s). • System tests are performed in the lab environment (i.e. fully controlled environment), meaning that no operational users are involved. Tests are strictly related to the technical working of the equipment.
2. Integration tests • During the integration tests phase focus is placed on the end-to-end working of the connected system components and the ‘functionality’. • Integration tests are performed in the lab environment, meaning that no operational users are involved. Tests are strictly related to the technical working of the equipment.
3. Performance tests • During the performance tests phase focus is placed on the process of handling an eCall. Attention is paid to following aspects: usability and performance criteria. • During the performance tests the eCall test fleet is involved in drive testing. Performance tests are performed by operational users. • The key performance criteria that are used during the performance tests phase are: o Percentage of success (also known as ‘Success rate’) with variation in signal strength, caused by variation in clutter density o Duration as defined by the time table Tests are always based on a predefined set of test scenarios. Each set of test scenarios is defined before a particular test cycle is started. Next to the system tests, integration tests and performance tests in a laboratory environment testing by driving a pre-defined route will be done. The emphasis is on testing realistic combinations of eCalls under different circumstances. The eCalls will be generated in the field, all at a time. Emphasis is also on the processes in PSAP and TMC with the information generated by the eCall. In every scenario all the KPIs will be taken into account.
D4.1 Draft KPIs, test specification and methodology . The following different test scenarios will be tested: Environment Highway Type of eCall 2 automatic eCalls Dangerous goods involved No dangerous goods Environment Rural Type of eCall 1 automatic eCalls, 4 manual eCalls Dangerous goods involved No dangerous goods Environment Highway Type of eCall 2 automatic eCalls Dangerous goods involved Dangerous goods present in pick-up truck Environment Rural Type of eCall 1 automatic eCall Dangerous goods involved No dangerous goods Environment Urban Type of eCall 1 manual eCall Dangerous goods involved No dangerous goods Environment Highway
D4.1 Draft KPIsHighway Type of eCall 2 automatic eCalls, 16 manual eCalls from cars passing by Dangerous goods involved Dangerous goods in second truck Environment Highway Type of eCall 2 manual eCalls Dangerous goods involved Dangerous goods in truck Environment Highway Type of eCall 8 automatic eCalls, 3 manual eCalls after 1st collision. Dangerous goods involved Dangerous goods in pick-up truck Environment Rural Type of eCall 2 manual eCalls by cars passing by Dangerous goods involved No dangerous goods Environment Urban Type of eCall 1 automatic eCall Dangerous goods involved No dangerous goods Manual 1 1 Multi manual 1 1 2 1 Multi automatic 2 2 1 Combination manual/ automatic 1 2 3 2 To refer the future situation with eCall to the current situation it will be necessary to map the current situation. This is why a pre-pilot measurement is being carried out. The baseline will be a quick-and-dirty measurement. The idea is to monitor for 1 or 2 days all incoming reports into the national 112-PSAP (PSAP 1), the regional PSAPs (PSAP 2) and TMC, and to describe in an exact way the tracking, timing and procedures. It should be clear what messages are actually sent to the TMC and how much time the different procedures take to complete. During the eCall pilot such measurements also will be carried out. At the conclusion of the tests it will be possible to draw conclusions about the effects of eCall on the incident duration, the different phases of incident management and the success rate. • In one or more scenario’s a foreign car(s) must be involved • Detailed planning of the scenario test specification must be done • Test duration and methodologyfrequency must be discussed • Test registration must be discussed
Appears in 1 contract
Sources: D4.2 Kpis, Test Specification and Methodology Final