Recommendations on data collection. One important issue that arose in relation to the collection of data is the use of surveys, particularly for assessing the relevance, utility, added-value and effectiveness of measures. Currently the use of surveys is very limited and not favoured by some Commission officials since they are considered to be expensive and probably disproportionate for measures with a small budget. The experience of other colleagues suggests that this is not necessarily the case and discussion at the workshops provided support for a more extensive use of them. It is not possible to propose a uniform template or standard approach to surveys since there are a number of differences between the surveys suggested in terms of the size and significance of the particular measure, the targets addressed and the level of detail required in the answers. Nonetheless, there are some common features across similar kinds of survey and a series of suggestions are made about how these common elements might be taken advantage of. The EIPC can also play a positive role in supporting and developing surveys and other instruments to provide feedback on developments at the Member State level, especially in relation to the take-up of recommendations and best practice. More generally, the timing of data collection is important. Some indicators do not need to be presented on an annual basis and are most usefully presented after an action has been completed. However, if evaluation is to be a continuous process throughout the project cycle, it is necessary to look for preliminary indications of results and outcomes, even if these are provisional in nature. The Interim and Ex-post Evaluations mark important points in the whole process and present an opportunity for an overview but the on-going indicator exercise should measure itself against the general evaluation process and contribute to it, especially by providing evidence of change over time.
Recommendations on data collection. A final task of the evaluation is to recommend mechanisms and instruments for the collection of information on indicators corresponding to the different EIP measures. The analysis of the existing indicators and processes used for the data collection and the discussion with Commission officials has led to a number of conclusions some of which were discussed further during the workshops. One important question is the use of survey mechanisms or tools for the collection of qualitative information. Currently the use of surveys is very limited and not favoured by some Commission officials since they are considered expensive and probably disproportionate for the small budget measures. The experience of other colleagues suggests that this is not necessarily the case and discussion at the workshops provided support for a more extensive use of them. In the recommendations above relating to specific sets of indicators across the range of measures, surveys were suggested on a number of occasions, especially as a way of providing information that could feed into assessments of measures’ relevance, utility, added-value and effectiveness. In fact, there were a number of differences between the surveys suggested in terms of the size and significance of the particular measure, the targets addressed and the level of detail required in the answers. On some occasions, the targets for instance, would be SMEs, on others, Member States, while in yet other cases the targets would be much more restricted, relating, for example, to participants in a particular conference. With such differences it is not possible to propose a uniform template or standard approach to surveys. Nonetheless, there are some common features across similar kinds of survey and for some of the more substantial requirements it may also be possible to have joint surveys providing information for indicators in different areas. The following considerations reflect some of the common features: • Any survey conducted with a view to providing information for the indicator system should reflect the new orientations of the system as a whole and, in particular, align the information sought with the aim of throwing light on the main evaluation issues (relevance, utility, efficiency, effectiveness, added-value and sustainability) • Short surveys can be particularly useful in providing information on relevance and utility • More substantial surveys can contribute to assessments of efficiency, effectiveness, added-value ...
