What do I monitor or evaluate?

The monitoring of a transition programme has to be tailored to the specific circumstances. The priorities among different goals and target groups determine the precise focus of the monitoring. But what you will monitor also depends on the actual project or programme and its environment and on ideas (hypotheses) about the spill-over effects of the transition. It is therefore impossible to describe the questions you should ask or the indicators you should use in general terms.

An additional factor is that the monitoring and evaluation of system innovation projects or programmes is usually not confined to the instruments of reflexive monitoring. It may be necessary and useful to employ more traditional forms of monitoring and evaluation. On this point, see also: 'What are in general objectives of monitoring and evaluation?'  

The following steps can be taken to determine where your focus will lie:

1) Start by clarifying the objectives, the questions to be addressed and the target groups for the monitoring and evaluation. This will give you a first impression of where you should focus. 

2) Specify the future vision envisaged by the project or programme.

• In which system?
• What will be fundamentally different in the future?
• What will you do to reach the new situation?
• How will your interventions gradually contribute to a system innovation and what might be good ‘milestones' for the purpose of monitoring and evaluation.

3) Translate the choices you make into indicators.

Indicators

There are various types of indicators. Below is are examples. The appropriate indicators for you are determined in part by the answers you have given to the above questions.

Process indicators

• Changes in the behaviour, perceptions and attitudes of stakeholder
• The composition of the network; the existence of new networks or relationships within networks
• Convergence of visions.

Early signs of practices representing a system innovation

• The scale of the use of new technological applications
• Numbers of innovative experiments
• The existence of new routines and standards.

More traditional indicators of sustainability

• For profit: examples include economic indicators, such as productivity, efficiency or profit
• For planet: examples include CO emissions; concentrations of particulate matter, biodiversity parameters or the use of space
• For people: examples include social indicators, such as well-being, income, employment or child labour. 

Objectives and questions that are difficult to reconcile

During the monitoring and evaluation you may be confronted with objectives and questions that are difficult to reconcile. You might therefore be required to consult the client again about further choices and/or about how different monitoring and evaluation activities can be carried out over time. On this point, see also ' Can learning be reconciled with accountability?'