Monitoring Interreg

In regard to monitoring, I would like to touch upon two issues:

1. explanation of inter-dependence between definitions of hierarchic gradation of outcome, result and impact, and
2. relation of the activity to the overall objectives as basis for monitoring at a project level.

While the concept itself - to distinguish consequences of an activity into hierarchic grades is very helpful to monitoring, I think that the presentation of the terms has been lost in general talk and has become rather confusing when coming to essence, especially between the definitions of outcome and result. The slides are however quite clear:

As we are talking about planned activity, we would benefit from introducing definition of categorised objectives and activities along with the consequences into analysis for having something to measure these consequences against.

We would also need to look into a broadmonitoring.jpger context of definitions, before analysing and attempting to explain commonalities and differences between the categories of consequence: outcome, result, and impact.

We may begin from the sequence of (i) set of objectives, (ii) range of activities initiated to achieve these objectives, and (iii) consequences of each activity. Monitoring then may be defined as evaluation whether the consequences of the chosen activity, measuring them against the objectives as much as they enable to attain these objectives, to justify the share of total dedicated resources spent on this particular activity.

As well as categorising consequences of activity into outcome, result, and impact, one may be consistent and similarly categorise objectives and activities into certain hierarchies:

As it is not the task of this brief note to engage into deep semantic analyses, it would be enough to say that introducing sequential analysis of vector combining time and extent of direct control over consequences, measured against hierarchy of objectives, would give a two-dimensional matrix.

We may also assume, that, in turn, each of the components in this sequence can be viewed upon either as a cause or a sequel: when an objective is a cause, activity is its sequel; in its turn, activity becomes a cause of a consequence: outcome seen as sequel; further, outcome produces result, and result yields an impact, that, in its turn, generates e.g. change, and so forth.

Then, in an interchanging cause-consequence two-dimensional sequentiality of an action, outcome, result, and impact, measured against a hierarchy of objectives may be illustrated as follows:

Let us say that, following one of the examples presented at the workshop, an end is to fight a long term unemployment in an area, a goal being to promote skiing tourism, an aim is to provide adequate services, and a target is to educate a necessary number of skiing trainers and guides.

For this hierarchy of objectives, a certain programme is created e.g. funded from the structural funds of the community, and it finances a given tourism promotion project, consisting of tasks facilitating service provision, and one of the tasks is a training course for skiing trainers and guides.

This task may be split into a number of actions: designing the course, arranging the logistics of the workshop, choosing the lecturers, etc.

Let us say that workshop as an activity may be expected to have following consequences: an outcome of the course is 20 guides trained; having guides providing service may have an expected result of increased range of specialised service that can be marketed, that may give an impact to growth of tourism in medium term, and bring about a change in long term unemployment by providing workplaces to service growing influx of tourists.

Trying to explain the differences between levels of hierarchy of the consequences a good way is to explain what gives them a common denominator. In the case as presented in Interreg monitoring workshop, it is the more the extent of direct control the project has over the given hierarchic level of consequence. In that way, the project directly controls how many guides will be trained (outcome); whether the range of increased service will help marketing the area (result of task) depends on whether somebody will give an effort to that skilful marketing and therefore is less controlled by the task's consequences, but may be directly controlled by another task in the same project. Whether the tourism to the area will grow (impact of task) depends on a wide range of factors external to the project (still, partly by whether having service of skiing guides increases competitiveness of that particular area - so still by indirect partial influence of the project; and may be directly controlled by higher level of activity - e.g. same programme).

From this example one can clearly see, that possibility to quantify consequences exists on various levels of the hierarchy (e.g. also on impact level - how many more tourists in two years), and therefore is not a definitive quality of an outcome level, as it has been suggested by the consultants neither a common denominator (as it is irrelevant to the result level - increased marketability of an area in this example).

Another, very different in substance, example would be a workshop on transdisciplinarity in planning urban revitalisation, our IUPM project. There, consequences would hierarchy like this:
output: understanding of the nature of the phenomenon and how to employ it in practice among the project partners;
result: incorporation of concept into the planning approach in partner communities;
impact: synergies achieved in planning process towards sustainability.

So much about definitions. Now, the issue of monitoring, same example.
Let us say the tourism promotion project costs 1 MEURO, lasts 18 months and has 15 items on task level. Say other similar tasks are producing an area-marketing booklet, and creating a partnership of a range of inns, restaurants and skiing facilities to offer combined services. The forms would tell us that skiing guides training has taken 2 months to prepare, 2 weeks to implement, and cost 70.000, and it was planned to have 6 weeks for preparation and spend 55.000 Euro, and it changed as additional important training issues emerged while preparing the course and additional instructor costing 15.000 Euro had to be commissioned (as the completing the presented forms would tell us).

This data will not serve the objective of monitoring, unless we are able to establish (i) the adequacy of extent of an action to the problem and (ii) comparative importance of the outcome to achieve expected result.

As to adequacy, e.g. it may be enough to have trained 10 guides for 1 week for 30.000 Euro - it would have had the same result, as such is the project demand for quantity and quality of service. The action is easy to repeat any time in case the demand grows.

As to the comparative importance of outcomes of different activity partially producing same result and/or impact - let us speculate that given area would only attract sporty skiers who do not mind restaurants, therefore this task of training guides is vital and much more efficient while partnership of restaurants and inns costing 200.000 Euro to establish is contributing to expected impact (increased tourism) much less and is therefore inefficient despite e.g. Euro 100.000 decrease from planned cost of Euro 300.000.

Without the indication of relation to overall goals, only from information as in presented monitoring forms, one may assume that training task was less efficiently performed than partnership establishment task. It is therefore necessary to introduce brief references to this relation into the forms somehow to make these forms serve monitoring objectives.

Then, referring to the first chart, one would monitor: output against targets, results against aims, impacts against goals, etc.

So finally, in the tables, perhaps there should be a column of objectives, relating each activity to an objective, and indicating percentage how much it is expected to contribute to a certain objective; another column can indicate how much of an overall project success depends on that objective.