A note on the evaluation of British Council Activities

There is a general interest in the issue of ‘impact’ of an intervention [e.g. a seminar, exhibition, series of events, a showcase, visits] in BC activities.  The notion of impact might be considered at three broad levels which have a temporal dimension.  The first is awareness, the second is quality of the experience and the third is longer term impact [including off book activity].  The first two are evaluations which might be conducted during or immediately after an event.  The third is a longer term view on the impact of an intervention.  Generally speaking the first two types of evaluation are undertaken on a routine basis within the BC worldwide, the third is much more uneven.  In order to assess the multiplier effects including off- book gains of an intervention more follow-up evaluations of impact should be considered.

Awareness 
This concern is usually associated with cultural events or exhibitions.  The pre-occupation of the Office is usually to assess the extent to which advance publicity, publicity during the event and post event assessment is present in local media coverage.  This presence acts as an indirect indicator of public awareness of the event.  The assumption is that reading or seeing publicity of the event creates ‘event awareness’.  It is particularly important to penetrate media that communicates in the indigenous language.  Offices routinely undertake these types of analysis, particularly in countries with sophisticated or developed media infrastructures.  The difficulty is to identify a measure of what counts as strong media presence.

Quality of the experience
This broad concern is at the heart of the developmental potential of an evaluation and is aimed at assessing the way in which target groups, key or influential stakeholders, participants, BC staff etc experienced the event.  There are powerful evaluations which assess this experience against objectives identified at the planning stage.  However, it is quite possible to undertake useful evaluations which are more open in their approach and are interested in understanding more about a stakeholder’s experience for future reference.

Longer term impact 
The issue of impact is interesting in the case of the creative industries.  All BC activities must be understood in terms of the broad diplomatic mission of furthering British interests..This now includes an explicit concern with the commercial impact of arts activities.  This is not to say, however, that events or educational activities do not have an intrinsic value or contribution to local quality of life, but that there is always a parallel concern to position Britain positively.  An underlying concern then will always be to address the levels of attribution to the BC and positive association with the UK and now commercial outcomes.  The principle methodological difficulty is in the assumption that a positive experience, attributable to the BC and associated with the UK, will lead to changed behaviour or practice e.g. buying British. There is no logical connection between a positive experience of an event and future actions. We can assume, however, that a positive experience is a necessary condition for desired actions but it can rarely be sufficient.  Issues of timing and resources are often more important in determining actions.  This means that measuring the quality of experience of an event will not necessarily provide a good indicator for future actions. 
If the aim of an event is to promote a more positive view of a British good, service or aspect of culture with a view to actually changing behaviour then a positive experience of an event will not, in itself, indicate the future use of British goods or services etc on the part of participants.

If active measures like ‘increased use of UK resources’ or ‘more business’ are objectives then experience of the event is not sufficient as a measure.  We also need to develop measures of follow-up actions which might involve changed behaviour.  These might include 

•	Further participation in BC events
•	Further inquiries on subject matter associated with an event
•	Discussions with colleagues,friends,family on subject matter associated with an event
•	Planning/policy decisions with reference to subject matter derived from an event
•	Purchase of services or products associated with the subject matter of an event
	[includes UK consultancy, products, courses, training etc]

It is likely that an evaluation will address the three broad concerns of 

1.	Awareness
2.	Experience
3.	Impact:
•	Attribution
•	Association
•	Changed behaviour [including purchasing creative products]

There were some examples of follow up evaluations [This is a reference to the CIEP Evaluation] to pick up impact [the evaluations of the Edinburgh Showcase is a case in point] but not many.  Unfortunately there is no quick fix for this kind of evaluation.  As in the case of the evaluation of the Showcase, it involves ‘tracer studies’ of stakeholders [participants or invitees] to assess the extent of any follow-up activity.  These can be simple designs with two or three questions to address on-going involvement and can be electronic or by telephone.  Stakeholders can be sampled to gain indicative data.  The principle drawback is that this type of activity is resource heavy and can involve considerable person hours.

Murray Saunders
June 2000