Impact 

At the evaluation workshop there was some discussion about the nature of impact and the evaluation team at Lancaster offered to open discussion on the evaluation list on the subject. This short document brings together comments on impact from the original tender document with some comments from the workshop and further possible issues that might provide a basis for discussion.
1- Comments from original tender document

Aims

1.	to assess the effectiveness of LTSN in improving the co-ordination and coherence of activities which are intended to improve the quality of learning and teaching, (with consequences for the extent and impact of the dissemination of good practice in teaching and learning)

Note on impact
We understand impact as effects of activity resulting in changes in behaviour. These changes can be judged on a continuum from mere awareness to adaptive practice. Table 5.1 draws on a model we have used in other impact studies. It has origins in the work of Gene Hall and Susan Loucks (1978) on analysing reactions to a change or innovation in terms of ‘stages of concern’.  In the case of impact on academics as end users, we shall benchmark our impact judgements against a scale of this kind. The model describes the possible ‘connections’ an HE teacher might have with LTSN activities, on a continuum from ‘awareness’ to ‘refocusing’.  Strength or intensity of impact will be measured in terms of users’ positions with respect to this continuum.

Wider communities of users
Indicators of LTSN impact on HE practitioners and related bodies will include
•	knowledge of the work and activity of LTSN
•	participation in and experience of LTSN activity
•	attribution of changed practice to LTSN interventions
•	quality of LTSN sponsored events/activity
•	the nature of network activity
•	relationship between active participation in a network and changed practice (see Table 5.1 below).

7	Refocussing	attention now on adaptation, major changes, alternatives to original ideas, creativity	
6	Collaboration	coordinating and cooperating with others in using new practices	
5	Consequence	attention on impact on students of new practices, relevance, evaluation and implied changes	
4	Management	attention on difficulties in the processes and tasks involved in developing new practices	
3	Personal	begins to analyse effects and impacts on existing professional practice	
2	Informational	awareness and interest in knowing more	
1	Awareness	little concern or involvement	
0	No Awareness		
Table 5.1 Stages of concern model for impact assessment
2- Possible Issues

The evaluation workshop noted some potential issues concerning impact.

•	Impact could be intensive or extensive. The example used was a single intensive initiative, which could transform an individual's perception of teaching and learning against a wider initiative with broader but shallower impact.
•	LTSN Centres could be effective but lack impact. It is possible to conceive of an LTSN centre that was effective in carrying out its activities but failed to have significant impact. For example due to the resistance of its constituency or the weakness of the original conception of the activity.
•	Impact is difficult to ascribe to a particular cause. It was noted that impact on something as complex as teaching and learning was likely to be the result of a number of different factors rather than having a simple or singular cause.

Further issues can be developed from these initial concerns.

•	Impact is neither good nor bad. There is no value attached to impact, impact simply measures the effect of an initiative. It could be said that the policy of privatisation had a significant impact on the railways without endorsing the effects of the impact.
•	Impact may have a long or a short duration. The results of activity may be short lived. A participant in a workshop may be fired with enthusiasm on leaving but fail to sustain that momentum when faced with a resistant department.
•	Impact may require a critical mass. It is possible that impact on teaching and learning is constrained at the level of the individual academic and that impact might depend on attaining a critical mass in a Department, discipline or institution.