THE INTERCULTURE PROJECT

EVALUATION REPORT

Michael Byram, University of Durham

1.0 Scope

This report deals with those 'products' of the Project which are offered to students and their tutors involved in a period of residence abroad, in order to help them prepare for, profit from and reflect upon residence abroad as a cultural/intercultural experience. It therefore focuses on the 'outcomes' of the Project as presented on the website, with some attention to other elements of the website which in itself is a 'product' of the Project. The website can be used as an introduction to the issues in general but those, both students and tutors, who wish for an immediate response or at least a stimulus to responding to the issues of intercultural learning and residence abroad will find these in the part of the website which is labelled 'outcomes'. This is explained on the home page of the website although, in addition to the recommendations made to students interested in self-study, there could also be a parallel statement to tutors indicating where they should go first, e.g. to the module 'Acquiring intercultural competence', or to the article 'Raising intercultural awareness in preparation for periods of residence abroad: a review of current practice in UKHE', in one of the sub-projects and not easily found by chance in what is a large amount of material on the website as a whole

2.0 Approach to the evaluation

The approach I propose to take to the evaluation of outcomes is to establish a number of criteria from within the Project and to consider the different outcomes in the light of those criteria. In other words I shall attempt to judge in how far the Project has been able to meet its own intentions within the outcomes.

In the rationale for the Project, distinctions are made between 'developing knowledge about other cultures of a cognitive, declarative nature' on the one hand and 'intercultural awareness' and 'intercultural competence' on the other, the latter two also being distinguished from each other. One of the principal goals of the Project is to produce learning activities which develop intercultural awareness. It does not attempt to provide declarative knowledge about other cultures, although this is done incidentally in 'intercultural incidents' (see below).

Intercultural awareness is distinguished from 'cultural awareness', the former being described as a non-judgemental awareness of difference rather than the 'fixed representation' of another culture which is cultural awareness. An important part of intercultural awareness is 'reflection on the self and on one's own culture' , as well as reflection on the host culture. A further element of intercultural awareness is described as 'relativising skills' i.e. the ability to move away from single absolutist responses to a cultural interaction. There are then three key factors in intercultural awareness: non-judgemental awareness of difference; reflection on self as well as other; relativisation.

The difference between intercultural awareness and intercultural competence is described in terms of the former being passive and the latter being active, but also as the former being one of the pre-requisites for the latter. Intercultural competence is defined as 'the ability to interact successfully with others across cultural difference', and can only be 'developed and assessed in action'. It is therefore argued that the activities and resources produced by the Project have the primary goal of developing intercultural awareness, with intercultural competence being a consequence of residence abroad.

This means that the three factors of intercultural awareness identified above might be the basis for criteria to judge the quality of the activities and materials offered as outcomes. In fact, however, some of the materials, in particular the module 'Acquiring intercultural competence' appears to go further than developing intercultural awareness not only in its title but also in its content. That module uses as a 'framework' Byram and Zarate's model of intercultural competence (in a slightly amended form) which also includes an 'active' and not just 'passive' dimension to the definition and goal. Thus some elements of the outcomes appear to be more ambitious than the Project rationale would claim, and I shall attempt to bear this in mind in the following evaluation, considering to what extent the activities and materials might be used not just to develop intercultural awareness but also intercultural competence.

I shall consider each outcome in the order it is presented in the list of contents of 'ICP Outcomes'

3.0 Evaluation of 'ICP Outcomes'

3.1 Learning activities

3.1.1 SARA learning activities

The SARA database is a major product of the Project in terms of quantity of material available for students and tutors and makes full use of the potential of computer software programmes to search large amounts of data. Since these data are the accounts students give of their experience of residence abroad, they are a vicarious experience for those who have not yet resided abroad. The aims are thus emphasised as preparation and developing intercultural awareness as part of personal and professional development. There are also in fact suggestions for using the database for reflection on experience and follow-up although this is not emphasised in the list of aims.  The database thus offers a means of relating experiential and reflective learning both before the period of residence, albeit vicarious experience, and after.

The aims stated for these activities are consistent with the definition of intercultural awareness established above with the exception of reflection on self and one's own culture, and in practice I was not able to find data which could be used for this end. 

The particular activities, which are only intended as examples, are clearly explained and encourage tutors, for whom the instructions are intended, to allow inter-active learning both among students and between students and the database. The indications of which part of the database to use for specific activities are very helpful and time-saving for tutors. Tutors are also given suggestions for concrete outcomes for the activities. 

What might be added is a discussion of the ways in which tutors can evaluate the success of an activity by reference to what might count as evidence that students have learnt and/or become in some sense more interculturally aware. Ultimately, the website might also include examples of student products from these activities and discussions of how those outcomes might be evaluated. 

3.1.2  STEFE learning activities

The STEFE databases - extracts from interviews and entire diaries - contain the experience of French and British students on a jointly certificated initial teacher training course which takes place in France and England. As such it allows insight into the experience peculiar to this course, and the presentation of the interview extracts reflects the nature of the course and the pre-occupations of the students, with a majority of categories used referring to the issues of teaching (classroom skills, subject knowledge etc). 

The activities offered would be very useful for other students taking this course. Some are however potentially of more general use, such as an activity which asks students to reflect on co-operation with people of a different working style. The approach to the activities involves students in reflection and discussion, singly or in pairs, by same nationality or mixed nationality. In the course of the latter, there is potential for relativisation of and reflection on one's own presuppositions about the themes identified since an extract could well be seen differently from the two national perspectives. Perhaps the instructions for students working autonomously could include specific suggestions about reflection on the difference in interpretation and the origins of such difference in modes of socialisation within the education systems of the two countries.

The presentation of the data is very clear and it is easy to navigate through the database according to the pre-determined themes. There is however a potential yet to be realised for coding the material as was done for the SARA database which would allow a more flexible searching for key concepts. By chance I found the following extracts under different themes, all of them dealing with teaching in France, but one from a French student and the others British, two from the theme 'Classroom teaching skills' and the other from 'Positive outcomes'

Our teachers (in France) were really good apart from one # who wasn't good at all # I liked some courses which involved us in thinking a lot # just giving opinions about things ... this teacher for one exam she asked us to learn by heart 50 pages # and we just had to write you know by heart # it was stupid ... for a fortnight I just had to concentrate and learn by heart ... so I just had to forget everything else and then I had a good mark but it doesn't mean that I understood everything ... she was not really tolerant with the foreign students we had in the class so she expected them to speak in good French with no different accent just a French accent and she was really strict and she wasn't tolerant at all ... so I didn't like her # she didn't fit my expectation of a teacher (Anne-Marie, French, 'Classroom teaching skills')

Anthropology lessons ... the teacher was a very strange lady and # I didn't know how to react to her # she gave us texts to read for example and the next lesson she just picked names at random from the register to come up and write on the board what they had learnt and what they could say about the text and she was never satisfied # this was after about 4 weeks Nana just exploded really # she said why are you like this # why do you have to be like this with us # and then she did try and change # as soon as the exposés began it was much better (Denise, British, 'Classroom teaching skills')

In retrospect I did learn a lot and at the time I probably thought that I don't see the point of this but # a lot of the courses were useful # things like Christianisme and Islam and some anthropology units weren't anything to do with anything that we are doing here but it was quite interesting # and as far as things like the syntax and grammaire textuelle I found those at the time not very useful but # looking back I think that I have learnt quite a lot from that but nothing that I feel that I would be able to use in my classes now # because I'm in quite a difficult school # it was just totally irrelevant (Lucy, British, 'Positive outcomes')

It is not clear if they are all talking about the same classes and teachers, but students could be asked to analyse the difference between the perspective taken by Anne-Marie and the others, since Anne-Marie tries to see the teaching from a differentiated point of view and is aware of the ways in which the British students were experiencing the class she describes. The British students not only talk about the mode of teaching but also about the content and seem to agree that it was interesting but irrelevant. There are clearly interesting issues here concerned with perspective, with the use of anthropology, with styles of teaching and presuppositions about what is most appropriate, and about native speakers expectations of foreign speakers of a language.

At the moment, there is no possibility that I can see of using the database in this more flexible way, and it would be useful to have the data coded and made accessible using the approach taken to the SARA database as there are valuable opportunities for comparative treatment of topics from two national perspectives which are not available in the SARA database. 

The second database of whole diaries from four students does not have any activities specified 'as they are best read as a whole'. Since the diaries were written without any detailed specification, they include both interesting and sometimes trivial material e.g. 'To King's Lynn' (which may be a self-conscious entry). An edited version of the diaries would make them more readable and it would be useful to suggest activities which compare Simone's French perspective at particular moments of the course with the British perspective of the other three  -  with the task of speculating on the underlying presuppositions in the diaries but also in the way students read and respond to the diaries. 

In future developments of this work the diary guidelines developed within the Sheffield sub-project might be considered, with some modifications, as the teacher training students would then be encouraged to be more reflective in their writing.

(I had some difficulty in navigating this database as the use of the 'next' and 'previous' tab seems to jump from October to February missing out January.)

3.1.3  Other resources

The purpose of this part of the website is to present other materials not directly produced by the Interculture Project, but noted in the course of its work. It is heterogeneous in nature since it includes directions for tutors on organising classroom activities, extracts from other Projects, articles describing workshops and materials. It might be more user-friendly to put descriptive semi-academic articles into one category-space and directions for activities in another.

The difficulty in this section will be to know what deserves inclusion since users are invited to contribute further activities. On the one hand, it may not be necessary to evaluate an activity before it is included, but on the other hand it will be necessary to decide whether an activity falls within the focus on intercultural awareness/ competence which the Project has set itself. Secondly, there will be an issue of whether material should be made directly available on this site or whether users should be referred to other websites, as might have been the case for example with the ethnographic exercises from the LARA project, where they could be seen in context. A similar point might be made about the 'Portfolio' which is difficult to understand out of context. (On this point, the development of a Council of Europe Portfolio of language and culture learning could be noted as students during residence abroad would be able to use this portfolio.)

The presence of articles, with academic references and evaluation, alongside directions for activities in the classroom also raises the question of whether 'tips' of the latter kind should be provided without an underpinning. In the 
'Stereotype exercise' for example, it would be appropriate to select sections from Allport's 'The Nature of Prejudice' to support the experiential learning and help students to reflect critically on stereotypes, rather than just refer them to dictionary definitions. I suspect that neither tutors nor students in HE will be satisfied only with uninformed discussion, even though the activities might be useful as a starting point to bring out the questions for discussion. So the question of whether there should be some criteria for including activities may yet need to be discussed.

3.1.4  New ideas

This list of ideas intended as a stimulus includes suggestions about how students should be encouraged to reflect on their own culture. This is very much to be welcomed as it was not evident in the activities for the SARA database. The ideas are however very briefly described and, given the difficulty of making students decentre from their own culture(s), it would be helpful to have some more specific suggestions for appropriate techniques.

3.2  Module: Acquiring Intercultural Competence

In many respects this module can be seen as a crucial part of the Outcomes from the Project, at least as far as the user who is a tutor is concerned. The module aims to provide an integrated and reasoned approach to the intercultural dimension of preparation, experience and follow-up for a period of residence which ensures that it has pedagogical significance. The module draws in detail on the other Outcomes of the Project whilst offering tutors a model of how to develop a structured progression for students through the whole experience. I am particularly pleased to see that the module has a debriefing/follow-up element which ensures that the learning experienced during the period of residence is linked to work in the home university on return, as this is often a weakness in the ways universities handle the period of residence. The Ealing Project (Roberts et al in press) is the only other attempt that I know of to do this systematically.

The module also meets one of the problems raised earlier that although the Project as a whole includes reflection on self and own culture(s) as one of the aims of residence abroad, the SARA database does not seem to facilitate this in itself. The module however makes this one of its central aims: 'to equip students with the tools which enable them to gain greater awareness of their own cultural values and of intercultural issues, and to develop a working knowledge of life in the country or countries in which they intend to spend their period of residence abroad.' In fact the former aim is given more attention than the latter in the description of the module and activities suggested for it.

The module uses a modified version of a model of intercultural competence produced by Zarate and myself for the Council of Europe as a 'framework'. It is not possible to see direct links between the elements of the framework and the aims and objectives specified for each 'content area'. Although a too mechanistic approach to planning aims, objectives, outcomes and assessment criteria should be avoided, if the module were to be developed to meet the demands of this kind of planning, there would need to be further explication of the conceptual framework which undoubtedly underpins the module. (Some of this is evident in the 'Appendix: module as presented for validation at the University of Central Lancashire', although the issue of assessment is not expanded even there.)

I think what is particularly noteworthy is that the module attempts to develop a pedagogy of affective as well as cognitive learning by addressing issues of attitudes and expectations, stereotypes and self-analysis. This is a pedagogy which is crucial to all cultural learning and the development of intercultural competence, but it is one which is seldom pursued or articulated.  I think the proposal to have colleagues from Student Counselling Services involved in the teaching of such modules is an admirable innovation and one which needs to be promoted further. It is also an important and valuable aspect of the module that a link is made with postgraduate employment, with the help of the Careers Service, so that students might see the relationship more clearly between the period of residence abroad and their potential careers options.

Related to the issue of affective learning, the proposals for assessment clearly attempt implicitly to deal with personal development, and here again the inclusion of the Student Services Counsellor is an interesting innovation. However, it remains unclear what criteria should be used. Unlike the measurement of language acquisition during residence abroad, on which there have been a number of publications, there has been only a little empirical work on the assessment of cultural learning. The assessment of cultural learning and intercultural competence is an issue which might be taken forward in another research project.

The syllabus structure and descriptions of content areas, including suggestions for classroom activities, are described in some detail. There are suggestions of how to use other Outcomes from the Project, especially the SARA and STEFE databases. The general approach, given the aims of promoting reflection on self and own culture, seems intuitively right, my intuition being based on experience from the Ealing Project mentioned earlier. However there does appear to be much 'discussion' which does not always have a stimulus or clear starting and end point. Similarly, where suggestions are made that students should go out and observe, they probably need closer guidance than has been offered here. It is evident that this module is a plan rather than a report of a course already taught, and it would be valuable to have the mode up-dated with more teaching materials once it has been put into practice.

As pointed out earlier the module goes beyond the basic aims of the Project in that it attempts to develop intercultural competence and not just awareness. Insofar as the former is defined as active rather than passive, the attempts of the module to make students investigate and think about their own cultures and identities is valuable. There could be more emphasis on the notion of mediation and the intercultural mediator to help students not only to relativise their own preconceptions, but also to put values, practices and other dimensions of another culture in relation to their own. A comparative, juxtaposing methodology is needed and exercises which encourage students to develop as mediators.

3.3 Taxonomy

The taxonomy used to code the SARA and STEFE databases is interesting but does not appear to have pedagogical purpose, nor can I think of one myself. Perhaps this information would be better placed in the reports on activities rather than in this section on Outcomes.

On the other hand, the account of data collection in the activities section includes the questionnaires used with students on their return, and these could also have a pedagogical application and introduced into the Outcomes section (as is the case for some other data collection techniques.)

3.4  Bibliography

A bibliography for those tutors new to developing courses such as the module discussed above is clearly a very important tool. At the time of writing, the bibliography is very substantial  - over 40 pages  - and the intention to make it searchable online is laudable. It is not yet clear whether this will be by keyword, which involves a good deal of work in terms of categorisation of each item.

A comparison with another bibliography, supplied on line by the University of Minnesota, reveals some overlap but also discrepancy and gaps in the Lancaster bibliography (and also in the Minnesota bibliography, which however is inevitable as it claims to be only introductory and is much shorter). The work of Bennett is under-represented in the Lancaster bibliography as is Nostrand.

The exact purpose of the bibliography - whether to be introductory or comprehensive - needs to be clarified and some annotations or other means of access provided, as is planned.

3.5  Quizzes

The quizzes are a development from a well-known and much criticised approach to cross-cultural training which identifies situations in which foreigners may make mistakes and offers them a multiple-choice solution. The danger of this kind of training is that it leads to over-generalisation, although there are some routines of etiquette which they can encompass and illustrate. The quizzes as presented here are to be used more imaginatively as a stimulus for discussion, although it is also implied that they can be used for eliciting information about how much prior knowledge students' have of the country where they are going to reside, and in this respect might also be criticised for the superficiality of the information elicited. In this respect, too, the suggestion in the rationale for the Project that it is not concerned with declarative knowledge are somewhat belied

As stimuli for discussion, the quizzes are of varied value since the questions range from trivial issues, such as where to have one's rail ticket checked in France  (important information, but unlikely to lead to fruitful discussion), to questions about body space and non-verbal communication in Germany, which could raise many interesting questions abut unconscious response to people with whom one is communicating (see comments on non-verbal communication below). What is missing in these quizzes however is an approach which raises similar questions about expectations in students' own cultures since they would then begin to see the problematic nature of simplistic responses to complex issues  - and how they would react to a foreigner making a 'mistake' in etiquette or in non-verbal communication.

One quiz interestingly attempts to stimulate reflection on students' identity, their socialisation and issues of national identity as projected to foreigners. This is a valuable attempt to fulfil the aim of developing reflexivity, but the quiz is not given in full, and neither are the earlier ones.  I am not sure why this should be so as other materials in the Outcomes section have been made fully available.

The quizzes only partially contribute to the purpose of developing intercultural competence and could be said to undermine a view of this as a flexible and reflective capacity rather than an acquisition of sometimes over-generalised information. It would be helpful if more discussion of how these quizzes are best used for intercultural competence could be included.

3.6  Diaries

The diary guidelines are a very substantial product from the Project, offering a range of variations on the basic idea to cover different languages and different kinds of residence: as language assistant, in a work placement or as a student. They can be integrated into the module described above or they can be used independently. The team have provided tutor users of the website with all they need ready to download and print. 

The diaries are a significant means of making the period of residence a reflective experience since they encourage students to monitor their own learning  -  both linguistic and cultural  -  and juxtapose their own culture(s) with those they meet during residence. They also contain elements referring to personal development and, especially in the diaries for work-placements, the acquisition of new skills. this is a welcome recognition that the period of residence cannot be seen simply as an extension of classroom learning/acquisition of language and culture, but is also a major maturation process for many students.

Concentrating on those parts of the diaries which deal with cultural learning, I note that they integrate the different savoirs mentioned earlier under the analysis of the module, since they ask students not only to consider their own state of knowledge about specific aspects of the culture(s) they are about to enter  -  and then later to consider what they have learnt during residence  -  but they also engage students in investigation and analysis through two 'research tasks'. Students are given advice on how they might do their investigation,  but I wonder if they would find this sufficient without some preparation on the construction of 'questionnaires/surveys', on 'observation' and on 'an 'intercultural experiment'. Perhaps the website could include some examples from diaries of what might be expected, to make the idea of a research task less daunting, for both tutors and students. 

The sub-project has also presented a proposal for the assessment of diaries, including self-assessment by students. This is a detailed and separate specifications for tutors and students, with self-assessment counting for 30% of the mark. Although the specifications are detailed, it would be very helpful to have examples of diaries and the assessment made of them by both tutors and students to help those with little or no experience of this mode. 

Finally, there are suggestions for the use of diary extracts to prepare students for residence abroad. The questions suggested do not, I find, apply equally well to all the extracts, and  some piloting and experimentation with the extracts in a preparatory course would help to refine the suggestions. The material is however well chosen and speaks for itself in that it is likely to stimulate conversation and could be used  to ask students to consider why precisely these events or incidents are seen as significant by their authors and how the other parties in the events may have intended their actions to be understood. Further, students can then be asked to speculate on how the other parties would react to corresponding behaviour in Britain, in order to help the students to decentre and view their own culture(s) from the outside perspective.

The diaries and material related to them are clearly coherent with the Project's definition of intercultural awareness and its three elements: non-judgemental awareness of difference; reflection on self as well as other; relativisation. Of the three however, I think the third is least well realised; students could be asked more often and more pointedly to decentre and look at their own country from the perspective they gain during residence abroad and the perspective of the people around them in the foreign country.

3.7  Preparation courses

The notion of a short preparatory course is well established in some universities, but the presentation of a timetable and rationale for such a course is useful for those tutors responsible for organising residence abroad. It suggests the areas which might be dealt with and also indicates where the outcomes from the Project might be introduced. The material used on the Lancaster course in June 2000 but not yet on the site, includes a list of questions to stimulate reflection among students on their own socialisation and the characteristics of national identity. This is referred to as 'British', but this too needs to be reviewed and some students may not be happy with being called British/anglais/EngländerInnen etc and further questions on this issue need to be included. Nonetheless this is a useful ploy to emphasise the self-reflection which is crucial to intercultural awareness. 

Other handouts include multiple-choice responses on how to behave in social situations in specific countries. There is a danger of over-generalisation which is warned against in the introduction to the version 'with responses', but the responses remain, perhaps inevitably, recipe-like. On the other hand, the value of this kind of material is that it reminds linguists that interpersonal communication is to a large extent non-linguistic, and that 'intercultural incidents' (of which there are some examples in the handouts for discussion) may be caused by non-verbal miscommunication. I would suggest that the power of non-verbal communication should be made more explicit here and elsewhere throughout the site because linguists are often too focused on language.
 
It would be helpful to provide handouts from the course on the website, particularly those which were used with Project materials, on identity, encounters and incidents etc. For those users who come directly to this element of the website there need to be links to the diaries and other Project materials.

3.8  Intercultural incidents

This material is a development from the notion of 'critical incidents' which identify moments of misunderstanding and ask those about to go to another country what would be the 'correct' response. The Project's definition of intercultural awareness, and especially the concept of relativising, implies that 'correct' answers are not to be sought and the change of the name to 'intercultural incident' is an indication of this. There is clearly much material in the databases for creating such incidents, and some used in the preparation courses could be added to this part of the site. It would be helpful of tutors could be directed to this so that they could write their own incidents for preparatory courses.

The format presented here nonetheless could lead to the perception that there are 'correct' responses in some cases, particularly if a user comes straight to this part of the website without absorbing the notion of intercultural awareness. Either there should be fuller explanation of how this material should be used to encourage relativisation and decentring, or the open-ended instruction 'How should this be interpreted' should be used with all the incidents, as it is with one.

3.9 Card gallery

This collection of portrayals of intercultural incidents in caricature style is an intriguing approach to raising awareness of different perspectives on an incident. Some cards simply identify problems (e.g. what to take as a present if one is invited to dinner) and others portray potential misunderstandings. They are of different kinds and potential use. Humour may be a means of creating distance and reflection but it may also be directed at the other rather than at oneself and thus reinforce generalisations. The question of red-tape, for example, is one which also enrages foreign language assistants who come to Britain, as I discovered in my own research, and should not be seen as the sole preserve of foreign bureaucracies. 

There might be a link to the section on intercultural incidents to enlighten those users who come directly to this part of the website, on how intercultural incidents might be used to develop relativisation.

I am not persuaded that the cards should be used as postcards whilst students are abroad, as they might be misunderstood; the card 'Vegetarians beware!' is an example.

3.10  Focus groups and interviews

The value of debriefing or follow-up activities is often ignored by institutions organising residence abroad. Experiential learning benefits from a cycle of experience and reflection and whatever can be done to enable this during residence is important, although difficult to realise. The diaries probably help in this as they encourage students to 'step outside' their experience whilst they write. The suggestion that focus groups and interviews can be used after the period of residence is a useful one. 

If such interviews are to have a pedagogical function, to provide the moment of reflection and become an integral part of the learning process, they need to have clear pedagogical objectives. The guidelines provided at the moment are however not appropriate as they had a different purpose: to pursue the collection of data rather than develop a new perspective on experience. The extracts reveal too that the leaders of the focus groups were deliberately non-interventionist, whereas focus-groups/interviews with a different purpose might need to be steered more deliberately to pedagogical outcomes. The guidelines need to be revised and a discussion of the nature of the cycle of experiential learning as described by Kolb might be usefully included. Extracts are needed which show students taking a new perspective on their experience and not simply renewing and re-telling it.

4.0  The website

The website as a whole is easy to use. It is simple and uncluttered and movement within sections is quick, while it is easy to establish where one is within the site and possible to return to higher order sections. The databases have been commented on above. They too are easy to use and, putting aside the questions raised about the STEFE databases, I have not found any function I would like to follow which has not been available.

I have not considered in detail the sections of the site which are primarily reports on processes rather than outcomes, but it is clear that some of the reports would be useful for someone new to the field (see also below).

4.1  Sub-project reports

The reports from sub-projects are not presented directly as Outcomes of the Project, although they are referred to and links provided at various points. The reports on 'Raising Intercultural Awareness in preparation for periods of residence abroad. A review of current practice in UKHE' and 'Diaries as Learning Support for the Year Abroad' are both substantial pieces of work which would be useful for tutors coming new to the field and not satisfied simply with tips on what to do in the classroom. The reports provide clear reviews of the issues and significant bibliographies which would be helpful for tutors. A third report, on 'The Year Abroad Mailing List', describes the use of an open list allowing students and tutors in different countries to raise issues and compare their experiences, and in the second half of the year enabling students about to go abroad to ask questions of their predecessors whilst the latter are still there. The article provides technical information for others who might like to develop a mailing list or similar facility and gives examples of the conversations which took place. A future development might be on-line counselling for students with difficulties, although this would mean involving university counselling services too.

Although the first two reports can be reached through the Outcomes section of the site, it would be useful to have them as an integral part of it and it may be useful to have a separate section within Outcomes which provides academic and technical papers for tutors new to the field but interested in developing their understanding of it.

5.0  Conclusions

The Lancaster University Consortium application for funds to the FDTL identifies three aims (p. 3):
-  to identify and categorise examples of intercultural/socio-linguistic 'dysfunction' experienced by students studying and working abroad;
-  to describe and evaluate the guidance and support provided by a representative sample of HE institutions in the intercultural/socio-linguistic spheres;
-  to select and test 'models of good practice', to develop teachers' expertise in their application and to disseminate examples, descriptions and learning aids through electronic networks, reports and workshops.

My brief, to consider the website and in particular the Outcomes on the site does not correspond to all of the aims, and I cannot therefore comment on the effectiveness of workshops for example. I would also note that there is some suggestion in the application that the concern should not be just with students of Modern languages (p.6 para 2.3) but in practice the Project has focused mainly on Modern Languages students. There is potential for extending the experience gained during the Project to students of other subjects studying or working abroad.

It is noteworthy in retrospect that the concept 'intercultural/socio-linguistic' which, it was implied by reference to 'spheres' in the plural, were different but related dimensions of experience and competence, in practice have been treated under the notion of intercultural awareness/competence. I have found no direct reference to socio-linguistic competence. The Project as a whole has not attempted to impose a particular theory of competence, although the course proposed by one sub-project supports its rationale by reference to specific theories. It might be possible to develop a theory grounded in the analysis of the databases but the Project has deliberately left the databases open for tutors and students to explore and interpret themselves. 

This has advantages but I have pointed out that there are problems with openness in respect of the 'Other resources' which have been included or could be included in the future, and this is perhaps an issue elsewhere too.

As for the aims cited above, the website provides the material on the basis of which 'dysfunction' can be identified, again without theorising this. It also provides surveys of guidance and support in HE through the two sub-project reports, and there are 'models of good practice' not only from evaluations of current practice but also in the proposals for a module, in the diary study, in the various activities developed by the Project. It is however not possible to see from the website the extent of 'testing' of such models of good practice. 

The website provides for tutors the means of developing their expertise, and doubtless workshops and networks have also pursued this aim. However, although the Outcomes and other elements of the website are useful for tutors, the Project set itself the additional aim of providing for students too. This has been done largely through the databases and suggestions for their use by students. This is a welcome addition to the aims of the Project. There will be perhaps some need of targeted dissemination among students too to ensure that they realise that they can use the website autonomously as well as under the guidance of tutors. 

I have identified throughout a number of places where further work could be done within the Project as conceived. It is also possible to envisage a similar project for students of subjects other than Modern Languages, for whom study abroad is an option but who have quite different purposes.

Finally, I would like to return to a point raised earlier: the power of non-verbal behaviour. It is clear from the data, from incidents used for preparatory courses and reported on the mailing list, for example, that 'dysfunction' can be as much a consequence of non-verbal miscommunication as verbal. This may also be exacerbated by disparities in youth and gender cultures and by the immaturity of some students experimenting with the 'freedom' they feel in a society where they are not known, able to re-make their identity and test the boundaries of socially acceptable behaviour. There is little the Project can do about the latter; nor did it set out to do so. 

As for non-verbal communication per se, the Project did not set itself a specific aim in this area either, but there are data which could be used, and an Outcome which focuses on the non-verbal would be a welcome addition as well as a focus for future work.

Durham

Project response to Mike Byram's evaluation report

Various items were changed or developed on the website in response to Mike Byram's comments, as detailed on the schedule below.  Mike Byram was informed of the changes, and accepted the response, pointing out that his role throughout the project had been one of advisor rather than evaluator and that the report had been written in a spirit of formative evaluation rather than summative, with the intention that further work could be done as the project continued.

The page and paragraph references on the schedule refer to the Evaluation Report.

Schedule of Work done to the website

NOTE: Work which has not been undertaken now but which could be done in the future is indicated as "Future Research & Development".  

1.0 Scope. (p.1)

Home page:  added a recommendation to TUTORS that for background awareness  they should look at the sections on two sub-projects in particular: "Raising Intercultural Awareness etc.."  and the module on Acquiring Intercultural Competence.					

3.1.1 SARA learning activities(p.2)

Added to list of aims:  using database for reflection on experience & follow-up.		

Added statement to intro page re. reflection on self and own culture				

FUTURE RESEARCH & DEVELOPMENT (FR&D)

Discussion of ways in which tutors can evaluate the success of an activity.  
See report p. 3 para 2

3.1.2 STEFE learning activities (p.3)

FUTURE RESEARCH & DEVELOPMENT (FR&D)
Barry Jones responded that time constraints meant that the only possiblity of following the 
suggestions of the report would be under continuation funding, if this were available.

Note:  The comment in the report on difficulty of navigation (p.5 para 3) is due to a misunderstanding of the way the data is organised.

3.1.3. Other Resources (p5)

1.  Separated descriptive semi-academic articles and descriptions of actual learning activities into two categories.

2. Include the sub-project reports in the 'semi-academic articles' category

I didn't do this - because "other resources" refers to work from other projects.  The reports of the sub-projects are now much more extensively cross-referenced from other pages, however, particularly in the "outcomes" section; a new link on the navigation bar goes directly to the Reports page; and the Reports page has been re-ordered so that the sub-project reports appear first on this page.

The Other Resources section has been rewritten, to provide briefer descriptions with links or references to sources where possible, rather than to reproduce material too extensively from other sources. 
 
Some of the fuller descriptions have been kept and I have kept the old files of  scanned-in transcripts of papers, in case we re-think this again in the future!

LARA final pack 

The contents of the are listed and described briefly, with a link to the LARA website.

Portfolio

A link has been added  to Council of Europe Modern Languages Project.

FR&D

Devise criteria for including items in "other resources". 

3.1.4  New Ideas (p5)

Added an introductory statement that the ideas in this section were intended to give 
tutors ideas which they could develop themselves.  Ideally they would then send the
developed ideas back to the project for inclusion in the "other resources" section!

FR&D

Develop specific suggestions for appropriate techniques to exploit the ideas in the
 "new ideas" section.

3.2 Module (p. 6)

Small amendments made to the Module, particularly in the area of Assessment Strategy

3.3. Taxonomy (p. 7)

Move the taxonomy section from Outcomes to Project Activities				

NOTE:  This was not done, as Uschi Stickler used the Taxonomy in a Debriefing exercise described in the report Integration of the PRA and De-briefing and felt that it does have pedagogical value.  Added a reference to the report.

Move the questionnaires from the Data Gathering Section to combine with Focus Groups and Interviews, under Outcomes.

NOTE:  The Questionnaire material was not completely removed from Data Gathering (under Project Activities) - the samples of reponses were left under Data Gathering.  

Under the Outcomes section, the Interviews and Focus Groups section was extended and re-named to include Questionnaires.  The two sections with material on the questionnaires were cross-referenced.	

3.4  Bibliography (p7)

1.  A new introduction has been written.  								

2.  Searchable bibliography - development on this is progressing well and we expect the final version to be linked to the website by the meeting of 5th September.

3.5  Quizzes (p.8)

1.  Added suggested responses to the sample questions on web, to demonstrate that the quizzes are not designed to elicit 'one correct answer' but discussion of possibilities.	

2.  Still awaiting decision on final date for inclusion of all quiz material on web.

3.6  Diaries (p. 9)

FR&D

Add examples of diaries and the assessment made of them both by tutors and by students. (p9 para 5 italics)

Extend and refine the diary writing exercise. (p9 para 6).	

3.7 Preparation courses

1.  Altered identity quiz question to allow for non-British respondents

2.  Added links to materials on website which were referred to or used during the preparation course e.g. quizzes, intercultural incidents, diaries				

FR&D

Non-verbal communication.

3.8 Intercultural Incidents (p. 10)

1. Added link to database, and recommendation to tutors to consult database as source for creating their own intercultural incidents.							

2. Removed all suggested interpretations of intercultural incidents and replace with "How could this be interpreted?"							

3.9 Card Gallery (p.11)

1.  Linked to section on intercultural incidents							

2.  Asked MB for clarification of his reservation about the use of "vegetarians beware" postcard.	

3.10 Focus Groups & Interviews (p. 11)

Redrafted guidelines.

FR&D

Include extracts showing students taking a new perspective on their experience, 
not just renewing and re-telling it.

4.1 Sub-project reports (p. 12)

A direct link to the Reports page is now included on the navigation bar of the Outcomes section.  

Reports of the sub-projects are now much more extensively cross-referenced from other pages, and the Reports page has been re-ordered so that the sub-project reports appear first on this page.
		
FR&D

Develop on-line counselling for students with difficulties

5.0 Conclusions (p.12)

FR&D

Extend the experience gained during the Project to students of other subjects 
studying or working abroad. (NB  This is already the case with Sheffield students.)

Testing models of good practice.

Targetted dissemination to students to encourage self-access use of database. 
(Continuation activity).

GENERAL FR&D IDEAS

This section of the schedule collates the FR&D ideas above.

Assessment

•	assessment of cultural learning and intercultural competence
•	add examples of diaries and the assessment made of them both by tutors and by students.
•	
•	
•	Further extension of learning activities and materials
•	
•	update module with more teaching materials once it has been put into practice
•	methodology and exercises encouraging students to develop as mediators
•	extend and refine the diary writing exercise. (p9 para 6).
•	identify extracts showing students taking a new perspective on their experience, not just renewing and re-telling it.
•	develop on-line counselling for students with difficulties 
•	testing models of good practice.
•	targetted dissemination to students to encourage self-access use of database. (Continuation activity).

Non-verbal communication.

Broaden work beyond ML students

Extend the experience gained during the Project to students of other subjects studying or working abroad