CTImusic
News


Summer 1998

Conferences, publishing, evaluation, peer review: are we going astray?

Lisa Whistlecroft

In November 1997, I was fortunate to have the opportunity to attend the ATMI (Association for Technology in Music Instruction) annual conference, held in Cleveland, Ohio. My visit was undertaken with the joint aims of presenting the work of CTI Music, demonstrating the TLTP CD-Tour Generator, and finding out more about the Association and its work.

ATMI 97 was an almost all-American-continent conference: as far as I know, I was the only European attending. This made it an exciting, if sometimes confusing, experience, as entire areas of apparently common curricula (such as music arrangement for marching bands) were unfamiliar to me.

From a CTI point of view, I spent much of my time at the conference alternating between enthusiasm and despair. My own enthusiasm was generated by the hard-working and enthusiastic ethos of the community I had joined for four days, and by the many innovative solutions I saw to challenges that face music educators on both sides of the Atlantic. The need for imaginative presentation of potentially confusing or dry material, for instance, along with coping with lower levels of background knowledge in incoming students, and helping talented performers who have inadequate notation skills or knowledge of theory are familiar to many of us. My despair was brought on by two repeating frustrations with the work that was described in the papers: lack of evaluation and a startling lack of availability.

Evaluation

Anyone who has heard Diana Laurillard of the Open University speak on the topic of evaluation cycles will know of the deep frustration that is developing in the UK. This frustration stems from the way funding bodies of all types support projects and commission evaluation but never seem to have in place any mechanism for absorbing, or in some cases even considering, the outcomes of these evaluative studies. My impression from the ATMI conference is that in computer-assisted music education in the USA this problem has yet to be faced. Indeed, it seems as though, in some cases, evaluation is not written into project plans at all. I found it quite distressing to hear of the time and effort that had gone into an extensive software development programme and then to hear that no documented study had been made of the students' use of the end product. Evaluation by peers, a normal and, often, the most valuable outcome of a conference paper presentation, seemed to be even rarer - due mostly to an availability problem.

Availability

There has been much talk in the UK of the 'not-invented-here syndrome' as a possible explanation for the low usage of educational software by lecturers other than the actual author of the software. It describes the behaviour of lecturers who decide not to use a teaching aid produced by another department, on the grounds that it does not precisely meet their existing needs. Rather than selecting only the parts of the material that are useful, tailoring it to their use, or even changing their course, better to make use of the material, they condemn it for its inadequacies and say that they would have used it, if only it had been written by their own institution - which would inevitably have had a better understanding of their particular needs. It was refreshing to find no evidence of this attitude among my American colleagues, who were keen to make use of each other's materials. Ironically, what seemed to be far more common was for an institution to have developed a course or module that was clearly of wide interest and applicability, but to refuse (or be unable) to release it for wider use or even for evaluation.

Ethics

There is, in my opinion, an interesting educational-ethical issue here which urgently needs to be addressed, in the UK as well as overseas. It is the very nature of conferences that presenters are effectively publishing as well as publicising their work. In the humanities, a conference paper is often the precursor of a published book - an opportunity both to advertise the author's work and to receive crucial feedback and criticism that will inform the more substantial publication to follow. A colleague recently described it to me as an opportunity to make mistakes in public in order to avoid making them later in print! The full fruits of their labours will, in due time, be made public, albeit sometimes at a price.

In the sciences and engineering, whilst the end result may not be a book but either a journal article or a physical product, it is usual to expect the data that formed the basis or justification for any new finding or theory to be made available to other scholars, so that they may verify the conclusions against their own ideas and methods. When the data is withheld, it is usully because the product is available for sale.

In education, rather as in medicine and the social sciences, it may not always be appropriate to make student-related data available to outsiders. In these circumstances it is surely even more important to make available the materials themselves.

Sharing knowledge

An education conference - and by inclusion, each of the papers presented there - may serve either of two purposes:

1. to present interesting new findings on the processes of education: in these presentations the educational materials developed, the tools as it were, may be of less interest than the results obtained from their use.

2. to present a new tool or technique for teaching: in this case the tool itself is the subject of the paper.

In the latter case, it seems to me that there is little value in describing the tool at all if it cannot be made available to other potential users. What use is there in describing to colleagues the way in which a year's work produced a valuable course, if all that they gain is the knowledge that if they spend a year doing it for themselves, their own students might benefit too? Like children at the window of a sweet-shop after closing time, listeners are left with more of a sense of deprivation than of illumination.

I appreciate that it would be financial and professional suicide to spend a year writing a course for which your employers have paid you - and for which they will charge their students - and then to give it away to all comers. But there are deeper and educationally more interesting issues behind the withholding of new projects. Again this was brought home to me at ATMI when a presenter described what was clearly an innovative and effective on-line distance-learning programme. In the question time, after the technicalities and the results details were addressed, the overriding questions were 'May we see it?', 'May we use it?', 'Can we buy access to it?'. The answer was that the programme had not been made available yet because it was not completely finished.

In Music, of all subjects, this struck me as tragically ironic. If an instrumental student refused to play during term time because their performance and interpretation was not ready for their final recital, they would be reminded that the whole process of education was about sharing the draft version to gain from the experience and knowledge of their peers. Surely the same must be true of the development of new teaching materials. Moreover, I am reminded of the advice that Jim Ridgway of Lancaster University gave the TLTP at its start: 'Make mistakes and, more importantly, make them early'. My suspicion is that by the time the distance learning programme was deemed by its authors to be properly finished, they would have no desire to start modifying it in the light of any comments their colleagues in other institutions might make.

I hope that none of this polemic will be seen as criticism of ATMI, and still less of the talented and dedicated tutors who shared their work and experiences in Cleveland. Neither am I ignoring the fact that the overriding reason for not making musical materials widely available is usually one of copyright. In the UK too we are probably more hamstrung by this than by any other consideration. I am guilty of demonstrating the best of the Music TLTP modules, and of the NetMuse projects, only to tell my audience (whose teaching is paid for by the council which commissioned the software development) that they can only have the silent version! My hope is that funding bodies, be they institutional or national, will come to realise that there is just not time for everyone to make their own solutions, and that collaboration, peer review and sensible licensing are the most effective ways for the academic community world-wide to create the educational materials and techniques of the future.


CTImusic News is © 1998 CTImusic, Roger Bray, David Burnand, Lisa Whistlecroft. All rights reserved

CTI Music home page / Index / Previous Article