The role of evidence in close-up higher education research: Towards evaluations of social justice programmes in higher education
Sarah Goodier, Carren Field, Suki Goodman, University of Cape Town
Abstract
Abstract As a programme’s context is inextricably linked to any evaluation work that may be performed on it, evaluation should be linked to the real-world experience of the programme. A top-ranking South African university’s global citizenship programme (here called the GCSA Programme) was evaluated in 2015 and is used in this paper as an example to illustrate that a key component in evaluation work is a programme’s context. The GCSA Programme is embedded in the South African higher education (HE) context at a time where the #feesmustfall movement is highlighting social justice issues in the HE environment. The GCSA Programme, which focuses on social justice, cannot be separated from the South African HE context in which it operates and this informs what evaluation questions were most useful for the programme stakeholders. The levels in the Rossi et al. (2004) evaluation hierarchy were used as a theoretical framework to consider the evaluation questions that would best inform the GCSA Programme staff, the evidence that could be gathered to answer these questions and how this information could be leveraged to strengthen a working programme. In terms of this framework, a theory evaluation to critique the programmes’ conceptualisation and design was conducted. Such an evaluation involves investigating the theoretical underpinning of how a programme is supposed to work, considering the theory that underlies similar programmes. This evaluation found a situation where, as student participants bring their own unique backgrounds and skill sets to the programme when they join, different outcomes may be achieved for each student. One of the key learnings this evaluation revealed was that the individual- and pedagogical-related assumptions (participants’ backgrounds, skill sets, levels of engagement) are key influencers of the programme theory of the GCSA Programme. This emergent and flexible approach, centred on the individual student, is a key assumption underlying many HE programmes and understanding this can help towards the planning and design of an evaluation of the extent to which individuals can and do engage with the programme. The GCSA Programme evaluation findings regarding the challenge of measuring programme outcomes can serve as a lesson to all HE programmes, especially those that take cognisance of social justice issues, of the importance of a robust theory-based approach in programme design. For many HE programmes, considering their work through an evaluative lens has the potential to strengthen their offerings and promote critical reflection on the programme within its context.
Keywords
Theory evaluation, programme context, global citizenship programme, programme theory, programme design
Link to Full Paper (If submitted)
Conference Organisers
Paul Trowler
Lancaster University, UK
Alice Jesmont
Lancaster University, UK
Malcolm Tight
Lancaster University, UK
Paul Ashwin
Lancaster University, UK
Murray Saunders
Lancaster University, UK
Chrissie Boughey
Rhodes University, South Africa
Suellen Shay
University of Cape Town, SA
