5 October 2018
I will miss visiting Helsinki. Since last August, I have made 8 visits to Finland, including mid-summer and mid-winter. The reason? I have been chairing the international audit of a Finnish university. In this blog, I will offer a few personal reflections on the process.

The Finnish education system, including its universities, is rightly praised around the world.  One of the pillars of the system is the work of the Finnish Education Evaluation Centre (FINEEC). The first observation to make is that the Centre works across the whole spectrum of education, from pre-school to lifelong learning, offering an integrated, consistent approach to the evaluation of educational activity.  Given the scale of activity to be evaluated in Finland, this is practicable in a way that would not work in larger systems, but it is a key strength.

Each university must undergo an external audit at roughly 5/6 year intervals. The invitation from FINEEC to chair this particular audit came in January 2017.  My team comprised five members, three international and two Finnish; we were supported by two members of staff from FINEEC.  We met for the first time in August 2017 for a two-day training event, followed by a one-day meeting with representatives from the university to be audited.  These initial meetings were crucial, enabling us to bond as a team and helping to establish a strong working relationship with senior staff from the university.  Any audit or evaluation inevitably raises uncertainty, anxiety and even suspicion.  For me, an essential aspect of any evaluation process is for evaluators and evaluatees to work together with a common purpose of enhancing the quality of activity.  This initial meeting provided a strong basis of mutual respect and ensured a human face to the whole process.

The next stage was for the university to produce a detailed written submission.  This followed guidelines provided by FINEEC.  It is important to stress that the emphasis in preparing the submission – indeed, the emphasis throughout the audit – was not on outputs themselves, but on processes.  At no stage was I expected to assess a curriculum or a research paper. Rather, our role was to check that the university had in place effective procedures to ensure that it could measure and analyse its own level of performance with a view to enhancing the quality of activity.  Thus, the final submission prepared by the university described the operation of the institution’s quality system and its relationship with overall strategy, and detailed the operation of the system for teaching and education at different levels; research, development and innovation; societal impact and regional development; and an optional area of activity. We were provided with a wealth of paperwork explaining the responsibilities of committees and individuals, with examples of feedback from students, arrangements for ensuring and enhancing the quality of research, and details regarding the inputs from external stakeholders and the wider community.  All of this was provided in excellent English language, of course!

We then met as a team to consider the documentation, to decide who we wanted to meet during the university visit and to agree whether we needed any additional information.  As Chair of the team, I also visited the university to meet with senior staff and to address an open meeting of staff and students.  Again, this element of personal contact helped to cement good working relationships.

In November 2017, we visited the university.  Over three days, we had 17 meetings with staff, students and external stakeholders.  It was intense, but illuminating.  Again, the emphasis was on discovering how the university gathered information on quality across the range of its activities, and how the information was used.  As external “experts”, it was our task to test the effectiveness of the systems in place and to identify possible areas for improvement.  Central to the success of the evaluation was openness and trust; we were there to help and support, not to judge or catch them out.

On the day after the visit, we met again to consider our findings.  At this stage, we were especially guided by the framework applied by FINEEC which required us to rate each area of activity as “absent”, “emerging”, “developing” or “advanced” according to pre-prescribed criteria, with “absent” representing the weakest level of performance and “advanced” the strongest.  In total 10 ratings were agreed.  In order to pass the audit, a university must avoid any rating of “absent” and must be rated as at least “developing” for the quality system as a whole.  The university we considered successfully passed the audit.

Each member of the group participated in writing the final report and we met together again to sign off the final version.  We were required to set out clear evidence for our conclusions, to agree a set of recommendations for the university and to identify areas of good practice. The report was finalised in February 2018.  A version was sent to the university to correct any factual errors.  Then, I travelled to Helsinki to present the report to the FINEEC Evaluation Committee and to answer questions.  The report was approved, and then formally released to the university and published.

The final stage was to visit the university to present the report to another open meeting of staff and students, and to answer questions; in addition, I led a further session offering detailed ideas and feedback to senior staff.

The university is not required to take our advice, but the constructive, non-confrontational nature of the exercise gives me confidence that they will respond positively.  The audit was time-consuming for all concerned and, I am sure, expensive to organise; few countries can consider such an undertaking.  It is important that the costs of audit and evaluation are justified by the benefits.  However, I really do believe that we have helped to enhance the quality of education, research and societal impact in this particular university.

Hopefully, work will take me back to Finland soon; I miss those cinnamon buns!