• Home
• About
• Team Members
• Resource Toolkit
• Contact us

Activity

Information

Presentation

Website

Return to 'Toolkit' Structure: Ten features of evaluation

7 Evaluation Data Collection

There are a myriad of different methods of collecting data. There is no single method that will serve every situation or help you to collect data that will answer every evaluation question.

Answers to questions in the RUFDATA framework will all inform the type of data collected. Ideally an evaluation should seek to provide:

• rich and varied diagnostic evidence of how WP practice is achieving institutional and externally defined goals: specifically its capacity to discern differential effects on its target population;
• an approach that has a focus on diagnostic explanation and individual trajectories as well as an overarching set of practices that will provide a Higher Education Institution or Aimhigher Partnership with a model of embedded evaluation that could act as an exemplar for wider use.

Practically the questions concerning DATA, TIMING and AGENCY are helpful when thinking about what, when and how data will be collected.

Things to do Think about the evaluation questions you are trying to answer. The questions should inform your decisions about the data collection methods you choose. For example, are you looking to demonstrate level 1 satisfaction of participants undertaking a specific event or level 2 understanding of how the event has influenced their commitment to homework or a particular curriculum subject. (see 4D moving beyond the feedback form) Try to identify the factors that might influence your data collection. Many of these will relate to participant data (see section 6 evaluation practicalities) as well as data collected for research purposes Remember the the desirability versus feasibility continuum referred to by Phil Dent from Aimhigher Coventry and Warwickshire consultancy (see 2a pdf) When deciding on data collection methods think about the: time required to negotiate access to the participants time you have available to collect, process and analyse the data Are there ways in which you can incorporate the collection of evaluation data as part of an activity for example, a focus group discussion at the end of an event, or use of an alternative research method to use data produced during an activity.

The resources in the 'ECB toolkit' are intended to provide some initial starting points for those who are new to gathering quantitative and qualitative data. The information sheets provide details of other resources available and the presentations are designed to act as a prompt for an individual and as a basis for staff development sessions.

Participant Data

Evaluation Methods: Participant Data 7A An information sheet with reference to presentations and reports that discuss the strategies used to collect participant data

Quantitative – questionnaires and surveys

	Evaluation Methods: Asking Questions - Questionnaires 7B (Slides pdf 680kB) (Handouts pdf 215kB) A presentation that outlines points to consider when developing your questionnaires and surveys. Provides an ideal focus for discussion about your questionnaire.
	Evaluation Methods: Questionnaire Layouts 7C (word 75kb) This word document includes a series of tables and different layouts than can be copied and pasted into another document to help with layout of a questionnaire.

Interviews: individual, paired and group

	Evaluation Methods: Interviews 7D (Slides ppt 1MB) (Handouts pdf 210kB) This powerpoint presentation outlines points to help you decide what type of interview to conduct and includes an activity to illustrate how you can use a dialogic interview to help generate statements for a simple survey. It includes a sound file of a short dialogic interview that plays through windows media player.
	Evaluation Methods: Focus Group Interviews 7E (Slides pdf 820kb) (Handouts pdf 245kb) T his powerpoint presentation describes the roles and responsibilities for different participants in a focus group interview. It outlines the steps to follow to organise a successful focus group.
	Evaluation Methods: Telephone Survey and Interviews 7F (pdf 110kb) An information sheet with reference to resources outlining practicalities of undertaking telephone surveys and also practical step by step guide to preparing and undertaking a telephone interview

Using theory

Evaluation Methods: Using Theory 7G (Slides pdf 900kb) (Handouts pdf 435kb) This powerpoint presentation lists some of the reasons for using theoretical frameworks to help make informed decisions about your evaluation. It includes examples from the literature relating to the strategies outlined in the toolkit.

Return to 'Toolkit' Structure: Ten features of evaluation