• Home
• About
• Team Members
• Resource Toolkit
• Contact us

Activity

Information

Presentation

Website

Return to 'Toolkit' Structure: Ten features of evaluation

8 Evaluation Analysis

Having collected all your data and before you are in a position to disseminate your findings, you need to analyse the information you have collected and consider what messages are emerging. It is worth highlighting that the ideas on this page are a basic introduction to the issues of analysis, see 8A Evaluation Analysis: Further reading.

Factors influencing analysis

There are many ways to analyse data and the methods you choose will depend amongst other things on the:

• type of data you collect

  quantitative (largely numerical or answers to closed questions) or;

  qualitative (answers open questions, observations, materials produced during an activity, which may include written, audio, visual, video evidence)

• way you collect and store the data
• experience and preferences of those involved in the data analysis, for instance it maybe quicker and more effective to commission some one from outside with experience of statistics to analyse large quantitative data sets
• time you have available which might be influenced by the numbers of people within your team, if you have commissioned someone else to help with the analysis
• access you have to equipment and software packages for instance, the time required to transcribe an interview will depend on use of a transcribing machine, or the use of software to support the sorting, coding and analysis of the data.

Evaluation Analysis: Further Reading 8A (pdf slides 290kB) This tool includes further details about and links to websites and reading materials relating to data analysis

Things to do The first piece of advice is a ‘don’t’ rather than a ‘to do’! Don’t leave analysis till the end; think about it from the beginning. This involves: Identifying who will contribute to the process of analysing data - WHO Thinking about the data you collect - WHAT Considering how you will store, retrieve and analyse the data - HOW WHO?: Decide who will be involved in which steps of the data analysis process. If you haven’t time, experience, or interest in this aspect of the evaluation process consider working with a consultant to support the process of analysis. However, try to avoid simply handing over this element of the evaluation but discuss practical ways in which you can be involved in contributing to the sense-making process. WHAT?: The approaches used for data analysis will depend on whether it is QUANTITATIVE or QUALITATIVE data. See 8A for more details about the Learnhigher CETL ‘Analyse this!’ that includes a short interactive quiz to aid understanding about the difference. HOW?: Think about how you will collect the data and how you will record and store it for the purposes of data analysis. For instance: quantitative data gathered from paper questionnaires need entering into a spreadsheet; who will have responsibility for data entering? Is it possible for them to submit answers to an electronic survey? developing a coding system for recording answers can speed the process; however, it is important to ensure that you use consistent codes and approaches. For an example and template for setting up basic Excel and SPSS spreadsheets to help record questionnaire data for analysis see Greater Merseyside Aimhigher Evaluation Toolkit Analysing qualitative data also requires preparation; see the Evaluation analysis tools: 8B (using a matrix or template) which is a simple and straightforward method whereby you record the answers from a series of semi-structured interviews, or participants in a focus group discussion 8C (using a thematic approach) whereby you analyse the data using a priori (existing) codes or generate codes using a grounded theory approach based on emerging ideas.

	Evaluation Analysis: Using a Matrix 8B (pdf slides kB) This powerpoint presentation goes through the steps and gives examples of analysing data using a matrix
	Evaluation Analysis: Using a thematic 8C (pdf slides kB) This information sheet works through the steps taken to analyse qualitative data using a thematic approach.

Achieving a balance and finding time

As with all aspects of the evaluation process the challenge is achieving the balance between what is desirable and what is feasible. The aim is to try and make sense of the data. This sense-making (Knight, 2002) process involves you in trying to extract messages from the raw data you collect and represent them in a format that is suitable for decision making and dissemination.

There are advantages and disadvantages with all approaches to data analysis. For example:

• How you analyse the data will depend on what you have collected.
• What is convincing evidence for answering a question such as: ‘who participated in the activity?’ is not going to answer the question, ‘why did they participate in the activity?’

The process of analysis is also the stage at which you will bring the different pieces of the jigsaw (data) together to help you answer your original questions. Remember it is not necessary to evaluate everything. It is important you make clear what you have done and that you do that well. When you have analysed your data it is vital that you do not make claims for your data that you cannot justify.

Although analysis of the data predominantly happens towards the end of the evaluation cycle it is important that you leave enough time to analyse your findings and disseminate them to the appropriate audiences. This needs careful planning and it is sometimes necessary to recognise the need to be selective when collecting your data and focus your analysis and evaluation on answering the questions that will allow you to USE your findings demonstrate your accountability, aid your development or contribute to knowledge according to your original PURPOSE.

Return to 'Toolkit' Structure: Ten features of evaluation