We have written previously about how our outcomes assessment system, Stedfast, is able to support education and training providers in measuring programme performance. We wanted to continue this blog post series to delve deeper into measuring each standard's outcomes.
Levels of Outcomes Measurement
Most organisations will have a series of outcomes that they work towards. Typically this will be:
The process for measuring outcomes applies to any level of the organisation – it does not matter if you are reviewing performance at the organisation or programme level, you will still have the same considerations:
What is useful with apprenticeship standards is that some of the work has already been done for providers. All apprenticeship standards have three key areas of delivery:
Mission, Goals, Outcomes
When conducting a programme review it is worth considering some standard headings:
Mission – This provides a holistic vision of what you are setting out to deliver and what values you are working towards.
Goals – This should be a general statement about the overall skills, knowledge and behaviours expected by learners on a particular programme.
Outcomes – These are the outcomes contained in apprenticeship standards
Providers might also want to consider adding a statement about endpoint assessment, which may demand more than just achieving the learning outcomes listed in the standard.
Once you have an assessment plan in place then, the next consideration is how to measure performance against the outcomes.
Two types of evidence need to be considered:
Direct Evidence – This should focus on learner achievement and is easily measurable. For instance:
Indirect Evidence – This is typically evidence such as:
Also, consider whether the evidence being collected is:
Formative – evidence gathered during the programme
Summative – evidence gathered at the end of a programme
Findings and Gradings
Once you have identified how you intend to measure each outcome and have distributed this review, the next step is to add findings and grades. Adding findings is quite straightforward. The reviewer will have the expected outcome and the measure and can, therefore, make a judgment based on either direct or indirect evidence. Scoring or grading then tends to be a problem as without structure, this tends to be left to how the reviewer feels.
We recommend that a scoring matrix is used to provide guidance to reviewers. Each outcome should have scoring guidance for reviewers.
Review and then review again
Once a programme review has been conducted, and actions have been identified, staff need to review and review again to ensure that findings and changes are being implemented. There will be a host of findings, such as:
Additional Employer engagement
Individual learning plans
Changes to the curriculum
But, the key message is that it needs to be constantly reviewed.