The Brief
Adult education can improve individuals’ quality of life, self-confidence and opportunities, as well as providing employers with the skills they need for growth. The Adult Education Budget (AEB) funds training to help people in the West Midlands find jobs and to upskill those already in work.
Winning Moves was commissioned by the West Midlands Combined Authority (WMCA) to evaluate the impact of the Adult Education Budget (AEB) devolution and associated activities in each of the first four years of the programme, including assessment of the contribution that the AEB has made or will make to WMCA’s ambitions, as set out in the Regional Skills Plan and Local Industrial Strategy.
Each academic year, Winning Moves provide the West Midlands Combined Authority (WMCA) with an evaluation of their devolved Adult Education Budget (AEB) programme. Each year, the interim evaluation that has two main objectives:
- To identify, understand and critically review the processes that WMCA have implemented to support the management, delivery, and monitoring of AEB during the year
- To complete an early-stage assessment of programme impacts which WMCA can use to determine the ‘direction of travel’ and identify any immediate issues that could influence future achievement of programme priorities.
The final evaluation involves assessment of programme performance and achievement against a range of outcomes and impacts, which include developing responsive provision to meet the needs of residents and employers, increases in Level 3 provision, and jobs and wider outcomes for AEB learners.
The evaluation also requires assessment of a range of wider benefits, including but not limited to relationships between WMCA and AEB providers, learner progress to positive destinations – further education or sustainable employment, improved relationships and collaboration between WMCA, colleges and Independent Training Providers (ITPs) and enhanced skills and personal development opportunities for people currently employed in key sectors.
The solution
Winning Moves undertake a range of primary and secondary research and analysis to evaluate AEB, adapting the approach over the course of the four-year contract to reflect changes in the programme and WMCA’s evolving evidence needs and priorities.
The approach includes:
- Interviews with WMCA and AEB programme staff
- Interviews with training providers, including a mix of colleges, local authorities, and ITPs
- Interviews with wider stakeholders
- Review of data from WMCA’s monitoring dashboard
- Review of Individual Learner Records
- Annual surveys of AEB learners, administered by WMCA
- Non-learner surveys, designed and implemented by Winning Moves to provide a comparison group
- Quasi-Experimental Analysis (QEA) with Propensity Score Matching, to estimate the impact of the AEB programme / course provision by comparing outcomes for learners and non-learners with similar characteristics
- Drawing comparisons with regional and national data to consider the results in the context of measurable macro-level indicators, such as employment and unemployment rates, qualifications (highest level obtained), economic growth, number of hard-to-fill vacancies, and the proportion of people in low paid employment.
The outcome
The evaluation synthesises a wide range of quantitative and qualitative evidence to provide a comprehensive assessment of the programme, including robust assessment of impacts via the QEA comparison of learners and non-learners.
The evaluation also provides rich insights throughout to inform the further development and refinement of the AEB, including a range of observations and learnings to inform future adult education provision in the West Midlands.