The apprenticeship service passed its private beta assessment in December.
I’m Vanessa, Digital User Experience Manager at the Skills Funding Agency (SFA). I’d like to share how we worked with GDS to get the apprenticeship service to this stage and how we’ll incorporate feedback in public beta.
The apprenticeship service
Apprenticeships raise skill levels and deliver strong returns for the UK economy.
The apprenticeship reform programme makes apprenticeships more rigorous and responsive to the needs of employers. A key part of the reforms is the upcoming introduction of the apprenticeship levy in April 2017.
The levy will take effect on 6 April. It requires all employers operating in the UK, with an annual pay bill of more than £3 million, to invest in apprenticeships.
To handle that change, the SFA is developing a service to help employers manage apprenticeships. We started to build the service in 2015 with just 4 people. Today there are 100 people working on it.
Some elements of the service, such as Find apprenticeship training and Recruit an apprentice, have been delivered. The biggest change will be when a management tool for the levy goes live.
Apprenticeship service private beta
There was a risk that different elements of the apprenticeship service would be siloed. To prevent this, we ran a private beta across the whole service, creating a single user journey for the employers we were testing with.
There were several challenges we needed to overcome:
Selecting employers
Choosing 100 employers with a good cross-section of characteristics was a complicated process. We had to make sure all sizes and types of employers had their say.
Testing legislation that is not yet active
We wanted to test how the flow of data and funds would work, so we simulated the process. Since employers won’t be paying the apprenticeship levy until April 2017, we had no data to use. To test accurately, we created accounts containing simulated funds for the employer to spend. We collected data from employers, which allowed us to estimate the amount of levy they would pay, and gave them an appropriate amount to test with. This created a lot of manual work, but the results were realistic. We tested how the interfaces and processes will function in detail.
Testing behind-the-scenes processes
To manage the flow of funds, there are complicated processes that need to work in the background. We didn’t want to test the interfaces independently of these processes. We worked with our internal data teams to simulate month-ends, so we could show payments entering and leaving employers’ accounts, and test the reports that were then generated. We gave employers specific tasks to do. Co-ordinating them was no small task, so we took on a research assistant, Laura. Laura is an ex-apprentice who helped to manage the work with employers, then collated and analysed their responses.
How did we test?
We tested in several ways to get a rounded set of results. During each round of testing, 15 employers were picked for qualitative face-to-face sessions. We also conducted around 20 moderated sessions, usually over the phone. We sent out around 15 detailed diaries for participants to keep as they worked through the service. Everyone else was sent a survey to complete. This gave us a mix of qualitative and quantitative data to analyse during each round of testing.
Our assessment
Our private beta assessment was run by GDS. Unusually, our assessment happened before our private beta finished and some time before our service was due to go live. With a ministerial deadline looming, we wanted to ensure there was time to take in panel recommendations.
Preparation was essential. We shared our research with the GDS user researcher who would be on our panel. We chatted through the information he’d need for the assessment, which we hadn’t done for the previous one. That was really helpful. It meant he had enough background information before the assessment and we could focus on assessment-specific questions.
Public beta
We have already used the private beta feedback to make some important changes to the service. For example, we learned a lot about what our users need to understand before they register for the service, such as who will be on their apprenticeship team and where to find their log-in details. This has helped us to develop content to guide them through the process.
The service is now in public beta and selected employers are being invited to register in phases from this week.
The apprenticeship service is due to go live by May 2017. In the meantime, you can follow our progress on the SFA Digital blog.
Follow GDS on Twitter, and don’t forget to sign up for email alerts.
