https://gds.blog.gov.uk/2018/09/06/second-round-of-govtech-catalyst-challenges-revealed/

Second round of GovTech Catalyst challenges revealed

It’s been almost a year since the GovTech Catalyst was announced - the £20 million fund set up to help private sector innovators tackle public sector problems.

There are 3 rounds of the Catalyst and following the first round earlier this year, the 5 successful challenges have included tackling Daesh online imagery, tracking waste and addressing loneliness and rural isolation.

At the TechUK Smarter State conference today, Minister for Implementation Oliver Dowden announced the next 5 successful challenges from public sector organisations to get funding. These challenges are:

  1. How might we improve firefighter safety and operational response? Submitted by Mid and West Wales Fire and Rescue Service
  2. How might we make better use of data to guide public sector audits? Submitted by Northern Ireland Audit Office
  3. How might we automatically detect and identify illicit goods during the journey across the border without impacting fluidity of trade? Submitted by Home Office, Border Force Detection Services
  4. How might we understand the overlaps between business regulations? Submitted by Department for Business, Energy and Industrial Strategy, Better Regulation Executive
  5. How might we guarantee prescription continuity while people move between care providers? Submitted by Northern Ireland Prison Service & South Eastern Health and Social Care Trust

These 5 challenges will be launched as Small Business Research Initiative competitions for tech companies, with the first competition on keeping emergency responders safe launching mid October.

The full list of all Round 2 submitted challenges both successful and unsuccessful funded can be found on GOV.UK.

Simplifying the application process

We gathered feedback from our users who took part in the Round 1 application process and further simplified it for Round 2 applicants.

For example, it became apparent not all users had access to Microsoft Word - which was initially required to fill out the Expression of Interest form.

So for Round 2 - inspired by the GDS Open Standards Board - we published a list of questions on GOV.UK and asked our users to send us an email answering them. We also designed the questions around the main criteria for the GovTech Catalyst fund.

We removed a question which asked users to describe a solution to their problem, as the aim of the Catalyst is to explore and test multiple approaches to a problem.

And we also scrapped the requirement to upload a signature, as it was not strictly needed.

Making it easier to evaluate challenges

The evaluation of Round 1 challenges was lengthy. The cross-government review board of 12 people scored 51 submitted challenges against 6 categories, 2 of which overlapped. The GovTech Catalyst team then followed up with a round of 1 hour interviews asking a further 14 questions.

The final scores clustered around a very narrow range instead of a naturally forming bell curve, so it was difficult to have a clear idea which challenges met the criteria particularly well and to differentiate between those in the middle.

A graph showing Round 1 challenge scores, which do not fall into a natural bell curve

To simplify this process for Round 2 we formed a smaller, multi-disciplinary assessment panel of 7 specialists from across central government.

We split evaluation into 2 areas: questions that could be answered yes or no, and then just 3 categories of weighted scoring, which put more value on understanding of user needs above over other factors in deciding where to invest public money.

The criteria included: The challenge describes an important public sector problem which could result in a new digital solution from business with a significant improvement in efficiency, policy or cost reduction for the public body? Yes/No . Demonstration of evidenced, clear user needs within the scope of public policy? Maximum points 40. Understanding of the market, including approaches that have been tried in the past, in order to demonstrate the opportunity for innovation? Maximum points 30. Demonstration of an empowered client-side team with sufficient time, money and people available to invest in the solution and plans to procure final product? Maximum points 30.
The criteria used to evaluate the challenges

What we learned

The scores this time were a lot more differentiated and followed the expected normal distribution pattern, making it easier to determine the highest scoring challenges.

A graph showing Round 2 scores, falling more in line with a natural bell curve

We used a mathematical formula to work out how much the evaluation panel disagreed with each other about each challenge submission. This meant we had a much shorter moderation meeting and were able to focus on the challenges that had the largest amount of disagreement.

Incidentally, the overall level of disagreement between panel members was about the same as in Round 1.

The cross-government panel, which included members from the Department for Digital, Culture, Media and Sport, the Department for Business, Energy and Industrial Strategy, Innovate UK and HM Treasury, praised the efficiency of the new evaluation process and said the lessons learned made implementing the second round much smoother.

What’s next

We expect to launch at least 15 competitions to the market. Here's a summary of the current status of the three rounds of competitions.

Round 1

Competition 4 - Using data to better understand and respond to road congestion is open now.

The first GovTech Catalyst contracts are in the final stages of a competitive process to work on innovative solutions to the Daesh imagery challenge.

Round 2

Challenge 1 - How might we improve firefighter safety and situational awareness? This will go out to competition to tech suppliers on 15 October. The other four competitions will opening in November through to February 2019.

Round 3

We expect to open Round 3 - our final call for public sector challenges - this October.

Lisa Scott is the product manager and Breandan Knowlton is a technical architect working on the GovTech Catalyst.

GovTech meetups are held in various locations across the country every month, providing the opportunity to find out the latest updates on the GovTech challenges.

Start-ups are also invited to attend, specifically those working in emerging tech and actively looking to supply to the public sector.

The next meet-ups are 1 November in Newcastle and 22 November in Belfast.

Subscribe for blog updates.

3 comments

  1. Comment by Andy Jones posted on

    Why not create an online service to allow for submissions?

    Reply
    • Replies to Andy Jones>

      Comment by Lisa Scott posted on

      Thanks for the suggestion. We have thought about this. As we have 3 confirmed challenge rounds, the effort to build a digital end-to-end service was too great for the value. Our goal was first to simplify the questions, which we've done, and now we'll fine-tune the submission process. We expect to use an online form (with save and return functionality) to collect challenges for Round 3.

      Reply
  2. Comment by Ian Powling posted on

    Outstanding knowledge sharing. We will explore the application of this method to the evaluation of options for a strategic plan. Thank you, Lisa and Breandan.

    Reply

Leave a comment

We only ask for your email address so we know you're a real person