DfE Explore Education Statistics private beta.
Hive IT have successfully undertaken the Explore Education Statistics Private Beta project on behalf of the Department for Education Statistics (DfE). The purpose of the project was to build a data platform for the DfE to publish their statistics. The new platform is now more reliable, more accessible and easier to use. This improves the way the DfE publishes official statistics and empowers statisticians to publish more data and improved commentary with their releases.
- Department for Education
- Who they are
- Responsible for children's services and education
- What they do
- Their vision is to provide world-class education, training and care for everyone, whatever their background.
- UK Wide
In 2018, the Department for Education (DfE) started to think about how they could better provide the statistics and data which they regularly publish for a range of users. Following successful Discovery and Alpha phases with Hive IT, DfE wanted to build a platform that would easily allow users to search for and understand data.
We successfully built a new data platform which is currently in its Public Beta phase. The result is a platform that helps users easily find the statistics they are searching for and presents such statistics in a way that can be easily interpreted. The new platform is accessible for a range of users and helps analysts create and publish their statistics releases more easily.
“It feels like it’s all gone better than I could have dreamed it would go. It’s all just been amazing and the investment from every person in the team has been really special. Everyone seems to care a lot about what we are trying to do.”Laura Selby (Product Owner and Data Analyst at the Department for Education)
We worked with the DfE on both the Discovery and Alpha phases of the Exploring Education Statistics project. Our time working together led to us building a ‘one team’ ethos which really helped us collaborate. We would participate in daily stand-ups with the DfE everyday and this helped our relationship grow stronger over time.
We also had monthly show and tells where we would showcase our most up to date work to the DfE. Our show and tells helped us include and gain more support from a range of people within the DfE.
In Alpha we prototyped and did technical investigations into how we could find the best ways to build the data platform. In this Beta phase we built on top of the most effective prototypes, whilst discarding the less effective ones.
We used Azure, DfE’s main platform and on top of that we also used React and Next.js for our front-end development. Through iteration and testing, we ensured that the website was built so it would remain performant with a large amount of data passing through it. One of the driving factors of the technological choices we made was to make it transferable so that it could be easily supported by the DfE and align with their existing infrastructure and architectural principles.
We built the website so that it provides the ability to rapidly create repeatable infrastructure within the Azure Platform. This aids maintenance, iterative development and will help successfully transition service maintenance to the in-house DfE teams. We also worked with DfE’s IT infrastructure team to ensure everything was approved by their security guidelines and ran IT health checks as part of the assessment process.
Integral to building the new data platform was understanding the needs of end users. We went through an iterative process of constantly improving the beta development via user testing. We conducted user research sessions where a team would observe those testing the site in real time and analyse how they were using the service. This was useful in helping us see where people were struggling with the service or finding it easy and we took notes from this.
We then used the feedback to identify any issues and iterate the development. This quick feedback loop helped us iron out any issues and progress in a timely manner. We learnt from the data we collected from our small user research groups and then opened up to a wider, private Beta. We built the site on the GDS design system where you have all of the proven components.
We invited around 240 people to test our development and the results from this user research allowed us to continue iterating as well as proving the site would work. All the while, technical development was continuously supporting the development of the platform in-line with the feedback from our user testing.
Working with the DfE
In collaboration with the DfE, we worked with the analysts and statisticians that would be using the new system. It is the analysts and statisticians that produce the data that is published onto the site and so it was important to work with them to ensure we had new standards for what the data would look like. This also helped ensure that we would have the data, in a suitable format, available right away to publish to the public Beta site once available.
We were successful in engaging with multiple teams within the DfE who provided feedback that they were pleased to be involved in the user testing process and invited to the show and tells. The teams from the DfE really helped us understand how we could improve the platform and were a pleasure to work with throughout.
We look forward to continuing our work with the DfE to develop the site even further and ensure the site can be easily transferred over and managed by the in-house DfE teams. We also hope to be able to help in taking the site Live before the service is transitioned in house.
We used the following tools and technology to build the platform:
- React, Next.js - Frontend web apps
- .NET - Backend REST APIs
- Robot Framework and Selenium - Automated testing
- C#, TypeScript, Python - Languages
- Azure - Hosting
- Azure DevOps - Continuous integration pipeline
The finished product
The service has been well received, and we helped facilitate cultural change in how DfE manage their production processes and release statistics. The client’s success has been recognised, winning a Government Statistical Service award (2020) award and DfE Innovation Award (2021).
You can find the Explore Education Statistics Service at Explore Education Statistics Service.