In 2015, the Companies House service (CHS) was launched to provide the public with free and easy access to company data.
At the same time, the application programme interface (API) was also released. And this sits on the same architecture, providing a simple and consistent approach for application developers to read the data directly from our service using their own software.
We predicted an increase in data consumption when we launched the service. We were introducing a modern service that placed the user at the heart of the design, and also removing the previous payment barrier. That said, when the service was launched, the demand surprised even us.
Growing demand for data
In the months following the launch, the data request volumes reached daily averages of 25 million data requests per day, peaking at 30 million on some days.
This volume did even out over time, but the overall result was an increase from 300 million data requests per year in 2014/15 to 6.5 billion per year in 2018/19. Fortunately, we had deployed the service to public cloud so could cope with the increase.
Since CHS was launched we’ve undertaken lots of investigation into customer usage patterns and behaviour. We've also collected feedback which has enabled us to continually iterate the service design and rapidly respond to user need.
For developers, we provide a forum where they can talk to us and others about creating services using our APIs. From the user research, we know that the newest register data is often the most valued and therefore the most in demand.
A legacy problem
Despite the availability of our new services, there remained the demand for legacy bulk products which provide a snapshot of register data, including new data added to the register. We noticed that when these bulk products were output, a peak of traffic would hit CHS shortly afterwards, taking the data items included in the bulk product and other associated data.
Users were taking the legacy products to find out what new company information was available and then switching to CHS to retrieve the data.
With lots of data constantly changing on the register, this was the best avenue users had to identify the new data items. The downside of this method was that changes were difficult to identify in the large volumes of bulk data, plus the time lag meant the information was often superseded by new data that users were unaware of.
A transformational solution
To address this problem, and as part of our ongoing transformation of services, we’ve launched a streaming API. And we believe we’re the first in government to do so. Our digital team has been working hard to launch this new service in collaboration with colleagues from across the organisation.
The new streaming API is designed to help our users get the best out of the data we hold, by accessing real-time information about companies in a quick and easy way.
The streaming API allows users to tap into a specific ‘stream’ of data from the register and immediately know about any updates. The service pushes information to users as it changes, through a constantly running connection.
There are 4 different types of information customers can access currently:
- company information
- filing history
- charges (mortgages)
- insolvency cases
This service will be hugely beneficial to users who rely on our data, especially those who request the data in bulk. The streaming API will also be an alternative method for those users who currently trawl CHS for new data, and this will reduce the load on the web service.
After being tested with a small group of users for 6 months, the new streaming API service is up and running. If you want information about developing against the streaming API, take a look on the Companies House Streaming API developer pages.
To keep in touch, sign up to email updates from this blog or follow us on Twitter.
2 comments
Comment by ebrahim ebrahim posted on
Thanks.
Comment by Thanawat posted on
Wow, this is well explained.