05 Sep 2024
by Nick Turner, Ali Nicholl

Unlocking the power of Government data: Moving from Siloed information to smart public services

Government agencies are collecting unprecedented volumes of data, yet much of it remains untapped, trapped in silos that prevent effective decision-making and service improvements. This data holds immense potential to transform public services by enabling more accurate, timely insights into service delivery, effectiveness, and efficiency. However, the way data is currently collected, stored, and structured often renders it under-utilised or completely unused.

In this blog, co-authored by Ali Nicholl from IOTICS and Nick Turner from CACI, we explore the critical user requirements for a data-driven smarter state and propose a scalable, federated approach to data discovery, access, and sharing. By enabling real-time data access at the point of need, this approach not only empowers better public services but also provides a coherent AI-ready workflow that leverages existing legacy systems without disruptive centralisation, duplication, or increased complexity.

The Challenge: Making Data Work for Everyone

In today’s environment, where both government and industry are under pressure to do more with less, reduce complexity, and comply with stringent regulations, several pain points persist:

  • Breaking down data silos: Data is often trapped within departmental or organisational boundaries.
  • Improving data quality: Data accuracy and consistency are compromised without a coordinated approach.
  • Addressing data custodianship concerns: Worries around GDPR, security, and data misinterpretation hinder sharing.
  • Ensuring controlled access: Striking the right balance between open access and secure controls.
  • Managing costs: High expenses related to data transit, hosting, and maintenance.
  • Overcoming budget constraints: Investment in new systems while maximising returns on legacy assets.
  • Becoming AI-ready: Adopting new technologies without costly overhauls.

For any system to be truly data-driven, it needs a minimum standard for quality, availability, consistency, and interoperability—without sacrificing security and appropriate access control. It’s the organisations closest to the data sources that have the best insights into managing quality and availability. However, leaving consistency and interoperability solely in the hands of data owners can lead to fragmentation, while expecting any single organisation to manage all data ownership is unrealistic.

The Evolving Solution Space: Technology, Policy, and Attitude Shifts

Recent advances in technology and shifts in policy have begun to address these challenges. Nearly two decades ago, the Reuse of Public Sector Information Regulations (2005) and the Transparency Agenda (2010) laid the foundation for more open attitudes towards data sharing in the UK. The evolution of cloud technology and API-driven architectures has further improved data accessibility by reducing latency and enhancing interoperability.

For example, the UK Transport Sector has effectively used open data APIs to share real-time transport information with developers and service providers, resulting in over 600 apps that benefit millions of Londoners every day. However, while these methodologies improve access, they do not fully solve the “data silo problem”—where data remains fragmented and lacks context, limiting its utility for broader insights.

A Federated Approach: Keeping Data in Place While Maximising Its Value

Our combined experience at IOTICS and CACI has only reinforced how unsustainable current approaches are. A smarter state needs a smarter approach. A federated approach. A federated approach allows data to stay in situ within its existing silos, accessible through a controlled, consistent, and extensible framework. This approach eliminates the need for costly mass data migrations while still unlocking insights at the point of need. Creating a more equitable democratisation of decision-making by ensuring that the right data is available at the right time.

This methodology aligns with how Health Services in the UK have approached data integration in recent initiatives. Within Social Care Networks, for example, connecting existing systems rather than centralising all data has ensured the Healthcare sector maintains flexibility to access relevant information while adhering to security and privacy requirements.

Understanding Stakeholder Needs: Tailoring Solutions for Maximum Impact

Different stakeholders have different goals and challenges when it comes to leveraging data. Here’s how a federated approach such as ours addresses their specific pain points:

  • CIOs need timely, reliable data for informed decision-making. Our solution ensures up-to-date insights without the need for complex data migration, helping CIOs set policies and make strategic decisions with confidence.
  • Heads of Data and CDOs seek to maximise ROI from data assets. We provide enhanced data discoverability and governance, ensuring that those who need access can find and use data efficiently.
  • Service Owners focus on delivering policy or strategic outcomes. Our approach reduces the under-utilisation of data, enabling service improvements without significant operational disruption.
  • Data Analysts require consistent and high-quality data for accurate analysis. By maintaining data integrity and enabling seamless integration across sources, we empower analysts to deliver actionable insights.
  • End Users demand real-time access to relevant data without navigating multiple platforms. Our solution brings data closer to its source, maximising relevance and minimising inconsistency.

Building a Data-Driven Smarter State: The Path Forward

Creating a data-driven smarter state requires lowering the barriers for departments, organisations, and individuals to surface their data and enrich it with context, turning it into actionable insights. A federated approach represents a scalable, flexible, and low-risk path towards unlocking the full potential of government data. The journey from siloed information to integrated insight is not just about technology; it’s about creating an ecosystem where data flows seamlessly, fostering collaboration, innovation, and smarter decision-making across the public sector.

To build this future, we must prioritise accessible, context-rich data and scalable collaboration across stakeholders. The smarter state of tomorrow is within reach if we embrace these principles today.


 

Heather Cover-Kus

Heather Cover-Kus

Head of Central Government Programme, techUK

Heather is Head of Central Government Programme at techUK, working to represent the supplier community of tech products and services to Central Government.

Prior to joining techUK in April 2022, Heather worked in the Economic Policy and Small States Section at the Commonwealth Secretariat.  She led the organisation’s FinTech programme and worked to create an enabling environment for developing countries to take advantage of the socio-economic benefits of FinTech.

Before moving to the UK, Heather worked at the Office of the Prime Minister of The Bahamas and the Central Bank of The Bahamas.

Heather holds a Graduate Diploma in Law from BPP, a Masters in Public Administration (MPA) from LSE, and a BA in Economics and Sociology from Macalester College.

Email:
[email protected]
LinkedIn:
https://www.linkedin.com/in/heather-cover-kus-ba636538

Read lessmore

Ellie Huckle

Ellie Huckle

Programme Manager, Central Government, techUK

Ellie joined techUK in March 2018 as a Programme Assistant to the Public Sector team and now works as a Programme Manager for the Central Government Programme.

The programme represents the supplier community of technology products and services in Central Government – in summary working to make Government a more informed buyer, increasing supplier visibility in order to improve their chances of supplying to Government Departments, and fostering better engagement between the public sector and industry. To find out more about what we do, how we do this and how you can get involved – make sure to get in touch!

Prior to joining techUK, Ellie completed Sixth Form in June 2015 and went on to work in Waitrose, moved on swiftly to walking dogs and finally, got an office job working for a small local business in North London, where she lives with her family and their two Bengal cats Kai and Nova.

When she isn’t working Ellie likes to spend time with her family and friends, her cats, and enjoys volunteering for diabetes charities. She has a keen interest in writing, escaping with a good book and expanding her knowledge watching far too many quiz shows!

Email:
[email protected]
Phone:
020 7331 2015
Twitter:
@techUK,@techUK
Website:
www.techuk.org,www.techuk.org
LinkedIn:
https://bit.ly/3mtQ7Jx,https://bit.ly/3mtQ7Jx

Read lessmore

Austin Earl

Austin Earl

Programme Manager, Central Government, techUK

Austin joined techUK’s Central Government team in March 2024 to launch a workstream within Education and EdTech.

With a career spanning technology, policy, media, events and comms, Austin has worked with technology communities, as well as policy leaders and practitioners in Education, Central and Local Government and the NHS.

Cutting his teeth working for Skills Matter, London’s developer community hub, Austin then moved to GovNet Communications where he launched Blockchain Live and the Cyber Security and Data Protection Summit. For the last 3 years he has worked with leaders in Education across the state and independent schools sectors, from primary up to higher education, with a strong research interest in technology and education management.

Email:
[email protected]
Phone:
07891 743 932
Website:
www.techuk.org,www.techuk.org
LinkedIn:
https://www.linkedin.com/in/austin-spencer-earl/,https://www.linkedin.com/in/austin-spencer-earl/

Read lessmore

Ella Gago-Brookes

Ella Gago-Brookes

Team Assistant, Markets, techUK

Ella joined techUK in November 2023 as a Markets Team Assistant, supporting the Justice and Emergency Services, Central Government and Financial Services Programmes.  

Before joining the team, she was working at the Magistrates' Courts in legal administration and graduated from the University of Liverpool in 2022.  Ella attained an undergraduate degree in History and Politics, and a master's degree in International Relations and Security Studies, with a particular interest in studying asylum rights and gendered violence.  

In her spare time she enjoys going to the gym, watching true crime documentaries, travelling, and making her best attempts to become a better cook.  

Email:
[email protected]

Read lessmore

Annie Collings

Annie Collings

Programme Manager, Cyber Security and Central Government, techUK

Annie joined techUK as the Programme Manager for Cyber Security and Central Government in September 2023. In this role, she supports the Cyber Security SME Forum, engaging regularly with key government and industry stakeholders to advance the growth and development of SMEs in the cyber sector.

Before joining techUK, Annie was an Account Manager at a specialist healthcare agency, where she provided public affairs support to a wide range of medical technology clients. She also gained experience as an intern in both an MP’s constituency office and with the Association of Independent Professionals and the Self-Employed. Annie holds a degree in International Relations from Nottingham Trent University.

Email:
[email protected]
Twitter:
anniecollings24
LinkedIn:
https://www.linkedin.com/in/annie-collings-270150158/

Read lessmore

Central Government updates

Sign-up to get the latest updates and opportunities from our Central Government programme.

 


 

 

 

Authors

Nick Turner

Nick Turner

Head of MooD, CACI

Ali Nicholl

Ali Nicholl

Founding Team, IOTICS