Guest blog: (Tata Consultancy Services): a virtuous circle - using AI to help unlock the data needed to leverage AI
In early 2024, the TCS Thought Leadership Institute conducted a comprehensive global survey on the state of AI adoption and its impact on organisations across various industries. The study surveyed nearly 1,300 CEOs and Senior Executives spanning 12 industries and 24 countries around the globe. The results show the majority of senior executives are optimistic about AI’s potential impact on their business and 94% have active plans or have deployed it already in their organisation. However, our survey found that the number 1 challenge that companies face when integrating AI into their ways of working is their current IT Infrastructure and 54% admit that the primary barrier is the quality and availability of enterprise data.
Without high-quality, reliable data powered by adequate computational resources and processing, even the most advanced AI algorithms will struggle to deliver meaningful results. Organisations that want to succeed with AI must invest in building robust data infrastructure and governance processes that ensure data is accurate, consistent, and accessible across the organisation.
The work of the data office ranges from data management, data quality, data cataloguing, and data governance. Almost half of businesses (46%) have struggled to recruit for roles that require data skills based on research commissioned by the Department for Digital, Culture, Media and Sport.*
Can AI help unlock the data needed to leverage AI? Let’s explore this virtuous cycle.
At an Enterprise level, readying data for AI starts with governance, improving data quality, integrating, cleaning, formatting and augmenting data. This level of robust data management is what is needed to go beyond pilots and prototypes where organisations are experimenting with small use cases. To go big, you need to…
Implement a strong data governance framework so that data used to inform AI meets high standards. You need processes, controls and accountability to govern privacy, security and responsible AI following principles such as fairness, inclusiveness and transparency. Data governance helps protect your data privacy as well as avoid unreliable AI outcomes.
Improve data quality with tools and techniques such as data profiling, data cleansing, data validation and monitoring. Monitor your data pipelines for quality by identifying anomalies, changes and errors.
Centralise data for a consistent single source of the truth by integrating data into a unified view or repository. Use integration tools to load disparate data sources into a data lake or data warehouse.
Clean and format data - prepare data for training models by enriching with additional attributes. Classify, segment and label data with annotations or tags to make it understandable and useable for AI algorithms.
Augment data to reduce bias and improve data generalisation using methods such as data generation, data manipulation, data transformation and data synthesis.
HOW AI CAN HELP
AI is transforming Data Management with emerging tools and methods that apply AI to the day-to-day challenges of data management, including data quality, cataloguing and governance.
Data Governance - Use AI instead of a team of people for:
- real-time monitoring and automated compliance checks
- automated identification and protection of sensitive data
- applying encryption and access controls
- proactively predicting and managing risks through predictive analytics
Data Quality - AI models (regression, clustering) can do the imputation of missing data and the correction of anomalies. Computer vision and natural language processing algorithms can also tap into original data sources, such as PDF and word files, extract data and validate what has been typed in the system. Machine learning models can support the deduplication of data, estimating the probability that two records refer to the same entity.
Data Integration - AI algorithms can help map and reconcile different data structures, identify relationships between datasets, and create a common data model. AI can automatically transform data into the right format, apply calculations, and aggregate data. AI can continuously monitor data sources and trigger data ingestion and processing when changes are detected.
Format data to create easily searchable assets with AI powered classifiers that are pre-trained and tested on domain specific knowledge. Tagging with AI can be zero-shot (without any provided examples) or few-shot (with few provided examples) and can save laborious time preparing data for analysis.
Augment Data with deep learning models to generate new data points at speed with more detail, depth and complexity than is possible by obtaining the data from the real world. This leads to more complete data before modelling and analysis is carried out.
CONCLUSION
We need to remember that the primary value creating output of the data office is analytics and insight. I believe AI can enable organisations to do more of this and less of the drudgery related to it. Get started by reviewing appropriate AI tools that might help your Data Office against your strategic imperatives and select the ones that best align with your needs.
*Quantifying the UK Data Skills Gap - Full report - GOV.UK
techUK - Putting AI into Action
techUK’s Putting AI into Action campaign serves as a one stop shop for showcasing the opportunities and benefits of AI adoption across sectors and markets.
During this campaign, techUK will run a regular drumbeat of activity, including events, reports, and insights, to demonstrate some of the most significant opportunities for AI adoption in 2025, as well as working with key stakeholders to identify and address current barriers to adoption.
Visit our AI Adoption Hub to learn more, or find our latest activity below.
Upcoming AI Adoption events
Latest news and insights
Get our tech and innovation insights straight to your inbox
Sign-up to get the latest updates and opportunities from our Technology and Innovation and AI programmes.
Contact the team
Usman Ikhlaq
Usman Ikhlaq
Usman joined techUK in January 2024 as Programme Manager for Artificial Intelligence.
His role is to help techUK members of all sizes and across all sectors to adopt AI at scale. This includes identifying the barriers to adoption, considering solutions and how best to maximise AI's potential.
Prior to joining techUK, Usman worked as a policy, government affairs and public affairs professional in the advertising sector. He has also worked in sales and marketing and FinTech.
Usman is a graduate of the London School of Economics and Political Science (MSc), BPP Law School (GDL and LLB) and Queen Mary University of London (BA).
When he isn’t working, Usman enjoys spending time with his family and friends. He also has a keen interest in running, reading and travelling.
Learn more about our AI Adoption campaign:
Authors
Verity Greig
I’m a Leader who believes that technology can be a force for good for Public Services. As a Cloud Evangelist, I have the privilege of leading TCS' cloud strategy for the UK Public Sector. Recently TCS was named the Number One IT Services Provider in the UK by TechMarketView. A moment of pride for us but also a moment of obligation. Prior to that I was an Azure Product Manager at Microsoft tailoring the Azure service to meet Public Sector needs. As the mum of two daughters, I’m particularly passionate about inspiring more women into STEM industries.