Using Sentiment Analysis and Topic Modelling to Enhance Citizen Satisfaction in Local Public Services (Guest blog by Purple Beard)
Harnessing Data to Improve Public Services
In the digital era, the global data sphere is projected to reach a staggering 175 zettabytes by 2025. A significant portion of this data, such as open-ended feedback, remains untapped, especially in public services where citizen satisfaction is closely tied to service quality. Understanding public sentiment is, therefore, crucial for effective governance [1][2] .
After analysing council tax collection rates from LG Inform across UK councils from 2016 to 2021/22, they identified councils with lower average collection rates, suggesting potential dissatisfaction among citizens.
Figure 1: Council tax collection rate
A two-stage analysis as proposed on citizen reviews for these councils:
- Sentiment Analysis to understand negative sentiments.
- Topic Modelling to identify prevalent themes in these reviews.
For this demonstration, they used Google review data, but data from other platforms like Twitter, Facebook can be integrated as well.
Decoding Citizen Emotions
Sentiment Analysis
Sentiment analysis allows us to categorize the emotional tone of the reviews into three primary categories: Positive, Negative, and Neutral. Various approaches to sentiment analysis exist, including rule-based methods, which use predefined rules and linguistic patterns, lexicon-based methods, machine learning-based methods, and aspect-based sentiment analysis.
In this context, two types of sentiment analysis could be especially insightful:
- Temporal Sentiment Analysis: This approach allows us to visualise how sentiments change over time, giving us a sense of the evolving citizen sentiment towards local council services.
- Aspect-Based Sentiment Analysis: This approach focuses on analysing and extracting sentiment for specific aspects within the feedback. It involves techniques such as parts of speech tagging to extract noun or noun-adjective sequences, which can provide a more granular understanding of the sentiment related to specific elements of the council services.
Interestingly, researchers have also explored combining topic modelling and aspect term extraction to further enhance the extraction of aspects from text data, underscoring the adaptability and potential of these techniques. [3]
Topic Modelling with BERTopic
To extract common themes or 'topics' from negative sentiment review documents, they employed BERTopic, a Python-based topic modelling technique. However, traditionally K-means clustering and Latent Dirichlet Allocation (LDA) for topic modelling have been widely used.
While the two are proven methodologies, they make certain assumptions about the data. Primarily with K-means, assumption is that clusters are spherical and evenly sized. While HDBSCAN supported by BERTopic allows for more complex shape and size as it uses a density-based approach to identify clusters. In real-world data, these assumptions may not always hold. LDA requires fine tuning of hyperparameters and often produces soft overlapping clusters. BERTopic, on the other hand, automatically finds number of topics supports embedding so no prior pre-processing needed, is highly modular, supports hierarchical topic reduction.
A comparison study between LDA, NMF, and Top2Vec and BERTopic can be found here.
Figure 2: Process diagram
Findings and Recommendations for Better Service Delivery
After following the steps discussed above, we can find topic clusters for each council as well as reviews belonging to each topic. We are also able to understand subtopics within topics using hierarchical topic visualisation methods.
Figure 3: Topic clusters for Hackney council.
Figure 4: Reviews within each topic clusters for Blackpool council.
Figure 5: Reviews within topic clusters suggests complaints related to phone service for Liverpool city council.
In conclusion, this method not only helps identify areas of dissatisfaction but also brings forth the underlying reasons, enabling councils to address these issues proactively thus enabling them to transform feedback into actionable insights.
References:
- Muktafin, E. H., Pramono, & Kusrini. (2021). Sentiments analysis of customer satisfaction in public services using K-nearest neighbors algorithm and natural language processing approach. TELKOMNIKA Telecommunication, Computing, Electronics and Control, 19(1), 146-154. doi:10.12928/TELKOMNIKA.v19i1.17417
- Kowalski, Radoslaw & Esteve, Marc & Mikhaylov, Slava. (2020). Improving Public Services by Mining Citizen Feedback: An Application of Natural Language Processing. Public Administration. 98. 10.1111/padm.12656.
- Ozyurt, Baris & Akcayol, M. (2020). A new topic modeling based approach for aspect extraction in aspect-based sentiment analysis: SS-LDA. Expert Systems with Applications. 168. 114231. 10.1016/j.eswa.2020.114231.
Upcoming Local Public Services events
Insight NWT Simplifying the Complex - Accelerating the Public Sector roadmap for digital and data
techUK, 10 St Bride Street, London, EC4A4AD
Local Public Services updates
Sign-up to get the latest updates and opportunities from our Local Public Services programme.
Authors
Kapil Parekh
This article was written by Kapil Parekh, Data Scientist and a Skills Bootcamp Trainer at Purple Beard Ltd. With expertise in machine learning, natural language processing, and cloud computing. Kapil has successfully improved business efficiency through automation and data driven insights. He has over 500+ hours of teaching experience in Data Analysis, coached more than 200 students to develop tech-talents throughout the year. He enjoys building models for algorithmic trading and loves watching cricket!
