Automation is gradually being woven into the fabric of public services at both local and national levels.
From administrative decision-making to various assessment functions, automation promises a future where efficiency, cost savings, and enhanced service quality are the norm.
Public-facing systems and behind-the-scenes automated tools aim to streamline operations, freeing up valuable staff time.
But automation isn’t all rosy, and we shouldn’t be blindsided by potential benefits – as they certainly do not guarantee success. Public sector organisations must adopt a discerning and well-informed approach to avoid being misled by overhyped claims that it’s the saviour.
And why? Because the consequences of flawed automated systems can be severe, affecting both citizens and the public bodies responsible for their implementation.
The scope of automation
A myriad of functions are currently being automated across the public sector, illustrating the diverse applications and potential of these technologies.
Local Authorities: Experimentation with chatbots, automation in social care assessments, and data analytics for identifying at-risk families.
GP Practices: AI-powered triaging and appointment management trials.
NHS: Development of AI diagnostic tools to enhance the speed and accuracy of medical diagnoses.
Department for Work and Pensions (DWP): Use of automated systems to detect fraudulent claimants.
Home Office: Implementation of automated decision-making for identifying modern slavery victims and processing immigration decisions.
But despite the extensive application, it is crucial to recognise that not all populations are equally affected by automated decisions. Marginalised communities, those with uncertain immigration status, or those heavily reliant on state support are much more likely to experience the impact of these systems – and not always in a good way. The potential for bias and privacy violations in such systems can lead to significant, life-changing consequences.
Risks of public service automation
It starts with looking at effectiveness. The primary question is whether these systems achieve their intended goals. While technology companies often present compelling arguments for the efficacy of their solutions, it is essential for public bodies to critically assess these claims.
Real-world examples show that not all automated systems deliver as promised. For instance, Hackney Council and Bristol’s risk-based verification (RBV) system both halted their programmes due to inefficacy and adverse impacts.
The discussion around bias in technology isn’t a new one, but it’s one we should keep having. Automated systems can perpetuate existing biases present in their training data. Predictive policing tools and the A Level algorithm of 2020 are notable examples where biases led to unfair outcomes. West Midlands Police and the Home Office have both discontinued certain automated systems due to these concerns.
And what of privacy and data protection? Automated systems often require vast amounts of data, raising questions about data sources and citizens’ consent. The controversy over the Data Protection and Digital Information Bill highlights the tension between data usage and privacy rights.
We know that control is an issue. Many automated systems are developed by private companies, resulting in limited transparency. Public sector bodies sometimes lack detailed knowledge of how these systems work, which complicates accountability and citizen recourse. The Public Law Project’s register of automated systems underscores the pervasive opacity in this domain.
Trust is paramount. Without it, citizens are likely to resist data collection and the use of automated systems. The NHS’s digital transformation efforts and the Post Office/Horizon scandal exemplify the challenges in maintaining public confidence in technology.
Why Labour government must create AI-only specialist university
Guiding principles for automation
To harness the benefits of automation while mitigating risks, public bodies should adhere to the following principles. It’s critical to have a framework to know how best to make the most of new opportunities that technology brings.
Interrogate the need – examine the underlying reasons for automation. Ensure that it does not replicate or exacerbate existing issues.
Impact vs. risk – focus on areas where automation can deliver significant benefits with minimal risks.
Establish ‘red lines’ – identify decisions too risky for automation due to their potential impact on individuals’ lives.
Thorough safeguarding and evaluation – implement consistent impact evaluations and empower authorities to modify harmful systems.
Transparency – make the use and mechanics of automated systems public and ensure comprehensive impact assessments.
Upskill staff– equip public sector employees with the knowledge to understand and assess automated systems.
Citizen involvement – engage citizens and civil society groups in decision-making and monitoring of automated systems, building trust and adoption
We know that the path to successful automation in public services is fraught with challenges. But it can be a force for good. By adopting a careful, principled approach, public bodies can leverage automation’s potential benefits without compromising citizens’ rights and trust – resulting in success for all.