Predictive Policing Failed Quietly. The Model Is Still Expanding.
WIRED reported that Avon and Somerset Police and Bristol City Council developed the Think Family Database between 2016 and 2023 — a predictive policing system that collected sensitive data on nearly half a million residents, combining police reports, mental health records, school data, and other information into a risk-scoring platform. More than 23 predictive models were built, focused on areas including burglary, domestic abuse, and child exploitation. Then audits found that some of the models were not working as advertised. At least two models designed to predict child sexual and criminal exploitation were quietly discontinued after council staff lost confidence in their outputs, while several other models reviewed by the Eticas research group showed precision rates below 10 percent.
The system did not fail dramatically. It failed quietly, and the failure is instructive about what happens when public institutions adopt algorithmic tools before the evidence base, the transparency infrastructure, and the accountability mechanisms are in place to support them. Residents did not know their data was being used to generate risk scores. The public did not have meaningful access to the models’ logic. When the outputs proved unreliable, the response was to discontinue specific models rather than to account publicly for what the data had been used to produce and why the systems failed.
Predictive policing tools operate on data generated by existing policing systems. Those systems already reflect decades of unequal surveillance, unequal enforcement, and unequal institutional attention across communities. A model trained on that data does not correct for those inequalities — it encodes them and then presents the result as a technical output, which carries a different social weight than a human judgment call. The appearance of objectivity is part of the problem, not a feature.
The UK case is not an anomaly. It is a pattern: public agencies adopt AI systems that can shape consequential decisions, embed them into institutional practice, and then assess the consequences — or quietly discontinue the models — after affected communities have already been living inside the outputs. When government uses AI to predict risk, the public is owed a clear account of what data is being used, how the model actually performs, who bears the cost of errors, and whether the system is working well enough to justify operating at all.
