What happened
Police forces in England and Wales have been told to pause some uses of in criminal justice work, specifically including the preparation of court statements. This intervention follows concerns raised by the Police.AI centre that commercial AI systems are being deployed in situations where accuracy, provenance, and accountability are critical. The decision comes amid specific incidents, such as West Midlands Police using Microsoft Copilot to prepare material for a proposed ban on Maccabi Tel Aviv supporters, where the AI fabricated a reference to a match that had never taken place. Additionally, a Derbyshire detective is under criminal investigation for allegedly using AI to create evidential material, leading to the review of several rape convictions.
Police forces in England and Wales have received instructions to pause specific applications of in criminal justice operations. This includes halting the use of AI for preparing court statements, a move driven by concerns that commercial systems are being used in contexts requiring strict accuracy and provenance. Alex Murray, head of the Police.AI centre, emphasized the need for forces to slow down and ensure proper assessment of AI systems before their deployment in such sensitive areas.
The directive follows concrete examples of AI failure in policing. West Midlands Police used Microsoft Copilot to prepare material for a proposed ban on Maccabi Tel Aviv supporters attending a match against Aston Villa in November 2025. The AI output referenced disorder at a previous match between the two clubs, but that match had never occurred. This incident demonstrated how AI-generated information can appear credible even when entirely fabricated, posing a risk to the integrity of investigations.
More serious allegations have emerged in Derbyshire, where a police detective has been suspended and is under criminal investigation. The allegations state that the detective used an AI chatbot to produce case paperwork in ways intended to influence charging decisions. As a result, several rape convictions are currently being reviewed. The investigation is ongoing, and no charges have been brought yet, but the case illustrates the potential for AI-generated material to influence legal decisions before errors are detected.
The core issue is not just factual error but the lack of provenance. In criminal justice, evidence must have a traceable origin and a verifiable basis for accuracy. Large language models generate responses based on patterns rather than truth verification, and their explanations of how they reached an answer are often themselves generated, not reliable audit trails. This complicates legal challenges, as defense lawyers must now question whether AI was used, what inputs were provided, and if all claims were checked against original evidence.
Source details: nation.cymru ↗
Why it matters
This pause highlights a critical tension in the adoption of AI within high-stakes legal and law enforcement environments. While AI offers efficiency in administrative tasks like case preparation, its tendency to generate plausible but false information poses severe risks to the integrity of criminal proceedings. The incident underscores that human oversight cannot merely be nominal; it requires rigorous verification of AI-generated content against original evidence to ensure provenance and accountability. This development signals a broader industry shift toward stricter governance of AI in sectors where errors can impact individual liberty and legal outcomes.
The pause reflects a recognition that the efficiency gains from AI in administrative tasks like case preparation are outweighed by the risks to legal integrity if errors are not caught. In criminal cases, the stakes include individual liberty, making the requirement for human verification not just a quality control measure but a fundamental accountability safeguard.
This situation highlights the limitations of '' oversight if it is merely nominal. The source argues that a person reading AI output and deeming it 'reasonable' is not the same as verifying it against underlying evidence. True oversight requires establishing that the information is accurate, traceable, and fit for use, which may reduce the speed benefits of AI adoption.
The incident serves as a cautionary example for other sectors adopting AI in high-stakes decision-making. It underscores the need for robust governance frameworks that prioritize provenance and accountability over speed, particularly in areas where AI errors can lead to wrongful convictions or compromised investigations.
Interactive Mechanism: How It Actually Works
Explore the underlying technology behind this development interactively.
Why can ethical evaluation not be reduced to one model score?
What to watch next
Monitor the implementation of the Police.AI tools planned for pilots in 2026 and national rollout in 2027, specifically how they incorporate quality-checking mechanisms. Watch for the outcome of the criminal investigation into the Derbyshire detective and the subsequent review of related rape convictions. Additionally, observe whether other law enforcement agencies in the UK or internationally adopt similar pauses or stricter verification protocols for AI-generated evidence.
The Police.AI centre is developing tools to prepare and quality-check case files, with pilots beginning in 2026 and a national rollout planned for 2027. The effectiveness of these tools in preventing the types of errors seen in the West Midlands and Derbyshire cases will be a key indicator of whether AI can be safely integrated into criminal justice workflows.
The outcome of the criminal investigation into the Derbyshire detective and the review of the associated rape convictions will provide legal precedent on how courts handle AI-generated evidence and the responsibilities of officers who use such tools.
Further guidance from law enforcement bodies in the UK and potentially other jurisdictions on the acceptable use of AI in court proceedings and evidence preparation will be crucial for understanding the evolving regulatory landscape.