Report Finds 22 Percent of Home Office AI Asylum Summaries Failed Accuracy Checks

AI summaries fail accuracy tests

An investigation by the Open Rights Group (ORG) has found that 45 of 203 AI-generated asylum interview summaries checked by Home Office experts after the system rolled out in April 2026 failed accuracy tests, a 22 percent failure rate. By comparison, the rejection rate recorded during the original pilot was 9 percent. A further 19 summaries were flagged for additional review, and of 56 second-stage reviews, 30 did not meet the required standard.

The errors identified include factual inaccuracies and misrepresentations of what asylum seekers said during interviews. ORG also found that summaries could pass checks even when they omitted important details, provided the information they did include was accurate.

Processing times worsen despite efficiency promises

The report raises questions about whether the tools are saving time. ORG says the average monthly number of initial substantive asylum decisions since April 2026 fell by about 42 percent compared with the same period in 2025, while the mean time from interview to decision rose by 29 percent. The group cautions that these figures do not prove the AI caused the slowdown, but they offer little evidence of the productivity gains ministers have promised.

Relatively few summaries have been examined overall. ORG estimates that Home Office experts checked only 0.3 percent of summaries between April and July, based on the department's projected monthly caseload of 17,515.

Oversight gaps in both systems

The tools in question are Asylum Case Summarisation (ACS), which summarizes asylum interviews, and Asylum Policy Search (APS), a chat-style tool for searching country policy information. Both use OpenAI's GPT-4o model through Microsoft's Azure AI Foundry. The government insists humans remain responsible for asylum decisions, but ORG questions how reliably caseworkers can spot mistakes in AI-generated material.

Oversight of APS appears particularly lacking. The Home Office confirmed in August that expert testing had stopped, despite previously telling MPs that checks were continuing, and ORG says no summary of the testing results was produced. Asylum applicants are also not specifically told when AI has been used in their cases and cannot inspect or correct the summaries generated from their interviews.

Calls for suspension and independent audit

ORG warns that the department's own data protection assessment says ACS automates work previously done by technical specialists who reviewed and summarized asylum cases before passing them to decision-makers, potentially removing an experienced layer of scrutiny rather than simply saving staff time. The group is calling for both tools to be suspended and independently audited, arguing that current safeguards are inadequate for decisions that determine whether someone can remain in the UK.

The findings follow other technology controversies at the department. In June, more than 60 rights groups urged ministers to abandon plans for AI-powered age estimation of asylum-seeking children, and an inspection found the Atlas immigration system was failing to provide caseworkers with feedback from asylum appeals. The Home Office did not respond to The Register's questions.

Sources: Theregister