On "human in the loop" and automation bias
On the role of the "human in the loop" in AI: I keep coming back to a widely circulated exchange between American journalist Shane Harris and Anthropic's Claude, and I think it says something important that applies well beyond its original context.
What the exchange was actually about
Harris, a national security journalist at The Atlantic, questioned Claude about its own reported use by the US military in target identification during operations against Iran in early 2026 — specifically in connection with the strike on the Shajarah Tayyebeh school in Minab, which killed more than 150 people, many of them children. Multiple outlets, including the Wall Street Journal and the Washington Post, reported that Claude was used within the Pentagon's targeting infrastructure for intelligence synthesis, scenario simulation, and target selection during that operation. Anthropic's own CEO later told Bloomberg that the company did not know exactly how its models were used in the strike, while maintaining that the use case, as understood, did not violate Anthropic's policies — policies built around the principle that a human makes the final decision.
Harris put that principle to Claude directly, and asked whether "a human makes the final decision" genuinely resolves the ethical problem. Claude's response, as read out in the exchange:
"I don't think that the frame of 'humans make the final decision' fully resolves the ethical problem. When a system like Maven generates hundreds of targeting recommendations and humans spend roughly the equivalent of a glance approving each one, the human is not really making a decision in any meaningful sense. They are ratifying an algorithmic output under time pressure, with incomplete information, and institutional pressure to move fast. [...] That's not human judgement, that's automation bias with a human signature attached."
Spot on. And worth remembering the next time you approve an AI-generated document without thoroughly confirming and questioning the analysis and output underneath it.
Why this generalises far beyond military targeting
Automation bias — the well-documented tendency to favour automated recommendations over contradictory information, even when that information is correct — isn't new, and isn't specific to any one domain. What's changed is the volume and speed at which AI systems now generate outputs for a human to nominally review. A "human in the loop" who is reviewing hundreds of recommendations at the pace of a glance each isn't exercising judgment in any meaningful sense; they're providing a signature that lets an institution say a human was involved.
That's exactly the mechanism the EU AI Act tries to guard against in its own high-risk categories, which include:
regulated products or safety components thereof
biometric identification and categorisation
critical infrastructure
education and vocational training
employment and workers management
access to essential private and public services
law enforcement
migration, asylum, and border control
administration of justice and democratic processes
The uncomfortable gap in that list
Notice what's missing. Under Article 2(3) of the AI Act, systems placed on the market, put into service, or used exclusively for military, defence, or national security purposes fall entirely outside the Regulation's scope — regardless of the entity carrying out those activities. It's a deliberate, legally grounded exclusion, tied to the division of competence between the EU and its Member States on defence and national security matters, and it isn't unique to the EU AI Act; comparable frameworks carve out military use in similar ways.
What should we take from the fact that exactly the use case at the centre of the Harris exchange — AI-assisted military targeting — sits explicitly outside the regulatory framework built to guard against the automation-bias failure mode that use case exemplifies? I don't think there's a tidy answer. The legal logic for the carve-out is coherent on its own terms. But it does mean the single most consequential domain for "does a human really make the final decision" questions is also the one domain this legislation was never designed to reach.
New guidance on classification, for those of us in health
On a related but more constructive note: the European Commission released draft guidelines on the classification of high-risk AI systems in May 2026, covering both the Annex I (regulated product) and Annex III (use case) routes to high-risk status. For healthtech specifically, the most useful clarification is on the definition of "safety component" — the guidelines confirm this is an autonomous, cross-sectoral definition under Article 3(14): a component qualifies if it either performs a safety function, or its failure or malfunction could endanger the health or safety of persons or property, and this definition takes precedence over any differing definitions in sector-specific legislation like MDR.
In practice, that clarification helps narrow down which AI functionality inside a health product actually triggers high-risk classification and which doesn't — a genuinely useful addition for anyone trying to scope this precisely rather than defaulting to "probably high-risk" out of caution. Beyond that, though, there isn't much in the draft with specific relevance to health use cases. Does that absence suggest a sector-specific guideline is still in the works? I'd read it that way, but it's speculation on my part until the Commission says otherwise.
Full interview of Shane Harris on AI at War
References
Bloomberg, Anthropic CEO Doesn't Know If Claude Used in Iran School Strike, June 10, 2026
Reuters and The Washington Post, reporting on Claude's use in Pentagon targeting infrastructure and the Minab school strike investigation, March 2026
European Commission, Draft Commission Guidelines on the Classification of High-Risk AI Systems under Article 6 of Regulation (EU) 2024/1689, published 19 May 2026, consultation extended to 23 July 2026
EU AI Act, Article 2(3) and Recital 24, on the military, defence, and national security exclusion
EU AI Act, Article 3(14), definition of "safety component"
Methodology note: This article is based on my original LinkedIn post (link), reflecting my professional perspective on human oversight requirements in AI regulation and the EU AI Act's classification framework. AI assisted in elaborating the topic into a broader article by integrating background research and verifying the underlying reporting on the referenced exchange and regulatory developments. All analysis and regulatory perspectives are my own, and all content has been reviewed by me for accuracy. Given the gravity of the events referenced in this piece, every effort has been made to represent the underlying facts accurately and without sensationalism.
