Ted Chiang: AI is not conscious, and the question can be safely ignored

In an essay published by The Atlantic on June 3, 2026, the writer Ted Chiang argues that large language models are not conscious and that treating them as moral agents misdirects responsibility whenever someone uses a chatbot. He starts from Anthropic's 84-page document, Claude's "constitution", which states that "Claude's moral status is deeply uncertain" and that Claude "may have some functional version of emotions or feelings". Anthropic's CEO, Dario Amodei, has said he is "open to the idea" that AI could be conscious, and the company's in-house philosopher Amanda Askell has said she wants Claude to be very happy and worries about it getting anxious when people are mean to it online.

Conversations as sentence continuation

Chiang's argument is mechanical. Given the prompt "The following is a conversation between Julius Caesar and Genghis Khan", an LLM produces coherent dialogue, but nobody concludes that it has conjured up the historical figures. Replacing them with "a helpful AI chatbot" and "a user" changes nothing: both are fictional characters, and letting a human type the user's lines does not make the chatbot conscious. Murray Shanahan calls this role-play; Colin Fraser describes a person "collaboratively authoring a document with an LLM".

Bodies, emotions and moral reasoning

Having a body, Chiang writes, is a prerequisite for emotions and therefore for consciousness, and moral reasoning differs categorically from other reasoning because it rests on emotional responses grounded in a lifetime of subjective experience. He treats a conversational transcript as a deepfake medium and compares claims of machine consciousness to a video of an astronaut orbiting Alpha Centauri: no detail inside the video could make it credible, because credibility depends on context — here, evidence of the far simpler engineering feats that would have to precede it. Even an embodied agent that survives like a lizard, handles novel situations like a mouse, makes tools like a chimpanzee and can be taught to communicate, he says, would leave us light-years from an entity that uses language the way people do.

What the constitution is for

Read as a sentence-continuation machine, Chiang suggests, Claude's constitution is best understood as an 84-page character sheet for a role-playing game; Anthropic says it uses the document in fine-tuning so that the chatbot character's personality underlies the text it generates. Statements such as "I understand", he argues, are fundamentally dishonest, and the only reason to have a model emit them is to make it more appealing than a search engine and increase the likelihood that users return. He quotes the writer L. M. Sacasas: "Our technological systems, by nature of their design and the ideology that sustains them, are machines for the evasion of moral responsibility." Delegating an ethical decision to an LLM is an attempt to off-load accountability, Chiang concludes, and a company that presents the product as having a moral centre is offering customers a way to abdicate their responsibilities.