Will the real Anthropic please stand up

It also warns that there’s a fair chance that its tech could “erode trust, harm society and destabilize the geopolitical order” and, if things don’t go to plan, “pose catastrophic or existential risks to humanity.”

“We encourage you, as a prospective owner of our stock, not to think of your purchase in purely transactional terms,” the company writes. “We hope you will see it instead as an opportunity to be part of something greater. The advent of machine intelligence is one of the seminal moments in human history.”

This kind of public messaging is a master class in narrative capture. But Anthropic has been honing its storytelling skills for years. Earlier this month, Jack Clark, the company’s cofounder and head of policy, gave a speech at the IMF meeting in Bangkok in which he announced to the assembled economists, bankers, and lawmakers that the singularity is already here.

Introduced by the science fiction writer and futurist Vernor Vinge around 40 years ago (and long debated on the nerdier fringes of the internet), the singularity is supposed to be the point at which technology gets so good that it outstrips the abilities of its makers.

Humans are left in the dust, the idea goes, no longer able to compete with—or even understand—the fruits of their former creation.

And this fictional world that many of us have imagined since we were kids? Apparently we’re already living in it: “I believe that this year we have gained convincing evidence that it has now begun,” Clark told his audience.

The company tells a vivid (and in many ways scary) story about a future it portrays as inevitable but one that has not yet happened, often co-opted from well-worn science fiction memes. It presents its technology as mysterious and dangerous, and insists that it is the only company trustworthy enough to tame it. (What investor wouldn’t want in on that?)

If you skim through the long sections about risks in the IPO prospectus, it’s clear this is no ordinary offering and no ordinary company. The way Anthropic tells it, business success depends not only on the firm’s ability to translate its technology into profitable products but also to save the world from what it has made.

Take another example. Last year, Chris Olah, another Anthropic cofounder, was invited to address the Vatican, where he told Pope Leo XIV and other religious scholars and officials what his company was learning about its technology. “I will be honest: We keep finding things that are mysterious, even unsettling,” he said. “We find evidence of introspection. We find internal states that functionally mirror joy, satisfaction, fear, grief, and unease.”

Never mind that many researchers would scoff at such an overt suggestion that Anthropic’s software has humanlike inner experience. This view of its technology makes its way into the prospectus as one more risk to its business. Anthropic states that: “In addition, given our focus on developing frontier AI, and the way we approach model sentience and moral status, we may face heightened scrutiny or pressure in connection with these debates and could elect to adopt policies, safeguards, or limitations regarding model development, deployment, or modification that go beyond existing legal requirements, which could constrain our operations, increase costs, or limit the commercial potential of our models, platform, and applications.”

In other words, if Claude wakes up, it could be a problem.

I’m not saying they’re necessarily wrong about any of this (although it’s pretty wild to read about machine sentience in an IPO prospectus.) But in a world where nobody knows what’s coming, companies like Anthropic get to fill the vacuum. When so little seems certain, what it says is snatched up and clung to like pieces of a sinking ship.

It’s worth taking a moment to ask: What is Anthropic? It might sound like a starter question on a family quiz show, but it’s far harder to answer than you might think.

The question has been on my mind since reading an exchange on X last week kicked off by Nathan Calvin, who is chief strategy officer at Encode AI, an organization based in Washington, DC, that advocates for regulation. Calvin was unhappy about the legislation that Anthropic proposed in a draft bill it circulated in secret to US lawmakers (which was then leaked ).

Anthropic wants to be seen as the good guy, Calvin said. But why should we trust it? “What is Anthropic?” he asked.

Here’s one answer: Anthropic is a company whose IPO values it at more than $1.5 trillion, netting its cofounders, including Clark and Olah, vast wealth.

But Anthropic has always insisted it is more than that. As Calvin noted: Anthropic might instead be considered an AI safety organization focused on developing powerful technology that is reliable and (almost) risk-free. Or it might be considered a group of people hell-bent on creating a technology with superhuman capabilities before anyone else—maybe because they believe they’re the most responsible people to do so, or maybe because of their egos.

In fact, Anthropic is all of these things. As CEO Dario Amodei put it in a reply to Calvin’s post: “Our strategy has always been what I’ve called the ‘race to the top’—by showing how it’s possible to build the technology responsibly, we can raise the bar for the entire industry. The forces that drive the race to the top are hugely more effective if the company practicing them is at the frontier; that’s one lesson I’ve learned repeatedly. And looking at that strategy as a whole, I’m proud of our track record and stand by it.”

Amodei makes it sound as if he’s got it all tied up, but playing multiple roles is hard. There are huge tensions between the priorities of an AI safety organization and a profit-driven megacorp, for example. (Amodei admits in his post that he has made mistakes.)

Ultimately, the stories Anthropic tells itself should help us understand the stories it tells us. They should help us figure out how much to trust it to self-regulate or whether we should be forcing it to give external safety testers access to its models on our terms rather than theirs. They should help us give it the benefit of the doubt (or not) when what looks like a self-serving piece of legislation comes to light.

But I think the stories reveal a multiheaded hydra. Anthropic is many things to many people, and I’m not sure that makes it more trustworthy. In many ways, a good old-fashioned technology company—without the science fiction role-play, without the woo-woo—would be far easier to read.

This article has been updated with additional details from the IPO prospectus.