OpenAI considered dropping token pricing for next model, says Sam Altman

OpenAI considered ending token-based pricing for its next model and instead charging a flat fee per submitted prompt, but negative customer feedback led the company to back away, CEO Sam Altman said. He discussed the idea on the Cheeky Pint podcast with Stripe President John Collison. Altman said the billing would have worked "like AWS," with OpenAI sending a bill after the work rather than metering tokens.

Flat-fee pricing idea

Altman recalls that OpenAI "almost made what would've been a bad call" to stop pricing its next model in tokens altogether, moving instead to an arrangement where a developer hands over a prompt and OpenAI hands back a price. He does not name the model in question or give a timeline for when the change was under consideration. The billing would have worked "like AWS," he explained, where you "just do something and we send you a bill" rather than metering the exchange token by token.

Why customers pushed back

The idea ran into resistance when OpenAI tried to gauge how customers felt about receiving a price with no breakdown. "The customer response on that is so negative," Altman recounts, with buyers telling the company, "I need to know what I'm paying for somehow." Developers already spend plenty of time tracking input and output tokens to keep costs down and estimate what they'll spend as usage grows. Getting a price without knowing how it was calculated would only make that harder.

Preference for visible tokens

What makes the admission worth a second look is that, according to Altman, the people he consulted would rather pay more to keep that visibility. He put the question to "a few people," asking whether they would take a higher rate for the same output as long as it stayed priced in consistent tokens, and the answer, he said, was that they would "much rather you inflate the tokens and pay more," because "I know there's a real thing there." Developers resist a single opaque price, he reasons, because they suspect that over the long run they are "gonna get screwed somehow," the same instinct that once had buyers paying for CPU hours instead of for a finished job. The proposal could have had significant implications for developers building agents, whose multistep workflows already consume unpredictable amounts of compute and make spending difficult to forecast even under today's token-based model. Altman does not explain how OpenAI would have set those per-job prices, or whether customers could have weighed one charge against another.

Tokens remain despite limitations

The subject came up when Collison pressed Altman on whether tokens were the right way to measure AI usage. Altman acknowledged the limitations of token pricing but had no clear alternative to offer. "Tokens are a kind of fake unit," he said. "I mean, they're real in some sense, but they're not a good unit." He balked at Collison's half-serious pitch to coin an intelligence-equivalent token, comparing it to the way chipmakers still market nanometer figures that no longer correspond to any physical measurement on the chip. Altman admits developers incur real overhead when choosing between a cheaper model and a more capable one, especially when they also have to guess how much reasoning a given task will demand. "That's just cognitive overhead that I don't think actually adds any value," he said. The exchange comes as OpenAI continues to rely on token pricing, even as Altman has argued that the cost of a given level of AI falls roughly tenfold a year. For now, token pricing remains in place, even as Altman acknowledges its shortcomings and OpenAI struggles to find a better alternative.

Sources: The New Stack