When Operai presented its “Reasoning” O3 model in December, the company associated with the creators of Arc-Agi, a reference point designed to test highly capable AI, to show the abilities of O3. Months later, the results have been reviewed, and now they look a little less impressive than initially.
Last week, the ARC award foundation, which maintains and manages Arc-AGI, updated its approximate computing costs for O3. The organization originally estimated that the best performance configuration of O3 tested, O3 Alto, cost around $ 3,000 to solve a single ARC-AGI problem. Now the arc award foundation believes that the cost is much higher, Possibly around $ 30,000 per task.
The review is remarkable because it illustrates how expensive the most sophisticated models of today can end up being for certain tasks, at least from the beginning. Operai does not yet have to set the price of O3, or release it, even. But the Arc Prize foundation believes that the O1-PRO model of OpenAI is a reasonable proxy.
For the context, O1-Pro is the most expensive model of OpenAi to date.
“We believe that O1-Pro is a closer comparison of the true cost of O3 … due to the amount of proof time computation used,” said Mike Knoop, one of the co-founders of the Arc award foundation, to TechCrunch. “But this is still a proxy, and we have kept O3 labeled as a preview in our classification to reflect uncertainty until the official price is announced.”
A high price for O3 high would not be out of the matter, given the amount of computer resources, according to reports, the model uses. According to the Arc Prize foundation, O3 High used 172x more computing than O3 Low, the lowest configuration of O3, to address ARC-AGI.
In addition, rumors have flown for quite some time on the expensive plans that Operai is considering presenting to business clients. At the beginning of March, the information reported that the company may be planning to charge up to $ 20,000 per month for specialized AI, such as a software developer agent.
Some might argue that even Openai’s most expensive models will cost well what a typical contractor or human employee would order. But as a researcher at the Toby Ord indicated in an X publicationModels may not be so efficient. For example, O3 High needed 1,024 attempts in each task in Arc-Agi to achieve its best score.
