AArtificial intelligence executives have big plans — and they aren’t cheap. In a recent interview with TIME, Dario Amodei, CEO of AI company Anthropic, predicted that the cost to develop the next generation of AI systems due out later this year would be around $1 billion. This trend suggests that the next generation would cost around $10 billion.
Amodei is not the only one preparing for a spending spree. Microsoft and OpenAI are supposedly planning to build a $100 billion supercomputer to build and run AI models. Asked about this plan, Google DeepMind CEO Demis Hassabis he said that your company will invest more over time.
See more information: Inside Anthropic, the AI company betting that security can be a winning strategy
In a new to study released on Monday, two researchers at Stanford University and three researchers at Epoch AI, a nonprofit research institute that focuses on predicting how AI will develop, published the most complete analysis yet on how the cost to train The most capable AI systems have evolved over time, and what is driving the rising costs of the AI arms race among technology companies. Their results suggest that the costs of training the most advanced AI systems have been rising for years as a result of the increasing amount of computing power used to train these systems, and that employee compensation also contributes significantly to the cost of AI.
“The cost of major AI training has been growing two to three times a year since 2016, and that puts billion-dollar price tags on the horizon by 2027, perhaps sooner,” says Epoch researcher Ben Cottier. AI who led the study. This will mean that only very well financed companies will be able to compete, consolidating the power of already powerful companies, he warns.
The cost of computing
To calculate the cost of computing power needed to train a given AI model, Epoch AI researchers took historical data on the cost of purchasing the specialized semiconductor chips needed and then depreciated that value over the time the chips were used. necessary to function. to train the AI model.
The researchers found that the cost of computing power needed to train the models doubles every nine months. This is a prodigious rate of growth – at this rate, the cost of the hardware and electricity needed to build cutting-edge AI systems alone would be billions by the end of this decade, without accounting for other costs such as remuneration. of employees.
However, the plans put forward by Amodei and others surpass even this rapid rate of growth. Costs could very well rise beyond the historic rate over the next two years before returning to their original long-term trend, Cottier says.
Better salaries
Accessing and powering the necessary computing power is only part of the cost involved in developing the most sophisticated AI systems. Companies must also pay the researchers who develop the algorithms. To estimate labor costs, Epoch AI researchers took the number of researchers who developed a given model, as indicated by the number of co-authors on the paper announcing its release, and multiplied that number by an estimate of the average compensation for an AI. researcher and an estimate of the time researchers spent developing the AI model.
They estimated labor costs for four AI models—OpenAI’s GPT-3 and GPT-4 models, a replication of GPT-3 developed by Meta researchers called OPT-175B, and Google DeepMind’s Gemini Ultra 1.0—and found that compensation of employees ranged from 29% to 49% of the total development cost.
While much discussion has focused on the rising costs of accessing the specialized semiconductor chips needed to train and operate advanced AI systems, Amodei assigned its astronomical projections of future costs for chips – Epoch AI’s results suggest that trade-off is also a significant cost factor. However, if companies continue to train AI models with ever-increasing amounts of computing power, Cottier expects labor costs to decrease as a proportion of total costs.
Winners take all
It is unclear whether the trend documented in the study will continue. The effort to build more computationally intensive AI systems could be hampered by the intense power needs of the largest clusters of semiconductor chips or a lack of training data. Some commentators to discuss that it will not make commercial sense for companies to continue training larger models in the future, given the high expenses and marginal incremental benefits of additional scale.
But if the trend continues, only well-resourced organizations will be able to keep pace. This includes tech giants – Google, Amazon, Microsoft and Meta – and smaller companies backed by the tech giants, such as OpenAI and Anthropic, and some other well-funded groups, such as the UAE government-funded Technology Innovation Institute. . Even within this narrow set of competitors, there are signs of imminent consolidation. In March, Inflection AI was “eaten alive” by its largest investor, Microsoft, with most of its leadership team and employees joining the technology giant.
Given that such large investments are likely to produce remarkably capable AI systems, the paper’s authors warn that the “concentration of such a powerful technology among a few key players raises questions about responsible development and implementation. Both AI creators and policymakers must engage with these issues and consider the tradeoffs involved.”
Correction, June 3
The original version of this story misstated the affiliations of researchers who contributed to the study. Two of the five researchers are affiliated with Stanford University and three with Epoch AI; they’re not all with Epoch.
This story originally appeared on Time.com read the full story
