Groq, a startup that develops chips to run generative AI models faster than conventional processors, said Monday it has raised $640 million in a new round of funding led by Blackrock. Neuberger Berman, Type One Ventures, Cisco, KDDI and Samsung Catalyst Fund also participated.
The installment, which brings Groq’s total raised to more than $1 billion and values the company at $2.8 billion, is a big win for Groq, which was reportedly originally looking to increase US$300 million in one a little lower ($2.5 billion) valuation. This more than doubles Groq’s previous valuation (~$1 billion) in April 2021, when the company raised $300 million in a round led by Tiger Global Management and D1 Capital Partners.
Meta’s chief AI scientist Yann LeCun will serve as a technical advisor to Groq and Stuart Pann, former head of Intel’s foundry business and former CIO at HP, will join the startup as chief operating officer, Groq also announced today . LeCun’s appointment is a bit unexpected given Meta’s position investments on its own AI chips – but it undoubtedly gives Groq a powerful ally in a cutthroat space.
Groq, what emerged from stealth in 2016, is creating what it calls an LPU (language processing unit) inference engine. The company claims its LPUs can run existing generative AI models similar in architecture to OpenAI GPT Chat It is GPT-4o at 10x the speed and a tenth the energy.
Groq CEO Jonathan Ross’ fame is helping to invent the tensor processing unit (TPU), Google’s custom AI accelerator chip used to train and run models. Ross teamed up with Douglas Wightman, an entrepreneur and former engineer at Google parent company Alphabet’s X moonshot lab, to co-found Groq nearly a decade ago.
Groq provides an LPU-based developer platform called GroqCloud that offers “open” models like Meta’s Llama 3.1 family, Google’s Gemma, OpenAI’s Whisper, and Mistral’s Mixtral, as well as an API that allows customers to use its chips in instances of cloud. (Groq also hosts a playground for AI-powered chatbots, GroqChat, launched late last year.) As of July, GroqCloud had more than 356,000 developers; Groq says part of the proceeds from the round will be used to scale capacity and add new models and features.
“Many of these developers are at large companies,” Groq COO Stuart Pann told TechCrunch. “By our estimates, more than 75% of the Fortune 100 companies are represented.”
As the generative AI boom continues, Groq faces increasing competition from both rival AI chip startups and Nvidia, the formidable incumbent in the AI hardware space.
Nvidia controls about 70% to 95% of the market for AI chips used to train and deploy generative AI models, and the company is taking aggressive steps to maintain its dominance.
Nvidia has committed to releasing a new AI chip architecture every year, rather than every two years as has historically been the case. And yours supposedly establishing a new business unit focused on designing custom chips for cloud computing companies and others, including AI hardware.
In addition to Nvidia, Groq competes with Amazon, Google and Microsoft, which all offer – or will soon offer – custom chips for cloud AI workloads. Amazon has its Trainium, Inferentia and Graviton processors available through AWS; Google Cloud customers can use the aforementioned TPUs and, over time, Google TPUs Axion splinter; and Microsoft recently released Azure instances in preview for your Cobalt 100 CPU, with Maia 100 AI Accelerator instances arriving in the coming months.
Groq could also consider Arm, Intel, AMD and a growing number of rival startups in an AI chip market that could reach $400 billion in annual sales within the next five years, according to some analysts. Arm and AMD in particular have booming AI chip businesses, thanks to increased capital expenditure by cloud providers to meet capacity demand for generative AI.
D-Matrix at the end of last year created US$110 million to commercialize what it characterizes as a first-of-its-kind inference computing platform. In June, recorded emerged from secrecy with $120 million for a custom processor to accelerate the currently dominant generative AI model architecture, the Transformer. SoftBank’s son Masayoshi is supposedly seeking to raise $100 billion for a chip venture to compete with Nvidia. And OpenAI is he said be in talks with investment firms to launch an AI chip manufacturing initiative.
To carve out its niche, Groq is investing heavily in corporate and government outreach.
In March, Groq acquired Definitive Intelligence, a Palo Alto-based company offering a range of business-oriented AI solutions, to form a new business unit called Groq Systems. Within Groq Systems’ purview is serving organizations, including U.S. government agencies and sovereign nations, that wish to add Groq chips to existing data centers or build new data centers using Groq processors.
More recently, Groq partnered with Carahsoft, a government IT company, to sell its solutions to public sector customers through Carahsoft’s reseller partners, and the startup has a letter of intent to install tens of thousands of its LPUs at the European company Earth Wind & Power, from Norway. Data center.
Groq is also collaborating with Saudi consultancy Aramco Digital to install LPUs in future data centers in the Middle East.
While building customer relationships, Mountain View, Calif.-based Groq is marching toward the next generation of its chip. Last August, the company announced that it would hire semiconductor company Global Foundries to manufacture 4nm LPUs, which are expected to provide performance and efficiency gains over Groq’s first-generation 13nm chips.
Groq says it plans to deploy more than 108,000 LPUs by the end of Q1 2025.
