MLQ.ai
About Sign in Subscribe
Groq logo

Groq

Private company. Information Technology. Based in Mountain View, California.

Founded 2016
Employees 400
Website groq.com/
What does Groq do?
Visit website →
AI ChipAI InfrastructureAI Data CenterInferenceML PlatformSovereign AI

Groq is an AI hardware and inference cloud company that designs and builds the Language Processing Unit (LPU), a chip architecture purpose-built for low-latency AI inference. The company operates GroqCloud, a public inference API service offering token-throughput rates substantially higher than conventional GPU-based inference. The strategic positioning is structurally distinct from NVIDIA's general-purpose GPU approach: Groq's LPU sacrifices the flexibility of GPU compute in exchange for substantially faster inference throughput on the supported language model architectures.

The LPU architecture is structurally distinctive. Where conventional GPUs use SIMD execution and depend on external memory bandwidth, the LPU is a tensor streaming processor that executes operations deterministically without runtime memory contention. The deterministic execution model produces predictable, ultra-low latency inference performance that is structurally different from the GPU performance envelope. The architectural tradeoff is that the LPU is optimized for inference of pre-trained models rather than training; Groq does not compete with NVIDIA in training workloads.

GroqCloud, the public inference service, has positioned the company as the highest-throughput commercial inference provider for the supported model families. The service offers token rates that can exceed 500 tokens per second for some Llama, Mistral, and DeepSeek models, materially faster than conventional GPU-based inference services. The pricing model is usage-based per million tokens, with the headline value proposition that ultra-low latency unlocks new application categories (real-time agentic workflows, voice-mode AI, real-time multimodal inference) that conventional inference latency cannot serve.

The product portfolio combines GroqCloud (the public inference service), GroqRack (data center hardware sold to enterprise and sovereign customers), and Groq Inference at scale (large dedicated inference deployments for AI-native enterprises and platform customers). The customer base concentrates among AI-native enterprises building real-time AI applications, sovereign and government customers requiring on-premises inference at scale, and developer adoption of GroqCloud for production inference workloads.

The strategic relationships include a substantial sovereign partnership in the Kingdom of Saudi Arabia for inference infrastructure deployment, partnerships with several major OEMs for hardware distribution, and customer relationships with frontier AI labs and AI-native enterprises consuming GroqCloud at scale. Groq has also raised substantial capital from BlackRock, Cisco, Samsung Catalyst Fund, Tiger Global, Saudi Aramco's venture arm, and various strategic investors.

The principal exposures include competition with NVIDIA's dominant accelerator architecture for inference workloads, the structural reality that GroqCloud serves a curated set of open-weights and partner-distributed model families rather than every model, the capital intensity of scaling LPU manufacturing through partner foundries, and the broader question of whether deterministic inference architecture can sustain pricing power as NVIDIA's GPU inference economics improve through Vera Rubin and successor generations.

Founders
Jonathan Ross
Founder, CEO

Hardware engineer and architect who previously led the design of Google's Tensor Processing Unit (TPU) program before founding Groq in 2016. Has served as CEO continuously since founding. Engineering background in custom AI accelerator design.

Douglas Wightman
Co-founder

Co-founded Groq in 2016 with Jonathan Ross. Engineering and product background in AI accelerator design.

Customers
MetaMistral AISaudi Arabia's Public Investment FundAramcoVolkswagen GroupRiyadh AirArgonne National Laboratory

Groq's customer base spans AI-native enterprises, sovereign and government customers, AI labs, and developers. AI-native enterprises building real-time inference applications (voice mode AI, real-time agents, conversational interfaces) are a primary GroqCloud customer category, drawn by token-throughput rates that conventional GPU inference cannot match. Sovereign customers (most prominently the Kingdom of Saudi Arabia, where Groq has announced a substantial inference infrastructure partnership) deploy GroqRack hardware for on-premises inference at scale. Frontier AI labs (Meta, Mistral, others) consume GroqCloud as a distribution channel for their open-weights models, particularly where ultra-low latency is required. Developers building applications on top of open-weights model families use GroqCloud through the public API on usage-based pricing.

Competitors
Dominant accelerator architecture and the structural reference point for inference performance. NVIDIA's CUDA ecosystem depth and broader software stack are central barriers to Groq adoption for general-purpose AI workloads.
AMD Instinct accelerators compete with Groq LPU in inference, particularly in workloads where memory capacity favors non-NVIDIA architectures.
Direct competitor in low-latency, high-throughput inference. Cerebras's wafer-scale architecture and Groq's LPU architecture both target inference workloads where conventional GPU economics are unfavorable. Cerebras emphasizes throughput on very large models; Groq emphasizes ultra-low latency per token.
SambaNova Systems
AI hardware company with reconfigurable dataflow architecture targeting similar inference and on-premises enterprise customers as Groq.
Gaudi accelerators compete with Groq LPU in inference, though Gaudi has not approached NVIDIA or AMD in market share.
Custom AI ASIC partnerships with hyperscalers (Google TPU, Meta MTIA) compete for overall inference accelerator wallet at hyperscaler scale, though these are not directly substitutable for Groq's standalone LPU architecture in merchant inference.
Frequently asked questions about Groq
What does Groq do?
Groq designs and builds the Language Processing Unit (LPU), a chip architecture purpose-built for low-latency AI inference. The company operates GroqCloud, a public inference API service offering token-throughput rates substantially higher than conventional GPU-based inference. Products also include GroqRack hardware sold to enterprise and sovereign customers and large dedicated inference deployments for AI-native enterprises.
Who founded Groq?
Groq was founded in 2016 by Jonathan Ross (CEO, previously led the design of Google's Tensor Processing Unit / TPU program) and Douglas Wightman. The founding thesis was that AI inference workloads require structurally different chip architecture than GPU compute, oriented around deterministic execution and ultra-low latency.
What is an LPU?
The Language Processing Unit (LPU) is Groq's chip architecture purpose-built for AI inference. Unlike conventional GPUs that use SIMD execution and depend on external memory bandwidth, the LPU is a tensor streaming processor that executes operations deterministically without runtime memory contention. The deterministic execution model produces predictable, ultra-low latency inference performance structurally different from the GPU performance envelope. The tradeoff is that the LPU is optimized for inference of pre-trained models rather than training.
What is GroqCloud?
GroqCloud is Groq's public inference API service. The service offers token rates that can exceed 500 tokens per second for some Llama, Mistral, and DeepSeek models, materially faster than conventional GPU-based inference services. Pricing is usage-based per million tokens. The headline value proposition is that ultra-low latency unlocks new application categories (real-time agentic workflows, voice-mode AI, real-time multimodal inference) that conventional inference latency cannot serve.
Who are Groq's competitors?
In merchant AI accelerators, Groq competes with NVIDIA (the dominant inference architecture) and AMD (Instinct). In low-latency inference specifically, Cerebras Systems is a direct competitor (wafer-scale architecture targeting throughput) and SambaNova Systems competes in on-premises AI hardware deployments. Intel Gaudi competes in broader inference accelerator market. Hyperscaler custom inference silicon (AWS Inferentia, Google TPU inference variants) reduces Groq's addressable share at hyperscaler-anchored customers.
What is Groq's relationship with Saudi Arabia?
Groq has announced a substantial sovereign partnership in the Kingdom of Saudi Arabia for inference infrastructure deployment, with Saudi Aramco's venture arm and the Public Investment Fund both as investors and strategic partners. The partnership includes large GroqRack deployments for Saudi sovereign AI infrastructure. The relationship is structurally important for Groq's revenue scale and a public demonstration of LPU deployment at substantial sovereign scale.
Why is Groq's architecture different from a GPU?
GPUs use SIMD (single-instruction-multiple-data) execution and depend on external memory bandwidth, which creates variable latency depending on memory access patterns. The LPU uses tensor streaming with deterministic execution, where every operation completes in a known number of cycles without runtime memory contention. The deterministic model produces predictable ultra-low latency per token. The tradeoff is that the LPU is purpose-built for inference rather than training, and the supported model architectures are determined by what Groq's compiler and runtime support.
Contacts at Groq
groq.com
🔒

Contacts are a premium feature

Upgrade to MLQ.ai premium to search verified business contacts at Groq by department, seniority, and role.

Upgrade to premium
The Power Delivery GapReport
Research

The Power Delivery Gap

The interconnection queue holds more than 1,100 GW of power. The grid delivers renewables quickly and firm gas slowly, if at all. A model of what the US grid can actually deliver, and when, built on MLQ's queue and generator data.

No spam. Unsubscribe anytime.