Groq
More info
- Creator
- Jonathan Ross (ex-Google TPU)
- Released
- Type
- AI inference hardware (LPU) and managed API
- Website
- groq.com
Hardware startup with an "LPU" (Language Processing Unit) — chip designed specifically for LLM inference, 5–10× faster than GPUs per token.
Jonathan Ross founded Groq after leading the TPU project at Google. Their hosting API: Llama 3, Mixtral, Whisper and others at 300–700+ tokens/s (compared to ~30–80 on GPT-4 Turbo). Deterministically scheduled — every token's production time is known precisely. Competition: Cerebras Inference (wafer-scale chip, also very fast), SambaNova, Etched (transformer-specific ASIC), Tenstorrent (Jim Keller). Not to be confused with Elon Musk's "Grok" LLM.