The Work Now Within Reach

OpenAI's new custom inference chip, Jalapeño, significantly enhances AI capabilities by offering 1.5 to 1.9 times more peak token throughput per watt and 1.7 to 3.6 times lower end-to-end latency compared to commercial systems in InferenceX tests. This development, alongside accelerators from NVIDIA, AMD, and other partners, aims to expand what people and businesses can achieve with AI, making previously impractical ideas feasible and opening doors to new breakthroughs and possibilities. Deployment is planned to begin by year-end.

This report is the first time OpenAI has announced its own custom inference chip, Jalapeño, which outperforms commercial systems in throughput and latency.