What happened
OpenAI published initial test results for Jalapeño, its first custom inference chip. The company says the system delivered 1.5 to 1.9 times more AI work per watt at peak throughput and 1.7 to 3.6 times lower end-to-end latency than comparison systems across GPT-OSS 120B, DeepSeek R1 670B and Kimi K2.5 1T.
OpenAI says Jalapeño is its first custom inference chip and that the results are based on testing the chip together with the system built around it. The company evaluated the system on InferenceX, a public benchmark from SemiAnalysis that measures the full process of serving an AI request. OpenAI says it compared Jalapeño with commercially available AI systems over operating points ranging from high-throughput serving to highly interactive, low-latency use.
Across GPT-OSS 120B, DeepSeek R1 and Kimi K2.5, OpenAI reports 1.5 to 1.9 times more AI work per watt at peak throughput and 1.7 to 3.6 times lower end-to-end latency than comparison systems. For highly interactive workloads, it reports 2.1 to 4.1 times higher performance. In the appendix, the company compares Jalapeño with a GB200 for GPT-OSS 120B and with a GB300 for DeepSeek R1 and Kimi K2.5. OpenAI says Jalapeño reached the Pareto frontier across the tested operating points, meaning it found a better combination of throughput, latency and power efficiency within those comparisons.
The company reports that Jalapeño has a 700-watt package power rating, while measured sustained power stayed at or below 550 watts on the tested workloads. The published comparisons use package power ratings, including 1,200 watts for the GB200 and 1,400 watts for the GB300. For GPT-OSS 120B, OpenAI reports approximately 1.9 times higher peak mixed tokens per second per kilowatt and 1.7 times lower end-to-end latency than the comparison system. For DeepSeek R1, it reports approximately 1.7 times higher peak performance per watt and 3.6 times lower latency. For Kimi K2.5, it reports approximately 1.5 times higher peak performance per watt and 3.4 times lower latency.
OpenAI attributes the results to a full-stack design that combines the chip, memory, network, software and rack-scale system around language-model workloads. The company says the architecture keeps model state, including the KV cache used during generation, local where possible and reduces communication delays between cores and chips. It describes prefill as more compute-intensive and decode as more constrained by memory bandwidth, and says Jalapeño is designed to handle both phases. OpenAI also says AI helped move the chip from initial design to tapeout in nine months and helped optimize arithmetic circuits, but the source gives no independent account of that timeline or a full breakdown of which design work was automated.
Why it matters
The results suggest that inference performance may increasingly depend on designing models, chips, memory, networking and software as one system. If the gains hold at production scale, they could reduce the power and hardware needed to serve AI responses and make interactive agent workloads more responsive.
The central significance is a potential shift in how AI infrastructure is optimized. OpenAI’s account argues that a general-purpose accelerator can lose performance when computation, memory access and communication are treated as separate problems. Jalapeño instead combines those elements around the timing and data-movement patterns of language-model inference. That approach matters because serving a response involves both processing the prompt and generating tokens sequentially, while agentic systems may repeat those steps many times in one task.
The reported latency gains could have practical effects for interactive AI products if they survive production deployment. OpenAI says faster inference can support more responsive agents and make it possible to run ultra-fast workloads at efficiencies previously associated with faster or more heavily batched modes. The source does not demonstrate those effects in a live customer product, however. It presents benchmark results and planned infrastructure deployment, so the public impact remains prospective.
Power efficiency also has operational significance. OpenAI says producing more useful work from the same power and hardware could help it serve demand at lower cost and improve operating leverage. That claim is plausible as a business objective, but the source does not provide per-query costs, total system costs, cooling requirements, capital expenditure, utilization assumptions or evidence that the reported benchmark ratios translate directly into lower prices for users.
The development process described by OpenAI is itself notable. The company says its models helped design and bring up Jalapeño, while newer models are being used to optimize and program it. Using Codex with GPT-Astra, the team reportedly brought three open-weight models that were not part of the original production plan to high performance within two months. OpenAI also says AI-generated implementations were 1.5 to 1.8 times faster than human-written implementations for selected GPT-OSS attention and mixture-of-experts blocks. Those figures apply only to selected blocks, not complete models, and do not establish that AI can independently design or optimize an entire production chip.
What to watch next
OpenAI plans to begin deploying Jalapeño in its compute infrastructure by the end of 2026, but says production qualification, software maturation, scale operations and broader model validation are still underway. The source does not establish independent verification, operating cost, production volume or how the chip will perform across workloads beyond those tested.
The first test is whether Jalapeño reaches the planned deployment milestone. OpenAI says it intends to begin deploying the chip within its compute infrastructure by the end of 2026, while production qualification and software work continue. Deployment would establish that the system has moved beyond benchmark demonstrations, but the source does not specify the initial scale, locations, workloads or whether users will notice a product-level change.
More evidence is needed on reproducibility and comparison methodology. OpenAI used a public benchmark, but the source does not provide the complete benchmark data, the number of chips or systems tested, software versions, utilization levels, cooling and facility power, or independent validation of the comparison results. Because the headline ratios are normalized using published chip power ratings while Jalapeño’s sustained power is also reported separately, readers should distinguish package-level efficiency from total data-center efficiency.
Model coverage will be important. OpenAI reports results for GPT-OSS 120B, DeepSeek R1 670B and Kimi K2.5 1T, and says internal testing showed a wider advantage on frontier OpenAI models. The source does not identify those internal models or publish their results. It also says each additional model family requires new kernels and model-specific optimization, which means the chip’s flexibility may depend on continuing software work rather than hardware alone.
OpenAI says Gen 2 is deep in development and Gen 3 is taking shape, while also emphasizing that it will continue deploying accelerators from NVIDIA and other partners for training and inference. The meaningful unknown is therefore not simply whether Jalapeño is faster in the reported tests, but how it fits into a mixed infrastructure strategy, how much capacity it will represent, and whether future generations preserve the balance between throughput, latency, efficiency and support for changing model architectures.