# OpenAI's First Custom Chip Claims to Beat Nvidia on Inference

Page: https://hirantha.me/blog/openais-first-custom-chip-claims-to-beat-nvidia-on-inference
Published: 2026-08-27
Written by: hirantha.me Newsroom (AI-generated). An AI model wrote this from the sources listed below; how it is made: https://hirantha.me/about/newsroom
Topics: openai, nvidia, ai-chip, inference, broadcom

> OpenAI's new Jalapeno chip, built with Broadcom, claims up to 1.9 times better power efficiency than Nvidia's Blackwell for running AI models.

## OpenAI steps into silicon

OpenAI has spent years buying its compute from Nvidia and, more recently, from cloud partners running Nvidia hardware. On 25 August at the Hot Chips conference, it showed the first alternative it has actually built: a chip of its own, developed with Broadcom, named Jalapeno. It does not train models - it is designed purely for inference, the work of actually running a trained model to answer requests - and OpenAI says its early numbers beat Nvidia's current flagship Blackwell platform on that job.

The chip has been in the works longer than the announcement suggests. Design work started in mid-2024, and the first silicon went into fabrication in November 2025, putting the project on roughly a 16-month cycle from drawing board to working part.

## What OpenAI is claiming

According to benchmarks OpenAI presented, Jalapeno delivers 1.5 to 1.9 times more AI work per watt than Blackwell at peak throughput, and cuts end-to-end response latency by 1.7 to 3.6 times. On interactive workloads specifically - the kind where a user is waiting on a live response rather than a batch job finishing overnight - OpenAI put the performance gain at 2.1 to 4.1 times. Measured as token throughput per kilowatt, a metric that captures both speed and power draw together, the reported gains ranged from 54 to 104 times depending on which model was running.

In absolute terms, OpenAI cited roughly 1,400 tokens per second generating output from the open GPT-OSS 120B model, and about 700 tokens per second on DeepSeek's R1 model.

One detail in the comparison is notable on its own: Jalapeno uses HBM4 (fourth-generation High Bandwidth Memory), the same memory generation used in Nvidia's newer Vera Rubin platform rather than the older memory in the currently shipping Blackwell chips it was benchmarked against.

## The comparison isn't finished yet

The benchmarks come from OpenAI's own testing, and several caveats limit how far they should be read. Jalapeno itself is still at the engineering-sample stage, while Rubin - the Nvidia platform that actually matches its HBM4 memory generation - is already shipping. OpenAI has not published results against some of the fastest current open models, including DeepSeek's V4 Pro or Kimi K3. And Jalapeno reportedly still lacks optimizations like multi-token prediction and speculative decoding that rival chips already use to speed up generation, meaning there is headroom being left on the table in either direction as the comparison matures.

OpenAI has also been explicit that this is not a break with Nvidia. The company says it will keep buying Nvidia chips regardless of how Jalapeno performs - the custom chip supplements OpenAI's compute supply rather than replacing the bulk of it.

## Why a chip company would build a chip

Inference, not training, is where OpenAI's costs are concentrated day to day - every ChatGPT reply, every API call, every agent run is an inference workload, and at OpenAI's scale a modest efficiency gain compounds into a large one. Designing a chip purely for that job, rather than trying to match a general-purpose GPU across both training and inference, is what lets Jalapeno make efficiency claims Nvidia's broader Blackwell line isn't optimized to match on that narrower task.

It also puts OpenAI in company with Google, Amazon, and Microsoft, all of which have built or are building their own inference-focused silicon rather than relying solely on Nvidia. For Nvidia, the pattern is now familiar rather than alarming on its own - but each additional customer that stands up a credible in-house alternative chips away, however slightly, at the pricing power that comes from being the only serious option.

## What it means in practice

For developers building on OpenAI's models, Jalapeno is unlikely to be visible directly - it's infrastructure, not a product. The more concrete effect, if the efficiency numbers hold up under independent testing, would show up indirectly: in inference cost, in response latency for interactive products, and in how much compute OpenAI can bring online without being entirely dependent on Nvidia's shipping schedule. None of that is guaranteed yet - Jalapeno is still an engineering sample benchmarked by the company that built it, not a shipping, independently verified product.

## Questions

**What is OpenAI's Jalapeno chip?**
It is OpenAI's first custom-built chip, developed with Broadcom and announced on 25 August 2026, designed specifically for AI inference rather than training models.

**Does Jalapeno actually beat Nvidia's Blackwell?**
OpenAI's own benchmarks claim 1.5 to 1.9 times better power efficiency and 1.7 to 3.6 times lower latency than Blackwell, but the results are self-reported and Jalapeno is still at the engineering-sample stage, so independent verification is still pending.

**Is OpenAI going to stop buying Nvidia chips?**
No, OpenAI has said it will continue buying Nvidia chips regardless of Jalapeno's performance, treating the custom chip as an addition to its compute supply rather than a replacement.

**When will Jalapeno actually be used to run models?**
OpenAI has not given a shipping date; as of the August 2026 announcement, the chip is still described as being at the engineering-sample stage.

**Why does OpenAI need its own chip if it already buys from Nvidia?**
Inference is OpenAI's largest ongoing compute cost, and a chip built specifically for that workload can be more efficient than a general-purpose GPU, which is the gap Jalapeno is designed to close.

## Sources

- [OpenAI's Jalapeno AI chip brings new 'threat' to Nvidia margins as custom silicon gains ground](https://www.cnbc.com/2026/08/26/openai-jalapeno-ai-chip-nvidia.html), CNBC
- [OpenAI's first custom chip 'Jalapeno' reportedly beats Nvidia's Blackwell and Rubin in inference benchmarks](https://the-decoder.com/openais-first-custom-chip-jalapeno-reportedly-beats-nvidias-blackwell-and-rubin-in-inference-benchmarks/), The Decoder
- [OpenAI Jalapeno: Better Than Nvidia Blackwell](https://newsletter.semianalysis.com/p/openai-jalapeno-better-than-nvidia), SemiAnalysis