OpenAI's Jalapeño chip outperforms Nvidia's superchips on AI inference benchmark tests

OpenAI has unveiled its first custom AI chip, Jalapeño, which demonstrated superior performance in AI inference benchmarks compared to Nvidia's superchips. Developed in partnership with Broadcom, the chip is designed to provide faster responses and higher throughput with lower latency, offering a 1.5 to 1.9 times improvement in AI work per watt across several models. Richard Ho, OpenAI's hardware vice president, stated that Jalapeño offers the "best of both worlds" by balancing throughput and latency. While the chip poses a significant threat to Nvidia's inference margins, analysts suggest that Nvidia GPUs will remain essential for large-scale model training. The administration announced that OpenAI plans to deploy Jalapeño in small volumes by the end of this year, with a full ramp-up into 2027. The chip's success highlights a broader trend of tech giants like Google, Meta, and AWS developing custom silicon to reduce reliance on Nvidia. Analysts predict that custom ASIC chips like Jalapeño may exceed GPUs in volume by 2028.

Sources

• The Verge
  OpenAI says its Jalapeño chip can power faster AI responses than the competition
• OpenAI
  Jalapeño’s first results show industry-leading speed and efficiency in AI inference
• CNBC
  OpenAI’s Jalapeño AI chip brings new 'threat' to Nvidia margins as custom silicon gains ground
• SemiAnalysis
  OpenAI Jalapeño: Better Than Nvidia Blackwell
• Bloomberg
  OpenAI Claims Its New Chips Can Outperform Nvidia Processors in Tests