Nvidia isn’t the only company that makes AI accelerators for training and inference, it’s a space that Intel is aggressively competing and excelling in too with its Intel Gaudi 2 technology, according to new research. Databricks conducted new research that is being released today, revealing that Intel Gaudi 2 provides strong performance competition against the industry-leading AI accelerators from Nvidia.
Match for Language Model Inference
The Databricks research found that for large language model (LLM) inference, Gaudi 2 matched the latency of Nvidia H100 systems on decoding and outperformed the Nvidia A100. The research found that Gaudi 2 inference achieves higher memory bandwidth utilization than H100 and A100.
Training Performance and Cost Efficiency
Nvidia still provides more training performance on its top-end accelerators. Using the Databricks MosaicML LLM foundry for training, the researchers found that Gaudi 2 achieved the second fastest single-node LLM training performance after NVIDIA H100, with more than 260 TFLOPS/chip. Overall, the Databricks research reported that based on public cloud pricing, Gaudi 2 has the best dollar-per-performance for both training and inference compared to A100 and H100.
“We were impressed by the performance of Gaudi 2, especially the high utilization achieved for LLM inference,” Abhinav Venigalla, lead NLP architect at Databricks, told VentureBeat.
Intel has been providing its own testing results on Gaudi 2 via the MLcommons MLperf benchmark for both training and inference. The new data from Databricks provides further validation for Intel on the performance of its Gaudi technology, from a third party.
“It’s always good to get validation of what we say,” Eitan Medina, COO at Habana Labs, an Intel company, told VentureBeat. “Since many people say that the Gaudi is kind of Intel’s best kept secret it’s actually important to have these sorts of publication reviews being made available so more and more customers know that Gaudi is a viable alternative.”
Intel acquired AI chip startup Habana Labs and its Gaudi technology back in 2019 for $2 billion and has been steadily improving the technology in the years since then.
