MLCommons Expands MLPerf Benchmarks
MLCommons, a neutral organization, is enhancing its MLPerf AI benchmarks, now including large language model (LLM) inference testing. These benchmarks aim to provide a fair platform for AI performance assessment across vendors.
MLPerf Inference 3.1
The latest update, MLPerf Inference 3.1, is the second major release this year and comprises a substantial dataset with over 13,500 performance results. Contributors range from ASUSTeK to xFusion, showcasing the industry-wide adoption of AI benchmarks.
“While there are multiple types of testing and configurations for the inference benchmarks, many submitters improved their performance by 20% or more over the 3.0 benchmark.”
Embracing Generative AI
The evolving benchmark suite reflects the rising influence of generative AI, especially large language models. MLPerf Inference 3.1 tackles tasks like text summarization, making AI more accessible to various organizations.
Diverse Hardware Representation
Intel and Nvidia are actively participating in the MLPerf Inference 3.1 benchmarks. Intel demonstrates the suitability of CPUs for inference, highlighting their performance on tasks like news summarization. Nvidia, on the other hand, introduces its GH200 Grace Hopper Superchip, designed for demanding AI workloads.
Nvidia’s L4 GPUs also shine in the benchmarks, outperforming x86 CPUs significantly.
These benchmarks signify the expansion of AI deployment options, catering to a wide array of compute preferences among enterprises and organizations.
