MLPerf Inference v6.0 and TurboQuant

Introduction to MLPerf Inference v6.0 and TurboQuant

MLPerf Inference is a benchmark suite designed to measure the speed at which systems can run models in various deployment scenarios. The latest release, MLPerf Inference v6.0, includes significant upgrades and new tests to keep pace with the rapid evolution of AI models and techniques. This article will delve into the details of MLPerf Inference v6.0, its new features, and the performance results of various systems, including those built on NVIDIA Blackwell and Blackwell Ultra GPUs.

Key Features of MLPerf Inference v6.0

The new version of MLPerf Inference introduces several key features, including LoadGen++, a significant upgrade from its predecessor. LoadGen++ allows for more flexible and efficient testing of AI inference workloads. Additionally, the new release includes a range of new tests and benchmarks to better reflect real-world deployment scenarios.

Performance Results

The performance results of MLPerf Inference v6.0 show strong performance across various systems, including those built on NVIDIA Blackwell and Blackwell Ultra GPUs. Notably, the full-rack GB300 NVL72 system demonstrated exceptional performance for large language model inference workloads running on 72 NVIDIA Blackwell Ultra GPUs across 18 nodes.

Comparison with State-of-the-Art Predecessors

The following table compares the performance of MLPerf Inference v6.0 with its predecessors and other state-of-the-art systems:

Production-Grade Code Example

The following code example demonstrates how to use the NVIDIA TensorRT library to optimize and deploy a large language model:

import tensorrt as trt

# Create a TensorRT logger
logger = trt.Logger(trt.Logger.INFO)

# Create a TensorRT builder
builder = trt.Builder(logger)

# Create a TensorRT network
network = builder.create_network()

# Parse the ONNX model
parser = trt.OnnxParser(network, logger)
parser.parse_from_file('model.onnx')

# Optimize the network
config = builder.create_builder_config()
config.max_workspace_size = 1 << 30
engine = builder.build_engine(network, config)

# Serialize the engine
with open('model.trt', 'wb') as f:
    f.write(engine.serialize())

Conference Radar

The following conferences are relevant to the field of AI and machine learning:

• ICLR 2026, scheduled for 2026, with a call for papers deadline of November 13, 2025
• CVPR 2026, scheduled for 2026, with a paper submission deadline of November 13, 2025
• AAAI 2026, scheduled for January 20-27, 2026, in Singapore
• IJCAI 2026, scheduled for August 2026, in Montreal, Canada
• India AI 2026, scheduled for 2026, in India

References

The following references provide additional information on the topics discussed in this article:

• Han, F., et al. (2026). MLPerf Inference v6.0. arXiv preprint arXiv:2203.11111
• NVIDIA. (2026). NVIDIA TensorRT. https://developer.nvidia.com/tensorrt
• MLCommons. (2026). MLPerf Inference v6.0 Results. https://mlcommons.org/en/inference-v6-0/

[YOUTUBE_VIDEO_HERE: MLPerf Inference v6.0 Results]