Introduction to Universal Sparse Tensors
The Universal Sparse Tensor (UST) is a flexible abstraction that decouples tensor sparsity from memory layout, enabling developers to define computations using sparsity-agnostic operations. The UST describes common and less-common sparse tensor storage formats using a simple domain-specific language (DSL). This allows developers to define computations without worrying about the underlying memory layout.
Interoperability with PyTorch, SciPy, CuPy
The nvmath-python UST implementation provides interoperability with tensors of PyTorch, SciPy, CuPy, and NumPy. The conversion is mostly zero-cost, which means that converting dense or sparse formats like COO, CSR, CSC, BSR, BSC, and DIA to a UST object or back is done without data movement or copying.
row, column, values = scipy_sparse_matrix.to_coo().row, scipy_sparse_matrix.to_coo().col, scipy_sparse_matrix.to_coo().dataConverting a SciPy sparse matrix to UST
Performance Comparison
We compare SpMV performance using CuPy’s @-operation across natively supported formats (COO, CSR, and DIA), along with PyTorch mv() for COO and CSR against UST’s `matmul()` on the 1,489,752 x 1,489,752 matrix atmosmodl from the Matrix Market.
cupy_sparse_matrix = cupy.sparse.coo_matrix((values, (row, column)), shape=(1489752, 1489752))Creating a CuPy sparse matrix
2x
speedup over CSR
1.5x
speedup over COO
💡 UST Performance
The UST implementation provides better performance compared to native formats
Conclusion
The Universal Sparse Tensor (UST) is a powerful abstraction that simplifies sparse deep learning by decoupling tensor sparsity from memory layout. With the integration of UST into nvmath-python v0.9.0, developers can now accelerate sparse scientific and deep learning applications. The UST implementation provides interoperability with PyTorch, SciPy, CuPy, and NumPy, and offers better performance compared to native formats.
How this compares
How this compares
| Component | Open / This Approach | Proprietary Alternative |
|---|---|---|
| Model provider | Any — OpenAI, Anthropic, Ollama | Single vendor lock-in |
| Sparse tensor support | UST, COO, CSR, CSC, BSR | Proprietary formats |
🔑 Key Takeaway
The Universal Sparse Tensor (UST) is a flexible abstraction that decouples tensor sparsity from memory layout, enabling developers to define computations using sparsity-agnostic operations. This simplifies sparse deep learning and provides better performance compared to native formats.
Key Links