A unified library of state-of-the-art model optimization techniques such as quantization, pruning, distillation, speculative decoding, etc. It compresses deep learning models for downstream deploym…

A simple, performant and scalable Jax LLM!

A pytorch quantization backend for optimum

A retargetable MLIR-based machine learning compiler and runtime toolkit.

Transformer related optimization, including BERT, GPT

The Torch-MLIR project aims to provide first class support from the PyTorch ecosystem to the MLIR ecosystem.

Backward compatible ML compute opset inspired by HLO/MHLO

Easy and lightning fast training of 🤗 Transformers on Habana Gaudi processor (HPU)

Milvus is a high-performance, cloud-native vector database built for scalable vector ANN search

State-of-the-Art Text Embeddings

Blazing fast training of 🤗 Transformers on Graphcore IPUs

a debugger for async rust!

AIMET is a library that provides advanced quantization and compression techniques for trained neural network models.

Hydra is a framework for elegantly configuring complex applications

SOTA low-bit LLM quantization (INT8/FP8/INT4/FP4/NF4) & sparsity; leading model compression techniques on TensorFlow, PyTorch, and ONNX Runtime

Dapr user documentation, used to build docs.dapr.io

[ARCHIVED] The C++ Standard Library for your entire system. See https://github.com/NVIDIA/cccl

Simple Python client for the Hugging Face Inference API

The Learning Interpretability Tool: Interactively analyze ML models to understand their behavior in an extensible and framework agnostic interface.

OpenVINO™ is an open source toolkit for optimizing and deploying AI inference

Cross-platform CLI and Python drivers for AIO liquid coolers and other devices

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

Visualizer for neural network, deep learning and machine learning models

ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator

The Triton Inference Server provides an optimized cloud and edge inferencing solution.

Open standard for machine learning interoperability