MLCommons

All

102 repositories

mlperf_inference_unofficial_submissions_v5.0
Public
These are automated test submissions for validating the MLPerf inference workflows
mlperf mlperf-inference mlperf-inference-results
Mermaid
•1•0•0•0•Updated Dec 23, 2024Dec 23, 2024
mlperf-automations
Public
Python
•
Apache License 2.0
•4•0•49•1•Updated Dec 22, 2024Dec 22, 2024
mlperf_inference_test_submissions_v5.0
Public
Mermaid
•1•0•0•0•Updated Dec 22, 2024Dec 22, 2024
inference
Public
Reference implementations of MLPerf™ inference benchmarks
benchmark machine-learning
Python
•
Apache License 2.0
•539•1.3k•160•14•Updated Dec 22, 2024Dec 22, 2024
medperf
Public
An open benchmarking platform for medical artificial intelligence using Federated Evaluation.
Python
•
Apache License 2.0
•32•148•52•23•Updated Dec 21, 2024Dec 21, 2024
modelbench
Public
Run safety benchmarks against AI models and view detailed reports showing how well they performed.
Python
•
Apache License 2.0
•12•68•174•7•Updated Dec 21, 2024Dec 21, 2024
algorithmic-efficiency
Public
MLCommons Algorithmic Efficiency is a benchmark and competition measuring neural network training speedups due to algorithmic improvements in both training algorithms and models.
Python
•
Apache License 2.0
•71•339•30•11•Updated Dec 21, 2024Dec 21, 2024
ailuminate
Public
Apache License 2.0
•0•0•0•0•Updated Dec 20, 2024Dec 20, 2024
dynabench
Public
Python
•
MIT License
•16•21•7•4•Updated Dec 20, 2024Dec 20, 2024
mlperf_inference_submissions
Public template
Shell
•0•0•0•0•Updated Dec 20, 2024Dec 20, 2024
GaNDLF-Synth
Public
Extension for GaNDLF [gandlf.org] to enable synthesis.
medical-imaging synthesis
Python
•
Apache License 2.0
•2•4•6•1•Updated Dec 20, 2024Dec 20, 2024
ck
Public
Collective Knowledge (CK) and Collective Minds (CM): community-driven projects to learn how to run AI, ML and other emerging workloads in a more efficient and cost-effective way across diverse models, datasets, software and hardware using MLPerf, CM and CK
education automation optimization best-practices collaboration cm modularity workflows portability reusability
Python
•
Apache License 2.0
•121•613•13•0•Updated Dec 20, 2024Dec 20, 2024
GaNDLF
Public
A generalizable application framework for segmentation, regression, and classification using PyTorch
machine-learning framework deep-learning regression pytorch medical-imaging classification segmentation data-augmentation biomedical-image-processing
Python
•
Apache License 2.0
•80•167•15•2•Updated Dec 20, 2024Dec 20, 2024
inference_policies
Public
Issues related to MLPerf™ Inference policies, including rules and suggested changes
Apache License 2.0
•52•57•18•2•Updated Dec 18, 2024Dec 18, 2024
policies
Public
General policies for MLPerf™ including submission rules, coding standards, etc.
Python
•
Apache License 2.0
•54•28•35•38•Updated Dec 18, 2024Dec 18, 2024
power-dev
Public
Dev repo for power measurement for the MLPerf™ benchmarks
Python
•
Apache License 2.0
•25•18•10•3•Updated Dec 17, 2024Dec 17, 2024
mobile_app_open
Public
Mobile App Open
C++
•
Apache License 2.0
•24•49•29•2•Updated Dec 17, 2024Dec 17, 2024
tiny
Public
MLPerf™ Tiny is an ML benchmark suite for extremely low-power systems such as microcontrollers
C
•
Apache License 2.0
•91•362•13•5•Updated Dec 13, 2024Dec 13, 2024
training_results_v4.1
Public
This repository contains the results and code for the MLPerf™ Training v4.1 benchmark.
Python
•4•3•1•0•Updated Dec 11, 2024Dec 11, 2024
mlperf-automations_archived
Public archive
This repository contains the automations and scripts used to automate MLPerf benchmarks (mainly MLPerf inference for now)
automation mlperf
Python
•
Apache License 2.0
•24•0•46•0•Updated Dec 11, 2024Dec 11, 2024
mlperf_client
Public
MLPerf Client is a benchmark for Windows and macOS, focusing on client form factors in ML inference scenarios.
benchmarking machine-learning ml
C++
•
Apache License 2.0
•0•5•0•0•Updated Dec 10, 2024Dec 10, 2024
logging
Public
MLPerf™ logging library
Python
•
Apache License 2.0
•46•31•41•2•Updated Dec 10, 2024Dec 10, 2024
ck-mlops
Public
A collection of portable workflows, automation recipes and components for MLOps in a unified CK format. Note that this repository is outdated - please check the 2nd generation of the CK workflow automation meta-framework with portable MLOps and DevOps components here:
machine-learning automation optimization artificial-intelligence open-science systems reproducibility workflows portability co-design
Python
•
Apache License 2.0
•8•17•0•0•Updated Dec 10, 2024Dec 10, 2024
cm4mlops
Public
A collection of portable, reusable and cross-platform CM automations to modularize MLPerf and simplify the process of building and benchmarking AI systems across diverse models, data sets, software and hardware
workflow devops machine-learning automation cross-platform reusable workflow-automation mlops mlperf
Python
•
Apache License 2.0
•24•17•7•0•Updated Dec 10, 2024Dec 10, 2024
chakra
Public
Repository for MLCommons Chakra schema and tools
Python
•
Apache License 2.0
•40•75•25•9•Updated Dec 6, 2024Dec 6, 2024
croissant
Public
Croissant is a high-level format for machine learning datasets that brings together four rich layers.
machine-learning schema-org json-ld datasets
Python
•
Apache License 2.0
•45•477•120•19•Updated Dec 5, 2024Dec 5, 2024
datasets-knowledge-graphs
Public
Apache License 2.0
•0•0•0•0•Updated Dec 3, 2024Dec 3, 2024
cm4abtf
Public
CM interface and automation recipes for ABTF
workflow automation
Python
•
Apache License 2.0
•6•2•2•0•Updated Nov 29, 2024Nov 29, 2024
common-crawl-dmlr
Public
Apache License 2.0
•0•0•0•1•Updated Nov 26, 2024Nov 26, 2024
cm4mlperf-inference
Public
Apache License 2.0
•35•1•2•2•Updated Nov 21, 2024Nov 21, 2024