Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix intmm benchmark script #141

Merged
merged 12 commits into from
Apr 16, 2024
Merged

Fix intmm benchmark script #141

merged 12 commits into from
Apr 16, 2024

Conversation

msaroufim
Copy link
Member

@msaroufim msaroufim commented Apr 16, 2024

There were a few bugs in this script that I fixed

Test plan was adding this as a github action

name: Run Benchmarks

on:
  push:
    branches:
      - main
  pull_request:
    branches:
      - main
  workflow_dispatch:

jobs:
  benchmark:
    strategy:
      matrix:
        include:
          - name: CUDA 2.2.2
            runs-on: 4-core-ubuntu-gpu-t4
            torch-spec: 'torch==2.2.2'
          - name: CUDA 2.3 RC
            runs-on: 4-core-ubuntu-gpu-t4
            torch-spec: 'torch==2.3.0 --index-url https://download.pytorch.org/whl/test/cu121'
          - name: CUDA Nightly
            runs-on: 4-core-ubuntu-gpu-t4
            torch-spec: '--pre torch --index-url https://download.pytorch.org/whl/nightly/cu121'
          - name: CPU 2.2.2
            runs-on: 32-core-ubuntu
            torch-spec: 'torch==2.2.2 --index-url https://download.pytorch.org/whl/cpu'
          - name: CPU 2.3 RC
            runs-on: 32-core-ubuntu
            torch-spec: 'torch==2.3.0 --index-url https://download.pytorch.org/whl/test/cpu'
          - name: Nightly CPU
            runs-on: 32-core-ubuntu
            torch-spec: '--pre torch --index-url https://download.pytorch.org/whl/nightly/cpu'
    runs-on: ${{ matrix.runs-on }}
    steps:
    - uses: actions/checkout@v2

    - name: Set up Python
      uses: actions/setup-python@v2
      with:
        python-version: '3.9'

    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        pip install ${{ matrix.torch-spec }}
        pip install -r requirements.txt
        pip install -r dev-requirements.txt
    - name: Install package
      run: |
        pip install .
    - name: Run benchmark
      run: |
        cd benchmarks
        python intmm.py --file_path intmm_shapes.csv
        python intmm.py --file_path sam_vit_b_shapes.csv

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Apr 16, 2024
@msaroufim msaroufim requested a review from cpuhrsch April 16, 2024 17:13
@msaroufim msaroufim changed the title Add benchmark.yml Fix benchmark script Apr 16, 2024
@msaroufim msaroufim changed the title Fix benchmark script Fix intmm benchmark script Apr 16, 2024
@msaroufim msaroufim merged commit d76ecc2 into main Apr 16, 2024
7 checks passed
@msaroufim msaroufim deleted the msaroufim/benchmarkci branch April 16, 2024 17:34
dbyoung18 pushed a commit to dbyoung18/ao that referenced this pull request Jul 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants