Skip to content
Self-hosted runner (nightly-past-ci-caller)

Self-hosted runner (nightly-past-ci-caller) #300

Sign in to view logs

Self-hosted runner (nightly-past-ci-caller)

Self-hosted runner (nightly-past-ci-caller) #300

Triggered via schedule December 20, 2024 02:39
@CemberkCemberk
⁠ 3c34382
main
Status Failure
Total duration 1m 18s
Artifacts 1

self-nightly-past-ci-caller.yml

on: schedule
Get number
0s
Get number
Matrix: PyTorch 1.11 / DeepSpeed CI / Setup
Waiting for pending jobs
Matrix: PyTorch 1.11 / Model CI / Setup
Waiting for pending jobs
Matrix: PyTorch 1.12 / DeepSpeed CI / Setup
Waiting for pending jobs
Matrix: PyTorch 1.12 / Model CI / Setup
Waiting for pending jobs
Matrix: PyTorch 1.13 / DeepSpeed CI / Setup
Matrix: PyTorch 1.13 / Model CI / Setup
Matrix: TensorFlow 2.10 / DeepSpeed CI / Setup
Waiting for pending jobs
Matrix: TensorFlow 2.10 / Model CI / Setup
Waiting for pending jobs
Matrix: TensorFlow 2.11 / DeepSpeed CI / Setup
Waiting for pending jobs
Matrix: TensorFlow 2.11 / Model CI / Setup
Waiting for pending jobs
Matrix: TensorFlow 2.5 / DeepSpeed CI / Setup
Waiting for pending jobs
Matrix: TensorFlow 2.5 / Model CI / Setup
Waiting for pending jobs
Matrix: TensorFlow 2.6 / DeepSpeed CI / Setup
Waiting for pending jobs
Matrix: TensorFlow 2.6 / Model CI / Setup
Waiting for pending jobs
Matrix: TensorFlow 2.7 / DeepSpeed CI / Setup
Waiting for pending jobs
Matrix: TensorFlow 2.7 / Model CI / Setup
Waiting for pending jobs
Matrix: TensorFlow 2.8 / DeepSpeed CI / Setup
Waiting for pending jobs
Matrix: TensorFlow 2.8 / Model CI / Setup
Waiting for pending jobs
Matrix: TensorFlow 2.9 / DeepSpeed CI / Setup
Waiting for pending jobs
Matrix: TensorFlow 2.9 / Model CI / Setup
Waiting for pending jobs
Matrix: PyTorch 1.11 / DeepSpeed CI / Examples directory
Waiting for pending jobs
Matrix: PyTorch 1.11 / DeepSpeed CI / TensorFlow pipelines
Waiting for pending jobs
Matrix: PyTorch 1.11 / DeepSpeed CI / PyTorch pipelines
Waiting for pending jobs
Matrix: PyTorch 1.11 / DeepSpeed CI / Torch CUDA extension tests
Waiting for pending jobs
Matrix: PyTorch 1.11 / Model CI / Examples directory
Waiting for pending jobs
Matrix: PyTorch 1.11 / Model CI / TensorFlow pipelines
Waiting for pending jobs
Matrix: PyTorch 1.11 / Model CI / PyTorch pipelines
Waiting for pending jobs
Matrix: PyTorch 1.11 / Model CI / Torch CUDA extension tests
Waiting for pending jobs
Matrix: PyTorch 1.12 / DeepSpeed CI / Examples directory
Waiting for pending jobs
Matrix: PyTorch 1.12 / DeepSpeed CI / TensorFlow pipelines
Waiting for pending jobs
Matrix: PyTorch 1.12 / DeepSpeed CI / PyTorch pipelines
Waiting for pending jobs
Matrix: PyTorch 1.12 / DeepSpeed CI / Torch CUDA extension tests
Waiting for pending jobs
Matrix: PyTorch 1.12 / Model CI / Examples directory
Waiting for pending jobs
Matrix: PyTorch 1.12 / Model CI / TensorFlow pipelines
Waiting for pending jobs
Matrix: PyTorch 1.12 / Model CI / PyTorch pipelines
Waiting for pending jobs
Matrix: PyTorch 1.12 / Model CI / Torch CUDA extension tests
Waiting for pending jobs
Matrix: PyTorch 1.13 / DeepSpeed CI / Examples directory
Matrix: PyTorch 1.13 / DeepSpeed CI / TensorFlow pipelines
Matrix: PyTorch 1.13 / DeepSpeed CI / PyTorch pipelines
Matrix: PyTorch 1.13 / DeepSpeed CI / Torch CUDA extension tests
Matrix: PyTorch 1.13 / Model CI / Examples directory
Matrix: PyTorch 1.13 / Model CI / TensorFlow pipelines
Matrix: PyTorch 1.13 / Model CI / PyTorch pipelines
Matrix: PyTorch 1.13 / Model CI / Torch CUDA extension tests
Matrix: TensorFlow 2.10 / DeepSpeed CI / Examples directory
Waiting for pending jobs
Matrix: TensorFlow 2.10 / DeepSpeed CI / TensorFlow pipelines
Waiting for pending jobs
Matrix: TensorFlow 2.10 / DeepSpeed CI / PyTorch pipelines
Waiting for pending jobs
Matrix: TensorFlow 2.10 / DeepSpeed CI / Torch CUDA extension tests
Waiting for pending jobs
Matrix: TensorFlow 2.10 / Model CI / Examples directory
Waiting for pending jobs
Matrix: TensorFlow 2.10 / Model CI / TensorFlow pipelines
Waiting for pending jobs
Matrix: TensorFlow 2.10 / Model CI / PyTorch pipelines
Waiting for pending jobs
Matrix: TensorFlow 2.10 / Model CI / Torch CUDA extension tests
Waiting for pending jobs
Matrix: TensorFlow 2.11 / DeepSpeed CI / Examples directory
Waiting for pending jobs
Matrix: TensorFlow 2.11 / DeepSpeed CI / TensorFlow pipelines
Waiting for pending jobs
Matrix: TensorFlow 2.11 / DeepSpeed CI / PyTorch pipelines
Waiting for pending jobs
Matrix: TensorFlow 2.11 / DeepSpeed CI / Torch CUDA extension tests
Waiting for pending jobs
Matrix: TensorFlow 2.11 / Model CI / Examples directory
Waiting for pending jobs
Matrix: TensorFlow 2.11 / Model CI / TensorFlow pipelines
Waiting for pending jobs
Matrix: TensorFlow 2.11 / Model CI / PyTorch pipelines
Waiting for pending jobs
Matrix: TensorFlow 2.11 / Model CI / Torch CUDA extension tests
Waiting for pending jobs
Matrix: TensorFlow 2.5 / DeepSpeed CI / Examples directory
Waiting for pending jobs
Matrix: TensorFlow 2.5 / DeepSpeed CI / TensorFlow pipelines
Waiting for pending jobs
Matrix: TensorFlow 2.5 / DeepSpeed CI / PyTorch pipelines
Waiting for pending jobs
Matrix: TensorFlow 2.5 / DeepSpeed CI / Torch CUDA extension tests
Waiting for pending jobs
Matrix: TensorFlow 2.5 / Model CI / Examples directory
Waiting for pending jobs
Matrix: TensorFlow 2.5 / Model CI / TensorFlow pipelines
Waiting for pending jobs
Matrix: TensorFlow 2.5 / Model CI / PyTorch pipelines
Waiting for pending jobs
Matrix: TensorFlow 2.5 / Model CI / Torch CUDA extension tests
Waiting for pending jobs
Matrix: TensorFlow 2.6 / DeepSpeed CI / Examples directory
Waiting for pending jobs
Matrix: TensorFlow 2.6 / DeepSpeed CI / TensorFlow pipelines
Waiting for pending jobs
Matrix: TensorFlow 2.6 / DeepSpeed CI / PyTorch pipelines
Waiting for pending jobs
Matrix: TensorFlow 2.6 / DeepSpeed CI / Torch CUDA extension tests
Waiting for pending jobs
Matrix: TensorFlow 2.6 / Model CI / Examples directory
Waiting for pending jobs
Matrix: TensorFlow 2.6 / Model CI / TensorFlow pipelines
Waiting for pending jobs
Matrix: TensorFlow 2.6 / Model CI / PyTorch pipelines
Waiting for pending jobs
Matrix: TensorFlow 2.6 / Model CI / Torch CUDA extension tests
Waiting for pending jobs
Matrix: TensorFlow 2.7 / DeepSpeed CI / Examples directory
Waiting for pending jobs
Matrix: TensorFlow 2.7 / DeepSpeed CI / TensorFlow pipelines
Waiting for pending jobs
Matrix: TensorFlow 2.7 / DeepSpeed CI / PyTorch pipelines
Waiting for pending jobs
Matrix: TensorFlow 2.7 / DeepSpeed CI / Torch CUDA extension tests
Waiting for pending jobs
Matrix: TensorFlow 2.7 / Model CI / Examples directory
Waiting for pending jobs
Matrix: TensorFlow 2.7 / Model CI / TensorFlow pipelines
Waiting for pending jobs
Matrix: TensorFlow 2.7 / Model CI / PyTorch pipelines
Waiting for pending jobs
Matrix: TensorFlow 2.7 / Model CI / Torch CUDA extension tests
Waiting for pending jobs
Matrix: TensorFlow 2.8 / DeepSpeed CI / Examples directory
Waiting for pending jobs
Matrix: TensorFlow 2.8 / DeepSpeed CI / TensorFlow pipelines
Waiting for pending jobs
Matrix: TensorFlow 2.8 / DeepSpeed CI / PyTorch pipelines
Waiting for pending jobs
Matrix: TensorFlow 2.8 / DeepSpeed CI / Torch CUDA extension tests
Waiting for pending jobs
Matrix: TensorFlow 2.8 / Model CI / Examples directory
Waiting for pending jobs
Matrix: TensorFlow 2.8 / Model CI / TensorFlow pipelines
Waiting for pending jobs
Matrix: TensorFlow 2.8 / Model CI / PyTorch pipelines
Waiting for pending jobs
Matrix: TensorFlow 2.8 / Model CI / Torch CUDA extension tests
Waiting for pending jobs
Matrix: TensorFlow 2.9 / DeepSpeed CI / Examples directory
Waiting for pending jobs
Matrix: TensorFlow 2.9 / DeepSpeed CI / TensorFlow pipelines
Waiting for pending jobs
Matrix: TensorFlow 2.9 / DeepSpeed CI / PyTorch pipelines
Waiting for pending jobs
Matrix: TensorFlow 2.9 / DeepSpeed CI / Torch CUDA extension tests
Waiting for pending jobs
Matrix: TensorFlow 2.9 / Model CI / Examples directory
Waiting for pending jobs
Matrix: TensorFlow 2.9 / Model CI / TensorFlow pipelines
Waiting for pending jobs
Matrix: TensorFlow 2.9 / Model CI / PyTorch pipelines
Waiting for pending jobs
Matrix: TensorFlow 2.9 / Model CI / Torch CUDA extension tests
Waiting for pending jobs
Matrix: PyTorch 1.11 / DeepSpeed CI /
Waiting for pending jobs
Matrix: PyTorch 1.11 / DeepSpeed CI /
Waiting for pending jobs
Matrix: PyTorch 1.11 / Model CI /
Waiting for pending jobs
Matrix: PyTorch 1.11 / Model CI /
Waiting for pending jobs
Matrix: PyTorch 1.12 / DeepSpeed CI /
Waiting for pending jobs
Matrix: PyTorch 1.12 / DeepSpeed CI /
Waiting for pending jobs
Matrix: PyTorch 1.12 / Model CI /
Waiting for pending jobs
Matrix: PyTorch 1.12 / Model CI /
Waiting for pending jobs
Matrix: PyTorch 1.13 / DeepSpeed CI /
Waiting for pending jobs
Matrix: PyTorch 1.13 / DeepSpeed CI /
Matrix: PyTorch 1.13 / Model CI /
Waiting for pending jobs
Matrix: PyTorch 1.13 / Model CI /
Matrix: TensorFlow 2.10 / DeepSpeed CI /
Waiting for pending jobs
Matrix: TensorFlow 2.10 / DeepSpeed CI /
Waiting for pending jobs
Matrix: TensorFlow 2.10 / Model CI /
Waiting for pending jobs
Matrix: TensorFlow 2.10 / Model CI /
Waiting for pending jobs
Matrix: TensorFlow 2.11 / DeepSpeed CI /
Waiting for pending jobs
Matrix: TensorFlow 2.11 / DeepSpeed CI /
Waiting for pending jobs
Matrix: TensorFlow 2.11 / Model CI /
Waiting for pending jobs
Matrix: TensorFlow 2.11 / Model CI /
Waiting for pending jobs
Matrix: TensorFlow 2.5 / DeepSpeed CI /
Waiting for pending jobs
Matrix: TensorFlow 2.5 / DeepSpeed CI /
Waiting for pending jobs
Matrix: TensorFlow 2.5 / Model CI /
Waiting for pending jobs
Matrix: TensorFlow 2.5 / Model CI /
Waiting for pending jobs
Matrix: TensorFlow 2.6 / DeepSpeed CI /
Waiting for pending jobs
Matrix: TensorFlow 2.6 / DeepSpeed CI /
Waiting for pending jobs
Matrix: TensorFlow 2.6 / Model CI /
Waiting for pending jobs
Matrix: TensorFlow 2.6 / Model CI /
Waiting for pending jobs
Matrix: TensorFlow 2.7 / DeepSpeed CI /
Waiting for pending jobs
Matrix: TensorFlow 2.7 / DeepSpeed CI /
Waiting for pending jobs
Matrix: TensorFlow 2.7 / Model CI /
Waiting for pending jobs
Matrix: TensorFlow 2.7 / Model CI /
Waiting for pending jobs
Matrix: TensorFlow 2.8 / DeepSpeed CI /
Waiting for pending jobs
Matrix: TensorFlow 2.8 / DeepSpeed CI /
Waiting for pending jobs
Matrix: TensorFlow 2.8 / Model CI /
Waiting for pending jobs
Matrix: TensorFlow 2.8 / Model CI /
Waiting for pending jobs
Matrix: TensorFlow 2.9 / DeepSpeed CI /
Waiting for pending jobs
Matrix: TensorFlow 2.9 / DeepSpeed CI /
Waiting for pending jobs
Matrix: TensorFlow 2.9 / Model CI /
Waiting for pending jobs
Matrix: TensorFlow 2.9 / Model CI /
Waiting for pending jobs
PyTorch 1.11  /  ...  /  Extract warnings in CI artifacts
PyTorch 1.11 / DeepSpeed CI / Extract warnings in CI artifacts
PyTorch 1.11  /  ...  /  Extract warnings in CI artifacts
PyTorch 1.11 / Model CI / Extract warnings in CI artifacts
PyTorch 1.12  /  ...  /  Extract warnings in CI artifacts
PyTorch 1.12 / DeepSpeed CI / Extract warnings in CI artifacts
PyTorch 1.12  /  ...  /  Extract warnings in CI artifacts
PyTorch 1.12 / Model CI / Extract warnings in CI artifacts
PyTorch 1.13  /  ...  /  Extract warnings in CI artifacts
0s
PyTorch 1.13 / DeepSpeed CI / Extract warnings in CI artifacts
PyTorch 1.13  /  ...  /  Extract warnings in CI artifacts
18s
PyTorch 1.13 / Model CI / Extract warnings in CI artifacts
TensorFlow 2.10  /  ...  /  Extract warnings in CI artifacts
TensorFlow 2.10 / DeepSpeed CI / Extract warnings in CI artifacts
TensorFlow 2.10  /  ...  /  Extract warnings in CI artifacts
TensorFlow 2.10 / Model CI / Extract warnings in CI artifacts
TensorFlow 2.11  /  ...  /  Extract warnings in CI artifacts
TensorFlow 2.11 / DeepSpeed CI / Extract warnings in CI artifacts
TensorFlow 2.11  /  ...  /  Extract warnings in CI artifacts
TensorFlow 2.11 / Model CI / Extract warnings in CI artifacts
TensorFlow 2.5  /  ...  /  Extract warnings in CI artifacts
TensorFlow 2.5 / DeepSpeed CI / Extract warnings in CI artifacts
TensorFlow 2.5  /  ...  /  Extract warnings in CI artifacts
TensorFlow 2.5 / Model CI / Extract warnings in CI artifacts
TensorFlow 2.6  /  ...  /  Extract warnings in CI artifacts
TensorFlow 2.6 / DeepSpeed CI / Extract warnings in CI artifacts
TensorFlow 2.6  /  ...  /  Extract warnings in CI artifacts
TensorFlow 2.6 / Model CI / Extract warnings in CI artifacts
TensorFlow 2.7  /  ...  /  Extract warnings in CI artifacts
TensorFlow 2.7 / DeepSpeed CI / Extract warnings in CI artifacts
TensorFlow 2.7  /  ...  /  Extract warnings in CI artifacts
TensorFlow 2.7 / Model CI / Extract warnings in CI artifacts
TensorFlow 2.8  /  ...  /  Extract warnings in CI artifacts
TensorFlow 2.8 / DeepSpeed CI / Extract warnings in CI artifacts
TensorFlow 2.8  /  ...  /  Extract warnings in CI artifacts
TensorFlow 2.8 / Model CI / Extract warnings in CI artifacts
TensorFlow 2.9  /  ...  /  Extract warnings in CI artifacts
TensorFlow 2.9 / DeepSpeed CI / Extract warnings in CI artifacts
TensorFlow 2.9  /  ...  /  Extract warnings in CI artifacts
TensorFlow 2.9 / Model CI / Extract warnings in CI artifacts
PyTorch 1.11  /  ...  /  Send results to webhook
PyTorch 1.11 / DeepSpeed CI / Slack Report / Send results to webhook
PyTorch 1.11  /  ...  /  Send results to webhook
PyTorch 1.11 / Model CI / Slack Report / Send results to webhook
PyTorch 1.12  /  ...  /  Send results to webhook
PyTorch 1.12 / DeepSpeed CI / Slack Report / Send results to webhook
PyTorch 1.12  /  ...  /  Send results to webhook
PyTorch 1.12 / Model CI / Slack Report / Send results to webhook
PyTorch 1.13  /  ...  /  Send results to webhook
12s
PyTorch 1.13 / DeepSpeed CI / Slack Report / Send results to webhook
PyTorch 1.13  /  ...  /  Send results to webhook
14s
PyTorch 1.13 / Model CI / Slack Report / Send results to webhook
TensorFlow 2.10  /  ...  /  Send results to webhook
TensorFlow 2.10 / DeepSpeed CI / Slack Report / Send results to webhook
TensorFlow 2.10  /  ...  /  Send results to webhook
TensorFlow 2.10 / Model CI / Slack Report / Send results to webhook
TensorFlow 2.11  /  ...  /  Send results to webhook
TensorFlow 2.11 / DeepSpeed CI / Slack Report / Send results to webhook
TensorFlow 2.11  /  ...  /  Send results to webhook
TensorFlow 2.11 / Model CI / Slack Report / Send results to webhook
TensorFlow 2.5  /  ...  /  Send results to webhook
TensorFlow 2.5 / DeepSpeed CI / Slack Report / Send results to webhook
TensorFlow 2.5  /  ...  /  Send results to webhook
TensorFlow 2.5 / Model CI / Slack Report / Send results to webhook
TensorFlow 2.6  /  ...  /  Send results to webhook
TensorFlow 2.6 / DeepSpeed CI / Slack Report / Send results to webhook
TensorFlow 2.6  /  ...  /  Send results to webhook
TensorFlow 2.6 / Model CI / Slack Report / Send results to webhook
TensorFlow 2.7  /  ...  /  Send results to webhook
TensorFlow 2.7 / DeepSpeed CI / Slack Report / Send results to webhook
TensorFlow 2.7  /  ...  /  Send results to webhook
TensorFlow 2.7 / Model CI / Slack Report / Send results to webhook
TensorFlow 2.8  /  ...  /  Send results to webhook
TensorFlow 2.8 / DeepSpeed CI / Slack Report / Send results to webhook
TensorFlow 2.8  /  ...  /  Send results to webhook
TensorFlow 2.8 / Model CI / Slack Report / Send results to webhook
TensorFlow 2.9  /  ...  /  Send results to webhook
TensorFlow 2.9 / DeepSpeed CI / Slack Report / Send results to webhook
TensorFlow 2.9  /  ...  /  Send results to webhook
TensorFlow 2.9 / Model CI / Slack Report / Send results to webhook
PyTorch 1.11  /  ...  /  
PyTorch 1.11 / DeepSpeed CI / Check new model failures /
PyTorch 1.11  /  ...  /  
PyTorch 1.11 / Model CI / Check new model failures /
PyTorch 1.12  /  ...  /  
PyTorch 1.12 / DeepSpeed CI / Check new model failures /
PyTorch 1.12  /  ...  /  
PyTorch 1.12 / Model CI / Check new model failures /
PyTorch 1.13  /  ...  /  
PyTorch 1.13 / DeepSpeed CI / Check new model failures /
PyTorch 1.13  /  ...  /  
PyTorch 1.13 / Model CI / Check new model failures /
TensorFlow 2.10  /  ...  /  
TensorFlow 2.10 / DeepSpeed CI / Check new model failures /
TensorFlow 2.10  /  ...  /  
TensorFlow 2.10 / Model CI / Check new model failures /
TensorFlow 2.11  /  ...  /  
TensorFlow 2.11 / DeepSpeed CI / Check new model failures /
TensorFlow 2.11  /  ...  /  
TensorFlow 2.11 / Model CI / Check new model failures /
TensorFlow 2.5  /  ...  /  
TensorFlow 2.5 / DeepSpeed CI / Check new model failures /
TensorFlow 2.5  /  ...  /  
TensorFlow 2.5 / Model CI / Check new model failures /
TensorFlow 2.6  /  ...  /  
TensorFlow 2.6 / DeepSpeed CI / Check new model failures /
TensorFlow 2.6  /  ...  /  
TensorFlow 2.6 / Model CI / Check new model failures /
TensorFlow 2.7  /  ...  /  
TensorFlow 2.7 / DeepSpeed CI / Check new model failures /
TensorFlow 2.7  /  ...  /  
TensorFlow 2.7 / Model CI / Check new model failures /
TensorFlow 2.8  /  ...  /  
TensorFlow 2.8 / DeepSpeed CI / Check new model failures /
TensorFlow 2.8  /  ...  /  
TensorFlow 2.8 / Model CI / Check new model failures /
TensorFlow 2.9  /  ...  /  
TensorFlow 2.9 / DeepSpeed CI / Check new model failures /
TensorFlow 2.9  /  ...  /  
TensorFlow 2.9 / Model CI / Check new model failures /
Fit to window
Zoom out
Zoom in

Annotations

6 errors
PyTorch 1.13 / Model CI / Setup (aws-g4dn-12xlarge-cache)
Unable to find self-hosted runner group: 'aws-g4dn-12xlarge-cache'.
PyTorch 1.13 / DeepSpeed CI / Torch CUDA extension tests (aws-g4dn-2xlarge-cache)
Unable to find self-hosted runner group: 'aws-g4dn-2xlarge-cache'.
PyTorch 1.13 / Model CI / Setup (aws-g4dn-2xlarge-cache)
Unable to find self-hosted runner group: 'aws-g4dn-2xlarge-cache'.
PyTorch 1.13 / DeepSpeed CI / Torch CUDA extension tests (aws-g4dn-12xlarge-cache)
Unable to find self-hosted runner group: 'aws-g4dn-12xlarge-cache'.
PyTorch 1.13 / DeepSpeed CI / Slack Report / Send results to webhook
Process completed with exit code 1.
PyTorch 1.13 / Model CI / Slack Report / Send results to webhook
Process completed with exit code 1.

Artifacts

Produced during runtime
Name Size
warnings_in_ci
162 Bytes