Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use cephfs-ci storageClass instead of ceph-filesystem #176

Merged
merged 2 commits into from
Nov 16, 2023

Conversation

furiosamg
Copy link
Collaborator

@furiosamg furiosamg commented Nov 14, 2023

Use cephfs-ci storageClass instead of ceph-filesystem

@furiosa-infra
Copy link
Contributor

Pip freeze result for all

pip freeze result
aiobotocore==2.7.0
aiofiles==23.2.1
aiohttp==3.8.6
aiohttp-retry==2.8.3
aioitertools==0.11.0
aiosignal==1.3.1
amqp==5.2.0
annotated-types==0.6.0
antlr4-python3-runtime==4.9.3
appdirs==1.4.4
asttokens==2.4.1
async-timeout==4.0.3
asyncssh==2.14.1
atpublic==4.0
attrs==23.1.0
billiard==4.2.0
boto3==1.28.64
botocore==1.31.64
celery==5.3.5
certifi==2023.7.22
cffi==1.16.0
charset-normalizer==3.3.2
click==8.1.7
click-didyoumean==0.3.0
click-plugins==1.1.1
click-repl==0.3.0
cmake==3.27.7
colorama==0.4.6
comm==0.2.0
configobj==5.0.8
contourpy==1.2.0
cryptography==41.0.5
cycler==0.12.1
Cython==3.0.5
debugpy==1.8.0
decorator==5.1.1
dictdiffer==0.9.0
diskcache==5.6.3
distro==1.8.0
dpath==2.1.6
dulwich==0.21.6
dvc==3.29.0
dvc-data==2.20.0
dvc-http==2.30.2
dvc-objects==1.2.0
dvc-render==0.6.0
dvc-s3==2.23.0
dvc-studio-client==0.15.0
dvc-task==0.3.0
entrypoints==0.4
exceptiongroup==1.1.3
executing==2.0.1
fastjsonschema==2.18.1
filelock==3.13.1
flatten-dict==0.4.2
flufl.lock==7.1.1
fonttools==4.44.0
frozenlist==1.4.0
fsspec==2023.10.0
funcy==2.0
furiosa-common==0.10.0
furiosa-models @ file:///app
furiosa-native-postprocess==0.9.0
furiosa-native-runtime==0.10.2
furiosa-quantizer==0.10.0
furiosa-quantizer-impl==0.10.1
furiosa-runtime==0.10.0
gitdb==4.0.11
GitPython==3.1.40
grandalf==0.8
gto==1.5.0
hydra-core==1.3.2
idna==3.4
importlib-metadata==6.8.0
importlib-resources==6.1.1
iniconfig==2.0.0
ipykernel==6.26.0
ipython==8.17.2
iterative-telemetry==0.0.8
jedi==0.19.1
Jinja2==3.1.2
jmespath==1.0.1
jsonschema==4.19.2
jsonschema-specifications==2023.7.1
jupyter_client==8.6.0
jupyter_core==5.5.0
kiwisolver==1.4.5
kombu==5.3.3
lit==17.0.5
markdown-it-py==3.0.0
MarkupSafe==2.1.3
matplotlib==3.8.1
matplotlib-inline==0.1.6
mdurl==0.1.2
mpmath==1.3.0
multidict==6.0.4
multipledispatch==1.0.0
nbclient==0.6.8
nbformat==5.9.2
nbmake==1.4.6
nest-asyncio==1.5.8
networkx==3.2.1
numpy==1.25.2
nvidia-cublas-cu11==11.10.3.66
nvidia-cuda-cupti-cu11==11.7.101
nvidia-cuda-nvrtc-cu11==11.7.99
nvidia-cuda-runtime-cu11==11.7.99
nvidia-cudnn-cu11==8.5.0.96
nvidia-cufft-cu11==10.9.0.58
nvidia-curand-cu11==10.2.10.91
nvidia-cusolver-cu11==11.4.0.1
nvidia-cusparse-cu11==11.7.4.91
nvidia-nccl-cu11==2.14.3
nvidia-nvtx-cu11==11.7.91
omegaconf==2.3.0
onnx==1.14.1
opencv-python-headless==4.8.0.76
orjson==3.9.10
packaging==23.2
pandas==2.0.3
parso==0.8.3
pathspec==0.11.2
pexpect==4.8.0
Pillow==10.1.0
platformdirs==3.11.0
pluggy==1.3.0
prompt-toolkit==3.0.41
protobuf==4.25.0
psutil==5.9.6
ptyprocess==0.7.0
pure-eval==0.2.2
py-cpuinfo==9.0.0
pyarrow==12.0.1
pycocotools==2.0.7
pycparser==2.21
pydantic==2.5.0
pydantic_core==2.14.1
pydot==1.4.2
pygit2==1.13.2
Pygments==2.16.1
pygtrie==2.5.0
pyparsing==3.1.1
pytest==7.4.3
pytest-asyncio==0.17.2
pytest-benchmark==4.0.0
python-dateutil==2.8.2
pytz==2023.3.post1
PyYAML==6.0.1
pyzmq==25.1.1
referencing==0.30.2
requests==2.31.0
rich==13.6.0
rpds-py==0.12.0
ruamel.yaml==0.18.5
ruamel.yaml.clib==0.2.8
s3fs==2023.10.0
s3transfer==0.7.0
scmrepo==1.4.1
semver==3.0.2
shortuuid==1.0.11
shtab==1.6.4
six==1.16.0
smmap==5.0.1
sqltrie==0.8.0
stack-data==0.6.3
sympy==1.12
tabulate==0.9.0
tomli==2.0.1
tomlkit==0.12.2
torch==2.0.1
torchvision==0.15.2
tornado==6.3.3
tqdm==4.66.1
traitlets==5.13.0
triton==2.0.0
typer==0.9.0
typing_extensions==4.8.0
tzdata==2023.3
urllib3==1.26.18
vine==5.1.0
voluptuous==0.14.0
wcwidth==0.2.10
wrapt==1.16.0
yarl==1.9.2
zc.lockfile==3.0.post1
zipp==3.17.0

@furiosa-infra
Copy link
Contributor

fc31a38 WIP: use new PVC and run regression tests


--------------------------------------------------------------------------------------------------------- benchmark: 9 tests --------------------------------------------------------------------------------------------------------
Name (time in ms)                                                 Min                   Max                Mean             StdDev              Median                IQR            Outliers       OPS            Rounds  Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_mlcommons_resnet50_accuracy                               1.2727 (1.0)         72.3006 (5.80)       5.2407 (1.0)       1.8776 (2.22)       4.8868 (1.0)       1.4968 (2.30)    5909;3718  190.8139 (1.0)       50000           1
test_mlcommons_ssd_mobilenet_with_native_rust_pp_accuracy      4.0567 (3.19)        12.4615 (1.0)        8.0037 (1.53)      0.8469 (1.0)        8.0417 (1.65)      0.6500 (1.0)       986;633  124.9424 (0.65)       5000           1
test_efficientnetb0_accuracy                                   4.5938 (3.61)     3,193.7534 (256.29)    16.2317 (3.10)     26.0472 (30.76)     13.5194 (2.77)      3.8784 (5.97)     781;2667   61.6079 (0.32)      50000           1
test_mlcommons_ssd_mobilenet_accuracy                          5.3209 (4.18)       226.5520 (18.18)     10.8848 (2.08)      3.2083 (3.79)      10.8549 (2.22)      0.8258 (1.27)       46;542   91.8716 (0.48)       5000           1
test_efficientnetv2s_accuracy                                  6.6764 (5.25)       714.4169 (57.33)     21.3216 (4.07)     23.4077 (27.64)     16.6419 (3.41)      5.1365 (7.90)    2419;4555   46.9009 (0.25)      50000           1
test_yolov5m_accuracy                                          7.9854 (6.27)        25.3976 (2.04)      11.0138 (2.10)      1.4750 (1.74)      10.7632 (2.20)      1.7777 (2.73)     2733;280   90.7952 (0.48)      10000           1
test_yolov5l_accuracy                                         10.8855 (8.55)        52.3222 (4.20)      14.7350 (2.81)      1.6811 (1.99)      14.4566 (2.96)      1.9766 (3.04)     2539;260   67.8657 (0.36)      10000           1
test_mlcommons_ssd_resnet34_with_native_rust_pp_accuracy      36.2243 (28.46)      762.0383 (61.15)     52.7862 (10.07)    17.0206 (20.10)     52.1312 (10.67)     3.0451 (4.68)        6;221   18.9444 (0.10)       5000           1
test_mlcommons_ssd_resnet34_accuracy                          93.7459 (73.66)    1,151.5121 (92.41)    123.1531 (23.50)    26.4997 (31.29)    120.1101 (24.58)    16.9555 (26.09)     244;148    8.1200 (0.04)       5000           1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
  Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
  OPS: Operations Per Second, computed as 1 / Mean

@furiosamg furiosamg force-pushed the use-cephfs-ci branch 2 times, most recently from 6324f52 to eeb9df2 Compare November 15, 2023 04:36
@furiosamg furiosamg self-assigned this Nov 15, 2023
@furiosamg furiosamg marked this pull request as ready for review November 15, 2023 05:01
@furiosamg furiosamg mentioned this pull request Nov 15, 2023
@furiosamg
Copy link
Collaborator Author

I've confirmed that it's a simple modification and it works, so will merge it.

@furiosamg furiosamg merged commit 81fcc88 into furiosa-ai:main Nov 16, 2023
1 check passed
@furiosamg furiosamg deleted the use-cephfs-ci branch November 16, 2023 05:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants