Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CI update] Try to use dockers and no cache #29202

Merged
merged 234 commits into from
May 6, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
234 commits
Select commit Hold shift + click to select a range
330f3b8
change cis
ArthurZucker Feb 22, 2024
7638069
Merge branch 'main' of github.com:huggingface/transformers into chang…
ArthurZucker Apr 26, 2024
8d83417
nits
ArthurZucker Apr 26, 2024
b2d4677
update
ArthurZucker Apr 26, 2024
0c5cd63
minor updates
ArthurZucker Apr 26, 2024
fcb03b5
[push-ci-image]
ArthurZucker Apr 26, 2024
f2497de
nit [push-ci-image]
ArthurZucker Apr 26, 2024
d7d09ae
nitsssss
ArthurZucker Apr 26, 2024
3a1a922
[build-ci-image]
ArthurZucker Apr 26, 2024
53c341c
[push-ci-image]
ArthurZucker Apr 26, 2024
1e3f97f
[push-ci-image]
ArthurZucker Apr 26, 2024
650e6a6
both
ArthurZucker Apr 26, 2024
c3c0221
[push-ci-image]
ArthurZucker Apr 26, 2024
bf29d95
this?
ArthurZucker Apr 26, 2024
dc9de14
[push-ci-image]
ArthurZucker Apr 26, 2024
4048bf8
pypi-kenlm needs g++
ArthurZucker Apr 26, 2024
2cb78cf
[push-ci-image]
ArthurZucker Apr 26, 2024
85c9d0d
nit
ArthurZucker Apr 26, 2024
226e952
more nits [push-ci-image]
ArthurZucker Apr 26, 2024
08fb481
nits [push-ci-image]
ArthurZucker Apr 26, 2024
bd16314
[push-ci-image]
ArthurZucker Apr 26, 2024
e0238df
[push-ci-image]
ArthurZucker Apr 26, 2024
32ff594
[push-ci-image]
ArthurZucker Apr 26, 2024
60b0e7c
add vision
ArthurZucker Apr 27, 2024
8b93057
[push-ci-image]
ArthurZucker Apr 27, 2024
e887c32
[push-ci-image]
ArthurZucker Apr 28, 2024
f48e924
add new dummy file but will need to update them [push-ci-image]
ArthurZucker Apr 28, 2024
dcb610b
[push-ci-image]
ArthurZucker Apr 28, 2024
3b22f18
show package size as well
ArthurZucker Apr 28, 2024
8379467
[push-ci-image]
ArthurZucker Apr 28, 2024
901571d
potentially ignore failures
ArthurZucker Apr 28, 2024
28db44a
workflow updates
ArthurZucker Apr 28, 2024
aa62903
nits [push-ci-image]
ArthurZucker Apr 28, 2024
fdb9e25
[push-ci-image]
ArthurZucker Apr 28, 2024
51bf712
fix consistency
ArthurZucker Apr 28, 2024
d405dd8
clean nciida triton
ArthurZucker Apr 28, 2024
3161df6
also show big packages [push-ci-image]
ArthurZucker Apr 28, 2024
46411b8
nit
ArthurZucker Apr 28, 2024
0341da8
update
ArthurZucker Apr 28, 2024
15505e2
another one
ArthurZucker Apr 28, 2024
cbde518
line escape?
ArthurZucker Apr 28, 2024
0431c02
add accelerate [push-ci-image]
ArthurZucker Apr 28, 2024
8d4e9d0
updates [push-ci-image]
ArthurZucker Apr 28, 2024
ccaa259
nits to run tests, no push-ci
ArthurZucker Apr 28, 2024
db6f724
try to parse skip reason to make sure nothing is skipped that should …
ArthurZucker Apr 28, 2024
a502cc1
nit?
ArthurZucker Apr 28, 2024
ca14571
always show skipped reasons
ArthurZucker Apr 28, 2024
69eaa61
nits
ArthurZucker Apr 28, 2024
3690fcb
better parsing of the test outputs
ArthurZucker Apr 28, 2024
fee82f0
action="store_true",
ArthurZucker Apr 28, 2024
fc38bcb
failure on failed
ArthurZucker Apr 28, 2024
6b1f4a2
show matched
ArthurZucker Apr 28, 2024
6485c26
debug
ArthurZucker Apr 28, 2024
d17dad9
update short summary with skipped, failed and errors
ArthurZucker Apr 29, 2024
d1eefa3
nits
ArthurZucker Apr 29, 2024
fef09d3
nits
ArthurZucker Apr 29, 2024
5fcb727
coolu pdates
ArthurZucker Apr 29, 2024
b4c9c1c
remove docbuilder
ArthurZucker Apr 29, 2024
8c89d25
fix
ArthurZucker Apr 29, 2024
78efc6f
always run checks
ArthurZucker Apr 29, 2024
55ab39e
oups
ArthurZucker Apr 29, 2024
6ea5196
nits
ArthurZucker Apr 29, 2024
f7fe098
don't error out on library printing
ArthurZucker Apr 29, 2024
ec6cd4f
non zero exi codes
ArthurZucker Apr 29, 2024
bfbd954
no warning
ArthurZucker Apr 29, 2024
c50c7ae
nit
ArthurZucker Apr 29, 2024
b896bc8
WAT?
ArthurZucker Apr 29, 2024
8e74d47
format nit
ArthurZucker Apr 29, 2024
a9f097c
[push-ci-image]
ArthurZucker Apr 29, 2024
c27e9bc
fail if fail is needed
ArthurZucker Apr 29, 2024
9b00c92
[push-ci-image]
ArthurZucker Apr 29, 2024
3d38101
sound file for torch light?
ArthurZucker Apr 29, 2024
cf0a318
[push-ci-image]
ArthurZucker Apr 29, 2024
af0c2be
order is important [push-ci-image]
ArthurZucker Apr 29, 2024
96f4be4
[push-ci-image] reduce even further
ArthurZucker Apr 29, 2024
57f850c
[push-ci-image]
ArthurZucker Apr 29, 2024
28aa1eb
use pytest rich !
ArthurZucker Apr 29, 2024
1fd0410
yes [push-ci-image]
ArthurZucker Apr 29, 2024
ae3cf06
oupsy
ArthurZucker Apr 29, 2024
de29ea2
bring back the full traceback, but pytest rich should help
ArthurZucker Apr 29, 2024
729cc07
nit
ArthurZucker Apr 29, 2024
660c153
[push-ci-image]
ArthurZucker Apr 29, 2024
39ea086
re run
ArthurZucker Apr 29, 2024
582720e
nit
ArthurZucker Apr 29, 2024
dc16067
[push-ci-image]
ArthurZucker Apr 30, 2024
7d03c73
[push-ci-image]
ArthurZucker Apr 30, 2024
d469c65
[push-ci-image]
ArthurZucker Apr 30, 2024
34e6913
empty push to trigger
ArthurZucker Apr 30, 2024
11eae2d
[push-ci-image]
ArthurZucker Apr 30, 2024
0fa8f98
nit? [push-ci-image]
ArthurZucker Apr 30, 2024
6ddae9b
empty
ArthurZucker Apr 30, 2024
9a7a394
Merge branch 'main' of github.com:huggingface/transformers into chang…
ArthurZucker Apr 30, 2024
7d289f0
try to install timm with no deps
ArthurZucker Apr 30, 2024
2abef24
[push-ci-image]
ArthurZucker Apr 30, 2024
5a0e43a
oups [push-ci-image]
ArthurZucker Apr 30, 2024
302a871
[push-ci-image]
ArthurZucker Apr 30, 2024
7931086
[push-ci-image] ?
ArthurZucker Apr 30, 2024
5f9f1c6
[push-ci-image] open ssh client for git checkout fast
ArthurZucker Apr 30, 2024
b2b1149
empty for torch light
ArthurZucker Apr 30, 2024
17a582b
updates [push-ci-image]
ArthurZucker May 1, 2024
2f24af9
nit
ArthurZucker May 1, 2024
1b5ffa2
@v4 for checkout
ArthurZucker May 1, 2024
693b54e
[push-ci-image]
ArthurZucker May 1, 2024
3ecf0ec
[push-ci-image]
ArthurZucker May 1, 2024
52b35f7
fix fetch tests with parallelism
ArthurZucker May 1, 2024
85ec7d8
[push-ci-image]
ArthurZucker May 1, 2024
ea49941
more parallelism
ArthurZucker May 1, 2024
c49e33f
nit
ArthurZucker May 1, 2024
fd9333b
more nits
ArthurZucker May 1, 2024
0c42346
empty to re-trigger
ArthurZucker May 1, 2024
6165b52
empty to re-trigger
ArthurZucker May 1, 2024
bff8160
split by timing
ArthurZucker May 1, 2024
87b31ae
did not work with previous commit
ArthurZucker May 1, 2024
a3cd9e4
junit.xml
ArthurZucker May 1, 2024
d08dc2c
no path?
ArthurZucker May 1, 2024
6f91c5e
mmm this?
ArthurZucker May 1, 2024
dd6832b
junitxml format
ArthurZucker May 1, 2024
f6810dc
split by timing
ArthurZucker May 1, 2024
1375b25
nit
ArthurZucker May 1, 2024
fa1e131
fix junit family
ArthurZucker May 1, 2024
00cdc41
now we can test if the xunit1 is compatible!
ArthurZucker May 1, 2024
18f8ac2
this?
ArthurZucker May 1, 2024
eb77eed
fully list tests
ArthurZucker May 1, 2024
f97e398
update
ArthurZucker May 1, 2024
5168304
update
ArthurZucker May 1, 2024
efc2ffb
oups
ArthurZucker May 1, 2024
6c70798
finally
ArthurZucker May 1, 2024
1e7c50a
use classname
ArthurZucker May 1, 2024
8e6fb98
remove working directory to make sure the path does not interfere
ArthurZucker May 1, 2024
ff21adc
okay no juni should have the correct path
ArthurZucker May 1, 2024
a494f69
name split?
ArthurZucker May 1, 2024
55ecc72
sort by classname is what make most sense
ArthurZucker May 1, 2024
a111c32
some testing
ArthurZucker May 1, 2024
eb32ef7
naem
ArthurZucker May 1, 2024
e6c0c7b
oups
ArthurZucker May 1, 2024
0eca262
test something fun
ArthurZucker May 1, 2024
736d197
autodetect
ArthurZucker May 1, 2024
3315804
18?
ArthurZucker May 1, 2024
eb65ff4
nit
ArthurZucker May 1, 2024
925c708
file size?
ArthurZucker May 1, 2024
a3fae80
uip
ArthurZucker May 1, 2024
1d9404b
4 is best
ArthurZucker May 1, 2024
db90351
update to see versions
ArthurZucker May 1, 2024
83a5d70
better print
ArthurZucker May 1, 2024
2737073
[push-ci-image]
ArthurZucker May 2, 2024
2875558
[push-ci-image]
ArthurZucker May 2, 2024
98ad885
please install the correct keras version
ArthurZucker May 2, 2024
8d611c0
[push-ci-image]
ArthurZucker May 2, 2024
b95e1af
[push-ci-image]
ArthurZucker May 2, 2024
727e9b6
[push-ci-image]
ArthurZucker May 2, 2024
30f712d
Merge branch 'main' into change-ci
ArthurZucker May 2, 2024
4557e60
[push-ci-image]
ArthurZucker May 2, 2024
9622d83
Merge branch 'change-ci' of github.com:huggingface/transformers into …
ArthurZucker May 2, 2024
bbffd3d
[push-ci-image]
ArthurZucker May 2, 2024
c426bb0
uv is fucking me up
ArthurZucker May 2, 2024
9152efe
[push-ci-image]
ArthurZucker May 2, 2024
86aa82e
[push-ci-image]
ArthurZucker May 2, 2024
f67feca
[push-ci-image]
ArthurZucker May 2, 2024
65fe05a
nits
ArthurZucker May 2, 2024
c4b58c9
[push-ci-image]
ArthurZucker May 2, 2024
150a3ce
[push-ci-image]
ArthurZucker May 2, 2024
7a03e44
install issues an pins
ArthurZucker May 2, 2024
a137067
tapas as well
ArthurZucker May 2, 2024
bdb4146
nits
ArthurZucker May 2, 2024
95a3c36
more paralellism
ArthurZucker May 2, 2024
2f13f7f
short tb
ArthurZucker May 2, 2024
39d5ff2
soundfile
ArthurZucker May 2, 2024
c8a86ce
soundfile
ArthurZucker May 2, 2024
9fb04d4
[push-ci-image]
ArthurZucker May 2, 2024
a115714
[push-ci-image]
ArthurZucker May 2, 2024
688d31f
[push-ci-image]
ArthurZucker May 2, 2024
8a73a9a
oups
ArthurZucker May 2, 2024
a97cfbd
[push-ci-image]
ArthurZucker May 2, 2024
91dbbe2
fix some things
ArthurZucker May 2, 2024
879f95a
[push-ci-image]
ArthurZucker May 2, 2024
afe96e5
[push-ci-image]
ArthurZucker May 2, 2024
9cc306e
[push-ci-image]
ArthurZucker May 2, 2024
900f779
[push-ci-image]
ArthurZucker May 2, 2024
8dda4c3
use torch-light for hub
ArthurZucker May 2, 2024
c46885e
small git lfs for hub job
ArthurZucker May 2, 2024
5c799ea
[push-ci-image]
ArthurZucker May 2, 2024
e2404ef
[push-ci-image]
ArthurZucker May 2, 2024
b167deb
[push-ci-image]
ArthurZucker May 2, 2024
ca253a0
[push-ci-image]
ArthurZucker May 2, 2024
bcd2deb
fix tf tapas
ArthurZucker May 2, 2024
e06df9c
[push-ci-image]
ArthurZucker May 2, 2024
483752e
nits
ArthurZucker May 2, 2024
02529b0
[push-ci-image]
ArthurZucker May 2, 2024
016f494
don't update the test
ArthurZucker May 2, 2024
43fd2e4
[push-ci-image]
ArthurZucker May 2, 2024
6f7ff62
[push-ci-image]
ArthurZucker May 2, 2024
7bef003
Merge branch 'main' of github.com:huggingface/transformers into chang…
ArthurZucker May 2, 2024
e8c33e4
[push-ci-image]
ArthurZucker May 2, 2024
47f52c1
no use them
ArthurZucker May 2, 2024
9166aec
[push-ci-image]
ArthurZucker May 2, 2024
a2fff68
[push-ci-image]
ArthurZucker May 2, 2024
27640dd
[push-ci-image]
ArthurZucker May 2, 2024
3237448
[push-ci-image]
ArthurZucker May 2, 2024
6419d34
update tf proba
ArthurZucker May 2, 2024
9388cd3
[push-ci-image]
ArthurZucker May 2, 2024
c04d7e3
[push-ci-image]
ArthurZucker May 2, 2024
a83168d
woops
ArthurZucker May 2, 2024
b4e21e6
[push-ci-image]
ArthurZucker May 2, 2024
1e21c9c
[push-ci-image]
ArthurZucker May 2, 2024
b688346
[push-ci-image]
ArthurZucker May 2, 2024
76524e3
[push-ci-image]
ArthurZucker May 2, 2024
3c1868d
[push-ci-image]
ArthurZucker May 2, 2024
66e12c2
[push-ci-image]
ArthurZucker May 2, 2024
f6c4867
test with built dockers
ArthurZucker May 2, 2024
96c7692
[push-ci-image]
ArthurZucker May 2, 2024
ff96332
skip annoying tests
ArthurZucker May 2, 2024
d651547
Merge branch 'main' of github.com:huggingface/transformers into chang…
ArthurZucker May 2, 2024
46b10e2
revert fix copy
ArthurZucker May 2, 2024
725db3e
update test values
ArthurZucker May 3, 2024
43db1f7
update
ArthurZucker May 3, 2024
eb8b69b
Merge branch 'main' of github.com:huggingface/transformers into chang…
ArthurZucker May 3, 2024
e71b18f
last skip and fixup
ArthurZucker May 3, 2024
f479bfd
nit
ArthurZucker May 3, 2024
a020478
ALL GOOOD
ArthurZucker May 3, 2024
896f0c0
quality
ArthurZucker May 3, 2024
27929ba
Update tests/models/layoutlmv2/test_image_processing_layoutlmv2.py
ArthurZucker May 6, 2024
89e3353
Update docker/quality.dockerfile
ArthurZucker May 6, 2024
8b094f8
Update src/transformers/models/tapas/modeling_tf_tapas.py
ArthurZucker May 6, 2024
1d6ceec
Apply suggestions from code review
ArthurZucker May 6, 2024
b6fc187
use torch-speed
ArthurZucker May 6, 2024
e4f4e23
updates
ArthurZucker May 6, 2024
15dde2c
[push-ci-image]
ArthurZucker May 6, 2024
de0b945
Merge branch 'change-ci' of github.com:huggingface/transformers into …
ArthurZucker May 6, 2024
4e4a2e0
[push-ci-image]
ArthurZucker May 6, 2024
b8ab028
[push-ci-image]
ArthurZucker May 6, 2024
514caeb
[push-ci-image]
ArthurZucker May 6, 2024
b663f61
fuck ken-lm [push-ci-image]
ArthurZucker May 6, 2024
23b48ac
[push-ci-image]
ArthurZucker May 6, 2024
d47d214
[push-ci-image]
ArthurZucker May 6, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 12 additions & 51 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ jobs:
# Ensure running with CircleCI/huggingface
check_circleci_user:
docker:
- image: cimg/python:3.8.12
- image: python:3.10-slim
parallelism: 1
steps:
- run: echo $CIRCLE_PROJECT_USERNAME
Expand All @@ -26,13 +26,11 @@ jobs:
fetch_tests:
working_directory: ~/transformers
docker:
- image: cimg/python:3.8.12
- image: huggingface/transformers-quality
parallelism: 1
steps:
- checkout
- run: pip install --upgrade --upgrade-strategy eager pip
- run: pip install -U --upgrade-strategy eager GitPython
- run: pip install -U --upgrade-strategy eager .
- run: uv pip install -U -e .
- run: mkdir -p test_preparation
- run: python utils/tests_fetcher.py | tee tests_fetched_summary.txt
- store_artifacts:
Expand Down Expand Up @@ -88,25 +86,22 @@ jobs:
echo "No tests to run, exiting early!"
circleci-agent step halt
fi
- run: cp test_preparation/generated_config.yml test_preparation/generated_config.txt
- store_artifacts:
path: test_preparation/generated_config.txt
path: test_preparation/generated_config.yml
- store_artifacts:
path: test_preparation/filtered_test_list_cross_tests.txt
path: test_preparation/filtered_test_list_cross_tests.txt
- continuation/continue:
configuration_path: test_preparation/generated_config.yml
configuration_path: test_preparation/generated_config.yml

# To run all tests for the nightly build
fetch_all_tests:
working_directory: ~/transformers
docker:
- image: cimg/python:3.8.12
- image: huggingface/transformers-consistency
parallelism: 1
steps:
- checkout
- run: pip install --upgrade --upgrade-strategy eager pip
- run: pip install -U --upgrade-strategy eager GitPython
- run: pip install -U --upgrade-strategy eager .
- run: uv pip install -e .
- run: |
mkdir test_preparation
echo -n "tests" > test_preparation/test_list.txt
Expand All @@ -126,32 +121,15 @@ jobs:
check_code_quality:
working_directory: ~/transformers
docker:
- image: cimg/python:3.8.12
- image: huggingface/transformers-quality
resource_class: large
environment:
TRANSFORMERS_IS_CI: yes
PYTEST_TIMEOUT: 120
parallelism: 1
steps:
- checkout
- restore_cache:
keys:
- v0.7-code_quality-pip-{{ checksum "setup.py" }}
- v0.7-code-quality-pip
- restore_cache:
keys:
- v0.7-code_quality-site-packages-{{ checksum "setup.py" }}
- v0.7-code-quality-site-packages
- run: pip install --upgrade --upgrade-strategy eager pip
- run: pip install -U --upgrade-strategy eager .[all,quality]
- save_cache:
key: v0.7-code_quality-pip-{{ checksum "setup.py" }}
paths:
- '~/.cache/pip'
- save_cache:
key: v0.7-code_quality-site-packages-{{ checksum "setup.py" }}
paths:
- '~/.pyenv/versions/'
- run: uv pip install -e .
- run:
name: Show installed libraries and their versions
command: pip freeze | tee installed.txt
Expand All @@ -167,32 +145,15 @@ jobs:
check_repository_consistency:
working_directory: ~/transformers
docker:
- image: cimg/python:3.8.12
- image: huggingface/transformers-consistency
resource_class: large
environment:
TRANSFORMERS_IS_CI: yes
PYTEST_TIMEOUT: 120
parallelism: 1
steps:
- checkout
- restore_cache:
keys:
- v0.7-repository_consistency-pip-{{ checksum "setup.py" }}
- v0.7-repository_consistency-pip
- restore_cache:
keys:
- v0.7-repository_consistency-site-packages-{{ checksum "setup.py" }}
- v0.7-repository_consistency-site-packages
- run: pip install --upgrade --upgrade-strategy eager pip
- run: pip install -U --upgrade-strategy eager .[all,quality]
- save_cache:
key: v0.7-repository_consistency-pip-{{ checksum "setup.py" }}
paths:
- '~/.cache/pip'
- save_cache:
key: v0.7-repository_consistency-site-packages-{{ checksum "setup.py" }}
paths:
- '~/.pyenv/versions/'
- run: uv pip install -e .
- run:
name: Show installed libraries and their versions
command: pip freeze | tee installed.txt
Expand Down
Loading
Loading