Skip to content

Commit

Permalink
Upgrade seqio to upstream to workaround the MaxText dependency issue …
Browse files Browse the repository at this point in the history
…for inference (#1225)

Default version built with MaxText image is `seqio==0.0.16` which
introduces error
```
0: Traceback (most recent call last):
0:   File "/opt/maxtext/MaxText/inference_microbenchmark.py", line 26, in <module>
0:     from jetstream.engine import token_utils
0:   File "/opt/jetstream/jetstream/engine/token_utils.py", line 24, in <module>
0:     from seqio.vocabularies import SentencePieceVocabulary
0:   File "/usr/local/lib/python3.12/dist-packages/seqio/__init__.py", line 18, in <module>
0:     from seqio.dataset_providers import *
0:   File "/usr/local/lib/python3.12/dist-packages/seqio/dataset_providers.py", line 60, in <module>
0:     @dataclasses.dataclass(frozen=True)
0:      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
0:   File "/usr/lib/python3.12/dataclasses.py", line 1258, in wrap
0:     return _process_class(cls, init, repr, eq, order, unsafe_hash,
0:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
0:   File "/usr/lib/python3.12/dataclasses.py", line 994, in _process_class
0:     cls_fields.append(_get_field(cls, name, type, kw_only))
0:                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
0:   File "/usr/lib/python3.12/dataclasses.py", line 852, in _get_field
0:     raise ValueError(f'mutable default {type(f.default)} for field '
0: ValueError: mutable default <class 'seqio.vocabularies.PassThroughVocabulary'> for field vocabulary is not allowed: use default_factory
```
Upgrade seqio to directly use upstream repo to workaround the issue.
(This is not the perfect but workable solution)

---------

Co-authored-by: Vladislav Kozlov <vkozlov@nvidia.com>
  • Loading branch information
gpupuck and DwarKapex authored Jan 5, 2025
1 parent 92cc80b commit 4f388bd
Showing 1 changed file with 8 additions and 1 deletion.
9 changes: 8 additions & 1 deletion .github/container/Dockerfile.maxtext
Original file line number Diff line number Diff line change
Expand Up @@ -68,10 +68,17 @@ for pattern in \
"s|protobuf==3.20.3|protobuf>=3.19.0|g" \
"s|tensorflow-datasets|tensorflow-datasets>=4.8.0|g" \
"s|grain-nightly|grain|g" \
"s|tensorboardx|tensorboardx>=2.6.2|g" \
"s|sentencepiece==0.1.97|sentencepiece>=0.2|g" \
; do
sed -i "${pattern}" ${SRC_PATH_MAXTEXT}/requirements.txt;
done
echo "tensorflow-metadata>=1.15.0" >> ${SRC_PATH_MAXTEXT}/requirements.txt
for requirement in \
"tensorflow-metadata>=1.15.0" \
"seqio@git+https://github.com/google/seqio.git" \
; do
echo "${requirement}" >> ${SRC_PATH_MAXTEXT}/requirements.txt
done
EOF

###############################################################################
Expand Down

0 comments on commit 4f388bd

Please sign in to comment.