Bump version to 0.7.0 #1063
Merged
Bump version to 0.7.0 #1063
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
irenedea
commented
Mar 26, 2024
* Remove hf_prefix_lm * Remove prefix_lm from mpt modeling * Remove bidirectional mask * Remove text denoising dataloading * Remove adapt tokenizer
irenedea
commented
Mar 26, 2024
dakinggg
reviewed
Mar 26, 2024
irenedea
force-pushed
the
bump_version_v0.7.0
branch
from
2c9ec60 to
09ab444
Compare
irenedea
force-pushed
the
bump_version_v0.7.0
branch
from
09ab444 to
a62600d
Compare
dakinggg
approved these changes
Mar 26, 2024
alextrott16
suggested changes
Mar 26, 2024
| is_subseq, batch['bidirectional_mask'][j] == | ||
| 1)], | ||
| skip_special_tokens=False, | ||
| clean_up_tokenization_spaces=True)) |
There was a problem hiding this comment.
You shouldn't delete these lines. Instead, find a different way to slice out the context. batch['bidirectional_mask'][j] == 1 was just a convenient way to do that. The context occurs where attention_mask==1 and labels==_HF_IGNORE_INDEX.
| tokenizer.decode(batch['input_ids'][ | ||
| j, batch['bidirectional_mask'][j] == 1], | ||
| skip_special_tokens=False, | ||
| clean_up_tokenization_spaces=True)) |
There was a problem hiding this comment.
See above comment. Don't get rid of this, just do slicing differently.
| @@ -55,43 +55,10 @@ def validate_config(cfg: DictConfig): | |||
| loaders.append(eval_loader) | |||
| for loader in loaders: | |||
| if loader.name == 'text': | |||
| if cfg.model.name in ['hf_prefix_lm', 'hf_t5']: | |||
| if cfg.model.name in ['hf_t5']: | |||
There was a problem hiding this comment.
Suggested change
| if cfg.model.name in ['hf_t5']: | |
| if cfg.model.name == 'hf_t5': |
KuuCi
pushed a commit
that referenced
this pull request
Apr 18, 2024
* Bump version * Remove triton (#1062) * Remove github action workflows for version bumps * Fix cpu test issues * code quality * Fix gpu tests * Fix gpu tests nicely * Remove z-loss (#1064) * Remove prefix lm and denoising (#1065) * Remove hf_prefix_lm * Remove prefix_lm from mpt modeling * Remove bidirectional mask * Remove text denoising dataloading * Remove adapt tokenizer * Remove llama attention patch (#1066) * Remove bidirectional mask in tests * Fix test_hf_config_override with patch
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Bumps version to 0.7.0 and removes deprecated features.
Removals
Regression tests all passed:
llm-foundry-regression-tests-runner-4bPcjdllm-foundry-regression-tests-runner-s1KZPH