Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add transformation method to hf_causal_lm #1383

Merged
merged 8 commits into from
Jul 23, 2024
Merged

Add transformation method to hf_causal_lm #1383

merged 8 commits into from
Jul 23, 2024

Conversation

irenedea
Copy link
Contributor

@irenedea irenedea commented Jul 22, 2024 •
edited
Loading

Refactors the ComposerHFCausalLM class to include a transform_model method and make get_peft_config public so that subclasses can override this to transform models prior to training.

Also creates and uses a new utility function, set_config_overrides.

@irenedea irenedea requested a review from a team as a code owner July 22, 2024 23:29
snarayan21
snarayan21 approved these changes Jul 23, 2024
View reviewed changes
Copy link
Contributor

@snarayan21 snarayan21 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm + offline discussion

llmfoundry/models/hf/hf_causal_lm.py Show resolved Hide resolved
@irenedea irenedea enabled auto-merge (squash) July 23, 2024 00:35
dakinggg
dakinggg approved these changes Jul 23, 2024
View reviewed changes
llmfoundry/models/hf/hf_causal_lm.py Show resolved Hide resolved
llmfoundry/models/hf/hf_causal_lm.py Show resolved Hide resolved
dakinggg
dakinggg approved these changes Jul 23, 2024
View reviewed changes
tests/models/hf/test_hf_transform.py Outdated Show resolved Hide resolved
@irenedea irenedea merged commit cefd616 into mosaicml:main Jul 23, 2024
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dakinggg dakinggg dakinggg approved these changes

@snarayan21 snarayan21 snarayan21 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@irenedea @snarayan21 @dakinggg