New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

refactor(data): 重构mask方式，sharegpt 支持更精细的mask控制 #6498

Open

zzc0430 wants to merge 1 commit into hiyouga:main from zzc0430:refactor/mask_history

zzc0430 commented Dec 31, 2024

What does this PR do?

重构mask方式，sharegpt 支持更精细的mask控制

Before submitting

Did you read the contributor guideline?
Did you write any new necessary tests?


          refactor(data): 重构mask方式，sharegpt 支持更精细的mask控制

b7779b6

Owner

hiyouga commented Dec 31, 2024

Where is the main difference between this PR and #6396?

Author

zzc0430 commented Dec 31, 2024

Where is the main difference between this PR and #6396?

Unified the two control methods for the mask: data_args(mask_history, train_on_prompt) and dataset_attr(sharegpt: train).
Do not reverse messages when mask_history is set to True

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet