Skip to content

Removed outdated warning about batch contamination #4423

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 6 commits into from
Nov 4, 2025
Merged

Removed outdated warning about batch contamination #4423

merged 6 commits into from
Nov 4, 2025

Conversation

@Harras3
Copy link
Contributor

@Harras3 Harras3 commented Nov 2, 2025 •
edited
Loading

What does this PR do?

Removed outdated warning about batch contamination

Fixes #4381
#4381

Files Changes

trl/docs/source/reducing_memory_usage.md

Before submitting

  • Was this discussed/approved via a GitHub issue? Please add a link
    to it if that's the case.
  • Did you read the contributor guideline,
    Pull Request section?

@qgallouedec
Copy link
Member

qgallouedec commented Nov 3, 2025

#4381 actually referred to

trl/docs/source/reducing_memory_usage.md

Lines 93 to 94 in ee5827c

> [!WARNING]
> Packing may cause batch contamination, where adjacent sequences influence one another. This can be problematic for some applications. For more details, see [#1230](https://github.com/huggingface/trl/issues/1230).

@Harras3
Copy link
Contributor Author

Harras3 commented Nov 3, 2025

#4381 actually referred to

trl/docs/source/reducing_memory_usage.md

Lines 93 to 94 in ee5827c

> [!WARNING]
> Packing may cause batch contamination, where adjacent sequences influence one another. This can be problematic for some applications. For more details, see [#1230](https://github.com/huggingface/trl/issues/1230).

Updated the PR

qgallouedec
qgallouedec approved these changes Nov 4, 2025
View reviewed changes
Copy link
Member

@qgallouedec qgallouedec left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks!

@HuggingFaceDocBuilderDev
Copy link

HuggingFaceDocBuilderDev commented Nov 4, 2025

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@qgallouedec qgallouedec merged commit 7f15a7f into huggingface:main Nov 4, 2025
1 check passed
qgallouedec added a commit to Harras3/trl that referenced this pull request Nov 4, 2025
@qgallouedec
Squashed commit of the following:
cab4108 
commit 7a9592b
Author: Quentin Gallouédec <45557362+qgallouedec@users.noreply.github.com>
Date:   Tue Nov 4 14:32:04 2025 -0700

    🐍 Drop Python 3.9 (huggingface#4183)

commit 7f15a7f
Author: Harras Mansoor <98635627+Harras3@users.noreply.github.com>
Date:   Wed Nov 5 02:06:31 2025 +0500

    Removed outdated warning about batch contamination (huggingface#4423)

commit 8b0a3ce
Author: Albert Villanova del Moral <8515462+albertvillanova@users.noreply.github.com>
Date:   Tue Nov 4 21:37:39 2025 +0100

    Update tokenizer apply_chat_template with return_dict=True default (huggingface#4448)

commit d9f9e2b
Author: Pramodith Ballapuram <16939722+pramodith@users.noreply.github.com>
Date:   Tue Nov 4 19:56:58 2025 +0000

    Support casting to fp32 when word embeddings are tied to lm_head (huggingface#4446)

commit 4e138ab
Author: Sergio Paniego Blanco <sergiopaniegoblanco@gmail.com>
Date:   Tue Nov 4 15:15:23 2025 +0100

    Upload notebook with T4 selected (huggingface#4449)
qgallouedec added a commit that referenced this pull request Nov 4, 2025
@qgallouedec
Squashed commit of the following:
6a2e980 
commit 4677cf2
Author: Harras Mansoor <98635627+Harras3@users.noreply.github.com>
Date:   Wed Nov 5 04:06:13 2025 +0500

    Removed Sentiment Tuning Examples (#4424)

commit 7a9592b
Author: Quentin Gallouédec <45557362+qgallouedec@users.noreply.github.com>
Date:   Tue Nov 4 14:32:04 2025 -0700

    🐍 Drop Python 3.9 (#4183)

commit 7f15a7f
Author: Harras Mansoor <98635627+Harras3@users.noreply.github.com>
Date:   Wed Nov 5 02:06:31 2025 +0500

    Removed outdated warning about batch contamination (#4423)

commit 8b0a3ce
Author: Albert Villanova del Moral <8515462+albertvillanova@users.noreply.github.com>
Date:   Tue Nov 4 21:37:39 2025 +0100

    Update tokenizer apply_chat_template with return_dict=True default (#4448)

commit d9f9e2b
Author: Pramodith Ballapuram <16939722+pramodith@users.noreply.github.com>
Date:   Tue Nov 4 19:56:58 2025 +0000

    Support casting to fp32 when word embeddings are tied to lm_head (#4446)

commit 4e138ab
Author: Sergio Paniego Blanco <sergiopaniegoblanco@gmail.com>
Date:   Tue Nov 4 15:15:23 2025 +0100

    Upload notebook with T4 selected (#4449)

commit 43253b2
Author: Pramodith Ballapuram <16939722+pramodith@users.noreply.github.com>
Date:   Mon Nov 3 21:07:31 2025 +0000

    Add On-Policy Distillation from thinking labs to paper index. (#4410)

    Co-authored-by: Quentin Gallouédec <45557362+qgallouedec@users.noreply.github.com>

commit 6f41b18
Author: Behrooz Azarkhalili <80390531+behroozazarkhalili@users.noreply.github.com>
Date:   Mon Nov 3 10:57:51 2025 -0800

    fix: Remove chat template setting from non-SFT trainer scripts (#4437)

    Co-authored-by: Quentin Gallouédec <gallouedec.quentin@gmail.com>
    Co-authored-by: Quentin Gallouédec <45557362+qgallouedec@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@qgallouedec qgallouedec qgallouedec approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Remove outdated warning about batch contamination

3 participants

@Harras3 @qgallouedec @HuggingFaceDocBuilderDev