Probably a more reasonable method of packing

[AIR-hl]

Feature request

Hi! Here i m again : ) , according to #1850 i noticed that the packing method of SFTTrainer truncates overly long data directly and roughly, it's obviously that this will cause a lot of text information to be destroyed.

When browsing the codes of LLaMA-Factory, i find they use a different elegant way for packing. In short, they use the greedy knapsack algorithm to fill each packing sequence as much as possible, maximizing its utilization.

Maybe trl can be modified accordingly in the future. What's ur idea?

Motivation

https://github.com/hiyouga/LLaMA-Factory/blob/main/src/llamafactory/data/processors/supervised.py#L130

[qgallouedec]

That's very interesting! It would be a nice improvement.

If you want to tackle this problem, you should be aware that packing will be implemented differently (in a simpler way) in the near future, see #2405. You should branch from there.