Skip to content

Issues: huggingface/trl

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

171 Open 1,231 Closed
171 Open 1,231 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Training Agents with GRPO 🏋 GRPO Related to GRPO
#2723 opened Jan 31, 2025 by August-murr
3
OOM for 7B model on A100 80Gb 🐛 bug Something isn't working
#2719 opened Jan 31, 2025 by JohnConnor123
5 tasks done
2
AttributeError: 'AutoModelForCausalLMWithValueHead' object has no attribute 'base_model_prefix' 🐛 bug Something isn't working ⚡ PEFT Related to PEFT 🏋 PPO Related to PPO
#2718 opened Jan 31, 2025 by Tarak200
GRPO for RL on agent trajectories 🏋 GRPO Related to GRPO 🏋 Reward Related to Reward modelling
#2715 opened Jan 31, 2025 by korbinian-hoermann
1
Isn't the reward *minimized* when len(completion)==20 if this is the reward function? 🏋 Reward Related to Reward modelling
#2714 opened Jan 31, 2025 by cfpark00
GRPO with tool calling 🏋 GRPO Related to GRPO 🏋 Reward Related to Reward modelling
#2712 opened Jan 31, 2025 by accupham
3 tasks
LoRA 'trainable params: 0' 🐛 bug Something isn't working ⚡ PEFT Related to PEFT
#2711 opened Jan 31, 2025 by shannonruxin
Examples in training VDPO on llava1.6 🏋 DPO Related to DPO ✨ enhancement New feature or request
#2710 opened Jan 31, 2025 by lucasjinreal
GRPO memory bottleneck from num_generations in compute_loss 🐛 bug Something isn't working 🏋 GRPO Related to GRPO ⚡ PEFT Related to PEFT
#2709 opened Jan 31, 2025 by willccbb
9
PPOTrainer + LoRA and Continued Training ⏳ needs more info Additional information or clarification is required to proceed ⚡ PEFT Related to PEFT 🏋 PPO Related to PPO
#2707 opened Jan 30, 2025 by kooryan
Multi-GPU sampling for vLLM in GRPO Trainer ✨ enhancement New feature or request 🏋 GRPO Related to GRPO
#2706 opened Jan 30, 2025 by nch0w
1
GRPO: Why does loss start at 0 for first K steps and then increase over time? 🏋 GRPO Related to GRPO ❓ question Seeking clarification or more information
#2703 opened Jan 30, 2025 by arnavgarg1
5 tasks done
3
Exposing GenerationConfig in the GRPO Trainer ✨ enhancement New feature or request 🏋 GRPO Related to GRPO
#2702 opened Jan 30, 2025 by Superskyyy
4
Allow pretokenized dataset in GRPO Trainer ✨ enhancement New feature or request 🏋 GRPO Related to GRPO
#2701 opened Jan 30, 2025 by Superskyyy
5
GRPO VLLM does not work with Lora 🏋 GRPO Related to GRPO ⚡ PEFT Related to PEFT
#2698 opened Jan 30, 2025 by gagan3012
5 tasks done
2
I cannot launch PPOTrainning script with accelerate launch ⚡accelerate Related to accelerate ⚡ PEFT Related to PEFT 🏋 PPO Related to PPO
#2696 opened Jan 30, 2025 by daehuikim
5 tasks done
3
OOM 8xH100 using latest GRPO code with vLLM 🐛 bug Something isn't working 🚀 deepspeed Related to deepspeed 🏋 GRPO Related to GRPO
#2688 opened Jan 30, 2025 by abacaj
5 tasks done
7
empty Cache after logps_per_token 🐛 bug Something isn't working 🏋 GRPO Related to GRPO
#2686 opened Jan 29, 2025 by shirinyamani
3
rewards_funcs set to eval mode 🏋 GRPO Related to GRPO ❓ question Seeking clarification or more information 🏋 Reward Related to Reward modelling
#2685 opened Jan 29, 2025 by shirinyamani
1
Support iterative GRPO ✨ enhancement New feature or request 🏋 GRPO Related to GRPO ⚡ PEFT Related to PEFT
#2684 opened Jan 29, 2025 by howardzhou
@shirinyamani
8
About the Implementation of GRPO 🏋 GRPO Related to GRPO ❓ question Seeking clarification or more information
#2681 opened Jan 29, 2025 by macheng6
1
Ability to provide a static completion for GRPO ✨ enhancement New feature or request 🏋 GRPO Related to GRPO
#2680 opened Jan 29, 2025 by Palmik
1
logging issue: generation_config in rloo_trainer.py's generate_completions() is not reflective of actual model generations 🐛 bug Something isn't working 🏋 RLOO Related to RLOO
#2678 opened Jan 28, 2025 by swkarlekar
5 tasks done
TypeError: type list doesn't define __round__ method - why I am getting this error 🐛 bug Something isn't working ⏳ needs more info Additional information or clarification is required to proceed 🏋 Reward Related to Reward modelling
#2674 opened Jan 28, 2025 by Tarak200
2
"None of the inputs have requires_grad=True" with online DPO and GRPO 🐛 bug Something isn't working 🏋 GRPO Related to GRPO 🏋 Online DPO Related to Online DPO
#2671 opened Jan 28, 2025 by benjamin-marie
5 tasks done
6
ProTip! Find all open issues with in progress development work with linked:pr.