New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

How does LoRA filter out the torch.nn.modules.Linear parameters in ViT? #13

Open

weilanzhikong opened this issue Oct 30, 2024 · 1 comment

weilanzhikong commented Oct 30, 2024

I find there linear parameters in ViT

Owner

2U1 commented Oct 30, 2024

Qwen2-VL-Finetune/src/training/train.py

Lines 81 to 82 in 80b59b8

    
           if not training_args.vision_lora: 
        
               training_args.lora_namespan_exclude += ["visual"]

This filters out the vision module linears. If you are not using the vision_lora.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment