[Performance 5/6] Prevent unnecessary extra networks bias backup

[huchenlei]

Description

According to lllyasviel/stable-diffusion-webui-forge#716 (comment) , network_apply always incur some overhead on copying tensors even when no extra networks are enabled. This PR prevents this behaviour. The performance gain is about 25ms/it.

Previously unnecessary weight copy was prevented in https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/12599/files . This PR is just a follow-up and apply the same approach on bias backup.

Screenshots/videos:

Checklist:

• I have read contributing wiki page
• I have performed a self-review of my own code
• My code follows the style guidelines
• My code passes tests

[AG-w]

is it possible to make this compatible with extension that changing lora weight during sampling?

https://github.com/cheald/sd-webui-loractl
https://github.com/AG-w/sd-webui-lora-block-weight

[huchenlei]

is it possible to make this compatible with extension that changing lora weight during sampling?

https://github.com/cheald/sd-webui-loractl https://github.com/AG-w/sd-webui-lora-block-weight

Do you mean that this change will cause dynamic lora weight not working?

[AG-w]

is it possible to make this compatible with extension that changing lora weight during sampling?
https://github.com/cheald/sd-webui-loractl https://github.com/AG-w/sd-webui-lora-block-weight

Do you mean that this change will cause dynamic lora weight not working?

the latest commit fixed it, thanks

[altoiddealer]

Whoa baby, I am SO looking forward to Forge speeds + ability to use loractl extension.

Absolutely amazing.