Split DoRA and LoRA weight merging logic

[ebsmothers]

No description provided.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/torchtune/1535

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 New Failures, 4 Cancelled Jobs

As of commit 88c4181 with merge base 66590b4 ():

NEW FAILURES - The following jobs have failed:

• GPU tests / gpu_test (3.10, stable) (gh)
  tests/torchtune/modules/peft/test_utils.py::TestGetMergedLoRACkpt::test_get_merged_lora_ckpt[True]
• Unit Test / unit_tests (3.10) (gh)
  tests/torchtune/modules/peft/test_utils.py::TestGetMergedLoRACkpt::test_get_merged_lora_ckpt[True]

CANCELLED JOBS - The following jobs were cancelled. Please retry:

• GPU tests / gpu_test (3.11, stable) (gh)
  ##[error]The operation was canceled.
• GPU tests / gpu_test (3.9, stable) (gh)
  tests/torchtune/modules/peft/test_utils.py::TestGetMergedLoRACkpt::test_get_merged_lora_ckpt[True]
• Unit Test / unit_tests (3.11) (gh)
  ##[error]The operation was canceled.
• Unit Test / unit_tests (3.9) (gh)
  ##[error]The operation was canceled.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[ebsmothers]

Closing now that #1315 has landed