do not review - llama 3 and 2 instantiate pos emb only once

[felipemello1]

Context

What is the purpose of this PR? Is it to

• add a new feature
• fix a bug
• update tests and/or documentation
• other (please add here)

same as in: #1554

Changelog

• llama 2 module list
• llama 3 module list

Test plan

no memory gains because fsdp nproc=8, so its 2GiB/8

tune run --nnodes 1 --nproc_per_node 8 lora_finetune_distributed --config llama2/7B_lora batch_size=8 max_steps_per_epoch=20 metric_logger=torchtune.training.metric_logging.WandBLogger gradient_accumulation_steps=1 epochs=1 compile=True log_peak_memory_stats=True

tune run --nnodes 1 --nproc_per_node 8 lora_finetune_distributed --config llama2/7B_lora batch_size=8 max_steps_per_epoch=20 metric_logger=torchtune.training.metric_logging.WandBLogger gradient_accumulation_steps=1 epochs=1 compile=True log_peak_memory_stats=True

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/torchtune/1556

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 New Failures, 1 Cancelled Job

As of commit 9d7347f with merge base 7dad2d6 ():

NEW FAILURES - The following jobs have failed:

• GPU tests / gpu_test (3.10, stable) (gh)
  tests/recipes/test_ppo_full_finetune_single_device.py::TestPPOFullFinetuneSingleDeviceRecipe::test_training_state_on_resume_with_optimizer_in_bwd
• GPU tests / gpu_test (3.11, stable) (gh)
  tests/recipes/test_ppo_full_finetune_single_device.py::TestPPOFullFinetuneSingleDeviceRecipe::test_training_state_on_resume_with_optimizer_in_bwd

CANCELLED JOB - The following job was cancelled. Please retry:

• GPU tests / gpu_test (3.9, stable) (gh)
  ##[error]The operation was canceled.

This comment was automatically generated by Dr. CI and updates every 15 minutes.