Fix initialized weights resetting in Fabric.setup()
when using FSDP
#12768
Job | Run time |
---|---|
24m 37s | |
11m 52s | |
13m 43s | |
29m 7s | |
13m 22s | |
14m 49s | |
21m 40s | |
13m 40s | |
14m 2s | |
21m 4s | |
10m 58s | |
13m 17s | |
18m 54s | |
12m 25s | |
14m 22s | |
21m 53s | |
11m 40s | |
13m 38s | |
19m 53s | |
11m 48s | |
13m 42s | |
5h 40m 26s |