fix(tests): update tests after torch 2.4.1 #26082

Job	Run time
build-packages / init	7s
build-packages / build-packages (lightning)	21s
build-packages / build-packages (fabric)	23s
build-packages / build-packages (pytorch)	21s
install-pkg (ubuntu-22.04, fabric, 3.9)	1m 36s
install-pkg (ubuntu-22.04, fabric, 3.11)	1m 34s
install-pkg (ubuntu-22.04, pytorch, 3.9)	1m 34s
install-pkg (ubuntu-22.04, pytorch, 3.11)	1m 45s
install-pkg (ubuntu-22.04, lightning, 3.9)	1m 36s
install-pkg (ubuntu-22.04, lightning, 3.11)	1m 43s
install-pkg (ubuntu-22.04, notset, 3.9)	1m 42s
install-pkg (ubuntu-22.04, notset, 3.11)	1m 35s
install-pkg (macOS-12, fabric, 3.9)	1m 1s
install-pkg (macOS-12, fabric, 3.11)	54s
install-pkg (macOS-12, pytorch, 3.9)	1m 25s
install-pkg (macOS-12, pytorch, 3.11)	1m 38s
install-pkg (macOS-12, lightning, 3.9)	1m 7s
install-pkg (macOS-12, lightning, 3.11)	1m 20s
install-pkg (macOS-12, notset, 3.9)	1m 10s
install-pkg (macOS-12, notset, 3.11)	1m 1s
install-pkg (windows-2022, fabric, 3.9)	2m 3s
install-pkg (windows-2022, fabric, 3.11)	2m 54s
install-pkg (windows-2022, pytorch, 3.9)	2m 26s
install-pkg (windows-2022, pytorch, 3.11)	2m 4s
install-pkg (windows-2022, lightning, 3.9)	2m 38s
install-pkg (windows-2022, lightning, 3.11)	2m 42s
install-pkg (windows-2022, notset, 3.9)	2m 28s
install-pkg (windows-2022, notset, 3.11)	2m 28s
	43m 36s

Provide feedback