Better graceful shutdown for KeyboardInterrupt #13272

Job	Run time
pl-cpu (macOS-11, lightning, 3.10, 2.0)	25m 11s
pl-cpu (ubuntu-20.04, lightning, 3.10, 2.0)	14m 22s
pl-cpu (windows-2022, lightning, 3.10, 2.0)	19m 3s
pl-cpu (macOS-11, lightning, 3.10, 2.1)	26m 55s
pl-cpu (ubuntu-20.04, lightning, 3.10, 2.1)	15m 4s
pl-cpu (windows-2022, lightning, 3.10, 2.1)	18m 7s
pl-cpu (macOS-11, lightning, 3.10, 2.2)	26m 34s
pl-cpu (ubuntu-20.04, lightning, 3.10, 2.2)	14m 32s
pl-cpu (windows-2022, lightning, 3.10, 2.2)	16m 18s
pl-cpu (macOS-14, lightning, 3.10, 2.3)	14m 35s
pl-cpu (ubuntu-20.04, lightning, 3.10, 2.3)	15m 21s
pl-cpu (windows-2022, lightning, 3.10, 2.3)	17m 7s
pl-cpu (macOS-12, pytorch, 3.11, 2.0)	22m 6s
pl-cpu (ubuntu-22.04, pytorch, 3.11, 2.0)	13m 21s
pl-cpu (windows-2022, pytorch, 3.11, 2.0)	17m 54s
pl-cpu (macOS-12, pytorch, 3.11, 2.1)	22m 34s
pl-cpu (ubuntu-22.04, pytorch, 3.11, 2.1)	15m 27s
pl-cpu (windows-2022, pytorch, 3.11, 2.1)	18m 39s
pl-cpu (macOS-11, lightning, 3.8, 2.0, oldest)	25m 10s
pl-cpu (ubuntu-20.04, lightning, 3.8, 2.0, oldest)	13m 19s
pl-cpu (windows-2022, lightning, 3.8, 2.0, oldest)	16m 7s
pl-cpu (macOS-11, pytorch, 3.8, 2.0)	24m 55s
pl-cpu (ubuntu-20.04, pytorch, 3.8, 2.0)	13m 29s
pl-cpu (windows-2022, pytorch, 3.8, 2.0)	16m 35s
	7h 22m 45s

Provide feedback