chore: remove caches from github actions #5725

Job	Run time
lint	2m 59s
Python Benchmark	13m 32s
Running with Python 3.8	4m 21s
Running with Python 3.9	4m 36s
Python Build (Python 3.10 PyArrow latest)	6m 14s
Python Build (Python 3.8 PyArrow 8.0.0)	12m 23s
PySpark Integration Tests	6m 8s
Running with Python 3.10	4m 31s
Running with Python 3.11	4m 25s
Running with Python 3.12	4m 15s
	1h 3m 24s

Provide feedback