[train] Fix train_multinode_persistence release test

[justinvyu]

Why are these changes needed?

This PR switches from using fsspec to the s3 and gs CLIs to download and delete the directory.

Uses: I want to download the contents of storage_path on cloud in order to assert that the contents are correct. I also want to delete the directory so that consecutive runs don't overlap with each other.

This is why we use CLIs instead of the filesystem implementations:

1. A pyarrow-wrapped version of s3fs will error on downloading. Fix attempt 1: use the pyarrow default filesystem to download.
2. The pyarrow default gce filesystem doesn't download all files. Fix attempt 2: Use the unwrapped fsspec filesystem implementations to delete/download files.
3. The unwrapped fsspec implementation for S3 does not delete nested directories properly. This PR's fix: use the CLI's instead of pyarrow or fsspec. Note that this is not really a concern for users on the "default" cloud path, where they don't specify a custom fsspec filesystem.

Related issue number

#39546

Checks

• I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
• I've run scripts/format.sh to lint the changes in this PR.
• I've included any doc changes needed for https://docs.ray.io/en/master/.
  • I've added any new APIs to the API Reference. For example, if I added a
    method in Tune, I've added it in doc/source/tune/api/ under the
    corresponding .rst file.
• I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
• Testing Strategy
  • Unit tests
  • Release tests
  • This PR is not tested :(