[CLI]: wandb artifact get broken for GCP references after 0.13.8 release

[omerferhatt]

Describe the bug

I get an error when I try to download artifact. But when i downgrade to 0.13.7, it works okay.
Error occurs at 0.13.8 and 0.13.9 versions

wandb artifact get ...

Traceback (most recent call last):ded...
  File "/home/omer/miniconda3/envs/alpha_trainer_tf/bin/wandb", line 8, in <module>
    sys.exit(cli())
  File "/home/omer/miniconda3/envs/alpha_trainer_tf/lib/python3.8/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/home/omer/miniconda3/envs/alpha_trainer_tf/lib/python3.8/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/home/omer/miniconda3/envs/alpha_trainer_tf/lib/python3.8/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/omer/miniconda3/envs/alpha_trainer_tf/lib/python3.8/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/omer/miniconda3/envs/alpha_trainer_tf/lib/python3.8/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/omer/miniconda3/envs/alpha_trainer_tf/lib/python3.8/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/home/omer/miniconda3/envs/alpha_trainer_tf/lib/python3.8/site-packages/wandb/cli/cli.py", line 102, in wrapper
    return func(*args, **kwargs)
  File "/home/omer/miniconda3/envs/alpha_trainer_tf/lib/python3.8/site-packages/wandb/cli/cli.py", line 1842, in get
    path = artifact.download(root=root)
  File "/home/omer/miniconda3/envs/alpha_trainer_tf/lib/python3.8/site-packages/wandb/apis/public.py", line 4739, in download
    pool.map(
  File "/home/omer/miniconda3/envs/alpha_trainer_tf/lib/python3.8/multiprocessing/pool.py", line 364, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/home/omer/miniconda3/envs/alpha_trainer_tf/lib/python3.8/multiprocessing/pool.py", line 771, in get
    raise self._value
  File "/home/omer/miniconda3/envs/alpha_trainer_tf/lib/python3.8/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/home/omer/miniconda3/envs/alpha_trainer_tf/lib/python3.8/multiprocessing/pool.py", line 48, in mapstar
    return list(map(*args))
  File "/home/omer/miniconda3/envs/alpha_trainer_tf/lib/python3.8/site-packages/wandb/apis/public.py", line 4835, in _download_file
    downloaded_path = self.get_path(name).download(root)
  File "/home/omer/miniconda3/envs/alpha_trainer_tf/lib/python3.8/site-packages/wandb/apis/public.py", line 4136, in download
    cache_path = manifest.storage_policy.load_reference(
  File "/home/omer/miniconda3/envs/alpha_trainer_tf/lib/python3.8/site-packages/wandb/sdk/wandb_artifacts.py", line 944, in load_reference
    return self._handler.load_path(artifact, manifest_entry, local)
  File "/home/omer/miniconda3/envs/alpha_trainer_tf/lib/python3.8/site-packages/wandb/sdk/wandb_artifacts.py", line 1110, in load_path
    return self._handlers[str(url.scheme)].load_path(
  File "/home/omer/miniconda3/envs/alpha_trainer_tf/lib/python3.8/site-packages/wandb/sdk/wandb_artifacts.py", line 1652, in load_path
    self.init_gcs()
  File "/home/omer/miniconda3/envs/alpha_trainer_tf/lib/python3.8/site-packages/wandb/sdk/wandb_artifacts.py", line 1625, in init_gcs
    self._client = storage.Client()
AttributeError: module 'google.cloud.storage' has no attribute 'Client'

Additional Files

No response

Environment

WandB version: 0.13.9

OS: Linux 5.15.79.1-microsoft-standard-WSL2 #1 SMP Wed Nov 23 01:01:46 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Python version: 3.8.16

Versions of relevant libraries: google-cloud-storage=2.7.0

Additional Context

No response

[thanos-wandb]

Hi @omerferhatt thanks so much for reporting this. Just some clarification before reporting it to the engineering team, is the bug only in CLI or do you get any errors when using the SDK:

import wandb
run = wandb.init()
artifact = run.use_artifact('entity/project/artifact:alias', type='artifact-type')
artifact_dir = artifact.download()

or the API:

api = wandb.Api()
artifact = api.artifact('entity/project/artifact:alias')
artifact_dir = artifact.download()

Also, are these reference artifacts?

[omerferhatt]

Hello, @thanos-wandb. Yes I confirm that artifact exists and can be downloaded with 0.13.7.

I tried to download with the script given below (variables replaced due to privacy) to download:

import wandb

if __name__ == "__main__":
    run = wandb.init()
    artifact = run.use_artifact('entity/project/artifact:alias', type='artifact-type')
    artifact_dir = artifact.download()

I tried with the API implementation too.

Results are same with the CLI. Traceback shows:

AttributeError: module 'google.cloud.storage' has no attribute 'Client'`

[omerferhatt]

For the sake of complete understanding, the problem doesn't happen exactly as soon as it starts downloading. While the download counter continues as 1...2...3, it stops at a random number and gives an error.

wandb: / 1 of 1000 files downloaded...
wandb: - 2 of 1000 files downloaded...
wandb: \ 3 of 1000 files downloaded...
wandb: | 4 of 1000 files downloaded...

It throws an error at a random step. When the download command is run again, it continues from where it crash.

wandb: / 4 of 1000 files downloaded...
wandb: - 5 of 1000 files downloaded...
wandb: \ 6 of 1000 files downloaded...
wandb: | 7 of 1000 files downloaded...

like that.

[thanos-wandb]

Hi @omerferhatt thanks a lot for the detailed information, I will add these to the internal ticket for our engineers. Found a similar error reported in Google API repo not sure if related to your miniconda env, could you try in a Colab too, or a different virtual environment?

[thanos-wandb]

Hi @omerferhatt just checking in here to see if you still experience this issue, and if you've tried from a different environment as well? thanks!

[omerferhatt]

Hi @thanos-wandb , sorry for the late reply. Unfortunately I didn't have time to try it. Currently I am still working with the same version. I'll get back to you as soon as I can after trying it.

Kind regards