Trying to use ai-models-aurora in free tier google colab

[sudhir2016]

As per my understanding the following three steps are required.

! pip install ai-models-aurora
! ai-models --download-assets aurora
!ai-models --input cds --date 20230110 --time 0000 --lead-time 6 aurora

The first step runs fine
In second step I get this error

2024-10-25 16:49:33,269 INFO Writing results to aurora.grib
2024-10-25 16:49:33,270 INFO Downloading /content/aurora-0.1-static.pickle
2024-10-25 16:49:33,270 INFO Downloading https://huggingface.co/microsoft/aurora/resolve/main/aurora-0.1-static.pickle
2024-10-25 16:49:33,587 INFO Total time: 12 seconds.
Traceback (most recent call last):
File "/usr/local/bin/ai-models", line 8, in
sys.exit(main())
File "/usr/local/lib/python3.10/dist-packages/ai_models/main.py", line 362, in main
_main(sys.argv[1:])
File "/usr/local/lib/python3.10/dist-packages/ai_models/main.py", line 310, in _main
run(vars(args), unknownargs)
File "/usr/local/lib/python3.10/dist-packages/ai_models/main.py", line 319, in run
model = load_model(cfg["model"], **cfg, model_args=model_args)
File "/usr/local/lib/python3.10/dist-packages/ai_models/model.py", line 610, in load_model
return available_models()[name].load()(**kwargs)
File "/usr/local/lib/python3.10/dist-packages/ai_models_aurora/model.py", line 257, in model
return modelsmodel_version
File "/usr/local/lib/python3.10/dist-packages/ai_models/model.py", line 97, in init
self.download_assets(**kwargs)
File "/usr/local/lib/python3.10/dist-packages/ai_models/model.py", line 173, in download_assets
download(self.download_url.format(file=file), asset + ".download")
File "/usr/local/lib/python3.10/dist-packages/multiurl/downloader.py", line 111, in download
return Downloader(url, **kwargs).download(target)
File "/usr/local/lib/python3.10/dist-packages/multiurl/base.py", line 126, in download
desc=self.title(),
File "/usr/local/lib/python3.10/dist-packages/multiurl/http.py", line 124, in title
value, params = parse_separated_header(headers["content-disposition"])
ValueError: too many values to unpack (expected 2)

In third step I get this error after 4 grip files are downloaded from cdsapi

2024-10-25 17:00:44,023 ERROR [Errno 2] No such file or directory: './aurora-0.1-static.pickle'
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/ai_models/main.py", line 335, in run
model.run()
File "/usr/local/lib/python3.10/dist-packages/ai_models_aurora/model.py", line 97, in run
with open(os.path.join(self.assets, self.download_files[0]), "rb") as f:
FileNotFoundError: [Errno 2] No such file or directory: './aurora-0.1-static.pickle'
2024-10-25 17:00:44,024 ERROR It is possible that some files required by aurora are missing.
2024-10-25 17:00:44,024 ERROR Rerun the command as:
2024-10-25 17:00:44,024 ERROR /usr/local/bin/ai-models --download-assets --input cds --date 20230110 --time 0000 --lead-time 6 aurora
2024-10-25 17:00:44,024 INFO Total time: 6 minutes 40 seconds.

Please help.

[Isaaciwd]

I get the same error with the download assets command, and cannot download assets.

[Isaaciwd]

Okay, I found a workaround. In your file "/usr/local/lib/python3.10/dist-packages/multiurl/http.py" replace the "def title(self):" function that should be at or near line 120 with this new title function:

def title(self):
    headers = self.headers()
    if "content-disposition" in headers:
        disposition = headers["content-disposition"]
        # Split the header by semicolon and process each part
        parts = [p.strip() for p in disposition.split(";")]
        params = {}

        # Iterate over parts to find "filename" or "filename*"
        for part in parts:
            if part.startswith("filename*="):  # Handle UTF-8 encoded filename
                filename = part.split("=", 1)[1].strip("UTF-8''")
                params["filename"] = filename.strip('"')
            elif part.startswith("filename="):  # Standard filename
                params["filename"] = part.split("=", 1)[1].strip('"')

        # Return filename if available
        if "filename" in params:
            return params["filename"]

This parses the header properly and should still work with the other models, but no guarantee on that. Also if you don't have a mars account, the download assets command will still throw up this error: ecmwfapi.api.APIException: "ecmwf.API error 1: User 'your-email@email.com' has no access to services/mars". Despite this error though, the command should have still worked and the assets should be downloaded. This error occurs when downloading the assets of any of the other models too.

Also I don't think it says this in the github yet, so be sure you're downloading the assets to the aurora model you're interested in. The options are aurora, aurora-0.1-finetuned, aurora-2.5-finetuned, aurora-2.5-pretrained. I think just running "ai-models --download-assets aurora" defaults to the aurora-0.1-finetuned model.