Build error, when downloading: https://naciscdn.org/naturalearth/110m/physical/ne_110m_coastline.zip

[chengzhuzhang]

The ne_110_coastline.zip file is required for cartopy plotting. But the servers has been unstable and resulted build errors in CI/CD in a few occasions.

 /usr/share/miniconda/envs/e3sm_diags_env_dev/lib/python3.9/site-packages/cartopy/io/__init__.py:241: DownloadWarning: Downloading: https://naciscdn.org/naturalearth/110m/physical/ne_110m_coastline.zip
  warnings.warn('Downloading: {}'.format(url), DownloadWarning)
Traceback (most recent call last):
  File "/usr/share/miniconda/envs/e3sm_diags_env_dev/lib/python3.9/urllib/request.py", line 1346, in do_open
    h.request(req.get_method(), req.selector, req.data, headers,
  File "/usr/share/miniconda/envs/e3sm_diags_env_dev/lib/python3.9/http/client.py", line 1253, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/share/miniconda/envs/e3sm_diags_env_dev/lib/python3.9/http/client.py", line 1299, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/share/miniconda/envs/e3sm_diags_env_dev/lib/python3.9/http/client.py", line 1248, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/share/miniconda/envs/e3sm_diags_env_dev/lib/python3.9/http/client.py", line 1008, in _send_output
    self.send(msg)
  File "/usr/share/miniconda/envs/e3sm_diags_env_dev/lib/python3.9/http/client.py", line 948, in send
    self.connect()
  File "/usr/share/miniconda/envs/e3sm_diags_env_dev/lib/python3.9/http/client.py", line 1415, in connect
    super().connect()
  File "/usr/share/miniconda/envs/e3sm_diags_env_dev/lib/python3.9/http/client.py", line 919, in connect
    self.sock = self._create_connection(
  File "/usr/share/miniconda/envs/e3sm_diags_env_dev/lib/python3.9/socket.py", line 843, in create_connection
    raise err
  File "/usr/share/miniconda/envs/e3sm_diags_env_dev/lib/python3.9/socket.py", line 831, in create_connection
    sock.connect(sa)
TimeoutError: [Errno 110] Connection timed out

[chengzhuzhang]

There is a fix merged in cartopy 6 days ago to address this issue by pointing to a new reliable server:
SciTools/cartopy#1833

[xylar]

That has made it to conda-forge as a back-port: conda-forge/cartopy-feedstock#116

[chengzhuzhang]

Thank you Xylar. CI is now green.

[xylar]

You can also include the (200+ MB) cartopy_offlinedata package so the data gets downloaded as part of the conda environment from the start: https://github.com/conda-forge/cartopy_offlinedata-feedstock. This prevents issues when trying to download cartopy data on machines with a firewall or on compute nodes without access to the internet.

[chengzhuzhang]

yes, Tom and I talked about this option. 200+ MB is no small. I guess let's see if the new server is stable enough and if we have the need to support machines without access to the online package.

[xylar]

At typical conda environment is about 1 GB so it's not in the noise but typically also not more than an additional 20%.

[xylar]

We have had a lot of trouble with MPAS-Analysis before we included this, but mostly because of the firewall at LANL.

[chengzhuzhang]

I see. yeah, in this case, it makes sense to follow what MPAS-Analysis does for including these offline data then...