Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cartopy unable to pull files from offline data directory #1072

Closed
puneet336 opened this issue May 20, 2018 · 3 comments
Closed

cartopy unable to pull files from offline data directory #1072

puneet336 opened this issue May 20, 2018 · 3 comments
Labels
Component: installation Type: Question

Comments

@puneet336
Copy link

puneet336 commented May 20, 2018 •
edited
Loading

Description

cartopy tries to pull data from internet while running/generating plot. As the machine has no access to internet, the download fails.
I was a query on similar issue at stack overflow
So i tried setting -
cartopy.config['data_dir']='/home/user/.local/share/cartopy'
cartopy.config['pre_existing_data_dir']='/home/user/.local/share/cartopy'

and copied the relevant zip file at -

[node1][/home/user/Post_process]> ls /home/user/.local/share/cartopy
ne_110m_coastline.zip  shapefiles

[node1][/home/user/Post_process]> ls /home/user/.local/share/cartopy/shapefiles/natural_earth/physical/
ne_110m_coastline.zip

, here are few lines from code -

import numpy as np  # Numerical python
import matplotlib.pyplot as plt  # General ploting
import matplotlib.cm as mpl_cm  # Colour maps
import matplotlib.ticker as mticker
import iris, json, os, sys, numpy, time
import iris.quickplot as qplt # Quickplot for Iris Cubes
#from cartopy.mpl.gridliner import LONGITUDE_FORMATTER, LATITUDE_FORMATTER
from mpl_toolkits.basemap import Basemap
import cartopy
import cartopy.crs as ccrs  # Cartography module used with Iris
import cartopy.feature as cfeature
from numpy import linspace
from numpy import meshgrid

#map = Basemap(projection='merc', lat_0 = 57, lon_0 = -135,
#    resolut   ion = 'h', area_thresh = 0.1,
#    llcrnrlon=-136.25, llcrnrlat=56.0,
#    urcrnrlon=-134.25, urcrnrlat=57.75)
cartopy.config['data_dir']='/home/user/.local/share/cartopy'
cartopy.config['pre_existing_data_dir']='/home/user/.local/share/cartopy'
.....

Traceback

/home/apps/SiteSoftwares/gnu/PYTHONPACKAGES/2.7.9/ucs4/gnu/4.8.5/CARTOPY/0.16.0/lib64/python2.7/site-packages/Cartopy-0.16.0-py2.7-linux-x86_64.egg/cartopy/io/__init__.py:260: DownloadWarning: Downloading: http://naciscdn.org/naturalearth/110m/physical/ne_110m_coastline.zip
  warnings.warn('Downloading: {}'.format(url), DownloadWarning)
Traceback (most recent call last):
  File "oper_vis_temp_rh_regional_space_1.py", line 227, in <module>
    plt.savefig(os.path.join(outdir, '%s_%s_%s.png' % (shortname, date, str(fhr).zfill(2))))
  File "/home/apps/SiteSoftwares/gnu/PYTHONPACKAGES/2.7.9/ucs4/gnu/4.8.5/MATPLOTLIB/2.2.2/lib64/python2.7/site-packages/matplotlib-2.2.2-py2.7-linux-x86_64.egg/matplotlib/pyplot.py", line 710, in savefig
    res = fig.savefig(*args, **kwargs)
  File "/home/apps/SiteSoftwares/gnu/PYTHONPACKAGES/2.7.9/ucs4/gnu/4.8.5/MATPLOTLIB/2.2.2/lib64/python2.7/site-packages/matplotlib-2.2.2-py2.7-linux-x86_64.egg/matplotlib/figure.py", line 2035, in savefig
    self.canvas.print_figure(fname, **kwargs)
  File "/home/apps/SiteSoftwares/gnu/PYTHONPACKAGES/2.7.9/ucs4/gnu/4.8.5/MATPLOTLIB/2.2.2/lib64/python2.7/site-packages/matplotlib-2.2.2-py2.7-linux-x86_64.egg/matplotlib/backends/backend_qt5agg.py", line 94, in print_figure
    super(FigureCanvasQTAgg, self).print_figure(*args, **kwargs)
  File "/home/apps/SiteSoftwares/gnu/PYTHONPACKAGES/2.7.9/ucs4/gnu/4.8.5/MATPLOTLIB/2.2.2/lib64/python2.7/site-packages/matplotlib-2.2.2-py2.7-linux-x86_64.egg/matplotlib/backend_bases.py", line 2263, in print_figure
    **kwargs)
  File "/home/apps/SiteSoftwares/gnu/PYTHONPACKAGES/2.7.9/ucs4/gnu/4.8.5/MATPLOTLIB/2.2.2/lib64/python2.7/site-packages/matplotlib-2.2.2-py2.7-linux-x86_64.egg/matplotlib/backends/backend_agg.py", line 513, in print_png
    FigureCanvasAgg.draw(self)
  File "/home/apps/SiteSoftwares/gnu/PYTHONPACKAGES/2.7.9/ucs4/gnu/4.8.5/MATPLOTLIB/2.2.2/lib64/python2.7/site-packages/matplotlib-2.2.2-py2.7-linux-x86_64.egg/matplotlib/backends/backend_agg.py", line 433, in draw
    self.figure.draw(self.renderer)
  File "/home/apps/SiteSoftwares/gnu/PYTHONPACKAGES/2.7.9/ucs4/gnu/4.8.5/MATPLOTLIB/2.2.2/lib64/python2.7/site-packages/matplotlib-2.2.2-py2.7-linux-x86_64.egg/matplotlib/artist.py", line 55, in draw_wrapper
    return draw(artist, renderer, *args, **kwargs)
  File "/home/apps/SiteSoftwares/gnu/PYTHONPACKAGES/2.7.9/ucs4/gnu/4.8.5/MATPLOTLIB/2.2.2/lib64/python2.7/site-packages/matplotlib-2.2.2-py2.7-linux-x86_64.egg/matplotlib/figure.py", line 1475, in draw
    renderer, self, artists, self.suppressComposite)
  File "/home/apps/SiteSoftwares/gnu/PYTHONPACKAGES/2.7.9/ucs4/gnu/4.8.5/MATPLOTLIB/2.2.2/lib64/python2.7/site-packages/matplotlib-2.2.2-py2.7-linux-x86_64.egg/matplotlib/image.py", line 141, in _draw_list_compositing_images
    a.draw(renderer)
  File "/home/apps/SiteSoftwares/gnu/PYTHONPACKAGES/2.7.9/ucs4/gnu/4.8.5/MATPLOTLIB/2.2.2/lib64/python2.7/site-packages/matplotlib-2.2.2-py2.7-linux-x86_64.egg/matplotlib/artist.py", line 55, in draw_wrapper
    return draw(artist, renderer, *args, **kwargs)
  File "/home/apps/SiteSoftwares/gnu/PYTHONPACKAGES/2.7.9/ucs4/gnu/4.8.5/CARTOPY/0.16.0/lib64/python2.7/site-packages/Cartopy-0.16.0-py2.7-linux-x86_64.egg/cartopy/mpl/geoaxes.py", line 385, in draw
    inframe=inframe)
  File "/home/apps/SiteSoftwares/gnu/PYTHONPACKAGES/2.7.9/ucs4/gnu/4.8.5/MATPLOTLIB/2.2.2/lib64/python2.7/site-packages/matplotlib-2.2.2-py2.7-linux-x86_64.egg/matplotlib/artist.py", line 55, in draw_wrapper
    return draw(artist, renderer, *args, **kwargs)
  File "/home/apps/SiteSoftwares/gnu/PYTHONPACKAGES/2.7.9/ucs4/gnu/4.8.5/MATPLOTLIB/2.2.2/lib64/python2.7/site-packages/matplotlib-2.2.2-py2.7-linux-x86_64.egg/matplotlib/axes/_base.py", line 2607, in draw
    mimage._draw_list_compositing_images(renderer, self, artists)
  File "/home/apps/SiteSoftwares/gnu/PYTHONPACKAGES/2.7.9/ucs4/gnu/4.8.5/MATPLOTLIB/2.2.2/lib64/python2.7/site-packages/matplotlib-2.2.2-py2.7-linux-x86_64.egg/matplotlib/image.py", line 141, in _draw_list_compositing_images
    a.draw(renderer)
  File "/home/apps/SiteSoftwares/gnu/PYTHONPACKAGES/2.7.9/ucs4/gnu/4.8.5/MATPLOTLIB/2.2.2/lib64/python2.7/site-packages/matplotlib-2.2.2-py2.7-linux-x86_64.egg/matplotlib/artist.py", line 55, in draw_wrapper
    return draw(artist, renderer, *args, **kwargs)
  File "/home/apps/SiteSoftwares/gnu/PYTHONPACKAGES/2.7.9/ucs4/gnu/4.8.5/CARTOPY/0.16.0/lib64/python2.7/site-packages/Cartopy-0.16.0-py2.7-linux-x86_64.egg/cartopy/mpl/feature_artist.py", line 137, in draw
    geoms = self._feature.intersecting_geometries(extent)
  File "/home/apps/SiteSoftwares/gnu/PYTHONPACKAGES/2.7.9/ucs4/gnu/4.8.5/CARTOPY/0.16.0/lib64/python2.7/site-packages/Cartopy-0.16.0-py2.7-linux-x86_64.egg/cartopy/feature.py", line 120, in intersecting_geometries
    return (geom for geom in self.geometries() if
  File "/home/apps/SiteSoftwares/gnu/PYTHONPACKAGES/2.7.9/ucs4/gnu/4.8.5/CARTOPY/0.16.0/lib64/python2.7/site-packages/Cartopy-0.16.0-py2.7-linux-x86_64.egg/cartopy/feature.py", line 191, in geometries
    name=self.name)
  File "/home/apps/SiteSoftwares/gnu/PYTHONPACKAGES/2.7.9/ucs4/gnu/4.8.5/CARTOPY/0.16.0/lib64/python2.7/site-packages/Cartopy-0.16.0-py2.7-linux-x86_64.egg/cartopy/io/shapereader.py", line 265, in natural_earth
    return ne_downloader.path(format_dict)
  File "/home/apps/SiteSoftwares/gnu/PYTHONPACKAGES/2.7.9/ucs4/gnu/4.8.5/CARTOPY/0.16.0/lib64/python2.7/site-packages/Cartopy-0.16.0-py2.7-linux-x86_64.egg/cartopy/io/__init__.py", line 222, in path
    result_path = self.acquire_resource(target_path, format_dict)
  File "/home/apps/SiteSoftwares/gnu/PYTHONPACKAGES/2.7.9/ucs4/gnu/4.8.5/CARTOPY/0.16.0/lib64/python2.7/site-packages/Cartopy-0.16.0-py2.7-linux-x86_64.egg/cartopy/io/shapereader.py", line 320, in acquire_resource
    shapefile_online = self._urlopen(url)
  File "/home/apps/SiteSoftwares/gnu/PYTHONPACKAGES/2.7.9/ucs4/gnu/4.8.5/CARTOPY/0.16.0/lib64/python2.7/site-packages/Cartopy-0.16.0-py2.7-linux-x86_64.egg/cartopy/io/__init__.py", line 261, in _urlopen
    return urlopen(url)
  File "/usr/lib64/python2.7/urllib2.py", line 154, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib64/python2.7/urllib2.py", line 431, in open
    response = self._open(req, data)
  File "/usr/lib64/python2.7/urllib2.py", line 449, in _open
    '_open', req)
  File "/usr/lib64/python2.7/urllib2.py", line 409, in _call_chain
    result = func(*args)
  File "/usr/lib64/python2.7/urllib2.py", line 1227, in http_open
    return self.do_open(httplib.HTTPConnection, req)
  File "/usr/lib64/python2.7/urllib2.py", line 1197, in do_open
    raise URLError(err)
urllib2.URLError: <urlopen error [Errno -3] Temporary failure in name resolution>
Full environment definition

Operating system

SLES 12

Cartopy version

0.16.0

please let me know if further information is required.
@pelson
Copy link
Member

pelson commented Dec 5, 2018

Looks like something changed along the way and there is no longer a ne_ prefix in the filename. You will also need to unzip the file I think (from memory).

This is really a question best suited for StackOverflow, but happy to help if I can be of assistance.

@pelson pelson closed this as completed Dec 5, 2018
@QuLogic
Copy link
Member

QuLogic commented Dec 5, 2018

It's the reverse, I think, in that #1037 added the prefix to the path that Cartopy used, because it was inconsistent with what NE provides. But with 0.16.0, you did have to rename things.

@puneet336
Copy link
Author

puneet336 commented Dec 5, 2018 •
edited
Loading

Thank you for replies,
I was able to figure out the "ne_" prefix part, though i now keep both files in $HOME/.local/share/cartopy/shapefiles/natural_earth/physical & cultural directories

-rw-r--r-- 1 user1 staff 5 Aug 1 17:23 10m_coastline.cpg -rw-r--r-- 1 user1 staff 5849675 Oct 30 16:37 ne_10m_admin_1_states_provinces_lines.zip -rw-r--r-- 1 user1 staff 81012 Oct 30 16:37 ne_10m_admin_1_states_provinces_lines.shx -rw-r--r-- 1 user1 staff 7365240 Oct 30 16:37 ne_10m_admin_1_states_provinces_lines.shp -rw-r--r-- 1 user1 staff 145 Oct 30 16:37 ne_10m_admin_1_states_provinces_lines.prj -rw-r--r-- 1 user1 staff 14534524 Oct 30 16:37 ne_10m_admin_1_states_provinces_lines.dbf -rw-r--r-- 1 user1 staff 5 Oct 30 16:37 ne_10m_admin_1_states_provinces_lines.cpg -rw-r--r-- 1 user1 staff 7 Oct 30 16:37 ne_10m_admin_1_states_provinces_lines.VERSION.txt -rw-r--r-- 1 user1 staff 29480 Oct 30 16:37 ne_10m_admin_1_states_provinces_lines.README.html -rw-r--r-- 1 user1 staff 81012 Oct 30 16:40 10m_admin_1_states_provinces_lines.shx -rw-r--r-- 1 user1 staff 7365240 Oct 30 16:40 10m_admin_1_states_provinces_lines.shp -rw-r--r-- 1 user1 staff 145 Oct 30 16:40 10m_admin_1_states_provinces_lines.prj -rw-r--r-- 1 user1 staff 14534524 Oct 30 16:40 10m_admin_1_states_provinces_lines.dbf -rw-r--r-- 1 user1 staff 5 Oct 30 16:40 10m_admin_1_states_provinces_lines.cpg -rw-r--r-- 1 user1 staff 7 Oct 30 16:41 10m_admin_1_states_provinces_lines.VERSION.txt -rw-r--r-- 1 user1 staff 29480 Oct 30 16:41 10m_admin_1_states_provinces_lines.README.html

@valeriupredoi valeriupredoi mentioned this issue Jan 23, 2019
Merged
@cpatrizio88 cpatrizio88 mentioned this issue Jul 30, 2019
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Component: installation Type: Question
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@QuLogic @pelson @puneet336