Any idea how I can get a functioning 'mapnik' for Ubuntu 17.10?

[kennykb]

I'm at wits' end here.

I recently upgraded to Ubuntu 17.10, and that seems to have no working mapnik. That is already reported here: https://bugs.launchpad.net/ubuntu/+source/python-mapnik/+bug/1734142

I tried the instructions (under Ubuntu 16.04) at https://github.com/mapnik/mapnik/wiki/UbuntuInstallation . The package appears to build, the tests appear to run (except that the PostgreSQL tests need a database that I don't have), and 'sudo make install' appears to work. But neither 'python' (the system-installed python, which is python3), nor explicitly choosing 'python2' or 'python3' on the command line can find the 'mapnik' package after installation.

Having seen that there are current issues for python3/scons3, I tried starting over from a fresh clone with an explicit PYTHON=/usr/bin/python2 throughout (including the make command), and once again, it appears to build, it appears to install, and the package is unavailable to me at run time.

What am I missing? Is there a different distribution channel that I should be trying? Or do I need to modify my python setup from the stock Ubuntu in order to find the material that the Mapnik installer has placed in /usr/local?

[talaj]

nor explicitly choosing 'python2' or 'python3' on the command line can find the 'mapnik' package after installation

Check if Python is looking into the place where Mapnik is installed:

>>> import sys
>>> sys.path
['', '/usr/lib/python2.7', '/usr/lib/python2.7/plat-arm-linux-gnueabihf', '/usr/lib/python2.7/lib-tk', '/usr/lib/python2.7/lib-old', '/usr/lib/python2.7/lib-dynload', '/usr/local/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages']
>>>

If not, try to set PYTHONPATH.

[kennykb]

I discovered that the python-mapnik that I thought I'd built was an ancient relic that happened still to be in /usr/local.

Starting from scratch:

$ git clone https://github.com/mapnik/python-mapnik --depth 10
$ cd python-mapnik/
$ virtualenv -p /usr/bin/python2 my-mapnik
$ source my-mapnik/bin/activate
$ export MASON_BUILD=true
$ python setup.py install
$ python setup.py test

The result is that I get a large number of errors like:

ERROR: Failure: ArgumentError (Python argument types in
    DatasourceCache.register_datasources(str)
did not match C++ signature:
    register_datasources(std::string))
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/kennykb/mapnik/python-mapnik/.eggs/nose-1.3.7-py2.7.egg/nose/loader.py", line 418, in loadTestsFromName
    addr.filename, addr.module)
  File "/home/kennykb/mapnik/python-mapnik/.eggs/nose-1.3.7-py2.7.egg/nose/importer.py", line 47, in importFromPath
    return self.importFromDir(dir_path, fqname)
  File "/home/kennykb/mapnik/python-mapnik/.eggs/nose-1.3.7-py2.7.egg/nose/importer.py", line 94, in importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "/home/kennykb/mapnik/python-mapnik/test/python_tests/webp_encoding_test.py", line 9, in <module>
    import mapnik
  File "/home/kennykb/mapnik/python-mapnik/mapnik/__init__.py", line 1071, in <module>
    register_plugins()
  File "/home/kennykb/mapnik/python-mapnik/mapnik/__init__.py", line 1053, in register_plugins
    DatasourceCache.register_datasources(path)
ArgumentError: Python argument types in
    DatasourceCache.register_datasources(str)
did not match C++ signature:
    register_datasources(std::string)

So... I stumble upon mapnik/python-mapnik#103
which suggests that this is the result of an incompatibility between the Mason boost libraries and the system ones. So I repeat this entire process adding

$ export BOOST_THREAD_LIB=boost_thread
$ export BOOST_SYSTEM_LIB=boost_system
$ export BOOST_PYTHON_LIB=boost_python

Now it builds and tests with only five failing tests.

So, still in the virtualenv, I go back to where I build the tiles and try to launch again.

It gets farther - now, inside Mapnik, I get an error like:

RuntimeError: failed to initialize projection with: '&EPSG-3857;' in Map of 'background.xml'

(Mind you, this is a .xml stack that used to work!)

The header of background.xml includes the DOCTYPE:

<!DOCTYPE Map [

<!ENTITY % common-entities SYSTEM "include/entities.xml">

%common-entities;

]>

and within that file there is

<!ENTITY EPSG-3857 "+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0 +units=m +k=1.0 +no_defs">

mapnik/mapnik#3630 suggests that this is an issue with the underlying libmapnik.so being linked with ptree rather than libxml2 - but I've no idea how to troubleshoot this in a Mason build. Can anyone suggest where to go from here? I really can't rework the XML to avoid <!ENTITY> declarations.

[talaj]

As you found, the problem with XML entities is caused by using ptree instead of libxml2 with Mason. You can set libxml2 for "Mason build" on this line.

But The Mason thing is used primarily for development of Mapnik, to provide particular versions of dependencies, because especially visual tests are very sensitive to versions of dependencies.
I would build Mapnik against system libraries for real mapping.

[kennykb]

@talaj Since the Mason build was the first thing mentioned in Readme.md, https://github.com/mapnik/python-mapnik, it was the first thing I tried. I'm not trying to set up a development environment for developing Mapnik, I'm simply trying to get a working build.

I did, in any case, manage to make a build that worked for me (except for a problem that I haven't yet tracked down that has identical symptoms to mapproxy/mapproxy#290 and appears to relate to a RasterSymbolizer trying to render a GDAL image at very low zoom levels). I can now live with the configuration I have. I'll open an issue for the RasterSymbolizer problem if I can manage to make it fail without needing to use the massive images (NLCD and NED-derived hillshading) that I see it failing on. In the meantime, simply disabling the raster images at the low zoom levels doesn't adversely affect the usability of my map.

I observe that libxml2 is not the default even when building from the system libraries, but must be explicitly requested on the configure command line. Fortunately, I stumbled upon mapnik/mapnik#3191 in the course of investigating this. This makes it clear to me that I should not expect XML entities to work in a standard Mapnik, and that I should instead use an external tool such as xmllint to preprocess the XML before giving it as input to Mapnik. (Since I wasn't familiar with xmllint prior to reading that issue, that was useful information.)

Apparently, the distributors of Xubuntu felt differently about libxml2 and ptree, and chose to enable that configuration item, so I never encountered trouble until resorting to a self-built Mapnik after Ubuntu released a non-working one (mismatched C++11 ABI settings between Boost and Mapnik). It's good to know that the feature actually cannot be depended on, and what the dependable substitute is.

[talaj]

Concerning the RasterSymbolizer problem, I saw a similar issue recently. Is the raster in the same projection as the map or are they different?