mplcairo-0.1

[cgohlke]

I noticed a few issues with mplcairo-0.1:

1. the Windows wheel is missing freetype.dll.

2. the file mplcairo.pth seems wrong, containing literal "\n" instead of line breaks.

3. on Windows, I get a segfault when closing the Python interpreter window after executing import mplcairo. I am unable to get a stacktrace.

4. the documentation claims that the cairo binaries at https://www.lfd.uci.edu/~gohlke/pythonlibs/ are not built with freetype support, however I am sure they are.

[anntzer]

the Windows wheel is missing freetype.dll.

I think "preshing's cairo.dll" (https://github.com/preshing/cairo-windows/releases) statically links it. See also (3).

the file mplcairo.pth seems wrong, containing literal "\n" instead of line breaks.

This is intentional (the pth trick, see e.g. https://www.reddit.com/r/Python/comments/1g94vc/are_pth_file_hacks_elegant_or_evil/; I use exec("some literal\nstring with newlines") to be able to exec, well, multiple lines).

on Windows, I get a segfault when closing the Python interpreter window after executing import mplcairo. I am unable to get a stacktrace.

The following works for me:
Create a clean conda env with just Py3.6, run pip install mplcairo (pulling in matplotlib as dependency), then:

set MPLBACKEND=module://mplcairo.tk
python -c "from pylab import *; plot(); show()"

(one can print(gcf().canvas) to check from the canvas type that mplcairo is indeed active); this also confirms that the freetype symbols are available. Similarly, qt works (pyqt needs to be installed from conda -- the PyPI wheels have never been compatible with conda's python in my experience.

One can also do

set MPLBACKEND=tkagg
set MPLCAIRO_PATCH_AGG=1
python -c "from pylab import *; print(gcf().canvas.renderer)"

to check that the pth file is working.

the documentation claims that the cairo binaries at lfd.uci.edu/~gohlke/pythonlibs are not built with freetype support, however I am sure they are.

Indeed. Did this change recently? That part of the docs was written a few months ago and I am happy to fix it (not that this changes anything to the build strategy I think(?), as I still cannot "steal" the symbols from _cairo.cpXX-win_amd64.pyd if I understand Windows symbol resolution correctly).

[cgohlke]

the Windows wheel is missing freetype.dll.

I think "preshing's cairo.dll"
(https://github.com/preshing/cairo-windows/releases) statically links
it. See also (3).

The _mplcairo C extension is linked to freetype.dll

the file mplcairo.pth seems wrong, containing literal "\n" instead
of line breaks.

This is intentional (the pth trick, see e.g.
https://www.reddit.com/r/Python/comments/1g94vc/are_pth_file_hacks_elegant_or_evil/;
I use |exec("some literal\nstring with newlines")| to be able to exec,
well, multiple lines).

I see. I missed that the multi-line statements were in an exec().

on Windows, I get a segfault when closing the Python interpreter
window after executing import mplcairo. I am unable to get a stacktrace.

The following works for me:

I was not reporting that mplcairo does not function (it seems to works well!) but that simply importing mplcairo leads to segfaults when closing the Python interpreter window without quitting the interpreter first (at least it does on my computer).

the documentation claims that the cairo binaries at
lfd.uci.edu/~gohlke/pythonlibs are not built with freetype support,
however I am sure they are.

Indeed. Did this change recently? That part of the docs was written a
few months ago and I am happy to fix it (not that this changes anything
to the build strategy I think(?), as I still cannot "steal" the symbols
from |_cairo.cpXX-win_amd64.pyd| if I understand Windows symbol
resolution correctly).

No, I did not change this recently. I was just wondering if there was a specific reason why those cairo binaries should not work. Anyway, it's probably best to leave it to the distributor of binaries which freetype enabled cairo library to link/ship.

[anntzer]

The _mplcairo C extension is linked to freetype.dll

So overall, is there a problem or something that needs to be fixed? I don't think I follow your point.

I was not reporting that mplcairo does not function (it seems to works well!) but that simply importing mplcairo leads to segfaults when closing the Python interpreter window without quitting the interpreter first (at least it does on my computer).

Oh, I see, I misread the original message. I can reproduce this (on Windows only), or rather I can get the "Python stopped working" window pop up. Likely some stuff that gets torn down in the wrong order? I don't have experience with debugging on Windows so any insights would be welcome here.

Edit: I figured out that this happens during deallocation of the MathTextParser class (tstate is NULL at line 1135 of cpython 3.6.6). I guess the class is deallocated too late in the Python shutdown so other machinery has already been torn down, but it's not clear why.

No, I did not change this recently. I was just wondering if there was a specific reason why those cairo binaries should not work. Anyway, it's probably best to leave it to the distributor of binaries which freetype enabled cairo library to link/ship.

pycairo binaries are not useful on Windows anyways because I can't steal the symbols from them (unlike on POSIX systems, where after importing (at the Python level) pycairo, the cairo symbols become available to everyone). So in a sense the whole discussion in the README was indeed written in a somewhat confusing way. What I should have referred to is anaconda and conda-forge's cairo (not pycairo), which on Windows is built without FreeType support and thus cannot be used. AFAICT you do not distribute cairo (not pycairo) binaries so I didn't need to mention you for that specific point. I have updated the README, does it look clearer now?

You can point to your cairo of choice by setting the LINK environment variable, as documented in https://github.com/anntzer/mplcairo#windows.

---

As a side note, if you have some time to spare and are interested in looking into it, generating wheels with Raqm support (for complex text layout, e.g. Arabic and Hebrew) would be greatly appreciated (likely relevant code in _raqm.cpp and _os.cpp, I think just dumping raqm.dll next to every other file should be sufficient).

[cgohlke]

the Windows wheel is missing freetype.dll.

The _mplcairo C extension is linked to freetype.dll

So overall, is there a problem or something that needs to be fixed? I don't think I follow your point.

freetype.dll needs to be included in the official wheels. Or linked statically.

[anntzer]

Why? The official wheels include cairo.dll, which statically link freetype.
If a downstream redistributor (e.g. yourself) wants to ship their own build of cairo.dll which does not statically link freetype, then they should start from the sdist and it is their job to also include freetype.dll in the wheel -- I think? (If you need to add an optional copy of freetype.dll at https://github.com/anntzer/mplcairo/blob/master/setup.py#L156, then I can do it of course.)

[cgohlke]

Why?

Because the _mplcairo.cp36-win_amd64.pyd C extension itself is linked to freetype.dll (in addition to cairo.dll) and does not load without freetype.dll

>py -3.6 -m pip install mplcairo
Collecting mplcairo
  Downloading https://files.pythonhosted.org/packages/6a/70/45f26f57cfd23f4bfaa63c24562cd523144071f3fcea11f7de85d3122a5b/mplcairo-0.1-cp36-cp36m-win_amd64.whl (1.3MB)
    100% |████████████████████████████████| 1.3MB 6.6MB/s
Requirement already satisfied: matplotlib>=2.2 in x:\python36\lib\site-packages (from mplcairo) (2.2.2)
Requirement already satisfied: pybind11>=2.2.3 in x:\python36\lib\site-packages (from mplcairo) (2.2.3)
Requirement already satisfied: python-dateutil>=2.1 in x:\python36\lib\site-packages (from matplotlib>=2.2->mplcairo) (2.7.3)
Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in x:\python36\lib\site-packages (from matplotlib>=2.2->mplcairo) (2.2.0)
Requirement already satisfied: kiwisolver>=1.0.1 in x:\python36\lib\site-packages (from matplotlib>=2.2->mplcairo) (1.0.1)
Requirement already satisfied: pytz in x:\python36\lib\site-packages (from matplotlib>=2.2->mplcairo) (2018.5)
Requirement already satisfied: six>=1.10 in x:\python36\lib\site-packages (from matplotlib>=2.2->mplcairo) (1.11.0)
Requirement already satisfied: numpy>=1.7.1 in x:\python36\lib\site-packages (from matplotlib>=2.2->mplcairo) (1.14.5+mkl)
Requirement already satisfied: cycler>=0.10 in x:\python36\lib\site-packages (from matplotlib>=2.2->mplcairo) (0.10.0)
Requirement already satisfied: setuptools in x:\python36\lib\site-packages (from kiwisolver>=1.0.1->matplotlib>=2.2->mplcairo) (39.2.0)
Installing collected packages: mplcairo
Successfully installed mplcairo-0.1

>py -3.6 -c"import mplcairo"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "X:\Python36\lib\site-packages\mplcairo\__init__.py", line 24, in <module>
    from ._mplcairo import antialias_t, operator_t, get_options, set_options
ImportError: DLL load failed: The specified module could not be found.

[anntzer]

conda create -yn tmpenv & activate tmpenv & pip install mplcairo & python -c "import mplcairo; print(mplcairo)" works for me...
Should I try a non-conda install?

[anntzer]

A clean non-conda install (from python.org) also works.

[cgohlke]

A clean non-conda install (from python.org) also works.

You probably have freetype.dll somewhere in the DLL search path. Try where freetype.dll.

[anntzer]

Ah, I see, I was implicitly relying on freetype.dll being provided by conda's matplotlib package (or rather its dependency freetype) that was installed in the root environment. Thanks for pointing that out!

I plan to just grab freetype.dll from conda and put it next to cairo.dll. Sounds good?

[anntzer]

How does the wheel at https://ci.appveyor.com/project/anntzer/mplcairo/build/artifacts look for you?

[anntzer]

Interestingly, on Linux, I can get the same segfault on Py3.7 (not Py3.6) by importing both mplcairo and pandas into the same IPython (not plain python), and then exiting IPython (again, tstate is null when deallocating MathTextParser).

[anntzer]

@cgohlke Can you check whether the following patch fixes the issue re: segfault at shutdown for you? It does fix the issue I described in the previous comment, and the suggested cause (see comment) at least seems plausible...

diff --git i/src/_mplcairo.cpp w/src/_mplcairo.cpp
index 8fbc1ef..bc90c6c 100644
--- i/src/_mplcairo.cpp
+++ w/src/_mplcairo.cpp
@@ -1514,19 +1514,22 @@ PYBIND11_MODULE(_mplcairo, m)
     py::module::import("matplotlib.mathtext")
     .attr("MathTextParser")("mplcairo");
 
-  auto cleanup = py::cpp_function{
-    [&](py::handle weakref) -> void {
-      FT_Done_FreeType(detail::ft_library);
-
-      // Make sure that these objects don't outlive the Python interpreter.
-      detail::UNIT_CIRCLE = {};
-      detail::PIXEL_MARKER = {};
-      GraphicsContextRenderer::mathtext_parser_ = {};
-
-      weakref.dec_ref();
+  py::module::import("atexit").attr("register")(
+    py::cpp_function{
+      [&]() -> void {
+        FT_Done_FreeType(detail::ft_library);
+        // Make sure that these objects don't outlive the Python interpreter.
+        // (It appears that sometimes, a weakref callback to the module doesn't
+        // get called at shutdown, so if we rely on that approach instead of
+        // using `atexit.register`, we may get a segfault when Python tries to
+        // deallocate the type objects (via a decref by the C++ destructor) too
+        // late in the shutdown sequence.)
+        detail::UNIT_CIRCLE = {};
+        detail::PIXEL_MARKER = {};
+        GraphicsContextRenderer::mathtext_parser_ = {};
+      }
     }
-  };
-  py::weakref(m, cleanup).release();
+  );
 
   // Export symbols.
 

[cgohlke]

Can you check whether the following patch fixes the issue re: segfault at shutdown for you?

Unfortunately not. I just tried mplcairo-0.1.post6+g5a4ec01.d20180814 with your patch on Python 3.7.