Question / Comment: How to ensure 1.17.4 installs through dependency on macOS 11.1, properly?

[martin-braun]

Hi, I try to install dependencies for a solution and it fails when installing PyMuPDF 1.17.4. I know that I have to install MuPDF 1.17.0 and replace the config.h with the _config.h from PyMuPDF 1.17.4 as it is described in this comment. However the installation still fails:

$% sudo -H pip3 install -r ./requirements.txt
Collecting pymupdf==1.17.4
  Using cached PyMuPDF-1.17.4.tar.gz (202 kB)
Collecting shapely==1.7.0
  Using cached Shapely-1.7.0.tar.gz (349 kB)
Using legacy 'setup.py install' for pymupdf, since package 'wheel' is not installed.
Using legacy 'setup.py install' for shapely, since package 'wheel' is not installed.
Installing collected packages: pymupdf, shapely
    Running setup.py install for pymupdf ... error
    ERROR: Command errored out with exit status 1:
     command: /opt/remarks/venv/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/tmp/pip-install-i49yvpm6/pymupdf/setup.py'"'"'; __file__='"'"'/private/tmp/pip-install-i49yvpm6/pymupdf/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/tmp/pip-record-bdzeg8ed/install-record.txt --single-version-externally-managed --compile --install-headers /opt/remarks/venv/include/site/python3.9/pymupdf
         cwd: /private/tmp/pip-install-i49yvpm6/pymupdf/
    Complete output (209 lines):
    running install
    running build
    running build_py
    creating build
    creating build/lib.macosx-11-x86_64-3.9
    creating build/lib.macosx-11-x86_64-3.9/fitz
    copying fitz/__init__.py -> build/lib.macosx-11-x86_64-3.9/fitz
    copying fitz/fitz.py -> build/lib.macosx-11-x86_64-3.9/fitz
    copying fitz/utils.py -> build/lib.macosx-11-x86_64-3.9/fitz
    copying fitz/__main__.py -> build/lib.macosx-11-x86_64-3.9/fitz
    running build_ext
    building 'fitz._fitz' extension
    creating build/temp.macosx-11-x86_64-3.9
    creating build/temp.macosx-11-x86_64-3.9/fitz
    clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -I/usr/local/include -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.0.sdk -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.0.sdk/usr/include -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.0.sdk/System/Library/Frameworks/Tk.framework/Versions/8.5/Headers -I/usr/local/include/mupdf -I/usr/local/include -I/usr/local/include -I/usr/local/opt/openssl@1.1/include -I/usr/local/opt/sqlite/include -I/opt/remarks/venv/include -I/usr/local/Cellar/python@3.9/3.9.0_5/Frameworks/Python.framework/Versions/3.9/include/python3.9 -c fitz/fitz_wrap.c -o build/temp.macosx-11-x86_64-3.9/fitz/fitz_wrap.o
    fitz/fitz_wrap.c:4381:30: warning: expression result unused [-Wunused-value]
                if (dest->alpha) *s++;
                                 ^~~~
    fitz/fitz_wrap.c:4380:19: warning: unsequenced modification and access to 's' [-Wunsequenced]
                    *s++ = 255 - *s;
                      ^           ~
    fitz/fitz_wrap.c:4403:11: warning: assigning to 'unsigned char *' from 'char *' converts between pointers to integer types with different sign [-Wpointer-sign]
            c = PyBytes_AS_STRING(imagedata);
              ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    fitz/fitz_wrap.c:4407:11: warning: assigning to 'unsigned char *' from 'char *' converts between pointers to integer types with different sign [-Wpointer-sign]
            c = PyByteArray_AS_STRING(imagedata);
              ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    fitz/fitz_wrap.c:4867:18: warning: unused variable 'popup' [-Wunused-variable]
            pdf_obj *popup = pdf_dict_get(ctx, annot->obj, PDF_NAME(Popup));
                     ^
    fitz/fitz_wrap.c:4959:14: warning: unused variable 'name' [-Wunused-variable]
        pdf_obj *name = NULL;
                 ^
    fitz/fitz_wrap.c:5052:12: warning: unused variable 'len' [-Wunused-variable]
        size_t len = 0;
               ^
    fitz/fitz_wrap.c:5696:47: warning: passing 'char *' to parameter of type 'const unsigned char *' converts between pointers to integer types with different sign [-Wpointer-sign]
        res = fz_new_buffer_from_copied_data(ctx, data, strlen(data));
                                                  ^~~~
    /usr/local/include/mupdf/fitz/buffer.h:83:81: note: passing argument to parameter 'data' here
    fz_buffer *fz_new_buffer_from_copied_data(fz_context *ctx, const unsigned char *data, size_t size);
                                                                                    ^
    fitz/fitz_wrap.c:6024:19: warning: unused variable 'pdf' [-Wunused-variable]
        pdf_document *pdf = pdf_get_bound_document(ctx, annot->obj);
                      ^
    fitz/fitz_wrap.c:6081:16: warning: unused variable 'res' [-Wunused-variable]
        fz_buffer *res = NULL;
                   ^
    fitz/fitz_wrap.c:6080:27: warning: unused variable 'js' [-Wunused-variable]
        pdf_obj *obj = NULL, *js = NULL, *o = NULL;
                              ^
    fitz/fitz_wrap.c:6080:39: warning: unused variable 'o' [-Wunused-variable]
        pdf_obj *obj = NULL, *js = NULL, *o = NULL;
                                          ^
    fitz/fitz_wrap.c:6486:64: warning: passing 'char [3]' to parameter of type 'const unsigned char *' converts between pointers to integer types with different sign [-Wpointer-sign]
                               fz_new_buffer_from_copied_data(ctx, "  ", 1),
                                                                   ^~~~
    /usr/local/include/mupdf/fitz/buffer.h:83:81: note: passing argument to parameter 'data' here
    fz_buffer *fz_new_buffer_from_copied_data(fz_context *ctx, const unsigned char *data, size_t size);
                                                                                    ^
    fitz/fitz_wrap.c:7408:40: warning: declaration of 'struct Document' will not be visible outside of this function [-Wvisibility]
    SWIGINTERN void delete_Document(struct Document *self){
                                           ^
    fitz/fitz_wrap.c:8030:17: warning: unused variable 'entry' [-Wunused-variable]
                int entry = 0;
                    ^
    fitz/fitz_wrap.c:8150:17: warning: unused variable 'page_n' [-Wunused-variable]
                int page_n = -1;
                    ^
    fitz/fitz_wrap.c:8380:68: warning: passing 'unsigned char *' to parameter of type 'const char *' converts between pointers to integer types with different sign [-Wpointer-sign]
                            LIST_APPEND_DROP(idlist, JM_UnicodeFromStr(hex));
                                                                       ^~~
    fitz/fitz_wrap.c:3448:41: note: passing argument to parameter 'c' here
    PyObject *JM_UnicodeFromStr(const char *c)
                                            ^
    fitz/fitz_wrap.c:8626:17: warning: unused variable 'cwlen' [-Wunused-variable]
                int cwlen = 0;
                    ^
    fitz/fitz_wrap.c:8630:36: warning: unused variable 'fb_font' [-Wunused-variable]
                fz_font *font = NULL, *fb_font= NULL;
                                       ^
    fitz/fitz_wrap.c:8627:17: warning: unused variable 'lang' [-Wunused-variable]
                int lang = 0;
                    ^
    fitz/fitz_wrap.c:8743:24: warning: unused variable 'len' [-Wunused-variable]
                Py_ssize_t len = 0;
                           ^
    fitz/fitz_wrap.c:9325:69: warning: passing 'char [3]' to parameter of type 'const unsigned char *' converts between pointers to integer types with different sign [-Wpointer-sign]
                                   fz_new_buffer_from_copied_data(gctx, "  ", 1), NULL, 0);
                                                                        ^~~~
    /usr/local/include/mupdf/fitz/buffer.h:83:81: note: passing argument to parameter 'data' here
    fz_buffer *fz_new_buffer_from_copied_data(fz_context *ctx, const unsigned char *data, size_t size);
                                                                                    ^
    fitz/fitz_wrap.c:9284:22: warning: unused variable 'page2' [-Wunused-variable]
                pdf_obj *page2 = NULL;
                         ^
    fitz/fitz_wrap.c:9361:26: warning: unused variable 'page2' [-Wunused-variable]
                    pdf_obj *page2 = pdf_lookup_page_loc(gctx, pdf, nb, &parent2, &i2);
                             ^
    fitz/fitz_wrap.c:9504:29: warning: unused variable 'seps' [-Wunused-variable]
                fz_separations *seps = NULL;
                                ^
    fitz/fitz_wrap.c:9943:104: warning: declaration of 'struct Colorspace' will not be visible outside of this function [-Wvisibility]
    SWIGINTERN struct Pixmap *Page__makePixmap(struct Page *self,struct Document *doc,PyObject *ctm,struct Colorspace *cs,int alpha,int annots,PyObject *clip){
                                                                                                           ^
    fitz/fitz_wrap.c:10204:142: warning: declaration of 'struct Graftmap' will not be visible outside of this function [-Wvisibility]
    SWIGINTERN PyObject *Page__showPDFpage(struct Page *self,struct Page *fz_srcpage,int overlay,PyObject *matrix,int xref,PyObject *clip,struct Graftmap *graftmap,char *_imgname){
                                                                                                                                                 ^
    fitz/fitz_wrap.c:10276:29: warning: unused variable 'seps' [-Wunused-variable]
                fz_separations *seps = NULL;
                                ^
    fitz/fitz_wrap.c:10494:35: warning: comparison of integers of different signs: 'int' and 'size_t' (aka 'unsigned long') [-Wsign-compare]
                        for (i = 0; i < n; i++) {
                                    ~ ^ ~
    fitz/fitz_wrap.c:10542:53: warning: declaration of 'struct Colorspace' will not be visible outside of this function [-Wvisibility]
    SWIGINTERN struct Pixmap *new_Pixmap__SWIG_0(struct Colorspace *cs,PyObject *bbox,int alpha){
                                                        ^
    fitz/fitz_wrap.c:10552:53: warning: declaration of 'struct Colorspace' will not be visible outside of this function [-Wvisibility]
    SWIGINTERN struct Pixmap *new_Pixmap__SWIG_1(struct Colorspace *cs,struct Pixmap *spix){
                                                        ^
    fitz/fitz_wrap.c:10621:53: warning: declaration of 'struct Colorspace' will not be visible outside of this function [-Wvisibility]
    SWIGINTERN struct Pixmap *new_Pixmap__SWIG_4(struct Colorspace *cs,int w,int h,PyObject *samples,int alpha){
                                                        ^
    fitz/fitz_wrap.c:10633:32: warning: comparison of integers of different signs: 'int' and 'size_t' (aka 'unsigned long') [-Wsign-compare]
                    if (stride * h != size) THROWMSG("bad samples length");
                        ~~~~~~~~~~ ^  ~~~~
    fitz/fitz_wrap.c:11683:19: warning: unused variable 'data' [-Wunused-variable]
                char *data = NULL;              // for new file content
                      ^
    fitz/fitz_wrap.c:11686:21: warning: unused variable 'size' [-Wunused-variable]
                int64_t size = 0;
                        ^
    fitz/fitz_wrap.c:12267:40: warning: declaration of 'struct Graftmap' will not be visible outside of this function [-Wvisibility]
    SWIGINTERN void delete_Graftmap(struct Graftmap *self){
                                           ^
    fitz/fitz_wrap.c:12284:42: warning: declaration of 'struct TextWriter' will not be visible outside of this function [-Wvisibility]
    SWIGINTERN void delete_TextWriter(struct TextWriter *self){
                                             ^
    fitz/fitz_wrap.c:12299:96: warning: declaration of 'struct Font' will not be visible outside of this function [-Wvisibility]
    SWIGINTERN PyObject *TextWriter_append(struct TextWriter *self,PyObject *pos,char *text,struct Font *font,float fontsize,char *language,int wmode,int bidi_level){
                                                                                                   ^
    fitz/fitz_wrap.c:12362:36: warning: declaration of 'struct Font' will not be visible outside of this function [-Wvisibility]
    SWIGINTERN void delete_Font(struct Font *self){
                                       ^
    fitz/fitz_wrap.c:12739:19: warning: incompatible pointer types passing 'struct Document *' to parameter of type 'struct Document *' [-Wincompatible-pointer-types]
      delete_Document(arg1);
                      ^~~~
    fitz/fitz_wrap.c:7408:50: note: passing argument to parameter 'self' here
    SWIGINTERN void delete_Document(struct Document *self){
                                                     ^
    fitz/fitz_wrap.c:16769:63: warning: incompatible pointer types passing 'struct Colorspace *' to parameter of type 'struct Colorspace *' [-Wincompatible-pointer-types]
        result = (struct Pixmap *)Page__makePixmap(arg1,arg2,arg3,arg4,arg5,arg6,arg7);
                                                                  ^~~~
    fitz/fitz_wrap.c:9943:116: note: passing argument to parameter 'cs' here
    SWIGINTERN struct Pixmap *Page__makePixmap(struct Page *self,struct Document *doc,PyObject *ctm,struct Colorspace *cs,int alpha,int annots,PyObject *clip){
                                                                                                                       ^
    fitz/fitz_wrap.c:17217:74: warning: incompatible pointer types passing 'struct Graftmap *' to parameter of type 'struct Graftmap *' [-Wincompatible-pointer-types]
        result = (PyObject *)Page__showPDFpage(arg1,arg2,arg3,arg4,arg5,arg6,arg7,arg8);
                                                                             ^~~~
    fitz/fitz_wrap.c:10204:152: note: passing argument to parameter 'graftmap' here
    SWIGINTERN PyObject *Page__showPDFpage(struct Page *self,struct Page *fz_srcpage,int overlay,PyObject *matrix,int xref,PyObject *clip,struct Graftmap *graftmap,char *_imgname){
                                                                                                                                                           ^
    fitz/fitz_wrap.c:17603:50: warning: incompatible pointer types passing 'struct Colorspace *' to parameter of type 'struct Colorspace *' [-Wincompatible-pointer-types]
        result = (struct Pixmap *)new_Pixmap__SWIG_0(arg1,arg2,arg3);
                                                     ^~~~
    fitz/fitz_wrap.c:10542:65: note: passing argument to parameter 'cs' here
    SWIGINTERN struct Pixmap *new_Pixmap__SWIG_0(struct Colorspace *cs,PyObject *bbox,int alpha){
                                                                    ^
    fitz/fitz_wrap.c:17638:50: warning: incompatible pointer types passing 'struct Colorspace *' to parameter of type 'struct Colorspace *' [-Wincompatible-pointer-types]
        result = (struct Pixmap *)new_Pixmap__SWIG_1(arg1,arg2);
                                                     ^~~~
    fitz/fitz_wrap.c:10552:65: note: passing argument to parameter 'cs' here
    SWIGINTERN struct Pixmap *new_Pixmap__SWIG_1(struct Colorspace *cs,struct Pixmap *spix){
                                                                    ^
    fitz/fitz_wrap.c:17777:50: warning: incompatible pointer types passing 'struct Colorspace *' to parameter of type 'struct Colorspace *' [-Wincompatible-pointer-types]
        result = (struct Pixmap *)new_Pixmap__SWIG_4(arg1,arg2,arg3,arg4,arg5);
                                                     ^~~~
    fitz/fitz_wrap.c:10621:65: note: passing argument to parameter 'cs' here
    SWIGINTERN struct Pixmap *new_Pixmap__SWIG_4(struct Colorspace *cs,int w,int h,PyObject *samples,int alpha){
                                                                    ^
    fitz/fitz_wrap.c:21495:19: warning: incompatible pointer types passing 'struct Graftmap *' to parameter of type 'struct Graftmap *' [-Wincompatible-pointer-types]
      delete_Graftmap(arg1);
                      ^~~~
    fitz/fitz_wrap.c:12267:50: note: passing argument to parameter 'self' here
    SWIGINTERN void delete_Graftmap(struct Graftmap *self){
                                                     ^
    fitz/fitz_wrap.c:21557:21: warning: incompatible pointer types passing 'struct TextWriter *' to parameter of type 'struct TextWriter *' [-Wincompatible-pointer-types]
      delete_TextWriter(arg1);
                        ^~~~
    fitz/fitz_wrap.c:12284:54: note: passing argument to parameter 'self' here
    SWIGINTERN void delete_TextWriter(struct TextWriter *self){
                                                         ^
    fitz/fitz_wrap.c:21678:59: warning: incompatible pointer types passing 'struct Font *' to parameter of type 'struct Font *' [-Wincompatible-pointer-types]
        result = (PyObject *)TextWriter_append(arg1,arg2,arg3,arg4,arg5,arg6,arg7,arg8);
                                                              ^~~~
    fitz/fitz_wrap.c:12299:102: note: passing argument to parameter 'font' here
    SWIGINTERN PyObject *TextWriter_append(struct TextWriter *self,PyObject *pos,char *text,struct Font *font,float fontsize,char *language,int wmode,int bidi_level){
                                                                                                         ^
    fitz/fitz_wrap.c:21817:15: warning: incompatible pointer types passing 'struct Font *' to parameter of type 'struct Font *' [-Wincompatible-pointer-types]
      delete_Font(arg1);
                  ^~~~
    fitz/fitz_wrap.c:12362:42: note: passing argument to parameter 'self' here
    SWIGINTERN void delete_Font(struct Font *self){
                                             ^
    49 warnings generated.
    clang -bundle -undefined dynamic_lookup -L/usr/local/lib -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.0.sdk build/temp.macosx-11-x86_64-3.9/fitz/fitz_wrap.o -L/usr/local/lib -L/usr/local/lib -L/usr/local/opt/openssl@1.1/lib -L/usr/local/opt/sqlite/lib -lmupdf -lmupdf-third -o build/lib.macosx-11-x86_64-3.9/fitz/_fitz.cpython-39-darwin.so
    ld: library not found for -lmupdf
    clang: error: linker command failed with exit code 1 (use -v to see invocation)
    error: command '/usr/bin/clang' failed with exit code 1
    ----------------------------------------
ERROR: Command errored out with exit status 1: /opt/remarks/venv/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/tmp/pip-install-i49yvpm6/pymupdf/setup.py'"'"'; __file__='"'"'/private/tmp/pip-install-i49yvpm6/pymupdf/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/tmp/pip-record-bdzeg8ed/install-record.txt --single-version-externally-managed --compile --install-headers /opt/remarks/venv/include/site/python3.9/pymupdf Check the logs for full command output.

In the comment I linked above, it was stated to generate MuPDF, but actually, I just moved the mupdf folder into /usr/local/include. Is this right? More details about my steps can be found here.

Why is the installation of of PyMuPDF 1.17.4 failing and how can I fix it?

$% sw_vers
ProductName:	macOS
ProductVersion:	11.1
BuildVersion:	20C5048k
$% python3 -V
Python 3.9.0
$% pip3 -V
pip 20.3.1 from /usr/local/lib/python3.9/site-packages/pip (python 3.9)

[JorjMcKie]

Your compile went okay - just the usual warnings.
The problem is the linker step: it does not find the required libs. You probably need to have another look into PyMuPDF's setup.py and adjust the lib folder names, parameter library_dirs.