macOS app throws "library load disallowed by system policy" error while loading 3rd-party Python `.so` modules

[FeodorFitsner]

Duplicate Check

• I have searched the opened issues and there are no duplicates

Describe the bug

Original report: #4514 (comment)

I got this error when I release it to test flight and test on my mac M4:

  File "/Users/simon/Library/Containers/com.diogenes.ai.chatbot.flet.diogenesaibutlerflet/Data/Library/Application Support/com.diogenes.ai.chatbot.flet.diogenesaibutlerflet/flet/app/__pypackages__/pydantic_core/__init__.py", line 6, in <module>
    from ._pydantic_core import (
ImportError: dlopen(/Users/simon/Library/Containers/com.diogenes.ai.chatbot.flet.diogenesaibutlerflet/Data/Library/Application Support/com.diogenes.ai.chatbot.flet.diogenesaibutlerflet/flet/app/__pypackages__/pydantic_core/_pydantic_core.cpython-312-darwin.so, 0x0002): tried: '/Users/simon/Library/Containers/com.diogenes.ai.chatbot.flet.diogenesaibutlerflet/Data/Library/Application Support/com.diogenes.ai.chatbot.flet.diogenesaibutlerflet/flet/app/__pypackages__/pydantic_core/_pydantic_core.cpython-312-darwin.so' (code signature in <17D6EF5C-4D71-3E06-912D-5464BC98FC00> '/Users/simon/Library/Containers/com.diogenes.ai.chatbot.flet.diogenesaibutlerflet/Data/Library/Application Support/com.diogenes.ai.chatbot.flet.diogenesaibutlerflet/flet/app/__pypackages__/pydantic_core/_pydantic_core.cpython-312-darwin.so' not valid for use in process: library load disallowed by system policy), '/System/Volumes/Preboot/Cryptexes/OS/Users/simon/Library/Containers/com.diogenes.ai.chatbot.flet.diogenesaibutlerflet/Data/Library/Application Support/com.diogenes.ai.chatbot.flet.diogenesaibutlerflet/flet/app/__pypackages__/pydantic_core/_pydantic_core.cpython-312-darwin.so' (no such file), '/Users/simon/Library/Containers/com.diogenes.ai.chatbot.flet.diogenesaibutlerflet/Data/Library/Application Support/com.diogenes.ai.chatbot.flet.diogenesaibutlerflet/flet/app/__pypackages__/pydantic_core/_pydantic_core.cpython-312-darwin.so' (code signature in <17D6EF5C-4D71-3E06-912D-5464BC98FC00> '/Users/simon/Library/Containers/com.diogenes.ai.chatbot.flet.diogenesaibutlerflet/Data/Library/Application Support/com.diogenes.ai.chatbot.flet.diogenesaibutlerflet/flet/app/__pypackages__/pydantic_core/_pydantic_core.cpython-312-darwin.so' not valid for use in process: library load disallowed by system policy)

Code sample

Code

# n/a

To reproduce

n/a

Expected behavior

No response

Screenshots / Videos

Captures

[Upload media here]

Operating System

macOS

Operating system details

Any version

Flet version

0.25.1

Regression

No, it isn't

Suggestions

No response

Logs

Logs

[Paste your logs here]

Additional details

No response

[dsx1986]

I do build with entitlements: flet build macos --info-plist LSApplicationCategoryType="public.app-category.productivity" --macos-entitlements com.apple.security.app-sandbox=True --build-number $(date +%y%m%d%H%M)

[FeodorFitsner]

OK, as it's sandbox my guess would be #2 from here: " it could be a problem with signing 3rd-party python packages (like pydantic) unpacked during the first run."

Will investigate.

[FeodorFitsner]

Note to developers: https://docs.flutter.dev/deployment/macos

[dsx1986]

I got another one, it is : .cpython-312-darwin.so' not valid for use in process: library load disallowed by system policy. It also relates to numpy. I tried this solution(flet-dev/serious-python#85 (comment)), but did not work.

Traceback (most recent call last):
File "/Users/simon/Library/Containers/com.diogenes.ai.chatbot.flet.diogenesaibutlerflet/Data/Library/Application Support/com.diogenes.ai.chatbot.flet.diogenesaibutlerflet/flet/app/pypackages/numpy/core/init.py", line 24, in
from . import multiarray
File "/Users/simon/Library/Containers/com.diogenes.ai.chatbot.flet.diogenesaibutlerflet/Data/Library/Application Support/com.diogenes.ai.chatbot.flet.diogenesaibutlerflet/flet/app/pypackages/numpy/core/multiarray.py", line 10, in
from . import overrides
File "/Users/simon/Library/Containers/com.diogenes.ai.chatbot.flet.diogenesaibutlerflet/Data/Library/Application Support/com.diogenes.ai.chatbot.flet.diogenesaibutlerflet/flet/app/pypackages/numpy/core/overrides.py", line 8, in
from numpy.core._multiarray_umath import (
ImportError: dlopen(/Users/simon/Library/Containers/com.diogenes.ai.chatbot.flet.diogenesaibutlerflet/Data/Library/Application Support/com.diogenes.ai.chatbot.flet.diogenesaibutlerflet/flet/app/pypackages/numpy/core/_multiarray_umath.cpython-312-darwin.so, 0x0002): tried: '/Users/simon/Library/Containers/com.diogenes.ai.chatbot.flet.diogenesaibutlerflet/Data/Library/Application Support/com.diogenes.ai.chatbot.flet.diogenesaibutlerflet/flet/app/pypackages/numpy/core/_multiarray_umath.cpython-312-darwin.so' (code signature in <4B244C37-40C5-380A-9FCF-3FE515E61AAA> '/Users/simon/Library/Containers/com.diogenes.ai.chatbot.flet.diogenesaibutlerflet/Data/Library/Application Support/com.diogenes.ai.chatbot.flet.diogenesaibutlerflet/flet/app/pypackages/numpy/core/_multiarray_umath.cpython-312-darwin.so' not valid for use in process: library load disallowed by system policy), '/System/Volumes/Preboot/Cryptexes/OS/Users/simon/Library/Containers/com.diogenes.ai.chatbot.flet.diogenesaibutlerflet/Data/Library/Application Support/com.diogenes.ai.chatbot.flet.diogenesaibutlerflet/flet/app/pypackages/numpy/core/_multiarray_umath.cpython-312-darwin.so' (no such file), '/Users/simon/Library/Containers/com.diogenes.ai.chatbot.flet.diogenesaibutlerflet/Data/Library/Application Support/com.diogenes.ai.chatbot.flet.diogenesaibutlerflet/flet/app/pypackages/numpy/core/_multiarray_umath.cpython-312-darwin.so' (code signature in <4B244C37-40C5-380A-9FCF-3FE515E61AAA> '/Users/simon/Library/Containers/com.diogenes.ai.chatbot.flet.diogenesaibutlerflet/Data/Library/Application Support/com.diogenes.ai.chatbot.flet.diogenesaibutlerflet/flet/app/pypackages/numpy/core/_multiarray_umath.cpython-312-darwin.so' not valid for use in process: library load disallowed by system policy)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/Users/simon/Library/Containers/com.diogenes.ai.chatbot.flet.diogenesaibutlerflet/Data/Library/Application Support/com.diogenes.ai.chatbot.flet.diogenesaibutlerflet/flet/app/pypackages/numpy/init.py", line 130, in
from numpy.config import show as show_config
File "/Users/simon/Library/Containers/com.diogenes.ai.chatbot.flet.diogenesaibutlerflet/Data/Library/Application Support/com.diogenes.ai.chatbot.flet.diogenesaibutlerflet/flet/app/pypackages/numpy/config.py", line 4, in
from numpy.core._multiarray_umath import (
File "/Users/simon/Library/Containers/com.diogenes.ai.chatbot.flet.diogenesaibutlerflet/Data/Library/Application Support/com.diogenes.ai.chatbot.flet.diogenesaibutlerflet/flet/app/pypackages/numpy/core/init.py", line 50, in
raise ImportError(msg)
ImportError:

IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.

We have compiled some common reasons and troubleshooting tips at:

https://numpy.org/devdocs/user/troubleshooting-importerror.html

Please note and check the following:

• The Python version is: Python3.12 from "/Applications/diogenesaibutlerflet.app/Contents/MacOS/diogenesaibutlerflet"
• The NumPy version is: "1.26.4"

and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.

Original error was: dlopen(/Users/simon/Library/Containers/com.diogenes.ai.chatbot.flet.diogenesaibutlerflet/Data/Library/Application Support/com.diogenes.ai.chatbot.flet.diogenesaibutlerflet/flet/app/pypackages/numpy/core/_multiarray_umath.cpython-312-darwin.so, 0x0002): tried: '/Users/simon/Library/Containers/com.diogenes.ai.chatbot.flet.diogenesaibutlerflet/Data/Library/Application Support/com.diogenes.ai.chatbot.flet.diogenesaibutlerflet/flet/app/pypackages/numpy/core/_multiarray_umath.cpython-312-darwin.so' (code signature in <4B244C37-40C5-380A-9FCF-3FE515E61AAA> '/Users/simon/Library/Containers/com.diogenes.ai.chatbot.flet.diogenesaibutlerflet/Data/Library/Application Support/com.diogenes.ai.chatbot.flet.diogenesaibutlerflet/flet/app/pypackages/numpy/core/_multiarray_umath.cpython-312-darwin.so' not valid for use in process: library load disallowed by system policy), '/System/Volumes/Preboot/Cryptexes/OS/Users/simon/Library/Containers/com.diogenes.ai.chatbot.flet.diogenesaibutlerflet/Data/Library/Application Support/com.diogenes.ai.chatbot.flet.diogenesaibutlerflet/flet/app/pypackages/numpy/core/_multiarray_umath.cpython-312-darwin.so' (no such file), '/Users/simon/Library/Containers/com.diogenes.ai.chatbot.flet.diogenesaibutlerflet/Data/Library/Application Support/com.diogenes.ai.chatbot.flet.diogenesaibutlerflet/flet/app/pypackages/numpy/core/_multiarray_umath.cpython-312-darwin.so' (code signature in <4B244C37-40C5-380A-9FCF-3FE515E61AAA> '/Users/simon/Library/Containers/com.diogenes.ai.chatbot.flet.diogenesaibutlerflet/Data/Library/Application Support/com.diogenes.ai.chatbot.flet.diogenesaibutlerflet/flet/app/pypackages/numpy/core/_multiarray_umath.cpython-312-darwin.so' not valid for use in process: library load disallowed by system policy)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "", line 47, in
File "", line 229, in run_module
File "", line 88, in _run_code
File "/Users/simon/Library/Containers/com.diogenes.ai.chatbot.flet.diogenesaibutlerflet/Data/Library/Application Support/com.diogenes.ai.chatbot.flet.diogenesaibutlerflet/flet/app/main.py", line 8, in
from nexa.gguf import NexaTextInference
File "/Users/simon/Library/Containers/com.diogenes.ai.chatbot.flet.diogenesaibutlerflet/Data/Library/Application Support/com.diogenes.ai.chatbot.flet.diogenesaibutlerflet/flet/app/pypackages/nexa/gguf/init.py", line 3, in
from .nexa_inference_vlm import NexaVLMInference
File "/Users/simon/Library/Containers/com.diogenes.ai.chatbot.flet.diogenesaibutlerflet/Data/Library/Application Support/com.diogenes.ai.chatbot.flet.diogenesaibutlerflet/flet/app/pypackages/nexa/gguf/nexa_inference_vlm.py", line 22, in
from nexa.gguf.llama.llama_chat_format import (
File "/Users/simon/Library/Containers/com.diogenes.ai.chatbot.flet.diogenesaibutlerflet/Data/Library/Application Support/com.diogenes.ai.chatbot.flet.diogenesaibutlerflet/flet/app/pypackages/nexa/gguf/llama/llama_chat_format.py", line 28, in
import numpy as np
File "/Users/simon/Library/Containers/com.diogenes.ai.chatbot.flet.diogenesaibutlerflet/Data/Library/Application Support/com.diogenes.ai.chatbot.flet.diogenesaibutlerflet/flet/app/pypackages/numpy/init.py", line 135, in
raise ImportError(msg) from e
ImportError: Error importing numpy: you should not try to import numpy from
its source directory; please exit the numpy source tree, and relaunch
your python interpreter from there.

[dsx1986]

Hi @FeodorFitsner , can you add openblas library to flet for numpy?

=======
Traceback (most recent call last):
File "/Users/simon/Library/Containers/com.diogenes.ai.chatbot.flet.diogenes-ai-butler/Data/Library/Application Support/com.diogenes.ai.chatbot.flet.diogenes-ai-butler/flet/app/pypackages/numpy/core/init.py", line 24, in
from . import multiarray
File "/Users/simon/Library/Containers/com.diogenes.ai.chatbot.flet.diogenes-ai-butler/Data/Library/Application Support/com.diogenes.ai.chatbot.flet.diogenes-ai-butler/flet/app/pypackages/numpy/core/multiarray.py", line 10, in
from . import overrides
File "/Users/simon/Library/Containers/com.diogenes.ai.chatbot.flet.diogenes-ai-butler/Data/Library/Application Support/com.diogenes.ai.chatbot.flet.diogenes-ai-butler/flet/app/pypackages/numpy/core/overrides.py", line 8, in
from numpy.core.multiarray_umath import (
ImportError: dlopen(/Users/simon/Library/Containers/com.diogenes.ai.chatbot.flet.diogenes-ai-butler/Data/Library/Application Support/com.diogenes.ai.chatbot.flet.diogenes-ai-butler/flet/app/pypackages/numpy/core/multiarray_umath.cpython-312-darwin.so, 0x0002): Library not loaded: @loader_path/../.dylibs/libopenblas64.0.dylib
Referenced from: <49126372-B133-329F-84A1-517025CF3C59> /Users/simon/Library/Containers/com.diogenes.ai.chatbot.flet.diogenes-ai-butler/Data/Library/Application Support/com.diogenes.ai.chatbot.flet.diogenes-ai-butler/flet/app/pypackages/numpy/core/multiarray_umath.cpython-312-darwin.so
Reason: tried: '/Users/simon/Library/Containers/com.diogenes.ai.chatbot.flet.diogenes-ai-butler/Data/Library/Application Support/com.diogenes.ai.chatbot.flet.diogenes-ai-butler/flet/app/pypackages/numpy/core/../.dylibs/libopenblas64.0.dylib' (mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64h' or 'x86_64')), '/usr/local/lib/libopenblas64.0.dylib' (no such file), '/usr/lib/libopenblas64_.0.dylib' (no such file, not in dyld cache)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/Users/simon/Library/Containers/com.diogenes.ai.chatbot.flet.diogenes-ai-butler/Data/Library/Application Support/com.diogenes.ai.chatbot.flet.diogenes-ai-butler/flet/app/pypackages/numpy/init.py", line 130, in
from numpy.config import show as show_config
File "/Users/simon/Library/Containers/com.diogenes.ai.chatbot.flet.diogenes-ai-butler/Data/Library/Application Support/com.diogenes.ai.chatbot.flet.diogenes-ai-butler/flet/app/pypackages/numpy/config.py", line 4, in
from numpy.core._multiarray_umath import (
File "/Users/simon/Library/Containers/com.diogenes.ai.chatbot.flet.diogenes-ai-butler/Data/Library/Application Support/com.diogenes.ai.chatbot.flet.diogenes-ai-butler/flet/app/pypackages/numpy/core/init.py", line 50, in
raise ImportError(msg)
ImportError:

IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.

We have compiled some common reasons and troubleshooting tips at:

https://numpy.org/devdocs/user/troubleshooting-importerror.html

Please note and check the following:

• The Python version is: Python3.12 from "/Users/simon/Documents/git/diogenesaibutlerflet/build/flutter/build/macos/Build/Products/Release/DiogenesAIButler.app/Contents/MacOS/DiogenesAIButler"
• The NumPy version is: "1.26.4"

and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.

Original error was: dlopen(/Users/simon/Library/Containers/com.diogenes.ai.chatbot.flet.diogenes-ai-butler/Data/Library/Application Support/com.diogenes.ai.chatbot.flet.diogenes-ai-butler/flet/app/pypackages/numpy/core/multiarray_umath.cpython-312-darwin.so, 0x0002): Library not loaded: @loader_path/../.dylibs/libopenblas64.0.dylib
Referenced from: <49126372-B133-329F-84A1-517025CF3C59> /Users/simon/Library/Containers/com.diogenes.ai.chatbot.flet.diogenes-ai-butler/Data/Library/Application Support/com.diogenes.ai.chatbot.flet.diogenes-ai-butler/flet/app/pypackages/numpy/core/multiarray_umath.cpython-312-darwin.so
Reason: tried: '/Users/simon/Library/Containers/com.diogenes.ai.chatbot.flet.diogenes-ai-butler/Data/Library/Application Support/com.diogenes.ai.chatbot.flet.diogenes-ai-butler/flet/app/pypackages/numpy/core/../.dylibs/libopenblas64.0.dylib' (mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64h' or 'x86_64')), '/usr/local/lib/libopenblas64_.0.dylib' (no such file), '/usr/lib/libopenblas64_.0.dylib' (no such file, not in dyld cache)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "", line 95, in
File "", line 229, in run_module
File "", line 88, in _run_code
File "/Users/simon/Library/Containers/com.diogenes.ai.chatbot.flet.diogenes-ai-butler/Data/Library/Application Support/com.diogenes.ai.chatbot.flet.diogenes-ai-butler/flet/app/main.py", line 6, in
from pages import (
File "/Users/simon/Library/Containers/com.diogenes.ai.chatbot.flet.diogenes-ai-butler/Data/Library/Application Support/com.diogenes.ai.chatbot.flet.diogenes-ai-butler/flet/app/pages/local_chat.py", line 4, in
from nexa.gguf import NexaTextInference
File "/Users/simon/Library/Containers/com.diogenes.ai.chatbot.flet.diogenes-ai-butler/Data/Library/Application Support/com.diogenes.ai.chatbot.flet.diogenes-ai-butler/flet/app/pypackages/nexa/gguf/init.py", line 3, in
from .nexa_inference_vlm import NexaVLMInference
File "/Users/simon/Library/Containers/com.diogenes.ai.chatbot.flet.diogenes-ai-butler/Data/Library/Application Support/com.diogenes.ai.chatbot.flet.diogenes-ai-butler/flet/app/pypackages/nexa/gguf/nexa_inference_vlm.py", line 22, in
from nexa.gguf.llama.llama_chat_format import (
File "/Users/simon/Library/Containers/com.diogenes.ai.chatbot.flet.diogenes-ai-butler/Data/Library/Application Support/com.diogenes.ai.chatbot.flet.diogenes-ai-butler/flet/app/pypackages/nexa/gguf/llama/llama_chat_format.py", line 28, in
import numpy as np
File "/Users/simon/Library/Containers/com.diogenes.ai.chatbot.flet.diogenes-ai-butler/Data/Library/Application Support/com.diogenes.ai.chatbot.flet.diogenes-ai-butler/flet/app/pypackages/numpy/init.py", line 135, in
raise ImportError(msg) from e
ImportError: Error importing numpy: you should not try to import numpy from
its source directory; please exit the numpy source tree, and relaunch
your python interpreter from there.

[FeodorFitsner]

@dsx1986 what dependencies do you have and what build command did you use?

[dsx1986]

I have langchain==0.3.14 -> numpy==1.26.4 and numpy needs openblas. libopenblas64_.0.dylib needs libgfortran.5.dylib

this is the command: flet build macos --info-plist LSApplicationCategoryType="public.app-category.productivity" --macos-entitlements com.apple.security.app-sandbox=True --build-number $(date +%y%m%d%H%M) -v

sandbox is preventing to reference from /usr/local/lib

and SIP forbide /usr/lib/