Fails without error in CI for certain import statements

[adigitoleo]

Problem Description

My pdoc runner is failing ever since I added the gmsh package as a dependency. Initially I thought I tracked down the failure to an import gmsh statement, however I still observe the behaviour after making gmsh an optional dependency and adding the necessary try/except statements.

Steps to reproduce the behavior:

I can't reproduce it locally, either on ubuntu or Arch Linux.
I can only reproduce in github using the following workflow:

name: GitHub Pages

on:
  push:
    branches:
      - main
  pull_request:
    branches:
      - main
  workflow_dispatch:

jobs:
  deploy:
    runs-on: ubuntu-latest
    permissions:
      contents: write
    concurrency:
      group: ${{ github.workflow }}-${{ github.ref }}
    steps:
      - name: Checkout repository
        uses: actions/checkout@v3

      - name: Setup Python
        uses: actions/setup-python@v4
        with:
          python-version: '3.12'

      - name: Build
        run: |
          sudo apt-get update
          sudo apt-get install -y python3-pip
          python3 -m pip install --upgrade pip
          python3 -m venv .venv-gh-pages
          source .venv-gh-pages/bin/activate
          python3 -m pip install "$PWD"[doc,test]
          pdoc -t docs/template -o html --math pydrex tests
      - name: Deploy
        uses: peaceiris/actions-gh-pages@v3
        # If you're changing the branch from main,
        # also change the `main` in `refs/heads/main`
        # below accordingly.
        if: ${{ github.ref == 'refs/heads/main' }}
        with:
          github_token: ${{ secrets.GITHUB_TOKEN }}
          publish_dir: ./html

System Information

It is installed via pip in the workflow, so that should get the latest version.

This is one of the log examples:

Click to expand

Run sudo apt-get update
17
Get:1 file:/etc/apt/apt-mirrors.txt Mirrorlist [142 B]
18
Hit:2 http://azure.archive.ubuntu.com/ubuntu jammy InRelease
19
Get:3 http://azure.archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB]
20
Get:4 http://azure.archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB]
21
Get:5 http://azure.archive.ubuntu.com/ubuntu jammy-security InRelease [110 kB]
22
Get:6 https://packages.microsoft.com/ubuntu/22.04/prod jammy InRelease [3632 B]
23
Get:7 http://azure.archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1502 kB]
24
Get:8 http://azure.archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [289 kB]
25
Get:9 http://azure.archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1619 kB]
26
Get:10 http://azure.archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [271 kB]
27
Get:11 http://azure.archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1059 kB]
28
Get:12 http://azure.archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [240 kB]
29
Hit:13 https://ppa.launchpadcontent.net/ubuntu-toolchain-r/test/ubuntu jammy InRelease
30
Get:14 http://azure.archive.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1286 kB]
31
Get:15 http://azure.archive.ubuntu.com/ubuntu jammy-security/main Translation-en [229 kB]
32
Get:16 http://azure.archive.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1591 kB]
33
Get:17 http://azure.archive.ubuntu.com/ubuntu jammy-security/restricted Translation-en [267 kB]
34
Get:18 http://azure.archive.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [851 kB]
35
Get:19 http://azure.archive.ubuntu.com/ubuntu jammy-security/universe Translation-en [162 kB]
36
Get:20 https://packages.microsoft.com/ubuntu/22.04/prod jammy/main amd64 Packages [136 kB]
37
Get:21 https://packages.microsoft.com/ubuntu/22.04/prod jammy/main armhf Packages [12.2 kB]
38
Get:22 https://packages.microsoft.com/ubuntu/22.04/prod jammy/main arm64 Packages [34.5 kB]
39
Fetched 9891 kB in 1s (7386 kB/s)
40
Reading package lists...
41
Reading package lists...
42
Building dependency tree...
43
Reading state information...
44
python3-pip is already the newest version (22.0.2+dfsg-1ubuntu0.4).
45
0 upgraded, 0 newly installed, 0 to remove and 27 not upgraded.
46
Requirement already satisfied: pip in /opt/hostedtoolcache/Python/3.12.2/x64/lib/python3.12/site-packages (24.0)
47
Processing /home/runner/work/PyDRex/PyDRex
48
  Installing build dependencies: started
49
  Installing build dependencies: finished with status 'done'
50
  Getting requirements to build wheel: started
51
  Getting requirements to build wheel: finished with status 'done'
52
  Installing backend dependencies: started
53
  Installing backend dependencies: finished with status 'done'
54
  Preparing metadata (pyproject.toml): started
55
  Preparing metadata (pyproject.toml): finished with status 'done'
56
Collecting numpy>=1.8 (from pydrex==0.1.dev1+g7d97d5e)
57
  Downloading numpy-1.26.4-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (61 kB)
58
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.0/61.0 kB 3.8 MB/s eta 0:00:00
59
Collecting numba>=0.57 (from pydrex==0.1.dev1+g7d97d5e)
60
  Downloading numba-0.59.1-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (2.7 kB)
61
Collecting scipy>=1.2 (from pydrex==0.1.dev1+g7d97d5e)
62
  Downloading scipy-1.12.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (60 kB)
63
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 60.4/60.4 kB 11.4 MB/s eta 0:00:00
64
Collecting pyyaml (from pydrex==0.1.dev1+g7d97d5e)
65
  Downloading PyYAML-6.0.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (2.1 kB)
66
Collecting meshio (from pydrex==0.1.dev1+g7d97d5e)
67
  Downloading meshio-5.3.5-py3-none-any.whl.metadata (11 kB)
68
Collecting cmcrameri (from pydrex==0.1.dev1+g7d97d5e)
69
  Downloading cmcrameri-1.8-py3-none-any.whl.metadata (4.8 kB)
70
Collecting pdoc (from pydrex==0.1.dev1+g7d97d5e)
71
  Downloading pdoc-14.4.0-py3-none-any.whl.metadata (7.5 kB)
72
Collecting pytest (from pydrex==0.1.dev1+g7d97d5e)
73
  Downloading pytest-8.1.1-py3-none-any.whl.metadata (7.6 kB)
74
Collecting matplotlib (from pydrex==0.1.dev1+g7d97d5e)
75
  Downloading matplotlib-3.8.3-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (5.8 kB)
76
Collecting llvmlite<0.43,>=0.42.0dev0 (from numba>=0.57->pydrex==0.1.dev1+g7d97d5e)
77
  Downloading llvmlite-0.42.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.8 kB)
78
Collecting packaging (from cmcrameri->pydrex==0.1.dev1+g7d97d5e)
79
  Using cached packaging-24.0-py3-none-any.whl.metadata (3.2 kB)
80
Collecting contourpy>=1.0.1 (from matplotlib->pydrex==0.1.dev1+g7d97d5e)
81
  Downloading contourpy-1.2.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (5.8 kB)
82
Collecting cycler>=0.10 (from matplotlib->pydrex==0.1.dev1+g7d97d5e)
83
  Downloading cycler-0.12.1-py3-none-any.whl.metadata (3.8 kB)
84
Collecting fonttools>=4.22.0 (from matplotlib->pydrex==0.1.dev1+g7d97d5e)
85
  Downloading fonttools-4.50.0-cp312-cp312-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (159 kB)
86
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 159.4/159.4 kB 23.3 MB/s eta 0:00:00
87
Collecting kiwisolver>=1.3.1 (from matplotlib->pydrex==0.1.dev1+g7d97d5e)
88
  Downloading kiwisolver-1.4.5-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.4 kB)
89
Collecting pillow>=8 (from matplotlib->pydrex==0.1.dev1+g7d97d5e)
90
  Downloading pillow-10.2.0-cp312-cp312-manylinux_2_28_x86_64.whl.metadata (9.7 kB)
91
Collecting pyparsing>=2.3.1 (from matplotlib->pydrex==0.1.dev1+g7d97d5e)
92
  Downloading pyparsing-3.1.2-py3-none-any.whl.metadata (5.1 kB)
93
Collecting python-dateutil>=2.7 (from matplotlib->pydrex==0.1.dev1+g7d97d5e)
94
  Downloading python_dateutil-2.9.0.post0-py2.py3-none-any.whl.metadata (8.4 kB)
95
Collecting rich (from meshio->pydrex==0.1.dev1+g7d97d5e)
96
  Downloading rich-13.7.1-py3-none-any.whl.metadata (18 kB)
97
Collecting Jinja2>=2.11.0 (from pdoc->pydrex==0.1.dev1+g7d97d5e)
98
  Downloading Jinja2-3.1.3-py3-none-any.whl.metadata (3.3 kB)
99
Collecting pygments>=2.12.0 (from pdoc->pydrex==0.1.dev1+g7d97d5e)
100
  Downloading pygments-2.17.2-py3-none-any.whl.metadata (2.6 kB)
101
Collecting MarkupSafe (from pdoc->pydrex==0.1.dev1+g7d97d5e)
102
  Downloading MarkupSafe-2.1.5-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.0 kB)
103
Collecting iniconfig (from pytest->pydrex==0.1.dev1+g7d97d5e)
104
  Downloading iniconfig-2.0.0-py3-none-any.whl.metadata (2.6 kB)
105
Collecting pluggy<2.0,>=1.4 (from pytest->pydrex==0.1.dev1+g7d97d5e)
106
  Downloading pluggy-1.4.0-py3-none-any.whl.metadata (4.3 kB)
107
Collecting six>=1.5 (from python-dateutil>=2.7->matplotlib->pydrex==0.1.dev1+g7d97d5e)
108
  Downloading six-1.16.0-py2.py3-none-any.whl.metadata (1.8 kB)
109
Collecting markdown-it-py>=2.2.0 (from rich->meshio->pydrex==0.1.dev1+g7d97d5e)
110
  Downloading markdown_it_py-3.0.0-py3-none-any.whl.metadata (6.9 kB)
111
Collecting mdurl~=0.1 (from markdown-it-py>=2.2.0->rich->meshio->pydrex==0.1.dev1+g7d97d5e)
112
  Downloading mdurl-0.1.2-py3-none-any.whl.metadata (1.6 kB)
113
Downloading numba-0.59.1-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (3.7 MB)
114
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.7/3.7 MB 113.8 MB/s eta 0:00:00
115
Downloading numpy-1.26.4-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (18.0 MB)
116
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 18.0/18.0 MB 107.5 MB/s eta 0:00:00
117
Downloading scipy-1.12.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (37.8 MB)
118
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 37.8/37.8 MB 50.3 MB/s eta 0:00:00
119
Downloading cmcrameri-1.8-py3-none-any.whl (277 kB)
120
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 277.4/277.4 kB 63.6 MB/s eta 0:00:00
121
Downloading matplotlib-3.8.3-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.6 MB)
122
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 11.6/11.6 MB 82.2 MB/s eta 0:00:00
123
Downloading meshio-5.3.5-py3-none-any.whl (166 kB)
124
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 166.2/166.2 kB 47.4 MB/s eta 0:00:00
125
Downloading pdoc-14.4.0-py3-none-any.whl (133 kB)
126
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 133.9/133.9 kB 37.3 MB/s eta 0:00:00
127
Downloading pytest-8.1.1-py3-none-any.whl (337 kB)
128
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 337.4/337.4 kB 41.3 MB/s eta 0:00:00
129
Downloading PyYAML-6.0.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (724 kB)
130
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 725.0/725.0 kB 65.8 MB/s eta 0:00:00
131
Downloading contourpy-1.2.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (303 kB)
132
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 303.9/303.9 kB 54.8 MB/s eta 0:00:00
133
Downloading cycler-0.12.1-py3-none-any.whl (8.3 kB)
134
Downloading fonttools-4.50.0-cp312-cp312-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.9 MB)
135
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.9/4.9 MB 112.7 MB/s eta 0:00:00
136
Downloading Jinja2-3.1.3-py3-none-any.whl (133 kB)
137
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 133.2/133.2 kB 31.0 MB/s eta 0:00:00
138
Downloading kiwisolver-1.4.5-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.5 MB)
139
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.5/1.5 MB 109.1 MB/s eta 0:00:00
140
Downloading llvmlite-0.42.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (43.8 MB)
141
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 43.8/43.8 MB 29.9 MB/s eta 0:00:00
142
Downloading MarkupSafe-2.1.5-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (28 kB)
143
Using cached packaging-24.0-py3-none-any.whl (53 kB)
144
Downloading pillow-10.2.0-cp312-cp312-manylinux_2_28_x86_64.whl (4.5 MB)
145
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.5/4.5 MB 71.5 MB/s eta 0:00:00
146
Downloading pluggy-1.4.0-py3-none-any.whl (20 kB)
147
Downloading pygments-2.17.2-py3-none-any.whl (1.2 MB)
148
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 60.8 MB/s eta 0:00:00
149
Downloading pyparsing-3.1.2-py3-none-any.whl (103 kB)
150
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 103.2/103.2 kB 25.9 MB/s eta 0:00:00
151
Downloading python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB)
152
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 229.9/229.9 kB 36.5 MB/s eta 0:00:00
153
Downloading iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
154
Downloading rich-13.7.1-py3-none-any.whl (240 kB)
155
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 240.7/240.7 kB 38.7 MB/s eta 0:00:00
156
Downloading markdown_it_py-3.0.0-py3-none-any.whl (87 kB)
157
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 87.5/87.5 kB 19.4 MB/s eta 0:00:00
158
Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
159
Downloading mdurl-0.1.2-py3-none-any.whl (10.0 kB)
160
Building wheels for collected packages: pydrex
161
  Building wheel for pydrex (pyproject.toml): started
162
  Building wheel for pydrex (pyproject.toml): finished with status 'done'
163
  Created wheel for pydrex: filename=pydrex-0.1.dev1+g7d97d5e-py3-none-any.whl size=338174 sha256=731b12a15b3438755d37339f4653dbd21e6aa88e5776158de9f1ca045b3da941
164
  Stored in directory: /tmp/pip-ephem-wheel-cache-i8p6akr0/wheels/10/e2/9b/83dc9a40653e818ab608edc1d4276de93ea79b4ec7b5c4e654
165
Successfully built pydrex
166
Installing collected packages: six, pyyaml, pyparsing, pygments, pluggy, pillow, packaging, numpy, mdurl, MarkupSafe, llvmlite, kiwisolver, iniconfig, fonttools, cycler, scipy, python-dateutil, pytest, numba, markdown-it-py, Jinja2, contourpy, rich, pdoc, matplotlib, meshio, cmcrameri, pydrex
167
Successfully installed Jinja2-3.1.3 MarkupSafe-2.1.5 cmcrameri-1.8 contourpy-1.2.0 cycler-0.12.1 fonttools-4.50.0 iniconfig-2.0.0 kiwisolver-1.4.5 llvmlite-0.42.0 markdown-it-py-3.0.0 matplotlib-3.8.3 mdurl-0.1.2 meshio-5.3.5 numba-0.59.1 numpy-1.26.4 packaging-24.0 pdoc-14.4.0 pillow-10.2.0 pluggy-1.4.0 pydrex-0.1.dev1+g7d97d5e pygments-2.17.2 pyparsing-3.1.2 pytest-8.1.1 python-dateutil-2.9.0.post0 pyyaml-6.0.1 rich-13.7.1 scipy-1.12.0 six-1.16.0
168
Warn: Suppressed execution of ['fc-list', '--help'] during import. Set PDOC_ALLOW_EXEC=1 as an environment variable to allow subprocess execution. (/opt/hostedtoolcache/Python/3.12.2/x64/lib/python3.12/subprocess.py:548)
169
Warn: Suppressed execution of 'lscpu' during import. Set PDOC_ALLOW_EXEC=1 as an environment variable to allow subprocess execution. (/opt/hostedtoolcache/Python/3.12.2/x64/lib/python3.12/subprocess.py:548)
170
Error: Process completed with exit code 1.

Initially it may look like it is caused by the subprocess issue discussed in #450 and I indeed thought so at first, however adding the flag as per the warning message doesn't help.

I wonder if anyone has a chance to try and repro on an ubuntu machine or if there are any further ideas about what could cause this. Thanks

[adigitoleo]

My current workaround was manual exclusion of the offending submodule.

[mhils]

This doesn't look like a pdoc bug, this looks like gmsh or one of its subdependencies is killing/crashing the process on import (for whatever reason).

[adigitoleo]

Have now reported this to the gmsh devs as well.

[adigitoleo]

In this particular case, I was in fact simply not installing the gmsh optional dep before running pdoc. (I've sinced tried again with installing gmsh in the yaml file, but there was in fact no change).

Either way, this can probably be closed as I'm happy with using !module.submodule exclude rules to cut out the offending submodules. The only thing I might suggest is giving some kind of warning if pdoc encounters an ImportError or missing dependency, rather than generic failure with "exit code 1".

[mhils]

Thanks for reporting back!