Executable not found in $PATH

[buehler]

When using the docker action of pandoc the following error occurs:

      - run: |
          mkdir public
          echo "FILELIST=$(printf '"%s" ' src/sections/*.md)" >> $GITHUB_ENV
      - name: Build Report to HTML
        uses: docker://pandoc/latex:2.11.4
        with:
          args: >-
            pandoc
            --filter pandoc-xnos
            --metadata-file=src/metadata.yaml
            --citeproc
            --bibliography=src/bibliography.bib
            --toc
            --standalone
            --output=public/index.html
            ${{ env.FILELIST }}

←[34m[Publish GitHub Pages/build-gh-pages ] ←[0m  ❌  Failure - Build Report to HTML
Error: Error response from daemon: OCI runtime create failed: container_linux.go:370: starting container process caused: exec: "--filter": executable file not found in $PATH: unknown

The problem seems to be that the ENTRYPOINT is not set anymore?

(note that pandoc-xnos is installed via PIP)

[buehler]

Now I ruled out that it is not the docker image. with act, the image does not run. Online the issue is, that the pandoc-xnos is not found nontheless.

[alerque]

This isn't an issue with the Action setup. The Docker image you are using doesn't have that filter baked in anywhere.

Have you seen building custom images? I think what you're going to need is a small shim Dockerfile in your projcet repository that loads the Pandoc image you want to use then adds the pandoc-xnos filter that you would like to use to the image in an appropriate place. Then you can use that in your GitHub Action workflow.

It would probably be a good idea to include an example of doing that properly the documentation for this repository.

[buehler]

That's a nice idea! I'm trying it out and if it works, I'm going to open a PR to add the example here.

[alerque]

Another solution that would work would be to use a container arrangement like in #11, but add a step that installs your filters to the container before using Pandoc. I don't think that's quite as well suited to solve this particular issue, but maybe worth mentioning in case other people with similar but not identical issues come by.

[buehler]

To be fair, what I already tried was:

jobs:
  build-gh-pages:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: actions/setup-python@v2
        with:
          python-version: '3.x'
      - name: Install Python Dependencies
        run: |
          python -m pip install --upgrade pip
          pip install pandoc-fignos pandoc-tablenos pandoc-secnos
      - run: |
          mkdir public
          echo "FILELIST=$(printf '"%s" ' src/sections/*.md)" >> $GITHUB_ENV
      - name: Build Report to HTML
        uses: docker://pandoc/latex:2.11.4
        with:
          args: >-
            --filter pandoc-xnos
            --metadata-file=src/metadata.yaml
            --citeproc
            --bibliography=src/bibliography.bib
            --toc
            --standalone
            --output=public/index.html
            ${{ env.FILELIST }}

So I basically installed python, pip and the executables. But it seems that they are not transferred into the docker execution.

[alerque]

What that effectively did was install Pip and your desired filters inside the ubuntu-latest container. Then you started up a Docker container inside Ubuntu, and no it does not have access to its host environment, just the current working directory files.

My second suggestion (re #11) was to use the container: instead of uses: method of calling the Pandoc Docker image. After that your uses: steps will actually run inside the Pandoc Docker image. This gives you great flexibility, but not that not all other actions will behave well (because they expect to be running a huge bloated Ubuntu image supplied by GitHub, not a slimmed down Alpine container with just Pandoc on hand). You would probably need to use a different way of installing the dependencies you wanted to use.

[buehler]

Yea that was what I imagined.. I'm going to try the customized Dockerfile approach

[alerque]

Just to save you a little time getting started chasing dependencies, you'll need something like this to get rolling with compiling pip stuff in Alpine Linux (which is what the Pandoc Docker image is based on):

FROM pandoc/latex:2.11.4

ENV PYTHONUNBUFFERED=1

RUN apk add --update --no-cache python3 python3-dev py3-pip py3-setuptools gcc musl-dev linux-headers
RUN pip install --no-cache --upgrade pandoc-fignos pandoc-tablenos pandoc-secnos

If this was not for one time use I would do a two stage thing where you build on on image then copy the result into a clean container without all the header baggage, but the use case discussed above in a one-off that needs to be built on every CI run, so this seems like the fastest way to get from point A to point B.

[alerque]

Following up on that, to use it in a project I think you'll want to save that as Dockerfile in the root of your project, then do something like this in your steps (untested):

- name: Build modified Pandoc Docker image
  run: docker build -t my-pandoc
- name: Use modified Pandoc
  run: >-
    docker run my-pandoc
      --filter pandoc-xnos
      --metadata-file=src/metadata.yaml
      --citeproc
      --bibliography=src/bibliography.bib
      --toc
      --standalone
      --output=public/index.html
      ${{ env.FILELIST }}

Obviously you can adjust paths to hide the Dockerfile somewhere or use docker-compose instead, but that's the gist of it.

[buehler]

That's a pretty clever idea. I was just googling on how to provide local docker images to "uses" clauses.

[alerque]

Just for completeness, there is another way to do it to. By including an actions.yml file that defines your project as an Action and configuring the action to use the Dockerfile, you could uses: user/project your own project as its own action. The tricky part about this is you'd have to do some serious shenanigans to get the versioning right so that each commit would build with the tooling matching that commit. Hence why I don't actually recommend this way.

Of course if this is something you use a lot you could make your own action and push the Docker images to either Docker Hub or GitHub Packages and reference it directly. That would save a little build time on each CI run in exchange for having to maintain another project.

[alerque]

I just thought you can probably avoid having a Dockerfile at all for this simple use case by piping the contents directly to the builder:

- name: Build modified Pandoc Docker image
  run: |
    cat <<- EOF | docker build -t my-pandoc
    FROM pandoc/latex:2.11.4
    ENV PYTHONUNBUFFERED=1
    RUN apk add --update --no-cache python3 python3-dev py3-pip py3-setuptools gcc musl-dev linux-headers
    RUN pip install --no-cache --upgrade pandoc-fignos pandoc-tablenos pandoc-secnos
    EOF
- name: Use modified Pandoc
  run: >-
    docker run my-pandoc
      --filter pandoc-xnos
      --metadata-file=src/metadata.yaml
      --citeproc
      --bibliography=src/bibliography.bib
      --toc
      --standalone
      --output=public/index.html
      ${{ env.FILELIST }}

This way you don't clutter up your repo with things that or only useful for CI and CI is contained in one workflow file.

[buehler]

Alright... I tried. I gave up now :)
Problem is, that you cannot build it and correctly use it since github actions call a ****load of arguments with docker run. Now I use the simpler approach, by building the image and publishing it go my github profile and the use the action in another repository. (https://github.com/buehler/mse-pandoc)

Only problem now is:

pandoc-secnos: Wrote the following blocks to header-includes.  If you
use pandoc's --include-in-header option then you will need to manually
include these yourself.

    %% pandoc-secnos: required package
    \usepackage{cleveref}

Error producing PDF.
! LaTeX Error: File `cleveref.sty' not found.

Type X to quit or <RETURN> to proceed,
or enter new name. (Default extension: sty)

Enter file name: 
! Emergency stop.
<read *> 
         
l.55 \ifxetex

😐

[buehler]

But this is now solved via tlmgr install. :-) Thanks for your support!

[alerque]

I appreciate that you've found a solution that works for you (and not a bad one at all), but would you mind if we keep this open until I or somebody hashes out how it can be done? I'm sure you're not the only one with this scenario to deal with, in fact I have a similar case coming up (which is why I had been thinking about ideas already).

[buehler]

Yes of course!

I have an idea.. maybe it would be an option to actually create a valid github action with pandoc baked in. Then one could use input fields to define additional packages to be installed (either latex packages with tlmgr or python ones via pip)

[alerque]

Yes, adding input argument(s) that install relevant packages from various ecosystems (don't forget Lua) as needed is an option. For several reasons I was hoping to not need to go there and that there would be add-hoc ways of doing this that could ultimately be more flexible, but if they aren't very user friendly that might be a reason to overcome some of the challenges with installing additional packages at runtime as a runner option.

[alerque]

For anyone interested or my future self if/when I get around to documenting this, you can run a GitHub action using the current repo as the action like this:

- uses: ./