PDF build issue

[JulienPalard]

Since #34 and #31 I have issues building PDFs on docs.python.org, it can easily be reproduced using https://github.com/python/docsbuild-scripts/ as:

./build_docs.py --build-root ./build_root --www-root ./www --log-directory ./logs --group $(id -g) --skip-cache-invalidation --language ja --branch 3.9

(you can easily try other branches by changing the --branch argument)

[JulienPalard]

This looks like to completly block the update of docs.python.org/ja/, as the make invocation fails, the docsbuild-script does not rsync the output.

See #35

[methane]

I do not have any latex experience. What I did is:

# Ubuntu 22.04
$ sudo apt-get install texlive-lang-japanese
$ cd build_root/cpython/Doc
$ make -C /home/methane/work/python/docsbuild-scripts/build_root/cpython/Doc PYTHON=/home/methane/work/python/docsbuild-scripts/build_root/venv-3.11/bin/python SPHINXBUILD=/home/methane/work/python/docsbuild-scripts/build_root/venv-3.11/bin/sphinx-build BLURB=/home/methane/work/python/docsbuild-scripts/build_root/venv-3.11/bin/blurb VENVDIR=/home/methane/work/python/docsbuild-scripts/build_root/venv-3.11 'SPHINXOPTS=-D latex_engine=platex -D latex_elements.inputenc= -D latex_elements.fontenc= -q -D locale_dirs=/home/methane/work/python/docsbuild-scripts/build_root/3.11/locale -D language=ja -D gettext_compact=0' SPHINXERRORHANDLING= autobuild-stable
...
Build finished; the LaTeX files are in build/latex.
Run `make all-pdf' or `make all-ps' in that directory to run these through (pdf)latex.
make[3]: Leaving directory '/home/methane/work/python/docsbuild-scripts/build_root/cpython/Doc'
sed -i 's/makeindex/makeindex -q/' build/latex/Makefile
(cd build/latex; make clean && make all-pdf && make FMT=pdf zip bz2)
make[3]: Entering directory '/home/methane/work/python/docsbuild-scripts/build_root/cpython/Doc/build/latex'
rm -f *.log *.ind *.aux *.toc *.syn *.idx *.out *.ilg *.pla *.ps *.tar *.tar.gz *.tar.bz2 *.tar.xz c-api.pdf distributing.pdf extending.pdf faq.pdf howto-annotations.pdf howto-argparse.pdf howto-clinic.pdf howto-cporting.pdf howto-curses.pdf howto-descriptor.pdf howto-enum.pdf howto-functional.pdf howto-instrumentation.pdf howto-ipaddress.pdf howto-isolating-extensions.pdf howto-logging-cookbook.pdf howto-logging.pdf howto-pyporting.pdf howto-regex.pdf howto-sockets.pdf howto-sorting.pdf howto-unicode.pdf howto-urllib2.pdf installing.pdf library.pdf reference.pdf tutorial.pdf using.pdf whatsnew.pdf c-api.dvi distributing.dvi extending.dvi faq.dvi howto-annotations.dvi howto-argparse.dvi howto-clinic.dvi howto-cporting.dvi howto-curses.dvi howto-descriptor.dvi howto-enum.dvi howto-functional.dvi howto-instrumentation.dvi howto-ipaddress.dvi howto-isolating-extensions.dvi howto-logging-cookbook.dvi howto-logging.dvi howto-pyporting.dvi howto-regex.dvi howto-sockets.dvi howto-sorting.dvi howto-unicode.dvi howto-urllib2.dvi installing.dvi library.dvi reference.dvi tutorial.dvi using.dvi whatsnew.dvi  *.fls *.fdb_latexmk
make[3]: Leaving directory '/home/methane/work/python/docsbuild-scripts/build_root/cpython/Doc/build/latex'
make[3]: Entering directory '/home/methane/work/python/docsbuild-scripts/build_root/cpython/Doc/build/latex'
latexmk -r latexmkjarc -pdfdvi -dvi- -ps-  'c-api.tex'
make[3]: latexmk: No such file or directory
make[3]: *** [Makefile:30: c-api.pdf] Error 127
...
$ sudo apt install latexmk
$ latexmk -r latexmkjarc -pdfdvi -dvi- -ps-  'c-api.tex'
Use of uninitialized value in concatenation (.) or string at (eval 10) line 1.
Use of uninitialized value in concatenation (.) or string at (eval 10) line 2.
Subroutine makeglo redefined at (eval 11) line 20.
Use of uninitialized value in concatenation (.) or string at (eval 11) line 1.
Rc files read:
  /etc/LatexMk
  latexmkrc
  latexmkjarc
Latexmk: This is Latexmk, John Collins, 20 November 2021, version: 4.76.
Latexmk: applying rule 'latex'...
Rule 'latex': File changes, etc:
   Changed files, or newly in use since previous run(s):
      'c-api.tex'
------------
Run number 1 of rule 'latex'
------------
------------
Running 'platex  -kanji=utf8  -recorder  "c-api.tex"'
------------
This is e-pTeX, Version 3.141592653-p3.9.1-210218-2.6 (utf8.euc) (TeX Live 2022/dev/Debian) (preloaded format=platex)
 restricted \write18 enabled.
entering extended mode
(./c-api.tex
pLaTeX2e <2021-11-15> (based on LaTeX2e <2021-11-15> patch level 1)
L3 programming layer <2022-01-21> (./sphinxmanual.cls
Document Class: sphinxmanual 2019/12/01 v2.3.0 Document class (Sphinx manual)
(/usr/share/texlive/texmf-dist/tex/platex/jsclasses/jsbook.cls
Document Class: jsbook 2021/06/28 jsclasses (okumura, texjporg)
(/usr/share/texlive/texmf-dist/tex/platex/jsclasses/jslogo.sty)))

! LaTeX Error: File `cmap.sty' not found.

Type X to quit or <RETURN> to proceed,
or enter new name. (Default extension: sty)

Enter file name: [ENTER]
(/usr/share/texlive/texmf-dist/tex/latex/amsmath/amsmath.sty
For additional information on amsmath, use the `?' option.
(/usr/share/texlive/texmf-dist/tex/latex/amsmath/amstext.sty
(/usr/share/texlive/texmf-dist/tex/latex/amsmath/amsgen.sty))
(/usr/share/texlive/texmf-dist/tex/latex/amsmath/amsbsy.sty)
(/usr/share/texlive/texmf-dist/tex/latex/amsmath/amsopn.sty))
(/usr/share/texlive/texmf-dist/tex/latex/amsfonts/amssymb.sty
(/usr/share/texlive/texmf-dist/tex/latex/amsfonts/amsfonts.sty))

! LaTeX Error: File `tgtermes.sty' not found.

Type X to quit or <RETURN> to proceed,
or enter new name. (Default extension: sty)

Enter file name: [ENTER]

! LaTeX Error: File `tgheros.sty' not found.

Type X to quit or <RETURN> to proceed,
or enter new name. (Default extension: sty)

Enter file name: [ENTER]
(./sphinx.sty (/usr/share/texlive/texmf-dist/tex/generic/ltxcmds/ltxcmds.sty)
(/usr/share/texlive/texmf-dist/tex/latex/graphics/color.sty
(/usr/share/texlive/texmf-dist/tex/latex/graphics-cfg/color.cfg)
(/usr/share/texlive/texmf-dist/tex/latex/graphics-def/dvipdfmx.def)
(/usr/share/texlive/texmf-dist/tex/latex/graphics/dvipsnam.def))
(/usr/share/texlive/texmf-dist/tex/latex/kvoptions/kvoptions.sty
(/usr/share/texlive/texmf-dist/tex/latex/graphics/keyval.sty)
(/usr/share/texlive/texmf-dist/tex/generic/kvsetkeys/kvsetkeys.sty))
(./sphinxoptionshyperref.sty
(/usr/share/texlive/texmf-dist/tex/latex/base/atbegshi-ltx.sty))
(./sphinxoptionsgeometry.sty)
(/usr/share/texlive/texmf-dist/tex/latex/base/textcomp.sty)

! LaTeX Error: File `float.sty' not found.

Type X to quit or <RETURN> to proceed,
or enter new name. (Default extension: sty)

Enter file name: [ENTER]

! LaTeX Error: File `wrapfig.sty' not found.

Type X to quit or <RETURN> to proceed,
or enter new name. (Default extension: sty)

Enter file name: [ENTER]

! LaTeX Error: File `capt-of.sty' not found.
...

I don't think errors I see is what happend on docs.python.org.
LaTeX is too difficult to me. I don't want to master it.
Please don't require PDF until some volunteer help it.

[take6]

I reproduced the error with Docker container based on Ubuntu 22.04. Here is the contents of Dockerfile. Essential part to reproduce the issue is a list of packages installed by apt-get.

FROM ubuntu:22.04

ENV DEBIAN_FRONTEND=noninteractive

WORKDIR /pydoc

RUN apt update \
    && apt-get install -y python3.11 python3-pip python3.11-venv git rsync zip \
    latexmk xindy texinfo \
    texlive-xetex texlive-latex-recommended texlive-fonts-extra texlive-lang-japanese \
    && apt-get clean
RUN python3.11 -m venv /pydoc/pydoc-venv

[take6]

It turned out that U+C4CF is Korean character, 쓏.

http://www.unicode-symbol.com/u/C4CF.html

So, I attempted to use package kotex by editing docsbuild-scripts/build_docs.py. This is quite ad hoc way but it appears to be working.

PLATEX_DEFAULT = (
    "-D latex_engine=platex",
    "-D latex_elements.inputenc=",
    "-D latex_elements.fontenc=",
    r"-D latex_elements.preamble=\\usepackage{kotex}",
)

However, the build on my environment was failed due to another error, "TeX capacity exceeded, sorry [input stack size=5000].", which may be indicating the lack of enough memory. Maybe I can open pull request to docsbuild-scripts repo later so that anyone can try this (awkward) fix.

[methane]

Curiously, there are no "\uC4CF" in Japanese html doc.
I don't know why LaTeX searches this character.

[m-aciek]

For reference, this also has been discussed here: texjporg/platex#84

[take6]

Curiously, there are no "\uC4CF" in Japanese html doc.
I don't know why LaTeX searches this character.

That is true. This is the reason why I call my fix "ad hoc way", which just avoid the error instead of fixing essential problem.

[take6]

Pushed branch.

https://github.com/take6/docsbuild-scripts/tree/fix-japanese-doc-build-error

Could anyone try if it works?

[take6]

Created pull request.

[methane]

I failed to build PDF with this error.

LaTeX Warning: Hyper reference `library/socket:module-socket' on page 4 undefin
ed on input line 167.

! TeX capacity exceeded, sorry [parameter stack size=10000].
\@inmathwarn #1->
                 \ifmmode \@latex@warning {Command \protect #1 invalid in ma...
l.170 ...tml\textgreater{}}{Emscripten Networking}
                                                  ^^M
If you really absolutely need more capacity,
you can ask a wizard to enlarge me.

[atsuoishimoto]

Building 3.9/10 branch(./build_docs.py ... --branch 3.9 or 3.10)

Causes following error. I confirmed the error in c-api and library, but it might happen in other document too.

"Improper discretionary list" error when creating divs

Building 3.11 branch

• Error on dvi->pdf conversion while building files following(Actual Unicode character may vary): howto-unicode.pdf, howto-regex.pdf, whatsnew.pdf

  ! LaTeX Error: Unicode character 顛 (U+C4CF)
  not set up for use with LaTeX

• Error on dvi->pdf conversion while building library.pdf

  library.pdf ! TeX capacity exceeded, sorry [input stack size=5000].