More doctests fix in sage_docbuild

[kiwifb]

A fix for a couple of doctest went missing from #33141.

sage -t --long --random-seed=22919418159437026337026400262276501942 /usr/lib/python3.10/site-packages/sage_docbuild/__init__.py
**********************************************************************
File "/usr/lib/python3.10/site-packages/sage_docbuild/__init__.py", line 110, in sage_docbuild.builder_helper
Failed example:
    try:  # optional - sagemath_doc_html
        build_many(build_ref_doc, [("docname", "en", "html", {})])
    except Exception as E:
        "Non-exception during docbuild: abort pool operation" in str(E)
Expected:
    True
Got:
    False
**********************************************************************
File "/usr/lib/python3.10/site-packages/sage_docbuild/__init__.py", line 211, in sage_docbuild.DocBuilder._doctrees_dir
Failed example:
    b._doctrees_dir()             # optional - sagemath_doc_html
Exception raised:
    Traceback (most recent call last):
      File "/usr/lib/python3.9/site-packages/sage/doctest/forker.py", line 695, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/usr/lib/python3.9/site-packages/sage/doctest/forker.py", line 1093, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage_docbuild.DocBuilder._doctrees_dir[2]>", line 1, in <module>
        b._doctrees_dir()             # optional - sagemath_doc_html
      File "/usr/lib/python3.9/site-packages/sage_docbuild/__init__.py", line 210, in _doctrees_dir
        os.makedirs(d, exist_ok=True)
      File "/usr/lib/python3.9/os.py", line 215, in makedirs
        makedirs(head, exist_ok=exist_ok)
      File "/usr/lib/python3.9/os.py", line 215, in makedirs
        makedirs(head, exist_ok=exist_ok)
      File "/usr/lib/python3.9/os.py", line 225, in makedirs
        mkdir(name, mode)
    PermissionError: [Errno 13] Permission denied: '/usr/share/doc/sage-doc-9999/doctrees'

Doctesting this file in is made more robust, and distro friendly by allowing and making SAGE_DOC to be a random temporary directory at the start of the doctesting of the file.

To allow SAGE_DOC to be set randomly for doctesting we cannot let it be imported globally in the whole file. Functions that uses it need to import it explicitly.

Depends on #33141

CC: @jhpalmieri

Component: distribution

Author: François Bissey

Branch/Commit: 3569408

Reviewer: John Palmieri

Issue created by migration from https://trac.sagemath.org/ticket/33538

[kiwifb]

New commits:

07545e9	Fix sage_setup doctests
aa971bd	Fix doctest in sage_docbuild/__init__.py. Most of the test rely on location that are writable or the presence of documentation sources.
54bd2f5	Merge branch 'develop' into distro_doctests
6c1dc29	replace optional build tag with sage_spkg
18308fc	get rid of the last optional build tag
e2c9eea	Merge branch 'develop' into distro_doctests
5382264	make doctest in find_extra_files more robust by sorting the result so that the order is stable.
3569408	Make SAGE_DOC random in doctests from sage_docbuild/__init__.py

[kiwifb]

Branch: u/fbissey/trac33538

[kiwifb]

Commit: 3569408

[jhpalmieri]

comment:3

Would it be too clumsy to conditionally import SAGE_DOC at the top of the file if it is not yet defined? That would require the doctests to set the environment variable SAGE_DOC before importing from sage_docbuild, but at the moment there seems to be only one affected doctest (or at least SAGE_DOC is only defined in one testing block), while the current branch requires five new from sage.env import SAGE_DOC statements.

[kiwifb]

comment:4

I tried various approaches. I am not sure about importing conditionally. Would you look at os.getenv? While I only defined SAGE_DOC once, that definition is carried to all subsequent doctests. And it affects the results of at least another doctest apart from the obvious one, the one at line 211 as stated in the description.

Now that I look at it with a bit of distance, there is one thing that could be interesting to try. Define SAGE_DOC in the doctest before the first import of sage_docbuild and see if that sticks.

[jhpalmieri]

comment:5

Having thought about it more, I think it's too fragile to have doctests depend on whether SAGE_DOC is defined before the relevant import statements. Let's leave the branch as it is — I don't have any better ideas for how to fix this.

Tests pass on Sage built from scratch, and I trust that they fix problems, or at least don't cause any new ones, for distros.

[jhpalmieri]

Reviewer: John Palmieri

[kiwifb]

comment:6

Thank you.

[vbraun]

Changed branch from u/fbissey/trac33538 to 3569408