Building documentation is broken in Sage 9.6.beta6

[tobiasdiez]

Building the documentation is currently broken and fails with the following error messages:

$ make doc-clean; make doc-uninstall; make

...
[sagemath_doc_html-none] [semirings] building [html]: targets for 3 source files that are out of date
[sagemath_doc_html-none] [semirings] dumping search index in English (code: en)... done
[sagemath_doc_html-none] [semirings] The HTML pages are in ../../home/gitpod/sage-local/share/doc/sage/html/en/reference/semirings.
[sagemath_doc_html-none] [semirings] Extension error (matplotlib.sphinxext.plot_directive):
[sagemath_doc_html-none] [semirings] Handler <function _copy_css_file at 0x7f0673973e50> for event 'build-finished' threw an exception (exception: [Errno 17] File exists: '/home/gitpod/sage-local/share/doc/sage/html/en/reference/semirings/_static')
[sagemath_doc_html-none] Build finished. The built documents can be found in /home/gitpod/sage-local/share/doc/sage/html/en/reference/semirings
...

CC: @kwankyu @strogdon @haraldschilly @mkoeppe

Component: documentation

Author: Kwankyu Lee

Branch: 0618873

Reviewer: Tobias Diez

Issue created by migration from https://trac.sagemath.org/ticket/33608

[kwankyu]

comment:1

This is not specific to gitpod. I am experiencing it too. It seem to have been introduced in beta6.

I guess that the following happens. plot_directive extension writes its own css files into _static directory for each sphinx project. In our case, _static is a symlink to ../static directory, which is shared with all projects. Hence when plot_directive attempts to write again, _static (or ../static) directory already has the files, which triggers "File exists" error.

What I don't understand why this mechanism was no problem before, but now it is...

[strogdon]

comment:2

Are

$ ./sage --docbuild tutorial html

and

$ ./sage --docbuild reference html

supposed to fail after a normal sagelib-9.6.beta6 build without doing anything else? If so, I don't see this here. Everything works.

[strogdon]

comment:3

I must add that I have the html docs installed. Does one have to do a make doc-clean doc-uninstall first?

[kwankyu]

comment:4

Replying to @strogdon:

I must add that I have the html docs installed. Does one have to do a make doc-clean doc-uninstall first?

Yes. This is also strange...

[kwankyu]

comment:5

In beta6, python3 was upgraded:

26574b0f82 Trac #33512: Update python3 to 3.10.3

If you build sage with python 3.10.3, there are no more those errors. So this might be some incompatibility in python system library and packages...

[kwankyu]

comment:6

Replying to @kwankyu:

In beta6, python3 was upgraded:

26574b0f82 Trac #33512: Update python3 to 3.10.3

If you build sage with python 3.10.3, there are no more those errors. So this might be some incompatibility in python system library and packages...

This may be wrong. In another build, I see those errors back even with python 3.10.3. I am utterly confused.

[kwankyu]

Author: Kwankyu Lee

[kwankyu]

Branch: u/klee/33608

[kwankyu]

comment:9

Symlinks pointing to a non-existing directory broke documentation building. These symlinks were no problem before. But some obscure changes in Sage 9.6.beta6 (possibly merge of python 3.10.3) now makes them problematic.

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

0618873

Ensure symlink points to an existing directory

[sagetrac-git]

Commit: 0618873

[kwankyu]

New commits:

0618873

Ensure symlink points to an existing directory

[strogdon]

comment:13

I don't want to get off topic, but since we're talking about symlinks, without this branch a fresh build of the html docs gives errors like

[reference] Handler <function _copy_css_file at 0x7f6177a57880> for event 'build-finished' threw an exception (exception: [Errno 17] File exists: '/local/sage-git/sage/local/share/doc/sage/html/en/reference/references/_static')

There are 156 of these errors and the referenced _static files are symlinks to an actual _static folder, i.e.

/local/sage-git/sage/local/share/doc/sage/html/en/reference/references/_static -> ../_static

[tobiasdiez]

comment:14

With this branch (on gitpod), I now get somewhat similar errors as reported by @strogdon (but for the inventories and not static files):

[tensor_fr] WARNING: failed to reach any of the inventories with the following issues:
[tensor_fr] intersphinx inventory '/home/gitpod/sage-local/share/doc/sage/inventory/en/reference/objects.inv' not fetchable due to <class 'FileNotFoundError'>: [Errno 2] No such file or directory: '/home/gitpod/sage-local/share/doc/sage/inventory/en/reference/objects.inv'
[tensor_fr] WARNING: failed to reach any of the inventories with the following issues:
[tensor_fr] intersphinx inventory '/home/gitpod/sage-local/share/doc/sage/inventory/en/reference/references/objects.inv' not fetchable due to <class 'FileNotFoundError'>: [Errno 2] No such file or directory: '/home/gitpod/sage-local/share/doc/sage/inventory/en/reference/references/objects.inv'

Also the issue for building the tutorial still persist.

[strogdon]

comment:15

With this branch after

make doc-clean doc-uninstall

both

./sage --docbuild tutorial html

and

./sage --docbuild reference html

fail as in the Description. Is --docbuild supposed to work out of the box in this way without doing something else?

[strogdon]

comment:16

Not related to this issue but I'm unable to start a new SageMath jupyter notebook. Before opening a new issue I wanted to make sure others are seeing this.

./sage -n jupyter

select New -> [SageMath](../wiki/SageMath) 9.6.beta6 and nothing happens. In the terminal

[I 12:39:27.730 NotebookApp] Creating new notebook in

with nothing else.

[kwankyu]

comment:17

Replying to @strogdon:

With this branch after

make doc-clean doc-uninstall

both

./sage --docbuild tutorial html

and

./sage --docbuild reference html

fail as in the Description. Is --docbuild supposed to work out of the box in this way without doing something else?

No. You must successfully build the docoumentation from scratch first, which means

make doc-clean; make doc-uninstall; make

[kwankyu]

comment:19

Replying to @tobiasdiez:

With this branch (on gitpod), I now get somewhat similar errors as reported by @strogdon (but for the inventories and not static files):

[tensor_fr] WARNING: failed to reach any of the inventories with the following issues:
[tensor_fr] intersphinx inventory '/home/gitpod/sage-local/share/doc/sage/inventory/en/reference/objects.inv' not fetchable due to <class 'FileNotFoundError'>: [Errno 2] No such file or directory: '/home/gitpod/sage-local/share/doc/sage/inventory/en/reference/objects.inv'
[tensor_fr] WARNING: failed to reach any of the inventories with the following issues:
[tensor_fr] intersphinx inventory '/home/gitpod/sage-local/share/doc/sage/inventory/en/reference/references/objects.inv' not fetchable due to <class 'FileNotFoundError'>: [Errno 2] No such file or directory: '/home/gitpod/sage-local/share/doc/sage/inventory/en/reference/references/objects.inv'

Also the issue for building the tutorial still persist.

Is this a documentation building from scratch?

[strogdon]

comment:20

Replying to @kwankyu:

Replying to @strogdon:

With this branch after

make doc-clean doc-uninstall

both

./sage --docbuild tutorial html

and

./sage --docbuild reference html

fail as in the Description. Is --docbuild supposed to work out of the box in this way without doing something else?

No. You must successfully build the docoumentation from scratch first, which means

make doc-clean; make doc-uninstall; make

OK, then I do not have this issue. From the develop branch w/o this ticket and after successfully building the documentation:

$ ./sage --docbuild tutorial html
[tutorial ] Merging environment/index files...
[tutorial ] ... done (0 todos, 23 index, 17 citations, 0 modules)
[tutorial ] no targets are out of date.
[tutorial ] The HTML pages are in local/share/doc/sage/html/en/tutorial.
Build finished. The built documents can be found in /local/sage-git/sage/local/share/doc/sage/html/en/tutorial

and

$ ./sage --docbuild reference html
[reference] no targets are out of date.
[reference] The HTML pages are in local/share/doc/sage/html/en/reference/references.
Build finished. The built documents can be found in /local/sage-git/sage/local/share/doc/sage/html/en/reference/references
[spkg     ] no targets are out of date.
[spkg     ] The HTML pages are in local/share/doc/sage/html/en/reference/spkg.
Build finished. The built documents can be found in /local/sage-git/sage/local/share/doc/sage/html/en/reference/spkg

...

[reference] no targets are out of date.
[reference] The HTML pages are in local/share/doc/sage/html/en/reference.
Build finished. The built documents can be found in /local/sage-git/sage/local/share/doc/sage/html/en/reference

I only see the issue if the documentation is not built first.

[tobiasdiez]

Reviewer: Tobias Diez

[tobiasdiez]

comment:21

I should have tested it with

make doc-clean; make doc-uninstall; make

in the first place, since the first issue I mentioned earlier is fixed by it.

Running this command on develop leads to the issues with _static, and with this branch these issues are fixed. Thus, lets get this in.

[jhpalmieri]

comment:22

If the documentation has not been built, then (for example) ./sage --docbuild tutorial html fails. I think this used to work, but I'm not sure and if it did, I don't know how long ago. Is there a ticket for this issue?

[kwankyu]

comment:23

Replying to @jhpalmieri:

If the documentation has not been built, then (for example) ./sage --docbuild tutorial html fails. I think this used to work, but I'm not sure and if it did, I don't know how long ago. Is there a ticket for this issue?

Perhaps no. It could have worked before. But when I try that now, it fails because the reference manual is not yet built. Hence I am not sure whether building an individual document from source without building the whole documentation is something we do efforts to support.

[vbraun]

Changed branch from u/klee/33608 to 0618873

[collares]

comment:25

Replying to Steven Trogdon:

I don't want to get off topic, but since we're talking about symlinks, without this branch a fresh build of the html docs gives errors like

[reference] Handler <function _copy_css_file at 0x7f6177a57880> for event 'build-finished' threw an exception (exception: [Errno 17] File exists: '/local/sage-git/sage/local/share/doc/sage/html/en/reference/references/_static')

There are 156 of these errors and the referenced _static files are symlinks to an actual _static folder, i.e.

/local/sage-git/sage/local/share/doc/sage/html/en/reference/references/_static -> ../_static

This may be fixed by matplotlib/matplotlib#23805

[collares]

Changed commit from 0618873 to none