BUG: templates_path in config was overriding value set in extension #566

AakashGfude · 2022-06-01T05:04:39Z

We update the templates_path config variable here https://github.com/executablebooks/sphinx-book-theme/blob/master/src/sphinx_book_theme/__init__.py#L198, to allow sidebars to be found in Jinja templates.

But if we set a value to templates_path in _config.yml, then sphinx overrides any previous value set to it, as can be seen from this function https://github.com/sphinx-doc/sphinx/blob/5.x/sphinx/application.py#L246. Resulting in issues like jupyter-book/jupyter-book#1660.

This PR does the updation of the templates_path in config-inited event instead, which is emitted after the sphinx config.init_values() function.

fixes jupyter-book/jupyter-book#1660

mmcky · 2022-06-01T06:45:15Z

thanks for working on this @AakashGfude. Ping me once the tests are passing and I'll review this for you.

E           sphinx.errors.ThemeError: An error happened in rendering the page index.
E           Reason: TemplateNotFound('sidebar-logo.html')

AakashGfude · 2022-06-07T10:03:40Z

@mmcky the tests are passing now. Have also updated the link to the sphinx-theme-builder filesystem layout.

mmcky

thanks @AakashGfude for fixing this bug. I had come across once as well.

mmcky · 2022-06-08T04:34:12Z

@AakashGfude do you think we need a test case for this?

AakashGfude · 2022-06-09T04:45:06Z

@AakashGfude do you think we need a test case for this?

@mmcky I think that's a good idea. I will create or either modify an existing test case, to test this.

choldgraf · 2022-06-09T06:49:35Z

src/sphinx_book_theme/__init__.py

+def update_general_config(app):
+    theme_dir = get_html_theme_path()
+    # Update templates for sidebar
+    app.env.config.templates_path.append(os.path.join(theme_dir, "components"))


Can you add a comment explaining why this is needed given that we already add the components at the link below?

https://github.com/executablebooks/sphinx-book-theme/pull/566/files#diff-323e2d703544f60dac082d2b47d8b6a2d8e1594eb3b8f49d569f48f768379252R205

Otherwise it will confuse people why we do this twice and not just once...

sure thing @choldgraf, I will add the explanation.

AakashGfude · 2022-06-10T07:47:29Z

@mmcky and @choldgraf this issue is apparently only with jupyter-book builds. As jupyter-book calls https://github.com/executablebooks/jupyter-book/blob/master/jupyter_book/sphinx.py#L7 , which is not in the case of SphinxTestApp. Also, the error is not getting triggered with sphinx-build, the reason for which I am not fully aware.

But, I had to add jupyter-book as a test dependency and create a jb project to test this out. Which does seem a bit too much to test this one thing.

mmcky · 2022-06-10T23:18:21Z

@AakashGfude a good reason to write a test. That is curious.

Also, the error is not getting triggered with sphinx-build, the reason for which I am not fully aware.

Thanks for the update. Could it be that the _templates are being overwritten by jupyter-book cli when it compiles sphinx-config? Or is this condition only met when trying to load the sidebars etc. as you've identified and hence should be fixed here? If you run as a myst-nb / sphinx project the issue doesn't show up.

Note: this is useful when running a project without using jb https://jupyterbook.org/en/stable/reference/cli.html?highlight=sphinx#jupyter-book-config-sphinx

AakashGfude · 2022-06-15T01:41:55Z

Sphinx overrides any previous config value based on confoverrides object in self.config.init_values() https://github.com/sphinx-doc/sphinx/blob/5.x/sphinx/application.py#L246 .

In jupyter-book build the confoverrides object is a copy of the entries in _config.yml https://github.com/executablebooks/jupyter-book/blob/master/jupyter_book/sphinx.py#L120. Which is not the case of sphinx-build, as it is always initialised with an empty object https://github.com/sphinx-doc/sphinx/blob/5.x/sphinx/cmd/build.py#L246 , and only filled with certain values before passed to sphinx App https://github.com/sphinx-doc/sphinx/blob/5.x/sphinx/cmd/build.py#L272 .

So, is it right to have a jb project here for testing @choldgraf @mmcky , as it might be necessary in some future issues as well? But also we can just not test this particular thing explicitly and save having an extra tests folder and 'jupyter-book' in testing dependencies.

mmcky · 2022-06-15T05:20:08Z

@AakashGfude so it doesn't seem clear to me if the fix to this templates path issue should be contained in this repository or the jupyter-book repository (i.e. should we be handling confoverides in a different way ).

I just don't want to introduce circular testing dependences as it can make releases really tricky if this depends on jupyter-book and jupyter-book depends on this. If we feel it is important to test then perhaps we can place a test in jupyter-book to test the _templates issue for sphinx-book-theme? Is this going to be an issue for any new theme we make?

AakashGfude · 2022-06-15T06:32:49Z

@mmcky you are right. I think we should reconsider how we are reading config in jupyter-book.

For instance, sphinx-build passes configuration to the sphinx app, using the confdir variable, which then reads the file https://github.com/sphinx-doc/sphinx/blob/5.x/sphinx/application.py#L202. But jupyter-book passes all configurations using confoverrides variable. Naturally, the variables will get overridden from their default values. Which we don't always want.

AakashGfude · 2022-06-15T06:57:54Z

@AakashGfude so it doesn't seem clear to me if the fix to this templates path issue should be contained in this repository or the jupyter-book repository (i.e. should we be handling confoverides in a different way ).

@mmcky I can think of only two ways we can solve this in jupyter-book. Either remove templates_path from confoverrides in jupyter-book or convert the _config.yml file to conf.py file and saves it in the confdir 😳. The first is probably more sensible for now.

choldgraf · 2022-07-05T19:25:20Z

@AakashGfude where are we on this one? Are you blocked on anything? Do we need to resolve questions before moving forward?

AakashGfude · 2022-07-06T01:14:43Z

@AakashGfude where are we on this one? Are you blocked on anything? Do we need to resolve questions before moving forward?

@choldgraf This PR itself is complete I think, except for the questionable move to test this (which cannot be done without introducing jb here). And we reached a consensus that this is not the right place to fix this.. And had suggested a fix in jupyter-book .

Either remove templates_path from confoverrides in jupyter-book

AakashGfude · 2022-07-06T01:15:44Z

I will just create a PR in jupyter-book and try to start the discussion again

AakashGfude · 2022-07-07T02:04:20Z

Actually @choldgraf, there is no easy way in jupyter-book to achieve this without some architectural changes. I think this might be a good enough place for this for now.

…inx-book-theme into templates_config

choldgraf

This looks good to me - do you know what circumstances "config-inited" is not called? I thought it was always triggered but I guess not. Either way let's merge this in to fix this bug!

- Change `update_general_config()` to run on `config-inited` event in `setup()` method - Add `config` parameter back to `update_general_config()` method

updating templates path

445b6c3

AakashGfude mentioned this pull request Jun 1, 2022

Including sphinx_book_theme in extensions breaks templates_path jupyter-book/jupyter-book#1660

Closed

AakashGfude added 4 commits June 7, 2022 14:51

moving to builder-inited and appending as well

d538598

is this necessary to keep?

a8befc2

config is needed

74f88ed

filesystem layout link was updated

2ad6c17

mmcky self-requested a review June 8, 2022 04:33

mmcky previously approved these changes Jun 8, 2022

View reviewed changes

choldgraf reviewed Jun 9, 2022

View reviewed changes

AakashGfude added 3 commits June 10, 2022 17:10

adding jb project in tests, comments in init func

e18a658

adding jb project in tests, comments in init func

da439ab

added _build to gitignore

e07eec3

AakashGfude dismissed mmcky’s stale review via e07eec3 June 10, 2022 07:15

have to add jupyter-book

93f8553

AakashGfude added 2 commits July 7, 2022 14:57

Merge branch 'master' into templates_config

12af95f

Merge branch 'master' into templates_config

111f84e

AakashGfude added 3 commits July 7, 2022 14:58

Merge branch 'templates_config' of https://github.com/AakashGfude/sph…

af3b4d2

…inx-book-theme into templates_config

removing test for jupyter-book

6e75ece

removing jb from dependency

4076fa6

melissawm mentioned this pull request Jul 13, 2022

Plausible tracking chanzuckerberg/napari-segmentation-workshop#40

Merged

AakashGfude added 3 commits July 15, 2022 10:58

Merge branch 'master' into templates_config

1a85b9c

removing en/latest

ef2282a

Merge branch 'templates_config' of https://github.com/AakashGfude/sph…

c1bf0aa

…inx-book-theme into templates_config

choldgraf approved these changes Jul 15, 2022

View reviewed changes

choldgraf merged commit b232ca4 into executablebooks:master Jul 15, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

BUG: templates_path in config was overriding value set in extension #566

BUG: templates_path in config was overriding value set in extension #566

AakashGfude commented Jun 1, 2022

mmcky commented Jun 1, 2022 •

edited

Loading

AakashGfude commented Jun 7, 2022

mmcky left a comment

mmcky commented Jun 8, 2022

AakashGfude commented Jun 9, 2022

choldgraf Jun 9, 2022

AakashGfude Jun 9, 2022

AakashGfude commented Jun 10, 2022

mmcky commented Jun 10, 2022

AakashGfude commented Jun 15, 2022 •

edited

Loading

mmcky commented Jun 15, 2022

AakashGfude commented Jun 15, 2022 •

edited

Loading

AakashGfude commented Jun 15, 2022 •

edited

Loading

choldgraf commented Jul 5, 2022 •

edited

Loading

AakashGfude commented Jul 6, 2022

AakashGfude commented Jul 6, 2022

AakashGfude commented Jul 7, 2022

choldgraf left a comment

BUG: templates_path in config was overriding value set in extension #566

BUG: templates_path in config was overriding value set in extension #566

Conversation

AakashGfude commented Jun 1, 2022

mmcky commented Jun 1, 2022 • edited Loading

AakashGfude commented Jun 7, 2022

mmcky left a comment

Choose a reason for hiding this comment

mmcky commented Jun 8, 2022

AakashGfude commented Jun 9, 2022

choldgraf Jun 9, 2022

Choose a reason for hiding this comment

AakashGfude Jun 9, 2022

Choose a reason for hiding this comment

AakashGfude commented Jun 10, 2022

mmcky commented Jun 10, 2022

AakashGfude commented Jun 15, 2022 • edited Loading

mmcky commented Jun 15, 2022

AakashGfude commented Jun 15, 2022 • edited Loading

AakashGfude commented Jun 15, 2022 • edited Loading

choldgraf commented Jul 5, 2022 • edited Loading

AakashGfude commented Jul 6, 2022

AakashGfude commented Jul 6, 2022

AakashGfude commented Jul 7, 2022

choldgraf left a comment

Choose a reason for hiding this comment

mmcky commented Jun 1, 2022 •

edited

Loading

AakashGfude commented Jun 15, 2022 •

edited

Loading

AakashGfude commented Jun 15, 2022 •

edited

Loading

AakashGfude commented Jun 15, 2022 •

edited

Loading

choldgraf commented Jul 5, 2022 •

edited

Loading