Merge branch 'main' into main

.github/dependabot.yml

@@ -0,0 +1,17 @@
+# To get started with Dependabot version updates, you'll need to specify which
+# package ecosystems to update and where the package manifests are located.
+# Please see the documentation for all configuration options:
+# https://docs.github.com/github/administering-a-repository/configuration-options-for-dependency-updates
+
+version: 2
+updates:
+  - package-ecosystem: "github-actions"
+    directory: "/"
+    schedule:
+      interval: "monthly"
+    groups:
+      actions:
+        patterns:
+          - "*"
+    labels:
+      - "dependencies"

.github/workflows/create-meeting-issue.yml

@@ -10,7 +10,7 @@ jobs:
   create-team-issue:
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@v2
+      - uses: actions/checkout@v4
       - uses: JasonEtco/create-an-issue@v2
         env:
           GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

.github/workflows/linkcheck.yml

@@ -7,8 +7,8 @@ jobs:
   check-links:
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@v3
-      - uses: actions/setup-python@v4
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
         with:
           python-version: '3.9'
           cache: 'pip'
@@ -21,7 +21,7 @@ jobs:
       # ref: https://github.com/lycheeverse/lychee-action
       # ref: https://github.com/lycheeverse/lychee#commandline-parameters
       - name: Link Checker
-        uses: lycheeverse/lychee-action@v1.5.4
+        uses: lycheeverse/lychee-action@v2.3.0
         env:
           GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN}}
         with:

.gitignore

@@ -1,4 +1,4 @@
-docs/_build
+_build
 *.ipynb_checkpoints
 .vscode/
 docs/gallery.txt

.readthedocs.yml

@@ -1,10 +1,15 @@
 version: 2
 
+build:
+  os: "ubuntu-22.04"
+  tools:
+    python: "3.11"
+
 python:
-  version: 3
   install:
     - requirements: requirements.txt
 
 sphinx:
   builder: dirhtml
   fail_on_warning: true
+  configuration: docs/conf.py

docs/blog/2020-08-07-announce-book.md

@@ -125,10 +125,10 @@ Jupyter Book and its related projects will continue to be developed as a part of
 
 If you'd like to learn more about Jupyter Book or get started building your own books, then check out [the new Jupyter Book documentation](https://jupyterbook.org). Jupyter Book is still in beta and is constantly being improved, so your [feedback and contributions are always welcome](https://github.com/executablebooks/jupyter-book/issues/new/choose).
 
-If there are particular features you'd like to see, open an issue or give a 👍 to a pre-existing issue, and [it will be bumped up on the feature request leaderboard](https://executablebooks.org/en/latest/feature-vote.html).
+If there are particular features you'd like to see, open an issue or give a 👍 to a pre-existing issue, and [it will be bumped up on the feature request leaderboard](https://executablebooks.org/en/latest/feature-vote).
 
 ## Get involved
 
-If you'd like to contribute to any of the projects listed in this post, you are highly encouraged to do so! The Executable Book Project (and thus Jupyter Book) is run as an open project that [welcomes contributions from others](https://executablebooks.org/en/latest/contributing.html).
+If you'd like to contribute to any of the projects listed in this post, you are highly encouraged to do so! The Executable Book Project (and thus Jupyter Book) is run as an open project that [welcomes contributions from others](https://executablebooks.org/en/latest/contributing).
 
 We are excited about all the new features and improvements in the new Jupyter Book, and look forward to seeing the new books that the community creates with this stack!

docs/blog/2023-03-07-call-for-myst-templates.md

@@ -161,8 +161,8 @@ Together let’s take the chore and re-work out of preparing a paper submission
 
 ```
 
-```{tip} Sign up for the Upcoming MyST Templates Tutorial Webinar
-:class: dropdown
+```{admonition} Sign up for the Upcoming MyST Templates Tutorial Webinar
+:class: dropdown, tip
 
 Interested in porting your first LaTeX template to a MyST template?
 

docs/blog/2023/new-project-members.md

@@ -0,0 +1,35 @@
+# Expanding our team and the next phase of Executable Books development
+
+```{post} 2023-11-28
+:author: Chris Holdgraf
+:image: 1
+:excerpt: 2
+```
+
+Over the past three years, the Executable Books team has focused its efforts on building a Sphinx-based technical stack underlying the Jupyter Book project. This has been extremely successful, and Jupyter Book and the associated MyST ecosystem in Sphinx have gained adoption across both scientific and open source communities. The MyST Parser for Sphinx averages around 350,000 downloads a month, which makes up about 13% of all Sphinx downloads ([https://www.pepy.tech/projects/myst-parser?versions=*](https://www.pepy.tech/projects/myst-parser?versions=*)). There are over 4000 Jupyter Books in public GitHub repositories ([https://github.com/search?q=%22format%3A+jb-book%22&type=code](https://github.com/search?q=%22format%3A+jb-book%22&type=code)), many of which are now featured at [gallery.executablebooks.org](https://executablebooks.org/en/latest/gallery/).
+
+![Stargazer counts over time for Jupyter Book and MyST Parser](../images/grant-star-timeline.png)
+
+Earlier this year, we shared some [exciting new work in the Javascript stack of the MyST ecosystem](https://executablebooks.org/en/latest/blog/2023-02-09-announce-mystjs/). This aims to bring the entire Executable Books stack into a more modern technical foundation, create a more sustainable and scalable community of development, and unlock new workflows in communicating with computational narratives, especially in scientific publishing.
+
+Over the past year we’ve continued to push this effort forward, and are excited to announce a few personnel changes to the project’s grant team in order to advance this effort further.
+
+
+## Rowan Cockett is joining the Executable Books steering council
+
+The Executable Books steering council has elected Rowan Cockett to join [our Steering Council](https://compass.executablebooks.org/en/latest/team/index.html#steering-council). Rowan will be the project’s first Steerco member that is not a Principle Investigator on the project’s original grant. Rowan brings expertise in modern web-based technology for scientific communication and computational narratives. He has made extensive contributions to the project over the past several years, and has led several major strategic efforts in defining new directions for our technical stack. He has also been an excellent collaborator and has exemplified our values of building an inclusive and participatory development culture. We’re excited to work with Rowan on the Steering Council, and look forward to his leadership in moving the project forward.
+
+
+## Angus Hollands is joining the 2i2c team
+
+We’re also pleased to announce that Angus Hollands will spend the next year dedicating his time to development and community management within the Executable Books project. He’ll do so via a new position with [2i2c](https://2i2c.org), one of the collaborating organizations represented on the Executable Books steering council. We'll use the final remaining funds in the project's grant to fund Angus' time.
+
+Angus will focus his efforts on three key areas of the Executable Books ecosystem:
+
+**Technical parity with Sphinx.** We wish to explore using MyST’s documentation engine as a back-end for Jupyter Book. As part of this, Angus will aim to bring MyST’s engine up to parity with the major functionality of Jupyter Book’s current implementation with Sphinx.
+
+**User community adoption.** As part of this effort, Angus will work with key user communities to adopt the MyST documentation back-end, make any necessary changes to their content to utilize the new MyST backend, and identify friction points that can be resolved with documentation or development.
+
+**Developer community growth.** Finally, Angus will work with other Steering Council and project members to lay a foundation for the Executable Books _organization_ moving forward, including identifying a fiscal home for the project and developing our initial contributing policies and guidelines as we begin opening up the project for formal participation from the broader community.
+
+We hope that this gives a picture of the progress that we’ve made in the last few years, where our project stands now, and where we are deploying the final resources of our grant in pursuit of our goals to create open tools for communicating computational narratives. We’re excited to have Rowan and Angus joining the team in a formal capacity, and look forward to the great work that they’ll do.

docs/blog/2024-05-20-jupyter-book-myst.md

@@ -0,0 +1,84 @@
+# Towards Jupyter Book 2 with MyST-MD
+
+```{post} 2024-05-19
+:author: Angus Hollands, Rowan Cockett, Chris Holdgraf
+:image: 1
+:excerpt: 2
+```
+
+## Jupyter Book and MyST - A Success Story
+
+Over [the past four years](https://executablebooks.org/en/latest/blog/2020-02-25-hello-world/), the [Executable Book(s)](https://executablebooks.org) (EB) project has been working to _improve workflows for writing and publishing with Jupyter Notebooks, along with the broader open source science ecosystem._
+
+Within this collaboration, much of the development effort has been spent on building [Jupyter Book](https://jupyterbook.org/), [which has been a tremendous success](https://executablebooks.org/en/latest/blog/2023/new-project-members/); there are at least 13,209 GitHub repositories using the tool according to the [dependency graph for the project](https://github.com/executablebooks/jupyter-book/network/dependents).[^1] Furthermore, the reach of EB has grown to encompass documentation writers and package authors who use EB tools to provide a richer experience for their audiences. For example, six of the flagship scientific python projects[^scipy] like [NumPy](https://numpy.org/doc/stable/), [Pandas](https://pandas.pydata.org/docs/), and [Dask](https://docs.dask.org/en/stable/) make use of at least one of the Sphinx-based tools we’ve developed such as the [PyData Sphinx Theme](http://pydata-sphinx-theme.readthedocs.io), the [Sphinx Book Theme](http://sphinx-book-theme.readthedocs.io), [MyST Markdown parser](https://myst-parser.readthedocs.io), or the [Jupyter Notebook parser](https://myst-nb.readthedocs.io).[^popular]
+
+[^scipy]: Here, Scientific Python means something distinct from "scientific python". The former refers to a community project <https://scientific-python.org/>.
+[^1]: These are repositories whose dependency information is sufficiently described such that GitHub is able to identify a dependency upon Jupyter Book. As such, the true figure may be higher.
+
+[^popular]: See [https://hugovk.github.io/top-pypi-packages](https://hugovk.github.io/top-pypi-packages) for the list we used.
+
+During this time, it has become clear that the [MyST Markdown language](https://executablebooks.org/en/latest/blog/2020-08-07-announce-book/#an-enhanced-flavor-of-markdown) has been at the heart of the project’s growth. In 2023, [this was recognized by designating MyST a top-level project](https://executablebooks.org/en/latest/blog/2023-02-09-announce-mystjs/#myst-is-now-a-top-level-project-in-executable-books) in Executable Books and launching the [MyST-MD](https://mystmd.org) (previously MyST-JS) in collaboration with [Curvenote](https://curvenote.com/). The JavaScript tools for parsing and rendering MyST Markdown on the web are now used by [JupyterLab MyST](https://github.com/executablebooks/jupyterlab-myst) to bring MyST Markdown to JupyterLab, and even [power the proceedings rendering at SciPy 2024](https://curvenote.com/news/curvenote-sponsors-scipy-proceedings-2024).
+
+:::{pull-quote}
+MyST-MD is able to _replace_ Sphinx in Jupyter Book [...].
+:::
+
+With MyST-MD, the project added a critical new capability: a **document structure and engine** in addition to the MyST markdown syntax. Rather than parsing MyST to Sphinx, MyST-MD provides its own [standard document format and structure](https://mystmd.org/spec/) (as JSON), and an [engine that exposes MyST documents](https://mystmd.org/guide) in a flexible and reusable way. As such, MyST-MD is able to _replace_ Sphinx in a growing number of scenarios; with the `mystmd` command line interface (CLI), it is now possible to use MyST-MD to produce [scientific PDFs](https://mystmd.org/guide/creating-pdf-documents), [documents](https://mystmd.org/guide/creating-word-documents), and [websites](https://mystmd.org/guide/website-templates).
+
+## Next steps for Jupyter Book
+
+The `1.x` release cycle across the Jupyter Book packages conveys our belief that many of the Sphinx-based tools that power Jupyter Book, such as `myst-nb` and `myst-parser` have matured to a point of _stability_, and are _feature-complete_ for Jupyter Book’s goals.[^2]
+
+[^2]: There is still plenty to improve in these tools for serving the open source developer community, and we anticipate that the maintainer community around them will continue to do so. Our definition of feature complete refers to meeting the needs of scientific and scholarly communication with Jupyter Book.
+
+Looking forward, there are still clear areas of improvements for Jupyter Book, such as simplifying the PDF export process, lowering the barrier to extensibility, improving the UX for authoring with Jupyter interfaces, and publishing to a wider range of platforms (such as scientific journals). **For the next phase of Jupyter Book, we believe that the new MyST-MD document engine is the right foundation to build on.**
+
+:::{pull-quote}
+Jupyter Book 2 will _be_ an opinionated distribution of MyST-MD.
+:::
+
+To continue to deliver on these aims and principles, we are building a new Jupyter Book experience on top of [MyST-MD](https://mystmd.org). Unlike Jupyter Book 1, which provides its own configuration and interface on top of Sphinx, Jupyter Book 2 will be an opinionated distribution of MyST-MD.
+
+By building on top of MyST-MD, we hope that:
+
+- Users will face lower barriers in producing high-quality PDF outputs (e.g. via [Typst PDF rendering](https://mystmd.org/guide/creating-pdf-documents#rendering-pdfs-with-typst)) and interactive websites.
+- Users will more easily discover, connect, reuse, and remix books published using MyST-MD / Jupyter Book, for richer and more reproducible communication. For example, via [cross-book embedding](https://mystmd.org/guide/embed).
+- Users will benefit from the latest developments in web tooling for interactive, rich, computational experiences across all of the MyST-tools e.g., JupyterLab, and the CLI.
+- Users will have a clear separation between the single source of truth for configuration ([https://mystmd.org](https://mystmd.org)), and the book-focused guide for the Jupyter Book distribution ([https://jupyterbook.org](https://jupyterbook.org)).
+- Developers will face a lower barrier to entry to contribute through the use of standards such as [unist](https://github.com/syntax-tree/unist) (see [this example of contributing a new feature](https://mystmd.org/guide/contribute-add-feature) to help you get started).
+- Developers will spend more time on enhancements and less time on maintenance, due to a more modern and streamlined infrastructure stack and lower technical debt.
+- Users and developers alike will have a tighter feedback loop through the centralized development of MyST and Jupyter Book.
+
+### Incorporation under the Jupyter project
+
+In order to more effectively focus our development efforts and ensure long-term sustainability, we intend to find a new organizational home for the MyST document engine, and focus Executable Books on the Sphinx-based ecosystem of tools for software developers. Our intention is to form a new sub-community within the [Jupyter Project](http://jupyter.org) to steward the MyST document engine. To follow along, see [this Jupyter Enhancement Proposal](https://github.com/jupyter/enhancement-proposals/pull/123) for creating a `jupyter-book` community within Jupyter.
+
+## What this means for Jupyter Book v1 users
+
+Our first priority is to make a transition to Jupyter Book v2 as frictionless as possible, by supporting v1 workflows and configuration where we can, and providing migration pathways for users (see the [principles we’re following for migration](https://github.com/executablebooks/mystmd/issues/1113) for more information). This will be a one-off, guided process that will produce a book that is _easier to maintain_ and _fully-compatible_ with MyST-MD.
+
+Below are the major areas where we anticipate users needing to perform steps to migrate.
+
+**Configuration.** Existing authors of Jupyter Books will need to migrate their books to the new `myst.yml` configuration format that replaces `config.yaml` and `_toc.yml`. This will be made easier using an interactive CLI tool that assists in performing the conversion.
+
+**Directives and roles.** We aim to support most Sphinx-based directives and roles, but some may be deprecated or change their configuration. Jupyter Book 2 will provide an interactive CLI tool that upgrades legacy syntax to the latest [MyST standard](https://mystmd.org/spec), such as `glossary` directives. This should be a mostly automatic process; although the presence of Sphinx was generally invisible in Jupyter Book 1, there are some places where it was not possible to hide this from the user. Such cases will require some human intervention to upgrade.
+
+**Functionality.** Broad functionality of Jupyter Book v2 should be nearly the same as v1, with additional capabilities as well. There may be some short-term regressions in functionality for less-used features, and we aim to grow these back over time as users request.
+
+To ease this process, the Executable Books team will author a migration guide and develop tooling to minimize the number of manual actions required to be taken by the user.
+
+## What this means for Sphinx extension users
+
+
+The [Executable Books](https://executablebooks.org) organization will remain the stewards of the Sphinx-based stack, and we anticipate continuing support and development of the Sphinx ecosystem of tools, likely with a focus on supporting the open source developer community rather than the broader “scientific communication” community. These projects are strongly supported by open-source contributors, and we hope that this continues alongside work from the Executable Books team. Some Executable Books tools will likely slow their development (e.g., [`MyST-NB`](https://myst-nb.readthedocs.io)), while more heavily-used tools will likely continue to evolve and improve (e.g., [`myst-parser`](https://myst-parser.readthedocs.io), which now powers a large part of the markdown experience in Sphinx). We recognize that there is an on-going need for these tools, and will continue working to ensure their long-lived success for the developer community.
+
+## How can I follow along?
+
+We’re excited about these new directions for the MyST project, and welcome others to join the conversation or get involved improving these tools.
+
+**For synchronous chat about the MyST engine:** See [our Discord chat room](https://discord.mystmd.org/).
+
+**For asynchronous discussion about the MyST engine:** See the [Discussions forum in the mystmd repository](https://github.com/executablebooks/mystmd/discussions).
+
+**For high-level information about the MyST engine:** See [mystmd.org](https://mystmd.org).
+