Possible regression: failed rendering of ReStructuredText files

[ml-evs]

Select Topic Area

Question

Body

Hi, we have a technical specification hosted on GitHub that makes heavy use of ReStructuredText (https://github.com/Materials-Consortia/OPTIMADE/blob/develop/optimade.rst). Up until today, our specification was nicely rendered when people viewed it on GitHub, but now we only have the option to view the raw files. Is this a global change, a bug, or something specific to our file (rendering timeouts perhaps)?

Thanks!

EDIT: please see many other examples below. In our case we could isolate the changes which cause the specification not to render (simply seems to arise from document length rather than any particular rst feature, see https://github.com/orgs/community/discussions/86715#discussioncomment-8117857)

[ml-evs]

Simple .rst files seem to work fine, so I guess we're just hitting some new complexity limit...

[fireattack]

This one does not render any more, either: https://github.com/mikf/gallery-dl/blob/master/docs/configuration.rst

[ml-evs]

At our end I've pinned it down to one patch that takes the file from rendering -> failing: https://github.com/Materials-Consortia/OPTIMADE/blob/450f5e4b22d044b30104784ff12bd8e3878c5470/optimade.rst vs https://github.com/Materials-Consortia/OPTIMADE/blob/b177978945e5755e9fbebb33557eec79be3e8746/optimade.rst (details in this PR Materials-Consortia/OPTIMADE#493)

[felixfontein]

There are many RST files in repositories having the same problem:

• https://github.com/ansible/ansible/blob/stable-2.15/changelogs/CHANGELOG-v2.15.rst
• https://github.com/ansible-community/ansible-build-data/blob/main/9/CHANGELOG-v9.rst
• https://github.com/ansible-collections/community.general/blob/stable-8/CHANGELOG.rst
• https://github.com/ansible-collections/community.docker/blob/main/CHANGELOG.rst
• https://github.com/getsops/sops/blob/main/README.rst (README.rst is not correctly rendered getsops/sops#1411)

[felixfontein]

I'm also getting Unicorn error messages (This page is taking too long to load. / Sorry about that. Please try refreshing and contact us if the problem persists.) sometimes, and sometimes the raw RST file.

(That was for https://github.com/ansible-community/ansible-build-data/blob/main/4/CHANGELOG-v4.rst)

[samccann]

Adding to this, the following are no longer even viewable (not getting the unicorn, getting error, something went wrong):

• https://github.com/ansible-community/ansible-build-data/blob/main/2.10/CHANGELOG-v2.10.rst
• https://github.com/ansible-community/ansible-build-data/blob/main/5/CHANGELOG-v5.rst

[gundalow]

I've filed this under ticket 2534050

[ml-evs]

I had a similar reply as the above on 17th Jan but no follow-up yet:

We have had a few reports of this bug over the last couple of days and our engineers are working on a fix. It's very useful for us to have more data points, so thank you for submitting this report! I'll pass it on to the team.

[evdcush]

Have y'all gotten any further engagement from GitHub (other than the initial canned/robot response in January)? 👀

[felixfontein]

Yes, there was https://github.com/orgs/community/discussions/86715#discussioncomment-8431598 in beginning of February, which wasn't very helpful, and https://github.com/orgs/community/discussions/86715#discussioncomment-9149558 in April, where actually something happened and it got a lot better. There are still a lot of broken things, but at least there's a rendering preview again that renders many things correctly (albeight still many not correctly or at all).

[fireattack]

it got better

I'm confused about this actually. Rendering RST is not a new feature, it worked totally fine before.

Did GitHub just change their rendering engine to a different one without any testing? And even if so, why not switch it back to the old one at least before the new one is good enough in internal testings?

[felixfontein]

I agree, but having rendered some is still better than having rendered nothing - which was the situation between early January and ... not sure anymore, but months later.

I'm also very much interested in an answer to the later question, but I guess we won't get any...

[KelSolaar]

We are experiencing similar issues with Colour since a few days? (Unsure when it started exactly) Our README which was rendering fine has stopped rendering: https://github.com/colour-science/colour. We are also using reStructuredText!

[adriandole]

Rendering is failing on all of our files above 1500-ish lines. There is noticeable lag and then the raw version is displayed. It breaks somewhere between 1000 and 1600 lines.

• 1057 lines, works

• 1627 lines, doesn't work

[hannes-ucsc]

This should be a high priority to fix.

https://github.com/HumanCellAtlas/dcp2/blob/main/docs/dcp2_system_design.rst

Note that the Preview tab is active in the above screenshot but the display is as if the Code tab were active.

[hannes-ucsc]

See also https://github.com/orgs/community/discussions/86715#discussioncomment-8434630

[gundalow]

GitHub support responded

Thanks for your patience while we investigate this problem. Our Engineers have found that a recent update to the underlying library that check if the rst file content can be rendered has made more restrictive tests. The files that aren't being rendered are failing the check for a valid rst file.

To test this we can make use of the rstcheck python library against the files you've noticed no longer render.

Our Engineering team continues to investigate solutions to return to the previous functionality. For now, rendering of rst files will work for files that pass the rstcheck test.

[stuij]

Seems like a file size issue to me more than time-out. I did some experimentation, and eventually pinpointed the exact character on which the page would fail: stuij/abi-aa@605b80b

For this document, this came down to 62804 characters or 62892 bytes. Reliably. If it'd be render time, i'd expect at least a bit of variation from render to render.

[stuij]

@gundalow : where did you file that ticket? Because I think it's clear that the reason they gave isn't applicable to the issue that some of us here are seeing. I'd like to try and put in another ticket.

[gundalow]

@stuij Thanks for this extra info, that's useful troubleshooting info. I've passed it on to GitHub Support.

I'll reply to this thread once I hear more.

[felixfontein]

If it is a file size problem, it's not really deterministic either. The file that works from my gist has 52654 bytes, the file that does not has 52687 bytes. Maybe it is a count of something else?

[RealEstateSaga]

How are you? If I delete this, will my site crash, or is this just for search engines?

/next-sitemap.config.js

Sorry, I am just in a little bit of bind, Google My business does not like something on it, and they emailed me about fixing it.

Thanks!

[stuij]

The reply I got from Github Support when I created a ticket on this:
We have received other reports of this and our team is currently working to fix this. Looking at the internal issue where this is being worked on it looks like we are close to being able to release this. We will let you know when this fix has been pushed to production.

[pierre-rouleau]

I hope they'll fix that soon. I'm also affected by this. I have a large file with several reStructuredText markup feature used and it was rendering fine: https://github.com/pierre-rouleau/pel/blob/master/doc/pel-manual.rst

[pierre-rouleau]

My document now renders... but sections lost their numbers. The table of content is only visible on the right side, in the Outline pane.

[pierre-rouleau]

One nice feature of the Outline pane is the ability to filter their section titles.

[Arpan3323]

The reStructuredText markup file here is not being rendered properly.

[stuij]

All of our .rst documents are now rendering again in the browser :D

I spot-checked some of the broken documents in this thread, and they also render again.

Hurray!

[felixfontein]

That's a huge improvement indeed! Unfortunately not all is well yet:

• Some documents still don't render:

  • https://github.com/ansible-community/ansible-build-data/blob/main/9/CHANGELOG-v9.rst
  • https://github.com/ansible-community/ansible-build-data/blob/main/8/CHANGELOG-v8.rst
  • https://github.com/ansible-community/ansible-build-data/blob/main/4/CHANGELOG-v4.rst
  • (likely all other Ansible changelogs) - fortunately we have MarkDown versions for the latest one now as well: https://github.com/ansible-community/ansible-build-data/blob/main/9/CHANGELOG-v9.md

• At least in one document, all code blocks render ... strangely:

  • https://github.com/getsops/sops/blob/main/README.rst - here all code lines are wrapped in <a>...</a>, which gives them underlines
  • It seems to happen as soon as you use .. code:: or .. code-block:: with some languages like bash or yaml or json. It does not happen with no language or with languages it apparently doesn't know about. In any case, there doesn't seem to be syntax highlighting.

[mmore500]

Same here! I am also having code lines wrapped in <a> tags https://github.com/mmore500/teeplot

[gundalow]

Thanks, I've passed this on to GitHub support.

[ml-evs]

Definitely an improvement, everything at https://github.com/Materials-Consortia/OPTIMADE/blob/develop/optimade.rst is working again, except the table of contents...

[stuij]

I also noticed that for lack of a better word sub-tables don't render anymore. So added columns or rows in a subsection of a table.

For example look at the source for the Fundamental Data Types table below:
https://github.com/ARM-software/abi-aa/blob/7c2fbbd6d32c906c709804f873b67308d1ab46e2/aapcs64/aapcs64.rst?plain=1#L507

This used to render like the equivalent ascii, and looks correctly in the genrated PDF:
https://github.com/ARM-software/abi-aa/releases/download/2023Q3/aapcs64.pdf

However currently you can clearly see that the table isn't parsed correctly when rendered on Github:
https://github.com/ARM-software/abi-aa/blob/main/aapcs64/aapcs64.rst#fundamental-data-types

I created a bug-report for it.

[smheidrich]

Related discussion about the underlined code issues people in here have mentioned: Code blocks appear as (underlined) links

[gykovacs]

Same problems here, previously nicely rendered README.rst broken: https://github.com/FalseNegativeLab/mlscorecheck

Is this discussion visible to GitHub developers?

[wdoekes]

Is this discussion visible to GitHub developers?

Not sure. I think mentioning this issue on Twitter or Mastodon with the appropriate tags might have more impact than doing a me-too here.

[stuij]

several of us have opened tickets, which unfortunately only seem to be visible between Github and the person oening the ticket. In my case they answered that internally they are aware and that they are looking into it. After which at least my customer tickets get closed.

…

On Sun, 14 Apr 2024 at 10:17, György Kovács ***@***.***> wrote: Same problems here, previously nicely rendered README.rst broken: https://github.com/FalseNegativeLab/mlscorecheck Is this discussion visible to GitHub developers? — Reply to this email directly, view it on GitHub <#86715 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABO6NLRIUV3M6ZDBWYLZSTY5JCTJAVCNFSM6AAAAABBYIXV4GVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4TCMBYGI2DM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[felixfontein]

I think the main takeaway (at least for me) is: if you want something rendered on GitHub, stick to MarkDown. Everything else can break at any moment, and then it can take month to get fixed - if ever. (I'm still waiting to be convinced that this isn't true, but... :( )

[ikvk]

Lets I try to guess: there is so big security hole at github because of .. code-block:: , that maybe we have never see fix for ReStructuredText.

[RealEstateSaga]

How are you? If I delete this, will my site crash, or is this just for search engines?

/next-sitemap.config.js

Sorry, I am just in a little bit of bind, Google My business does not like something on it, and they emailed me about fixing it.

Thanks!

[ikvk]

@RealEstateSaga I have no idia about it

[Graeme22]

Code blocks are now rendering correctly for me although there are still some things out of whack

[phetdam]

Code blocks also seem to be working for me too, e.g. .. code:: python3 blocks have syntax highlighting and no more underlining of text in the ..code :: blocks that I've seen in my projects. Definitely a welcome change.

[gundalow]

Hi,
I've been pushing GitHub support and they made some improvements yesterday.
They've asked for a list of things which are still broken.

If you have specific examples of what's still not rendering correctly, can you please reply to this message with a URL and a short description of the rendering bug. Thanks

[cdwilson]

pull-quote stopped working for me in https://github.com/golioth/reference-design-air-quality/blob/main/README.rst

For example:

.. pull-quote::
   [!IMPORTANT]

   This is important

[dolfinus]

Tables with merged cells cannot be rendered:
https://github.com/MobileTeleSystems/onetl/blob/c9b54df71643d6457704eea20be392158b11b647/README.rst

But Sphinx can render them propery:

[kourouklides]

.. contents:: stopped working on my repositories, too. It would be good if someone could fix it, please. Thanks.

[ale5000-git]

@gundalow
Hi, I have a similar issue.
Very simple example: https://github.com/ale5000-git/RSTtest/blob/main/README.rst
I have also reported it here: https://github.com/orgs/community/discussions/121163

[bartandrews]

@gundalow side-by-side figures are still not rendering correctly: https://github.com/HofstadterTools/HofstadterTools For reference, side-by-side figures in the corresponding sphinx docs are correct: https://hofstadter.tools/overview.html This used to work fine until fairly recently.

[JeanKossaifi]

I noticed the same issue with our repository: https://github.com/tensorly/tensorly/edit/main/README.rst
The table and some directives are no longer correctly rendered.

[AugustKarlstedt]

https://github.com/bazelbuild/bazel-gazelle/blob/master/README.rst is broken on GitHub, but renders perfectly well elsewhere, e.g. in this Online reStructuredText editor

[AugustKarlstedt]

[12rambau]

the image directive is still broken, it creates line breaks where it shouldn't.
example here: https://github.com/12rambau/sepal_ui/

[evdcush]

Related:
https://github.com/orgs/community/discussions/121163

[evdcush]

This bug/regression persists.

[evdcush]

This bug persists.

[evdcush]

Bug persists.

[pierre-rouleau]

Looks like some problems have recently been fixed. The list of tags at the top of my pages where recently sowing on separate lines. Now, they show one after the other, as they used to show. So...Good! Some improvements! All hope is not lost!

Ref: https://github.com/pierre-rouleau/pel#readme

[AugustKarlstedt]

Seeing a proper rendering of https://github.com/bazelbuild/bazel-gazelle/blob/master/README.rst now too!

[AndydeCleyre]

Hey, my table of contents is working again! 🙏🏼

[bartandrews]

My images are now showing side-by-side in the readme, which is how it should be! 😃 https://github.com/HofstadterTools/HofstadterTools?tab=readme-ov-file#quick-start

[stevepiercy]

Python console code blocks render correctly again! Yay!

https://github.com/zopefoundation/z3c.form/tree/master/src/z3c/form/browser#browser-support

[evdcush]

My rST README is rendered as it was before the bug! 🎉

[fireattack]

No one seems to mention that so far: is the root cause of all these due to GitHub switched from using sphinx to render rst to using pandoc to convert rst to md (or similar) and render from there?

There was a rendering issue I discussed above (https://github.com/orgs/community/discussions/86715#discussioncomment-9149986) and I later reported to pandoc (jgm/pandoc#9805) since they had the same issue. After the dev of pandoc fixed it, it was fixed on GitHub's end too now.

[AugustKarlstedt]

Good find, that does seem likely to be the root cause

[felixfontein]

I don't think they ever used Sphinx. They did use docutils (which Sphinx also uses) though: https://github.com/github/markup/blob/master/lib/github/commands/rest2html - but apparently they stopped using that script at the beginning of this year. (Or even before. That repo looks like it hasn't really been maintained for a longer time...)

[define-private-public]

It's working fine for me now. Thanks for fixing this!!

[github-actions[bot]]

🕒 Discussion Activity Reminder 🕒

This Discussion has been labeled as dormant by an automated system for having no activity in the last 60 days. Please consider one the following actions:

1️⃣ Close as Out of Date: If the topic is no longer relevant, close the Discussion as out of date at the bottom of the page.

2️⃣ Provide More Information: Share additional details or context — or let the community know if you've found a solution on your own.

3️⃣ Mark a Reply as Answer: If your question has been answered by a reply, mark the most helpful reply as the solution.

Note: This dormant notification will only apply to Discussions with the Question label. To learn more, see our recent announcement.

Thank you for helping bring this Discussion to a resolution! 💬