Add docutils 0.18 citation/footnote support using CSS grid

[agjohnson]

This is a fix for #1322 that continues our pattern for HTML5 support
using CSS grids, introduced for support for docutils < 0.17 HTML5
support.

At normal widths, the columns try to stay at a consistent width, including secondary paragraphs in the footnotes/citations:

At narrow widths, the paragraphs for each row shrink to accommodate the extra room for the lable/backrefs:

[agjohnson]

For comparison, here is the html5 writer output for docutils 0.17:

[agjohnson]

One new test that is not displaying as I would prefer is if we add a lot more backrefs:

I'd prefer this to wrap nicer, which is possible by limiting the backrefs width and wrapping them.

[benjaoming]

This is definitely the most correct fix. I'll merge test fixes in a separate PR and close the other effort.

[benjaoming]

If you are looking at this one more time, can you also expand the bottom margin of the footnote-list and citation-list to please Gestalt? :)

[agjohnson]

Given my last note, I went back and tested docutils < 0.18 with lots of backrefs too, and display looks rather bad there too.

Lots of backrefs isn't unreasonable, I've done close to this in the past, but it certainly isn't a common issue of course. So, seems like something to fix with this implementation.

My solution so far requires a reasonable length for the backrefs column to balance the row widths. Once I start reducing the column width here though, display gets worse. The docutils/Sphinx output here is using a non-breaking, comma separated list of backrefs in between parenthesis. This does not wrap in a consistent way, so I'm going to try getting fancier here.

The fix I think I lean towards is dropping all of the unnecessary punctuation and making this a superscript list of links instead:

I would make the html5 output match for both docutils < 0.18 and >= 0.18 both use these styles, however the CSS grid structure has to be slightly different for docutils < 0.18.

This version will look good for 2-6 references at normal viewport sizes. Once it wraps, it looks less good, but better than other options, and much better than the comma separated list.

[agjohnson]

I also think I can make the docutils < 0.18 html5 structure match visually here without much trouble, so might make this PR a bit farther reaching.

[agjohnson]

Here is Sphinx 4.5 + docutils 0.17 styling with the above fixes:

And here is the docutils 0.18 version:

[benjaoming]

The fix I think I lean towards is dropping all of the unnecessary punctuation and making this a superscript list of links instead.

There is already an issue open about accessibility in relation to footnotes: #1153

I wonder if using superscript for backrefs will make them more difficult to both read and click, thus worsening accessibility.

I also think that the former layout looked nice, I didn't miss anymore visual distinction between footnote labels and backrefs.

[agjohnson]

I can continue playing with this, I'm not 100% sold on using superscript here. But also, the default display with extra punctuation is both bad UX and displays poorly in a many cases. I'm not tied to it for such a minor use display element.

The backrefs have rather bad UX to them. There is no annotation or UI context as to what the links are. They are just a list of integers that happen to be links. Users that don't understand what that is won't be helped by extra punctuation.

The two options I'd prefer here are hiding the backrefs in a dynamic element -- popup or similar -- or putting the backrefs in the last row, under the footnote paragraph. Unfortunately, CSS grids does not support this on grids with implicit rows.

[benjaoming]

But also, the default display with extra punctuation is both bad UX and displays poorly in a many cases.

What is the extra punctuation? If it's the colon, I think that we agree to toss it 🗑️

[benjaoming]

Looking at the clock ticking towards Sphinx 6, I'd suggest to stick to the former version*) and postpone any further improvements to "lots of backrefs" scenarios for later? Since we are slowly trimming legacy support, these improvements might also be easier to lift in 2.0 and 3.0.

*) The former version was when you initially had everything looking like this:

[agjohnson]

What is the extra punctuation? If it's the colon, I think that we agree to toss it wastebasket

The parenthesis and commas. These are causing trouble wrapping as the output has nothing to line break on.

[benjaoming]

The parenthesis and commas. These are causing trouble wrapping as the output has nothing to line break on.

Feature or bug? :)

[agjohnson]

Hah yeah definitely a bug, the entire HTML structure of this output is deficient really. It's optimized for display without any styles, but that is not how anyone uses docutils, effectively. We can work with it, but it's not going to be good UX with the current structure.

The output that docutils 0.18 gives us:

<span class="backrefs">(<a role="doc-backlink" href="#id3">1</a>,<a role="doc-backlink" href="#id17">2</a>)</span>

Which, if it were optimizing for output that can be styled, and provides more context and a11y attention:

<span class="backrefs">
  <ul>
    <li><a role=".." href=".." title="Back reference 1 for footnote 1, for section Foo Bar">1</a></li>
    <li><a role=".." href=".." title="Back reference 2 for footnote 1, for section Foo Bar">1</a></li>
  </ul>
</span>

Not something we can change though.

[benjaoming]

Given a bit of creative well-meaning, the non-wrapping of backrefs can be seen as a feature. What I mean is that if forced onto the same line, parts of readability is maintained, while columns may stop aligning.

Very long lists of backrefs on narrow displays may not look nice, but the likelihood is low?

[benjaoming]

Also agreed that the DOM is less-than-optimal. Might worth raising the issue upstream, although also annoying to have to implement yet-another-DOM, especially if implemented in a way that makes CSS selectors ambiguous.

[benjaoming]

@agjohnson what do you think about maintaining the behavior in 47516ea and postpone improvements and decisions around superscript to a separate issue?

Edit: added the git commit hash to be explicit about what version of this PR I meant

[agjohnson]

What I mean is that if forced onto the same line, parts of readability is maintained, while columns may stop aligning.

Well, that's what I mean. The issue is more than just alignment, display is affected too. Multiple backrefs also bump the column out of the grid, because the backref column exceeds the display space that was allocated for it in the grid hack too:

As the screen narrows, it gets more exaggerated too (and clipped in addition), as the columns are percentage basis:

@agjohnson what do you think about maintaining the behavior in 47516ea and postpone improvements and decisions around superscript to a separate issue?

I'd like to resolve the bug above, but would agree we can come back to this at some point and spend more time on this. I can open up an issue to track the problem here as well.

[agjohnson]

Dropping the last column minmax value down drastically didn't seem to affect display on Firefox and allows for rather narrow display before the columns start breaking out of the grid -- at ~400px wide.

[agjohnson]

Okay, I feel this is as good as it will get, barring any more issues popping up on other browsers/etc. I introduced some pseudo class spacing in the backrefs and they at least wrap consistently now.

Sphinx with docutils 0.17 and HTML5 writer:

Sphinx with docutils 0.18 and HTML5 writer:

[benjaoming]

LGTM!

I've done a run of visual regression testing on all supported Sphinx versions and with htmlwriter4 and htmlwriter5.

With Sphinx 5.1 and htmlwriter4

With Sphinx 4.5 and htmlwriter5