Render Markdown using GitHub rendering engine

[dave-doty]

Describe the bug
By default, both Github and PyPi show the README.md page as the "home" page. Many things render differently between them, forcing me to put a disclaimer "This document is intended to be read on github. Some of what appears below does not render properly on other websites such as pypi.org.". I show a couple of examples.

Expected behavior
This table renders nicely on Github when put in backtick quotes, as you can see since you are reading this on the Github site:

╭──────────────┬─────────────┬─────────────────╮
│ First name   │ Last name   │ Email           │
├──────────────┼─────────────┼─────────────────┤
│ Alice        │ Johnson     │ alice@gmail.com │
│ Bob          │ Jackson     │ bob@nsf.gov     │
╰──────────────┴─────────────┴─────────────────╯

However, the length of the horizontal dash characters is not equal to others in the rendering engine used on PyPi, leading the above table to look bad. I don't have an example of that particular table, but look at the tables here: https://pypi.org/project/aggie-unterprise/

The top table on that page looks nice because it is rendered as a Markdown table. But the tables below it are put in backticks, and note how on PyPi they render poorly due to the shortened dashes in the first, third, and final lines of text:

compared to how it looks on Github:

Another example is that on Github, you can link to sections using the section name. For example, I have a table of contents here: https://github.com/dave-doty/aggie-unterprise that does not work here: https://pypi.org/project/aggie-unterprise/

To Reproduce
There's many examples of this, but they would all be solved if PyPi used the same Markdown rendering engine as GitHub (or maybe they do but have CSS styling slightly different or something). For a concrete example of how to reproduce, make a Github repo with a README.md file containing either use the tables above in backticks to see the difference, or create a table of contents like this:

# My Repo

## Table of contents

* [Overview](#overview)
* [Installation](#installation)

## Overview

## Installation

then publish that as a Python package on PyPi.org. Note how the relative links created when rendering the table of contants work on GitHub but not on PyPi.

My Platform
Chrome and Firefox both have this behavior.

Additional context
None

[di]

The top table on that page looks nice because it is rendered as a Markdown table. But the tables below it are put in backticks, and note how on PyPi (https://pypi.org/project/aggie-unterprise/), they render poorly due to the shortened dashes in the first, third, and final lines of text:

This seems more like an issue with the font used rather than with the rendering engine? FWIW, this is what it looks like on PyPI in my browser:

Another example is that on Github, you can link to sections using the section name.

This is a duplicate of pypa/readme_renderer#169, there are more details there.

There's many examples of this, but they would all be solved if PyPi used the same Markdown rendering engine as GitHub

We basically use a rendering engine that's as close as possible to GitHub's: readme_renderer uses https://pypi.org/project/cmarkgfm/ which are binding's to GitHub's fork of cmark, https://github.com/github/cmark-gfm, I don't think we can get any closer to what GitHub uses than that.

[dave-doty]

We basically use a rendering engine that's as close as possible to GitHub's: readme_renderer uses https://pypi.org/project/cmarkgfm/ which are binding's to GitHub's fork of cmark, https://github.com/github/cmark-gfm, I don't think we can get any closer to what GitHub uses than that.

Is it the CSS styling then that two different fonts are used between GitHub and PyPi?

[dave-doty]

Here's a clue as to the difference in styling:

PyPi website normally:

If I uncheck the font-family: Source Code Pro, monospace; styling, it instead uses monospace and fixes the rendering:

By contrast, here's GitHub font style for code elements:

I remain stumped why a font named "Source Code Pro" would not be a fixed-width font, but anyway it seems worth avoiding since one almost always wants fixed-width font for backtick Markdown environments.

[miketheman]

Something to consider: we use Google Fonts' implementation of Source Code Pro, see https://github.com/google/fonts/blob/43405b42bd150c0a667f82c53eeb226e5d789926/ofl/sourcecodepro/METADATA.pb
Most recent version was google/fonts#4010 from https://github.com/adobe-fonts/source-code-pro/releases/tag/2.030R-ro%2F1.050R-it
The source repo for Source Code Pro is here: https://github.com/adobe-fonts/source-code-pro and their most recent release: https://github.com/adobe-fonts/source-code-pro/releases/tag/2.042R-u/1.062R-i/1.026R-vf

Looks like you could submit a font upgrade to that repo - https://googlefonts.github.io/gf-guide/onboarding.html#font-upgrades

[di]

I could also be convinced to adopt a different monospace font if Source Code Pro isn't the most ideal one for us to use here as well.

[dave-doty]

I don't know much about fonts or styling, but here's a clue. If I go here and type the table

╭────────────────────────────────────┬─────────────┬─────────────┬────────────┬────────────┬────────────┬──────────────┬─────────────┬─────────────┬─────────────╮
│ Project Name                       │    Expenses │      Salary │     Travel │   Supplies │     Fringe │   Fellowship │    Indirect │     Balance │      Budget │
├────────────────────────────────────┼─────────────┼─────────────┼────────────┼────────────┼────────────┼──────────────┼─────────────┼─────────────┼─────────────┤
│ INDIRECT COST RETURN               │       $0.00 │       $0.00 │      $0.00 │      $0.00 │      $0.00 │        $0.00 │       $0.00 │     $904.00 │     $904.00 │
│ DISCRETIONARY FUNDS                │       $0.00 │       $0.00 │      $0.00 │      $0.00 │      $0.00 │        $0.00 │       $0.00 │   $2,500.00 │   $2,500.00 │
│ NSF Engineering DNA and RNA        │  $61,316.61 │  $34,800.00 │      $0.00 │    $133.40 │  $3,263.70 │        $0.00 │  $23,119.51 │ $318,683.39 │ $380,000.00 │
│ NSF CAREER Chemical Computation    │ $468,000.72 │ $211,746.21 │ $33,334.25 │  $8,847.54 │ $58,519.35 │    $5,166.81 │ $150,386.56 │  $17,180.28 │ $485,181.00 │
│ REU CAREER Chemical Computation    │  $44,062.63 │  $43,180.29 │      $0.00 │      $0.00 │    $882.34 │        $0.00 │       $0.00 │  $18,750.37 │  $62,813.00 │
│ DOE Office of Science Basic Energy │  $15,045.49 │   $8,642.86 │      $0.00 │      $0.00 │    $760.57 │        $0.00 │   $5,642.06 │  $51,372.51 │  $66,418.00 │
╰────────────────────────────────────┴─────────────┴─────────────┴────────────┴────────────┴────────────┴──────────────┴─────────────┴─────────────┴─────────────╯

into the normal text, it cannot render any of the frame symbols:

Those frame symbols are non-ASCII Unicode variants of dashes and so forth. So perhaps the issue is that Source Code Pro does not support non-ASCII Unicode? That also seems unimaginable in 2024.

But if it's true, perhaps the browser tries to style individual symbols at a time, instead of whole HTML elements (I don't know, I always assumed it would always use a single font for one HTML element), and then what's happening is that it applies Source Code Pro to the ASCII symbols, but switches to monospace for the non-ASCII symbols that Source Code Pro cannot handle, and then Source Code Pro and monospace use slightly different widths from each other.

[miketheman]

That also seems unimaginable in 2024.

[miketheman]

I could also be convinced to adopt a different monospace font if Source Code Pro isn't the most ideal one for us to use here as well.

I tinkered with another monospace font today - specifically GitHub Next's Monaspace Neon, and it looks even further emphasized, making me think that the underlying data has spacing in it that it shouldn't?

It looks like the project in question uses tabulate to create its renderings, and their project page renders fine to me with no tweaks or modifications

[miketheman]

I've also confirmed that the current release of Source Code Pro renders correctly, so I'd recommend going down the route of getting Google Fonts updated.

[dave-doty]

The format I used to generate the table in previous comments is rounded_grid.

Most of the table formats on the tabulate PyPi page look off in my browser (I suspect the ones using non-ASCII symbols):

When I remove Source Code Pro from the styling in my browser's dev tools, so that it goes to monospace, I notice that the rows with frame symbols such as ┌────────┬───────┐ do not appear to change, whereas the ones with ASCII symbols change (and then line up nicely with the non-ASCII rows). So I suspect that it is rendering the ASCII symbols with Source Code Pro and the non-ASCII with monospace.

[miketheman]

Blocked on google/fonts#8030
Alternative approach could be to change the font, but that would require more investigation as to the wider impact..