fix: fix entity specs

[UziTech]

Marked version: 4.2.2

Description

Fix Entity and numeric character references specs #28 and #34.

In order to fix the last 2 Entity and numeric character references specs we would need to convert html entities to percent encoded with this json map: https://html.spec.whatwg.org/entities.json. This would increase the size of marked significantly (almost double 48k -> 84k) and provide very little benefit to the actual output.

Contributor

• Test(s) exist to ensure functionality and minimize regression (if no tests added, list tests covering this PR); or,
• no tests required for this PR.
• If submitting new feature, it has been documented in the appropriate places.

Committer

In most cases, this should be a different person than the contributor.

• CI is green (no forced merge required).
• Squash and Merge PR following conventional commit guidelines.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Updated
marked-website	✅ Ready (Inspect)	Visit Preview	Nov 12, 2022 at 8:05PM (UTC)

[calculuschild]

Is there really no built-in JS function to convert html entities?

[UziTech]

Unfortunately no. The reason given is because they are HTML entities not JavaScript entities. 🫤

There is an easy way to convert them if you have a DOM available but NodeJS doesn't have a DOM so it wouldn't always be available.

And since they are automatically converted by the browser the broken tests don't mean anything is actually broken in marked, just that the output looks different.

[UziTech]

Any objections to merging this?

[styfle]

This would increase the size of marked significantly (almost double 48k -> 84k)

That would be unfortunate. Perhaps we could add the most common ones to escapeReplacements?

[styfle]

What about when href contains a ' or "?

Do we have a test for that?

[UziTech]

the href runs through encodeURI in the cleanURL

https://github.com/UziTech/marked/blob/57a203182cddb4dc1ae4ac8f7c72650eddf3442d/src/helpers.js#L98

And this is covered by CommonMark 503 test.

https://github.com/UziTech/marked/blob/57a203182cddb4dc1ae4ac8f7c72650eddf3442d/test/specs/commonmark/commonmark.0.30.json#L4032

[styfle]

Thanks!