[Consolidated proposal] Features to deprecate

[joshbruce]

Over the years the community has added features to Marked based on issues and user feedback; while not always a bad thing, sometimes it can cause a project to go beyond sustainable scope for itself (as we kinda saw). To try to rein things back in a bit, there are some features we may want to deprecate because they take Marked beyond its intended scope; thereby, making it more difficult for a small band to maintain.

This Issue is for capturing those suggestions, which will most likely be implemented during the 0.5.0 and beyond releases:

• ??

Use comments to discuss, will update the task list as we gain consensus and whatnot...that way we don't add too many issues that are future-planning.

[joshbruce]

Think we should deprecate header ids being handled by Marked. I don't think any of the supported flavors of specifications actually has this functionality and it's something that can be better performed as a secondary pass on the DOM, in my experience. Also, could be something a user could add as extended functionality to reduce our maintenance overhead.

Tagging @UziTech and @Feder1co5oave to make sure y'all see the ticket.

[Feder1co5oave]

My vote goes to deprecate the pedantic option 😆
I know chjj was in love with it, but it just doesn't make sense now that we have commonmark. Would be curious to see how many users actually enable it.

[joshbruce]

That's fair re pedantic. I don't know how many people know the Daring Fireball spec is even still out there. How different is it from CommonMark though?? Don't we get most of it "for free" by virtue of having CommonMark. Might be an extend + override path.

Pedantic = All these properties
CommonMark = Has Pedantic + these properties - these Pedantic properties.
GFM = Has CommonMark + these properties.
MultiMarkdown (this is totally made up at this point, just an example) = Has GFM

Know what I mean??

[Martii]

@joshbruce

Re:

Think we should deprecate header ids being handled by Marked.

I believe we currently utilize extending header ids to our own flavor for easier document traversal.

Will we be supporting syntax highlighting of languages such as c or r?

We have the ability to utilize the full compliment of languages however most ppl use js and diff (sometimes it automatches on html and css which is utilized as well). We match GitHubs implementation as close as possible which includes other identifiers of that nature.

[joshbruce]

Hey @Martii thanks for showing up!

Just to make sure I'm following, don't think we've interacted much, so pardon the bumps in communication, really glad to see a new face (though I feel like I should know you...been a lot of peopling for me recently, sorry).

we currently utilize... and We have the ability to utilize

https://openuserjs.org or are you taking on the voice of the marked community ?? (perfectly cool if you're taking on the voice of the community, I sometimes take on the voice of products...just wanna make sure I'm following the pronouns) 😉

own flavor for easier document traversal

Own flavor of Markdown or own flavor of marked?? We have a lot of forks that stand as separate products but credit marked as the inspiration.

(I guess what I'm asking is, would deprecating this feature break something for you and your users because you are dependent directly on marked...or is this a rationale for not deprecating...or something else entirely?)

We match GitHubs implementation as close as possible which includes other identifiers of that nature.

Again, just want to make sure I'm fully tracking you here. Are you saying you match the expected behavior based on using GitHub's text editor, saving, and how it is rendered on the final page? or, are you saying you follow the specification itself (https://github.github.com/gfm/)?

[Martii]

@joshbruce

Are you saying you match the expected behavior based on using GitHub's text editor, saving, and how it is rendered on the final page? or, are you saying you follow the specification itself (https://github.github.com/gfm/)?

These almost go hand in hand... so towards the goal of both.

https://openuserjs.org or are you taking on the voice of the marked community ??

Again a little bit of both here... messenger only from observation... I would have to say more towards forum i/o. Our establishing owner initially built his own software even before most packages existed to bridge any gaps that currently exist between forum software and non-forum software. He and his initial founding teams established a set of extending methods of this package marked and I'm the active maintainer (and co-owner since I have a bit of wisdom behind me I believe). One contributor went the exact opposite methodology and I'm somewhere in-between using custom versus outsourcing discrete methods such as marked.

Own flavor of Markdown or own flavor of marked?

Just extending existing methods for right now. We utilize this packages npmjs.com address for npm i but we do define some custom functions that we'd like to keep.

As I mentioned at an earlier point about a month ago we've been investigating alternatives to marked (Note: I always put packages in emphasis and use markdown and markup as respective counterparts) but prefer to keep the status quo on such a critical feature for communication and security (we have a term that all code must be blocked/fenced for example which most ppl follow)

As Code usually speaks better than explanations (especially when different terminology that means hopefully the same thing comes into play) our current extendings are at https://github.com/OpenUserJS/OpenUserJS.org/blob/master/libs/markdown.js . I've put most additional features on hold/tracking since there are some other technical issues with how our system runs but in general it works well for us. These extensions (which should be well commented and named for easier skimming) are what we currently utilize. Bringing up the c code fence coercion again... if that goes for some reason that's not something I'd like to see be removed. e.g. one never knows what "progress" will bring. Adaptability is a goal when a package is chosen along with how well the maintainers interact with others. (no ones perfect but there can be consistent good and consistent bad... this project case is consistent good from what I've observed so we prefer to continue here :)

I do understand the need to compartmentalize and modularize though for easier systems management but having the current feature availability present is a plus. Sometimes progress moves too quickly without gauging all the consequences ... a factor of knowing wisdom over knowledge/intelligence. (dating myself of course here ;)

Hope some of this explains our wishes for this issue and answers your questions.

[joshbruce]

tl;dr: Your users should not be negatively impacted by this deprecation. We should be able to allow for the continued use of the solution provided. The solution should probably be updated as it is not WCAG 2.0 compliant and appears to use Font Awesome 4, which is being deprecated in favor of the more accessible and robust solution offered by Font Awesome 5 (what appears to be used by GitHub.com as of this writing).

This whole thing can move as slowly or quickly as the users want it to but we're kinda at a point where folks need to decide on which option to go for (individually) and stick to it (we're getting a lot of mixed signals).

• Modernize: Fix and upgrade all the things.
• Euthanize: Focus on telling the world that eventually the package will no longer be available - period. (See LeftPad story by Kevlin Henney)
• Memorialize: Like the Tomb of The Unknown Soldier the core team becomes a set of sentries rotating shifts, broadcasting that the project is not active, and should be used primarily for "educational and accreditation purposes".

I will tend to push the same direction I am pulled by the users and community; constrained by my principles. Otherwise, I am perfectly content to sit and meditate, if there is no pull for something, I will not push for something. On the principle note, I am specifically referring to, the community principle under review as part of our possible code of conduct (#1094); therefore, if I see one, and only one, person killing themselves to try and modernize this project, I will do what I can or must to help in every way I can (responding to that pull).

These almost go hand in hand... so towards the goal of both.

I disagree.

Specification (documentation) is there to settle disputes of opinion (Judge Wapner, "get it in writing").

Back in the day ~1998 most browsers didn't follow the HTML specifications established by the w3c - they just did what they wanted to do (the blink tag was not a w3c specification but Microsoft did it anyway). That made it really hard on developers - but we tend to make things hard on developers it would seem - because our customers wanted their sites to look the same in all browsers. We, as developers, could blame the client (a lot of us did) or we could do honest and objective root cause analysis and blame (and complain to) the browser vendors for not following the standard established by the body who pretty much hold the keys to the kingdom of the internet.

So, we did complain...a lot.

Now, most browser vendors look to the specs to settle disputes while still trying to change and improve them.

• MS invented CSS Grid and put it in IE11 before submitting it to the w3c for adoption (IP transfer).
• The w3c modified the spec before approving it (they are the governing body after all).
• Now IE11 is no longer 100% compliant with the specification they started...while other browsers are close...with defects being found pretty regularly: https://github.com/rachelandrew/gridbugs - and MS has no intention of making IE11 compliant.
• Now IE11 will never support that specification, and is no longer supported by Microsoft (I think they don't even support security patches anymore)
• And, eventually, IE11 will be but a memory in the annals of history and most people won't even remember how horrible it was...just like IE6 (and the "IE6 Must Die" movement from web developers over a decade ago).

Right now, the core team of maintainers has decided on priorities laid out in #1006; namely:

• The only time-sensitive issues will be those of security vulnerabilities to ensure the safety of users unwilling or unable to move to a different product - for whatever reason.
• Meanwhile, we would also like to be able to know just how off the mark marked is based on the specifications; so, we can increase transparency and set expectations for new users trying to figure out whether or not to use marked in the first place (right now many new users think it's stable and can do what it promises without a hitch, which is not the case).

I believe we currently utilize extending header ids to our own flavor for easier document traversal.

Thank you for the code link! I really appreciate the implementation of the anchor generation (seems like ~L130). It's more robust than our example in the extensibility docs.

Having said that, I don't believe deprecating the capability marked currently has of adding ids to headers will negatively impact you or your users (tagging @intcreator, @davisjam, @UziTech, and @styfle to possibly crosscheck and verify). Further, while robust, the solution is not designed for modern web based solely on the HTML5 spec.

The HTML that gets generated by the extension should look something like this:

<h1>
  <!-- 1. The part that makes the browser jump to the right point on the page -->
  <a id="#something" rel="bookmark"></a> 

  <!-- 2. The string that will be presented to the user, after being sanitized - "checked for security" -->
  unsanitized something

  <!-- 3. The interactive element users actually use to get to #1 -->
  <a href="#something" class="anchor">  
  
    <!-- 4. A Font Awesome icon to display a glyph to users without vision impairments -->
    <i class="fa fa-link"></i>
  </a>
</h1>

Despite my earlier compliment regarding this solution being more robust than the one in our own documentation, it has some issues:

1. The biggest one for me is that it is not WCAG 2.0 Level A compliant as the Font Awesome icon (an image, essentially) does not have a text-based alternative.
2. Therefore, it should probably be hidden from people using assistive technologies per the w3c ARIA guidelines and Font Awesome 4's usage guidelines. Unfortunately, or fortunately, they are now on Font Awesome 5 and I can't seem to locate an example for you: aria-hidden=true is what you're looking for. (There's a problem here too, because this hides the interactive element from those using assistive technologies; so, you would still want to provide a text-based alternative.)
3. Having said that, to match the current GitHub experience, as the maintainer, you may want to consider doing some upgrades as GitHub seems to have (see image).

The reason I say you and, more importantly imho, your users should not be negatively impacted is because you are overriding the method that does this; so, you don't need the implementation details, you just need Marked to make a call to the method at some point.

If I'm reading the code correctly for marked, you should be able to find the call sight at L1047, the standard implementation at L822, which the code you presented is overriding anyway. The deprecation strategy could be to establish a contract for extenders of the marked renderer to optionally implement heading(), which we can then check to see if the method exists, and only if it does, actually make the call.

If we do this type of re-factoring and re-engineering to the marked code, we should be able to make the codebase smaller (thereby, making marked easier to maintain by most people with a cursory knowledge of JS), maintain the API you depend on, and make the core of marked potentially faster - the less process marked has to perform, the faster marked is.

Hence:

Over the years the community has added features to Marked based on issues and user feedback; while not always a bad thing, sometimes it can cause a project to go beyond sustainable scope for itself.

The problem with packages is that you typically don't just get the banana you wanted - you get the gorilla holding it as well - the deprecation roadmap is primarily to let users of marked pick and choose the gorillas they want to take on either themselves or from someone else.

Marked has a single responsibility laid out in our documentation, we are working at a sustainable pace to get there, based on pull. Further, we have milestones, epics, and issues to get us there.

If you would like it to happen faster, contribute issues and PRs asking for it. If you would rather it not, because things appear to be moving too fast, don't submit issues and PRs.

We will have three milestones: 0.x, 1.x, and 2.x

Per #1106

This particular deprecation event, will not occur until after 2.0, which could take a week, or may never come...because we base our actions on internal and external pull in order to follow some other principles near and dear to my heart:

Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.

Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.

See The Agile Manifesto.

With all sincerity, as was the case with this entire message, thank you for showing up. This comment has created enough pull for me to create an epic related to the 2.x milestone regarding feature deprecation; therefore, the issue itself will be closed (once the epic has been created).

Cheers,
Josh

ps. The code linked above indicates you use highlight.js and you should be able to avoid any negative impact to your users of that deprecation by implementing the concept found in our docs, which appears to have become "baked into" marked at some point (same with the heading thing). And, again, we could find a deprecation strategy to make sure the functionality exists.

[joshbruce]

Closing. See #1114 and #1115

[joshbruce]

Updated the following, by adding the parenthetical:

This whole thing can move as slowly or quickly as the users want it to but we're kinda at a point where folks need to decide on which option to go for (individually) and stick to it (we're getting a lot of mixed signals).

[Martii]

ps. The code linked above indicates you use highlight.js and you should be able to avoid any negative impact to your users of that deprecation by implementing the concept found in our docs, which appears to have become "baked into" marked at some point (same with the heading thing). And, again, we could find a deprecation strategy to make sure the functionality exists.

Introduction of jsdom introduced a rather large negative impact by about 15% per process per user on CPU usage just to get mentioning somewhat available (have other ideas but this isn't top priority). So separating the highlighting hook out of marked (which makes zero sense because it creates pre/code blocks for dynamic user content) would seriously impact productivity of any site including ours.

Euthanize: Focus on telling the world that eventually the package will no longer be available - period. (See LeftPad story by Kevlin Henney)

I'll skip that video. Theory doesn't always meet up with practicality not to mention it's way too long (and quite frankly boring and repetitive). ;) Also npmjs.com won't allow that... but as I mentioned about a month ago about the DDoS vulnerabilities on occasion of re's is a concern for us since not everyone out there does their best to be nice (this is putting it quite mildly).

Memorialize

This is where this project needs to decide if it is production worthy. If it's for educational purposes only then it should be advertised as such. I would prefer to see it continue to be developed but not hacked to bits (and it's somewhat bitrot that gets a boost every once in a while) because of lack of will for maintenance. You mention "baking"... I mention feature complete by adding whatever hooks are necessary to make it feature rich. If we wanted simplistic we wouldn't use some packages and use node itself (which we do in some areas)... same goes with MongoDB... some is core... some is packaged (hence the word here... marked is a package not a core item imho)

Most of the concern is the lack of fixes and a general move forward. I have been told that I'm infamous for being a contributing factor for getting change instigated... whether someone likes it or not... however stability with practicality is much better than instability with theory. I do like how you are outlining this projects goals and that is a major plus. Just remember not everyone is hyped up on coffee and sometimes it takes a while for ppl to respond (like me for example since I'm going through the flu this year right now so I'm just barely here e.g. set reasonable goals for responses... I tagged onto this about 28 days after because of computer network issues with SSD's failing for example... still working on those with data corruption that wasn't captured by the firmware... maintenance takes priority for me however I jumped back in during the beginning of my flu time just to put out some feelers... enough straying here). Back to marked.... Does marked want to be feature rich or an abstract for markdown? (rhetorical since commonmark already does this and there are already other feature rich packages that handle commonmark)

Font Awesome 5

Appreciate the heads up but I follow all dependencies including ones that have issues with later releases which is why our dependencies are fixed and static rather than dynamic in semver. Their current release is v4.x and when it rolls around we'll address that in a separate issue (definitely not here unless it merits it ;).

The w3c modified the spec before approving it (they are the governing body after all).

They are a current entity however not the only one... but that's off topic I think.

Anyhow... You put out a ton of responses which I'll examine later after I feel better. I will observe this project some more to see if it's still production worthy and I appreciate your responses.

P.S.

It's more robust than our example in the extensibility docs.

ecbf4b0 is a prime example of why one should try to utilize commit hashes. ;) Took me a bit to find that it was referencing something that changed. e.g https://github.com/markedjs/marked/blob/57681809993972c205bd48df26bd3c6cb46bf835/USAGE_EXTENSIBILITY.md