How to enable markdown extensions?

[MPvHarmelen]

Expected Behavior

I am looking for some configuration switch to tell Python Markdown which extensions to use, specifically something like PyMdown Extensions to enable task-lists and strike-through.

Actual Behavior

The list of extensions seems to be hard-coded and not amendable using templates (or consequently command-line arguments).

See:

pdoc/pdoc/html_helpers.py

Line 69 in 1709915

_md = markdown.Markdown(

Steps to Reproduce

N.A.

Additional info

• pdoc version: 0.7.4

[kernc]

It's non-amendable as nobody needed/proposed it yet. Seems reasonable. I think template tunables would allow for greater configurability (e.g. extension configs). A PR would be much appreciated. 👍

[MPvHarmelen]

I could spend some time on implementing this, but have no idea what you mean by "template tunables" and haven't worked with mako templates before. Could you point me in the right direction? Specifically I'm not quite sure in which of the templates this should be added.

I'm assuming you would want to move the whole list of extensions and config into some template, so users can also choose to leave out extensions, or are there any extensions that pdoc depends on, so must be kept in?

[kernc]

Great! Sorry, I meant config.mako template, which is consulted before rendering the real html/text/pdf templates.

users can also choose to leave out extensions, or are there any extensions that pdoc depends on

I'm not sure, best run the tests and see. toc is tested for sure. Re appending, I guess either way is fine, although the current defaults seem also quite fine, and I don't want to pollute user-facing config with pointless and never-to-be-customized internals. 🤔
Either way, with extensions defined in a template, you'll need a way to efficiently enable/cache them on the markdown converter instance, somewhat like the MathJax preprocessor is if latex_math config tunable is set:

pdoc/pdoc/html_helpers.py

Lines 356 to 365 in 1709915

	# Optionally register our math syntax processor
	if not latex_math and _MathPattern.NAME in _md.inlinePatterns:
	_md.inlinePatterns.deregister(_MathPattern.NAME)
	elif latex_math and _MathPattern.NAME not in _md.inlinePatterns:
	_md.inlinePatterns.register(_MathPattern(_MathPattern.PATTERN),
	_MathPattern.NAME,
	_MathPattern.PRIORITY)
	md = to_markdown(text, docformat=docformat, module=module, link=link)
	return _md.reset().convert(md)

I'm afraid creating a new markdown converter instance for each piece of docstring will be too inefficient.

The alternative is a CLI switch --markdown-extension-configs taking JSON dict with its keys marking extra extensions to enable. I maybe like this simpler option ...

[MPvHarmelen]

Okay, let's append things then. I think I like a template-based solution, as it keeps the functionality for CLI and programmatic usage the same.

Would handling it exactly the same as the LaTeX maths be an acceptable solution?
That would mean adding a list(-like?) variable to the config.mako template, propagating that to html_helpers.to_html in html.mako (like latex_math) and then registering the extensions exactly where the patterns for LaTeX maths are also registered.

pdoc/pdoc/templates/html.mako

Lines 17 to 18 in 1709915

	def to_html(text):
	return _to_html(text, module=module, link=link, latex_math=latex_math)

A problem with this solution is that you can't deregister an extension like you can deregister patterns, meaning that you can't use different extensions during the same run. Is this a problem? If so, the global Markdown instance must be made local in some way.

pdoc/pdoc/html_helpers.py

Lines 69 to 90 in 1709915

	_md = markdown.Markdown(
	output_format='html5',
	extensions=[
	"markdown.extensions.abbr",
	"markdown.extensions.attr_list",
	"markdown.extensions.def_list",
	"markdown.extensions.fenced_code",
	"markdown.extensions.footnotes",
	"markdown.extensions.tables",
	"markdown.extensions.admonition",
	"markdown.extensions.smarty",
	"markdown.extensions.toc",
	],
	extension_configs={
	"markdown.extensions.smarty": dict(
	smart_dashes=True,
	smart_ellipses=True,
	smart_quotes=False,
	smart_angled_quotes=False,
	),
	},
	)

Some stuff for (future) reference

Markdown.registerExtensions, to, well, register extensions...

https://github.com/Python-Markdown/markdown/blob/1de3083d82a5245015f0f0b9471a965916134d6d/markdown/core.py#L110-L120

[kernc]

exactly where the patterns for LaTeX maths are also registered.

pdoc/pdoc/templates/html.mako

Lines 17 to 18 in 1709915

	def to_html(text):
	return _to_html(text, module=module, link=link, latex_math=latex_math)

I must say, this looks one hella ugly piece of hack, sending values back and forth like insane. I'm not sure that wasn't a mistake. I wonder if most preferences couldn't instead be applied earlier, like in _render_template() where the config is read? 🤔

pdoc/pdoc/__init__.py

Lines 93 to 121 in 1709915

	def _render_template(template_name, **kwargs):
	"""
	Returns the Mako template with the given name. If the template
	cannot be found, a nicer error message is displayed.
	"""
	# Apply config.mako configuration
	MAKO_INTERNALS = Template('').module.__dict__.keys()
	DEFAULT_CONFIG = path.join(path.dirname(__file__), 'templates', 'config.mako')
	config = {}
	for config_module in (Template(filename=DEFAULT_CONFIG).module,
	tpl_lookup.get_template('/config.mako').module):
	config.update((var, getattr(config_module, var, None))
	for var in config_module.__dict__
	if var not in MAKO_INTERNALS)
	known_keys = set(config) | {'module', 'modules', 'http_server', 'external_links'} # deprecated
	invalid_keys = {k: v for k, v in kwargs.items() if k not in known_keys}
	if invalid_keys:
	warn('Unknown configuration variables (not in config.mako): {}'.format(invalid_keys))
	config.update(kwargs)
	try:
	t = tpl_lookup.get_template(template_name)
	except TopLevelLookupException:
	raise OSError(
	"No template found at any of: {}".format(
	', '.join(path.join(p, template_name.lstrip("/"))
	for p in tpl_lookup.directories)))
	try:
	return t.render(**config).strip()