Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for any language: #5

Closed
wants to merge 2 commits into from

Conversation

dmitry-timofeev
Copy link
Contributor

@dmitry-timofeev dmitry-timofeev commented Feb 14, 2020

Auto-detect the source language and the alias to use using the Pygments API.

Use the file extension as the language class for
highlight.js appears to support file extensions as language
classes, see: https://github.com/highlightjs/highlight.js#supported-languages
@dmitry-timofeev dmitry-timofeev changed the title Add support for any language: [WIP] Add support for any language: Feb 14, 2020
@dmitry-timofeev
Copy link
Contributor Author

It appears codehilite is used widely as an markdown extension, and it does not work with all file extensions (e.g., "cs" does not yield good enough results for C# — must be "csharp").

I'll see if the correct info can be extracted using the Pygments API: https://pygments.org/

@rnorth
Copy link
Owner

rnorth commented Feb 16, 2020

Thanks for looking into this - it's a good idea. I'd be interested to see if there's a suitable source for the extension->language mapping.

It must be more accurate than the file extension.
@dmitry-timofeev
Copy link
Contributor Author

It seems Pygments provides such a mapping through a pair of get_lexer_for_filename and aliases.
As Pygments is used by codehilite, it must be the best source of such mapping. I updated the code to use it and added more tests.

@dmitry-timofeev dmitry-timofeev changed the title [WIP] Add support for any language: Add support for any language: Feb 17, 2020
self.assertEquals('rust', get_lang_class('HelloWorld.rs'))
self.assertEquals('docker', get_lang_class('Dockerfile'))
self.assertEquals('xml', get_lang_class('HelloWorld.xml'))
self.assertEquals('toml', get_lang_class('HelloWorld.toml'))
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, the test seems to fail on this line - I wonder if there's varying support for the toml extension in different versions of pygments.

>       self.assertEquals('toml', get_lang_class('HelloWorld.toml'))
E       AssertionError: 'toml' != 'none'
E       - toml
E       + none

Copy link
Contributor Author

@dmitry-timofeev dmitry-timofeev Jun 21, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems I used Pygments~=2.5 or newer, which version fails for you? Would it make sense to require a certain version of it, or just remove this test?

Copy link
Owner

@rnorth rnorth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test failure when running this through pytest...

@rnorth
Copy link
Owner

rnorth commented Oct 18, 2020

Looks like this is included in #15
Trying to unpick these and forming a larger PR, #22, where I'll resolve the conflicts that have arisen

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants