Commas are not represented as a word node

[hangyav]

Hi,

Thanks for the awesome tool. I'm working on a project where I'd like to extract the text content of latex files using TS (https://github.com/hangyav/textLSP). I noticed that commas aren't part of the preceding word's node but instead only part of the parent node, in contrast to how dots behave. Here's an example:

Is that a bug or intentional for some reason? Thanks a lot!

[pfoerster]

Is that a bug or intentional for some reason?

Its mainly this way for historic reasons. This grammar was extracted from texlab and texlab does not really care about dots and the general sentence structure. However, commas are significant because they can be used to separate arguments to commands like \ref or \cite so they need special treatment. I guess it wouldn't hurt to extract the dot into a separate token, too.

[hangyav]

I see, thanks for the clarification! Dots are actually fine for me as they are now. What would be great for me is to have commas in some type of leaf nodes as well (either together with the preceding word or as an individual node) instead of contained only by one of the-non leaf nodes (the section node in the example).