Nabc ~ handling broken

[jakubjelinek]

Just installed 4.1 and noticed that tilde handling is broken.
...
nabc-lines: 1;
...
Test(g|po~)
displays normal porrectus like Test(g|po) rather than the liquescent diminutive one, something doesn't change the catcode to parse it properly. The *.gtex file contains correct stuff.

[eroux]

Will you have time to look at this or do you need someone else to do it?

[jakubjelinek]

I'm afraid I'll need help, this are of TeX I'm constantly struggling with.
Even using
\GreNABCChar{po~}%
somewhere after including a score doesn't show the right glyph.
If I apply patch
(which I believe was roughly what worked at some point), then \GreNABCChar{po~} already works, but \GreNABCNeumes{1}{po~} as part of the *.gtex still does not. So I'm lost on where the catcode is needed and why.

[jakubjelinek]

And it would be nice to cover this case in the testsuite once it is fixed, so that it doesn't reappear, the ~ parsing in TeX is a nightmare.

[jakubjelinek]

Worst case, if nothing else works, we could let gregorio replace the ~ in nabc strings with some other character (one that doesn't appear elsewhere in the nabc grammar, and doesn't have so many catcode issues as ), and adjust the nabc lua parser to treat ~ the same as the other character (i.e. replace all occurrences of the other character with ~ at the lua level before further parse_nabc processing).
Or use the catcode stuff, but store the argument into some lua variable earlier, and let parse_nabc just pop the argument ouf of that variable.
I've tried to replace the
\GreNABCNeumes{1}{po}%
in the *.gtex file with
{
\catcode`=12{}%
\GreNABCNeumes{1}{po}%
}
(that could be something gregorio could do if it detects ~ in nabc string), but unfortunately that doesn't work either.

[jakubjelinek]

Untested hack
Though of course, it would be cleaner if the ~ could be passed through the GregorioTeX macros instead. BTW, in the GregorioNabcRef document it seems to work properly, but it uses the two macros approach and doesn't use ~ in the scores, just in other macros.

[eroux]

I think you hack is what would work best. I've reduced the catcode problem to:

\def\mydefmacro#1{#1}

\def\macrowithtild{%
    \begingroup\catcode`\~=11{}%
    \domacrowithtild%
}

\def\domacrowithtild#1{%
    #1\endgroup %
}

\macrowithtild{a~b}

\mydefmacro{\macrowithtild{a~b}}
\bye

(to be compiled with luatex, not lualatex)

The problem is that we're using the second form (which doesn't work): we use \GreNABCNeumes inside \GreSyllable. If you use GreNABCNeumes alone it will work but if you use it inside a macro it won't.

Can you test your hack?

[henryso]

Alternately, It might be possible to use a variation of the code I use for sending headers directly into Lua. I escape troublesome characters into something that escapes into a character under Lua but doesn't have issues on the TeX side.

[eroux]

I'm not really sure about that... I think we would run into the exact same problem

[eroux]

oh sorry, you're right, I think that's even better indeed

[henryso]

Looking at the code, I also noticed that gregoriotex-nabc.tex uses \newcommand, and that should be changed to \def.

[eroux]

the nabc parser just needs to be able to parse \xyz strings where x, y and z are numbers

(you're right about the \newcommand that should be changed)

[jakubjelinek]

FYI, I've tested my patch and it seems to work.
As for \GreSyllable, would it be possible to break it up into the 2 macros (start + body) and change catcode in there? Does anything in GreSyllable arguments need the normal TeX handling of ~? If it is in some other GregorioTeX macro, would it work to change catcode of ~ in GreSyllable to make it normal character and then again in whatever macro might need the TeX handling of ~?

For the testsuite, we probably want:
potilde(c|po~) choralsign(e[cn:po~])
to cover both normal nabc and choral signs.

[henryso]

@eroux Lua will evaluate \xyz automatically. If we use my alternative, \grenabccharno should simply pass #1 (without escaping it) after processing (in gregorio) by tex_escape_text.

[eroux]

Indeed, I forgot to remove the escaping in my tests. Well, this solution is really good, let's use it!

[eroux]

I won't be able to work on this today anymore, I'll come back to this tomorrow if not done already

[henryso]

I'll prepare a pull request for this later this evening.