Create po4a branch for changes/corrections to EN and translations

[ignotus666]

As you know I've been setting up a branch using po4a to see if a better workflow can be implemented for translations. A key part of getting the branch ready is keeping all the wiki documents and their .po counterparts level with the "official" website docs. Right now they are at this state, but every time someone adds something to a document, I have to manually edit a number of files on my branch to stop them drifting out of sync. The alternative is waiting until a decision is made and aligning everything in bulk then, but that presents its own challenges and is a very time-consuming process. Either way, I can fix them as they happen or wait and do them all at once, but the work is still the same. If we were starting from scratch it would have been much easier, but as we had a significant amount of already-translated material, the process has been/is painstakingly tedious.

Now that the release is out and the website is live, I would like to propose making the switch while we have a blank slate and before stuff starts to get added to the 'changes' branch again. The transition wouldn't be all that traumatic as the only difference is that corrections to translations would be made using the .po files instead of directly editing .md files. Changes to EN files would happen as they always do. It may seem premature but I'm confident everything works as it should (I've been using it for a couple of weeks now without issue) and if it came to the worst, reverting to the previous system would just mean deleting a folder and a couple of files. But doing the switch now would save me from constantly playing catch-up... and my sanity. The hectic release process coupled with trying to keep my branch in a useful state has meant that when I saw a request today for someone to revise their translations I just moaned and thought... OMG please... no! ;-)

All the docs on the po4a branch are level with what's on the website now. To make the switch, it would only be a matter of "transplanting" the following from my po4a branch to 'changes' or whatever branch is created for the purpose:

• The wiki folder (replacing the one there).
• The two po4a scripts in the root directory (po4a-update-templates.sh and po4a-create-all-targets.sh).
• The 'translator-files' folder in the root directory.

The branch can't be used as-is because I've only updated the necessary files. I followed @ann0see's suggestion and eliminated the en- prefixes from all files (they are not necessary), and moved the translator-files folder out of wiki/ so translators have no need to go there and aren't tempted to edit .md files. There are readme docs in both wiki/ and translator-files/ with some explanations of how things would be done - though they are still WIP I think they are enough to get us going. The readme in the root directory would also need editing but that could be done in the coming days. The scripts haven't been automated via GH actions yet but that's not a biggie - I'll run them myself if need be until that can be done.

So what do you reckon? Am I getting ahead of myself? Doubts, questions?

[ann0see]

Yes. Moving to the process now would be the best move, I think.

If the branch can be deployed right now (and is in sync with the state on the Release branch, I would strongly suggest to:

1. Freeze release and changes
2. Merge your work to release (this will make the process easier in the mid term but dangerous for the short term; we can however always revert problematic commits)
3. This would result in your work being automatic synced back to changes
4. Remove/disable the automatic merge process from release to changes since this might eliminate unwanted merges
5. rename changes to nextrelease or next-release to avoid more confusion

[ann0see]

I think there are some questions regarding the process; I‘m sure we’ll sort them out soon.

e.g. Can we edit a translation on the release branch if there’s a typo?

[ignotus666]

It's not fully in sync with anything though - it was based on 'translate' from about a week ago and diverged from there. What's in sync are the wiki documents, but not the rest. The transition would involve manually adding the files I listed above.

e.g. Can we edit a translation on the release branch if there’s a typo?

Yes. You edit the typo in the corresponding .po file and run po4a-create-all-targets.sh

[ignotus666]

More problematic are typos in EN files. A correction to EN and then running po4a-update-templates.sh will cause the whole paragraph to appear as untranslated in all the .po files. They will insert a fuzzy match (if its 80% or more, which a typo correction will always be) but it has to be validated manually. This means all EN documents will have to be thoroughly proofread before running the script to avoid this.

[ann0see]

Sorry, should have written „as soon as“ instead of if (Could be a typical mistake for German speakers).

Could you please create a branch on your repo based on release, add the folders, commit & push it and then open a pull request to release?

You can have a look at https://github.com/jamulussoftware/jamulus/blob/master/TRANSLATING.md#command-line-git-tools

and

https://github.com/jamulussoftware/jamulus/blob/master/TRANSLATING.md#get-the-most-up-to-date-files-and-set-up-a-new-branch (of course, without all the translation stuff for the App and with jamulussoftware/jamuluswebsite instead of jamulussoftware/jamulus) for help with git.

[ignotus666]

Ok, I'll do it tomorrow morning.

[hoffie]

The hectic release process coupled with trying to keep my branch in a useful state has meant that when I saw a request today for someone to revise their translations I just moaned and thought... OMG please... no! ;-)

I agree, several parts in the process did not work out as expected for very different reasons, so I fully understand that feeling.

So what do you reckon? Am I getting ahead of myself? Doubts, questions?

The work you've done, the scripts you've written and the process as planned sound really good to me. I have to admit though that I haven't had a look in detail yet (and can't promise when I will). In general, I would be in favor of doing the change and doing it now.
However, I think there are two important roles whose members should confirm whether the process works for them:

• Authors of the English content: That would be mainly @gilgongo and @ann0see (I think), although this job may be more distributed in the future (e.g. if we ask feature contributors to add documentation themselves).
• Translators: Besides you, that would be @Helondeth, @ewarning, @jujudusud, @trebmuh and @dzpex

More problematic are typos in EN files. A correction to EN and then running po4a-update-templates.sh will cause the whole paragraph to appear as untranslated in all the .po files.

I think this may be solvable by manually editing the English string in the .po files, wouldn't it? We (well, mainly @softins) did similar things on the App translation stuff (which does not use .po files, but the process is comparable, I think).

[jujudusud]

Ok from my side.
I will test the process with linguist certainly tomorrow. I will let you know how it is going for me.

[gilgongo]

More problematic are typos in EN files.

BTW can we tell p04a to exempt certain paragraphs from translations? I'm thinking markdown code blocks mainly, but if for example if there's a paragraph with a file name in it or something and that needs updating (but not translating)?

[ignotus666]

If you update something, by default, without intervention by translators, the English paragraph will be shown, so in that case it's no problem.

[gilgongo]

Sorry I meant a change like this along with making changes to some other paras that do need translation. How does po4a-update-templates.sh know not to mark that paragraph as now only partially translated and then knock it all back to the English?

[ignotus666]

I think there are two important roles whose members should confirm whether the process works for them:
Authors of the English content

Of course their input is necessary, but their workflow will barely change at all. Editing/adding content will continue as up until now, on .md files.

I think this may be solvable by manually editing the English string in the .po files, wouldn't it?

This would work but I'm not sure if editing every raw .po file for every language by hand (.po editors won't let you change source segments) is any easier than just opening them with an editor and validating the inserted fuzzy matches.

[ignotus666]

Sorry I meant a change like this along with making changes to some other paras that do need translation. How does po4a-update-templates.sh know not to mark that paragraph as now only partially translated and then knock it all back to the English?

Paragraphs that have changed by more than 20% will appear as empty segments, those by less will have a fuzzy match inserted - in the editor. But without intervention, all of them will display the untranslated English version in the final document. Fuzzy match insertion is just to make the translator's work easier, as it's assumed you'll be able to "recycle" much of it.

Don't know if that answers your question.