Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Text_alignment / OCR] Syllables not being picked up for MS73 #1215

Open
JoyfulGen opened this issue Oct 10, 2024 · 2 comments
Open

[Text_alignment / OCR] Syllables not being picked up for MS73 #1215

JoyfulGen opened this issue Oct 10, 2024 · 2 comments
Labels
bug ON HOLD Text Alignment Issues pertaining to text alignment Rodan job

Comments

@JoyfulGen
Copy link
Contributor

JoyfulGen commented Oct 10, 2024

UPDATE: This might be in part due to a user mistake (here we are again), so please hold!

I've been starting to run some e2e OMR workflows with MS73 folios and the text part of the process doesn't seem to be working. Normally, the original image is separated into layers, and the text layer gets sent to the Text_Alignment job, which uses OCR to roughly find the syllables and then match them with the correct text that we provide. In Neon, the syllables will look like this (this is a Salzinnes folio):

Salzie good syllables example

However, this process doesn't seem to be working for MS73. So far, @kyrieb-ekat got this (enjoy the numbers):
Kyrie text nuggets

And I got no syllables at all:
MS73 054 no syllables

Because the syllable text is directly related to how the neumes are grouped into syllables, these errors result in the syllable groupings being completely wrong, which lengthens the correction time quite a bit.

I ran an e2e OMR workflow with an Einsie folio and the syllables were perfect, so this seems to be an MS73-specific problem. Could it simply be that the training model we've been using for Salzinnes and Einsie doesn't work for MS73? In the Text_Alignment job, the training model is built directly into the job, so I don't think this is something that I can change.

@JoyfulGen JoyfulGen added Priority: HIGH Text Alignment Issues pertaining to text alignment Rodan job bug and removed Priority: HIGH labels Oct 10, 2024
@JoyfulGen
Copy link
Contributor Author

ANOTHER UPDATE: This was indeed in part due to user error (you can always count on me). I mistakenly assigned the wrong layer output to the input of the Text_Alignment job, which is why my syllables came up completely empty.

However! Kyrie did not make that mistake, so her result is accurate. I tried running a couple more workflows after fixing my mistake and I'm getting something similar. There are syllables, but they are far too few and those that are there are not correct. I'm not sure at the moment what this is due to; it's possible that as our glyph classification training data improves, the syllable problem will lessen. I'll put this issue on hold for now until we know more.

@kyrieb-ekat
Copy link

I'm going to also be retracing some of the previous steps done on this, and test a few more pages of MS73. Also, to look into OCRopus and see what the text_alignment thought processes for the OCR models were.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug ON HOLD Text Alignment Issues pertaining to text alignment Rodan job
Projects
None yet
Development

No branches or pull requests

2 participants