[Nightly] - OCR Editor Fails to Copy Text to Selected Field in Instructions

[aureateflux]

First Check

• This is not a feature request
• I added a very descriptive title to this issue.
• I used the GitHub search to find a similar issue and didn't find it.
• I searched the Mealie documentation, with the integrated search.
• I already read the docs and didn't find an answer.

What is the issue you are experiencing?

This behavior occurs on the "Instructions" tab in the OCR editor:

The directions in the OCR editor indicate that the workflow should be to use the rectangular selection tool to select a portion of the image that contains the text you want to capture, then select the field you want the text to go into, then click the selection box again to copy the text to the field.

Despite following these instructions (assuming I interpreted them correctly), the text is not copied into the field. The text appears below the image, indicating that the OCR is working, but no text is added to the selected field.

I verified this on both Chrome and Firefox using both my own container and the demo instance on mealie.io.

Hypothesis on why this is happening:

I think what's going on is that the Steps fields are not being made editable on the page as they are when manually creating a recipe or after importing from a URL. Clicking on the Step 1 field that is there by default displays the green checkmark, as it does when the recipe card isn't in edit mode. This makes it appear that the field is being selected, when in fact it's just hiding the step. Also missing are any options to delete, merge, or reorder the fields.

This makes the Bulk Add option unusable as well. While it does capture the text and allow you to edit it, saving the bulk add leaves you with a number of steps filled in with no way to modify them to reorganize, consolidate, or format the text. The logic on how it creates and populates the fields seems unpredictable, and since you have to close the OCR editor to edit any of the fields, it's difficult to wrangle it. Editing the fields needs to be possible in the OCR editor because you don't have access to the OCR data or the image (unless it's also the cover image) outside the OCR editor.

Deployment

Unraid

Deployment Details

No response

[aureateflux]

It should be noted that the text capture and copy does work correctly on the Recipe and Ingredients tabs, since the fields on those tabs are editable.

[Miroito]

You have the right understanding on how it works and why the instructions are broken. I'm actually surprised and happy to hear that someone else than me uses the OCR feature.
I wrote this OCR feature some time ago and a change in the recipe instructions code broke the edit mode for it. And since the Mealie frontend is not tested (at all as far as I know), this issue got through.

The instructions have been broken for months if I remember correctly. I looked inside the code hoping to find an easy fix multiple times but the new way to manage edit mode is so convoluted that I couldn't find an easy way. And unfortunately it's not the only feature I wrote on this Recipe page that is broken now just a few months later.

Sorry your issue has been hanging for 2 weeks with no answer. Though I will unfortunately not try and fix it because I don't trust that whatever is done now will always work in the future due to lack of testing.

In the meantime, I work around this issue by using the bulk add button to get the text, save the recipe, then reorganize the instructions in edit mode by copy pasting and correcting the poorly recognized text / format.

[michael-genson]

the Mealie frontend is not tested

The frontend gets tested for every PR that affects it. The nightly build is, by definition, the "latest" and "most unstable" version. It's maintained primarily by one person (@hay-kot) so it's not like there's a QA team. That's why Mealie relies on bug reports

it's not the only feature I wrote on this Recipe page that is broken

Then you should file a bug report

I will unfortunately not try and fix it because I don't trust that whatever is done now will always work in the future due to lack of testing

Sorry, I don't understand this logic at all, but that's your choice

[hay-kot]

I wrote this OCR feature some time ago and a change in the recipe instructions code broke the edit mode for it. And since the Mealie frontend is not tested (at all as far as I know), this issue got through.

Though I will unfortunately not try and fix it because I don't trust that whatever is done now will always work in the future due to lack of testing.

Should be noted that you wrote the PR and you provided no tests for the components that you wrote.

---

Given the discussion in the original PR,

That said I'm okay merging if you cleanup the items I commented on with the caveat that I may remove it later down the line if no one is interested in improving or maintaining it to make it more useful than existing tooling.

link to a discussion or issues where people can provide direct feedback on usability and the interface. Then we can get some actionable feedback and make some improvements from there.

I had assumed that you would have been involved in some capacity instead of drive by committing and huge feature with a half-baked implementation that was supposed to be improved. If you wanted to start a feature and have someone else finish it for you, I would have appreciated you being up front about that.

---

And unfortunately it's not the only feature I wrote on this Recipe page that is broken now just a few months later.

Do you have an issue that you can point to here?

---

This is Free and Open Source Software that I had thought you had an intention in helping maintain - at least this specific feature, it's clear that this is not the case. I'm going to remove the OCR feature during my next round of updates unless someone else is willing to step up to maintain it.

[Miroito]

I understand that you feel defensive but my criticism is only fair. How did the PR that broke the edit mode on the instructions got through if the frontend is properly tested?

My unfortunate feeling (and I could be the only one here, take it as it is, just one random guy on the internet) is that Mealie's quality has been declining since I started using it and contributing to it more than a year ago.

You're ultimately the owner of the project, you are free to do what you want with it. Be it removing the OCR feature all together, if you feel it doesn't suit your project's vision anymore. Or provide feedback, if it is half-baked as you said, I didn't see any bug reports / feature requests for ideas on things that could be improved. I always assumed this was merged and no one used it until I saw this issue today.

I'm not here to start any argument, please take this feedback as it is, use it or ignore it. It's your project in the end and remember that if I really was completely unhappy with Mealie I wouldn't be here.

[michael-genson]

Mealie's quality has been declining since I started using it and contributing to it more than a year ago

I'm hesitant to comment, because this is wildly off topic, but I can't possibly see how this could even plausibly be true, and I would be shocked to find someone else with this opinion. It's incredibly disingenuous to minimize the work being done on Mealie because you implemented a feature and then never bothered to make improvements to it, despite acknowledging several times that it needs more work.

In the past year Mealie has squashed 284 issues, added tons of features (including many of my favorite ones, such as heavy UI improvements, a complete overhaul of the search, the last made/recipe timeline, recipe scaling support, and loads of shopping list improvements), fixed some of the biggest long-standing bugs (PWA/token time, SSR issues, broken mobile pages, major performance issues) and improved code quality immensely to the point where the old frontend code is completely unrecognizable.

The common thread with all of the successful changes done to Mealie (and any open source project, really) is community-driven changes, feedback, and maintenance. No one stays around an open source project forever, and that's totally okay, expected, in fact. But, to drop a huge feature, refuse to iterate on it, and respond to feedback with "I will unfortunately not try and fix it because I don't trust that whatever is done now will always work in the future" is very disrespectful to the project you claim to support.

[JvdMaat]

I just used the OCR feature and noted the same issues. Nothing was imported automatically, but I was able to use the tools to populate all the fields. Also noted that the settings (landscape, public recipe, disable comments, etc) did not inherit the group settings. So I'm logging a separate bug for that, but if that's all related, it can be pulled in to this one.

[JvdMaat]

Also of note. I initially tried to do this on my android phone, but the text select feature was unusable. I had to use my desktop so my mouse could draw rectangles on the text. Not sure if there was a trick I didn't find, or if it just doesn't work on mobile.

[Plevuus]

I'm going to remove the OCR feature during my next round of updates

😭 please don't.
As a mere end-user I would be very disappointed if the OCR functionality would disappear. I guess I'm a bit of a-typical user. I do not cook a lot from internet recipes, but we do have >200 cookbooks (on old-fashioned paper) and I enter the recipes we cooked with the addition of a picture, some extra info, the original scan and some notes do make it better next time we cook it. Almost all of the many recipes are entered via the OCR feature. If I would have to type them, the effort would not be worth the value and I would likely move away from Mealie. I agree that there could be many improvements to this feature, but I developed a method to work around all the quirks and in the end it just works for me. Unfortunately I do not have the skills to pick up the development of this feature. All I could do test it, file accurate bug reports and feature/improvement request and maybe sent some occasional support ;-)

[michael-genson]

FYI we have decided to remove OCR from Mealie. You can read more on #2838.

If you need to use OCR to import recipes, I recommend using external tools such as this online OCR tool to extract the text, which you then copy-paste into Mealie.

[Plevuus]

😞 Too bad.
I have to agree that the current implementation was half-baked and that suggested online tool does indeed a better job on the OCR itself. However, it's again more actions and dependence on online tool that might disappear overnight. I was just hoping that mealie would go the other direction and go for a full baked feature instead of removing the half-baked feature. It think that cookbookmanager.com sets a great example of how powerful good OCR features can be for a recipe manager.
Having expressed that, I also want to thank every contributor to mealie for creating, maintaining and improving it. I will keep on using it, at least for now.

[boc-the-git]

Thanks for the feedback @Plevuus.
This isn't to say we'll never have OCR in Mealie, just that at this time (for the reasons mentioned on the PR), removing it is the best option. Those things absolutely could change in future.