Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: 3897 - ingredients and packaging with multilingual OCR image+text #3911

Merged
merged 2 commits into from
Apr 27, 2023

Conversation

monsieurtanuki
Copy link
Contributor

What

  • This PR refactors the multilingual management already in place for product name in "Add basic details" page.
  • This PR also use that same multilingual management in the OCR pages (ingredients and packaging).
  • It looks successful on ingredients (e.g. for products 7300400481588 or 3760091722508)
  • But I'm afraid that there are currently no such things as multilingual packagings (cf. 'packaging_text_languages') on the server - at least I couldn't find an example.
  • For ingredients, I had to add a product field ('ingredients_text_languages'): if you don't see the language selector in the ingredients OCR page, please refresh the product first.
  • As for packaging, unless you find a product with multilingual packaging on the server, you won't be able to find the language selector on the OCR page.

Screenshots

App in French, on the ingredient OCR page, by default with the French settings:

French text before OCR French text after OCR
Screenshot_2023-04-26-15-13-57 Screenshot_2023-04-26-15-14-06

Same app in French, on the ingredient OCR page, after setting the page language to Italian:

Italian text before OCR Italian text after OCR
Screenshot_2023-04-26-15-14-17 Screenshot_2023-04-26-15-15-16

Part of

Files

New file:

  • multilingual_helper.dart: Helper for multilingual inputs (e.g. product name).

Deleted file:

  • ocr_widget.dart: moved code to edit_ingredients_page.dart, for better state refresh

Impacted files:

  • add_basic_details_page.dart: refactored with new class MultilingualHelper
  • edit_ingredients_page.dart: added multilingual management; moved code from ocr_widget.dart
  • ocr_helper.dart: refactored with multilingual management
  • ocr_ingredients_helper.dart: refactored with multilingual management
  • ocr_packaging_helper.dart: refactored with multilingual management
  • product_query.dart: added multilingual ingredients field; removed duplicate field

New file:
* `multilingual_helper.dart`: Helper for multilingual inputs (e.g. product name).

Deleted file:
* `ocr_widget.dart`: moved code to `edit_ingredients_page.dart`, for better state refresh

Impacted files:
* `add_basic_details_page.dart`: refactored with new class `MultilingualHelper`
* `edit_ingredients_page.dart`: added multilingual management; moved code from `ocr_widget.dart`
* `ocr_helper.dart`: refactored with multilingual management
* `ocr_ingredients_helper.dart`: refactored with multilingual management
* `ocr_packaging_helper.dart`: refactored with multilingual management
* `product_query.dart`: added multilingual ingredients field; removed duplicate field
@codecov-commenter
Copy link

Codecov Report

Merging #3911 (44c33d3) into develop (4e9349c) will decrease coverage by 0.02%.
The diff coverage is 0.00%.

@@             Coverage Diff             @@
##           develop    #3911      +/-   ##
===========================================
- Coverage    10.66%   10.64%   -0.02%     
===========================================
  Files          271      271              
  Lines        13455    13478      +23     
===========================================
  Hits          1435     1435              
- Misses       12020    12043      +23     
Impacted Files Coverage Δ
..._app/lib/pages/product/add_basic_details_page.dart 0.00% <0.00%> (ø)
...h_app/lib/pages/product/edit_ingredients_page.dart 0.00% <0.00%> (ø)
...oth_app/lib/pages/product/multilingual_helper.dart 0.00% <0.00%> (ø)
...kages/smooth_app/lib/pages/product/ocr_helper.dart 0.00% <ø> (ø)
..._app/lib/pages/product/ocr_ingredients_helper.dart 0.00% <0.00%> (ø)
...th_app/lib/pages/product/ocr_packaging_helper.dart 0.00% <0.00%> (ø)
packages/smooth_app/lib/query/product_query.dart 8.10% <ø> (ø)

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

Copy link
Member

@teolemon teolemon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@monsieurtanuki
Copy link
Contributor Author

Hi @teolemon, and thank you for your comment and example.

As far as I'm concerned, the PR seems to work too for multilingual packaging. But:

  1. the image was not relevant for OCR, therefore it failed (in French)
  2. my smartphone does not support the other language, that was Afar, and therefore the language could not appear as pre-selected as it virtually does not exist (at least for my smartphone) (for the record 76 off languages are recognized and 107 rejected) (feel free to open an issue if relevant)
  3. I managed to add a translation (directly, no OCR), in Dutch
language selector with only French failed OCR in French input in Dutch
Screenshot_2023-04-26-19-16-07 Screenshot_2023-04-26-19-16-17 Screenshot_2023-04-26-19-20-28

https://fr.openfoodfacts.org/api/v3/product/5400141625714?lc=fr&fields=packaging_text_languages

{
	"code": "5400141625714",
	"errors": [],
	"product": {
		"packaging_text_languages": {
			"aa": "test afar",
			"fr": "test fr tanuki",
			"nl": "test nl tanuki"
		}
	},
	"result": {
		"id": "product_found",
		"lc_name": "Produit trouvé",
		"name": "Product found"
	},
	"status": "success",
	"warnings": []
}

@teolemon teolemon requested review from g123k and M123-dev April 27, 2023 08:09
@monsieurtanuki monsieurtanuki merged commit e5bf1cb into openfoodfacts:develop Apr 27, 2023
@monsieurtanuki
Copy link
Contributor Author

Thank you @teolemon for your approval: that's another step towards a fully multilingual app.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging this pull request may close these issues.

3 participants