Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: new additives, non-nutritive sweeteners for new Nutri-Score #9005

Merged
merged 9 commits into from
Sep 20, 2023

Conversation

stephanegigandet
Copy link
Contributor

Added some newly listed additives from EU regulation: https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A02008R1333-20230720

Added detection of non nutritive sweeteners (needed for new Nutri-Score formula)

@stephanegigandet stephanegigandet requested a review from a team as a code owner September 15, 2023 14:52
@github-actions github-actions bot added API Issues related to the Open Food Facts API. More specific labels exist & should be used (API WRITE…) 🧬 Taxonomies https://wiki.openfoodfacts.org/Global_taxonomies 🥗 Ingredients 🧪 additives 🧪 tests 📖 Knowledge Panels https://wiki.openfoodfacts.org/Knowledge_panels 🥗🔍 Ingredients analysis https://wiki.openfoodfacts.org/Ingredients_Extraction_and_Analysis Food.pm 🧪 integration tests nutriscore labels Sep 15, 2023
@codecov-commenter
Copy link

codecov-commenter commented Sep 15, 2023

Codecov Report

Merging #9005 (68bf2f6) into main (28a3a84) will increase coverage by 0.13%.
Report is 17 commits behind head on main.
The diff coverage is 92.85%.

@@            Coverage Diff             @@
##             main    #9005      +/-   ##
==========================================
+ Coverage   46.12%   46.25%   +0.13%     
==========================================
  Files          64       64              
  Lines       19850    19883      +33     
  Branches     4795     4803       +8     
==========================================
+ Hits         9155     9197      +42     
+ Misses       9528     9522       -6     
+ Partials     1167     1164       -3     
Files Changed Coverage Δ
lib/ProductOpener/Tags.pm 41.10% <88.46%> (+0.49%) ⬆️
lib/ProductOpener/Food.pm 70.18% <100.00%> (+0.73%) ⬆️
lib/ProductOpener/Ingredients.pm 92.21% <100.00%> (+0.17%) ⬆️
lib/ProductOpener/Nutriscore.pm 67.18% <100.00%> (+2.08%) ⬆️

... and 4 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

Copy link
Member

@alexgarel alexgarel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great. Some comments on minor problems

Comment on lines 6055 to 6069
my %non_nutritive_sweeteners = (
"en:e950" => 1,
"en:e951" => 1,
"en:e952" => 1,
"en:e954" => 1,
"en:e955" => 1,
"en:e957" => 1,
"en:e959" => 1,
"en:e960" =>
1, # E960 is not listed in the Nutri-Score table as it was replaced by E960a/b/c/d, we assume it's E960a
"en:e960a" => 1,
"en:e961" => 1,
"en:e962" => 1,
"en:e969" => 1,
);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I find a bit odd to have that kind of data directly in the code ! (but I don't have a good cheap alternative).
Apart maybe from putting it in Config.pm ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An alternative approach would be to use a property in the additives taxonomy.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed the list of additives and created a new non_nutritive_sweeteners property in additives.txt instead.

Comment on lines 6085 to 6089
foreach my $additive (@{$product_ref->{'additives_tags'}}) {
if (exists $non_nutritive_sweeteners{$additive}) {
$product_ref->{with_non_nutritive_sweeteners} = 1;
last;
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think having a search_list and intersect_list methods for this kind of thing that comes over and over would be great

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@alexgarel I added generic functions in Tags.pm:

get_property_from_tags
get_inherited_property_from_tags
get_matching_regexp_property_from_tags

Comment on lines +6463 to +6464
elsif (defined $ingredient_ref->{percent_estimate}) {
$count += $ingredient_ref->{percent_estimate};
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

did you check it's not referenced in OpenAPI (if it is, change it there also, and add a description if there is none yet !)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in the doc we just said "estimate", not "minimum estimate", so the doc is correct.
I added the new "fruits-vegetables-legumes-estimate-from-ingredients" field.

taxonomies/additives.txt Show resolved Hide resolved
@@ -469,6 +520,7 @@ foreach my $test_ref (@tests) {
compute_serving_size_data($product_ref);
compute_field_tags($product_ref, $product_ref->{lc}, "categories");
extract_ingredients_from_text($product_ref);
extract_ingredients_classes_from_text($product_ref);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe we should add a comment at the place in the code that this test mimic to remember that if it is changed, it must be echoed here.

@github-actions github-actions bot added the 📚 Documentation Documentation issues improve the project for everyone. label Sep 20, 2023
@sonarqubecloud
Copy link

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
No Duplication information No Duplication information

@stephanegigandet stephanegigandet merged commit 691627f into main Sep 20, 2023
@stephanegigandet stephanegigandet deleted the nutriscore-sweeteners branch September 20, 2023 15:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🧪 additives API Issues related to the Open Food Facts API. More specific labels exist & should be used (API WRITE…) 📚 Documentation Documentation issues improve the project for everyone. Food.pm 🥗🔍 Ingredients analysis https://wiki.openfoodfacts.org/Ingredients_Extraction_and_Analysis 🥗 Ingredients 🧪 integration tests 📖 Knowledge Panels https://wiki.openfoodfacts.org/Knowledge_panels Tags 🧬 Taxonomies https://wiki.openfoodfacts.org/Global_taxonomies 🧪 tests 🧪 unit tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants