Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Return numbers as numbers (and not strings) in the API, store them as numbers in MongoDB, add tests #4389

Open
Tracked by #5523
stephanegigandet opened this issue Oct 21, 2020 · 3 comments
Labels
API Issues related to the Open Food Facts API. More specific labels exist & should be used (API WRITE…) MongoDB We have 2 mongodb collections: one for current products, and one for obsolete products ⏰ Stale This issue hasn't seen activity in a while. You can try documenting more to unblock it. typing

Comments

@stephanegigandet
Copy link
Contributor

We have this long running issue (see #639) of numbers being sometimes stored as strings in MongoDB or in the JSON returned by the API. We have been fixing many examples of it, but they keep coming back.

It has to do with the way Perl handles scalars: they are automatically converted to strings or to numbers as needed, based on what is being done with them.

So for instance if we use a number as a string (e.g. to display it, to output it to a file, to log it etc.) it will internally become a string.

Unfortunately, the whole world does not use Perl, so it is causing issues for apps that parse the JSON and expect a defined type for each field.

So we need to:

  1. Define exactly which fields are strings and which fields are numbers
  2. Look for, investigate and fix any inconsistent types
  3. Fix all inconsistent types in the database
  4. Whenever possible, add tests to make sure we don't break things (in particular for the API where we can do some processing before returning values, and where numbers can thus become strings)
@stephanegigandet stephanegigandet added the 🐛 bug This is a bug, not a feature request. label Oct 21, 2020
@aleene
Copy link
Contributor

aleene commented Oct 21, 2020

Is it possible to explicitly set the types for each and very field, instead of relying on Perl behaviour.

@stephanegigandet stephanegigandet added API Issues related to the Open Food Facts API. More specific labels exist & should be used (API WRITE…) MongoDB We have 2 mongodb collections: one for current products, and one for obsolete products labels Oct 21, 2020
@hangy
Copy link
Member

hangy commented Nov 1, 2020

Is it possible to explicitly set the types for each and very field, instead of relying on Perl behaviour.

Looks like it's possible with explicit typecasting. There's also document schemas in MongoDB, but I don't know how well this would work with all the dynamic language based fields in ProductOpener.

@github-actions github-actions bot added the ⏰ Stale This issue hasn't seen activity in a while. You can try documenting more to unblock it. label Jan 31, 2021
@teolemon teolemon added typing and removed ⏰ Stale This issue hasn't seen activity in a while. You can try documenting more to unblock it. labels Jul 19, 2021
Copy link
Contributor

This issue has been open 90 days with no activity. Can you give it a little love by linking it to a parent issue, adding relevant labels and projets, creating a mockup if applicable, adding code pointers from https://github.com/openfoodfacts/openfoodfacts-server/blob/main/.github/labeler.yml, giving it a priority, editing the original issue to have a more comprehensive description… Thank you very much for your contribution to 🍊 Open Food Facts

@github-actions github-actions bot added the ⏰ Stale This issue hasn't seen activity in a while. You can try documenting more to unblock it. label Feb 11, 2024
@teolemon teolemon moved this to To discuss and validate in 🍊 Open Food Facts Server issues Apr 23, 2024
@teolemon teolemon removed the 🐛 bug This is a bug, not a feature request. label Oct 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
API Issues related to the Open Food Facts API. More specific labels exist & should be used (API WRITE…) MongoDB We have 2 mongodb collections: one for current products, and one for obsolete products ⏰ Stale This issue hasn't seen activity in a while. You can try documenting more to unblock it. typing
Projects
Status: To discuss and validate
Status: To be triaged
Development

No branches or pull requests

4 participants