Display insight / predictions etc for a product or a particular filter

[alexgarel]

What

• It's quite difficult to know every predictions / insight that robotoff made for a product, its images included

Proposed solution

• I would like to have a simple url giving all the predictions / insights / annotations that robotoff has about a product.
  Also which were applied, when etc.
• This should be geared towards analyze by a human to understand what robotoff did, for debugging / auditing etc.
• Currently in hungergames we display the insights : (at https://hunger.openfoodfacts.org/insights)

But we would like to add predictions and all things related to images.

Add apis akin to /api/v1/insights (ProductInsightResource), with compatible filters (on product and type) for:

1. Predictions
2. Images
3. ImagePrediction - with info about image
4. LogoAnnotation - with info about image and annotation

The idea would be to use those APIs to append more tables in hunger game to have a complete view of why robotoff is predicting something.

Also add those filters to specific views:

• for Images filter out images that have at least one ImagePrediction
• for ImagePrediction filter out the one that have a corresponding logo annotation

Those filters will avoid repeating informations that are already in ImagePrediction or LogoAnnontation in hunger game.

• Related ticket for the Hunger Games part: Complete the insights view to add additional info openfoodfacts-hungergames#415

[ram690514]

yeah, its good to have an html page that shows all predictions/insights . It could be also used to show the recent updates and changes in the system

[ram690514]

I am interested to work on this issue, if no one is currently working on it

[alexgarel]

No let's go @ram690514 I think it's a good issue to start with !

[ashish-reddy-20-08]

Hey @alexgarel , is ram working on this, if not I can take it up, I will be able to manage all the insights and prediction stuff, just would need help at the last to post it in HTML.

[alexgarel]

I don't think @ram690514 is currently working on this.

I think the best is to have the data as json + as html on another endpoint (the html view can just exploit the same method to get data and build the html)

Here are the point from most difficult to least difficult (you can start doing them incrementally and submit the PR as draft).

For the json:

1. list all ProductInsight for this barcode - sort them in a predictable way : type - timestamp - id
2. list all Prediction for this barcode (same sorting order)
3. list all LogoAnnotation but with corresponding data from ImagePrediction and ImageModel: there again same order

For the html, you should find a way to present it all (sortable table ? Should we use vuejs ? Maybe @alexfauquette has a though on this).

It would be better to be able to filter on the type.

[alexfauquette]

@alexgarel Is that what you want?

https://hunger.openfoodfacts.org/insights#

If you click on a code (not the eye or the pen) you will filter by code

The associated request to robotoff:
https://robotoff.openfoodfacts.org/api/v1/insights?barcode=4026913121706&page=1&count=25

[ashish-reddy-20-08]

I don't think @ram690514 is currently working on this.

I think the best is to have the data as json + as html on another endpoint (the html view can just exploit the same method to get data and build the html)

Here are the point from most difficult to least difficult (you can start doing them incrementally and submit the PR as draft).

For the json:

1. list all ProductInsight for this barcode - sort them in a predictable way : type - timestamp - id
2. list all Prediction for this barcode (same sorting order)
3. list all LogoAnnotation but with corresponding data from ImagePrediction and ImageModel: there again same order

For the html, you should find a way to present it all (sortable table ? Should we use vuejs ? Maybe @alexfauquette has a though on this).

It would be better to be able to filter on the type.

Thanks @alexgarel I will work it on and get back to u, if any queries.

[ashish-reddy-20-08]

I don't think @ram690514 is currently working on this.

I think the best is to have the data as json + as html on another endpoint (the html view can just exploit the same method to get data and build the html)

Here are the point from most difficult to least difficult (you can start doing them incrementally and submit the PR as draft).

For the json:

1. list all ProductInsight for this barcode - sort them in a predictable way : type - timestamp - id
2. list all Prediction for this barcode (same sorting order)
3. list all LogoAnnotation but with corresponding data from ImagePrediction and ImageModel: there again same order

For the html, you should find a way to present it all (sortable table ? Should we use vuejs ? Maybe @alexfauquette has a though on this).

It would be better to be able to filter on the type.

Alex, I am not able to understand what exactly is the output required, should I be working on the data extraction from the link that [Alexandre Fauquette] commented ?
Can I please get a clear understanding of the project.

[alexgarel]

@ashish-reddy-20-08

I think the first part is to be done first. Create a new API route that return a json with

1. list all ProductInsight for this barcode - sort them in a predictable way : type - timestamp - id

2. list all Prediction for this barcode (same sorting order)

3. list all LogoAnnotation but with corresponding data from ImagePrediction and ImageModel: there again same order

4. (if it exists), list all ImagePrediction that have no LogoAnnotation with corresponding ImageModel

So it's all about doing right request in the database and putting info in nested dicts. (and the https://robotoff.openfoodfacts.org/api/v1/insights?barcode=4026913121706&page=1&count=25 api is a good starting point)

For the second part, it should be done in hungergame, completing what is already done to add those new data (prediction, logo annotation, image prediction) --> we should open a new ticket.

[alexgarel]

I completely changes the ticket description, I think going for separate api is more simple and more compatible with what exists in hungergame.

So please refer to current ticket description.

[alexgarel]

Comment to @Jagrutiti (already told on slack)

In image list (ImageCollection)
I would like to have a filter, which is on by default, that just list images without ImagePrediction linked. that is:

• without parameters, it would only return images without prediction images linked
• with parameter with_predicted=1 it would return all images

The only other filter you can put there is the barcode

[alexgarel]

@Jagrutiti

Let's move to ImagePredictionCollection !

This is the same pattern as with ImageCollection.

• we have the barcode filter (but you will have to follow the "image" relation to get barcode)
• we also have the "type" filter
• This time instead of with_prediction, we have a with_logo (defaults to False), if False it discards all ImagePrediction for which we have a logo annotation.

Should be really the same id as for ImageCollection.

[alexgarel]

Next is LogoAnnotationCollection

It's the same thing, with as previous view but for LogoAnnotation:
filters on:

• barcode
• type filter (a list of type) keep_types (filter on annotation_type)
• value_tag (filter on annotation_value_tag)

[alexgarel]

We identified with @Jagrutiti that we have to:

• add the server_domain to all "Collection" class
• by default, always sort on freshest item (higher id ?) first

[alexgarel]

@alexfauquette I verified, we have pagination on all those APIs, thanks to @Jagrutiti 😎

But we forgot to add the documentation ! Opened #896 for that…

[alexgarel]

This issue is finished :-) thank @Jagrutiti 🎉