docs(frontend): adding a use-case for private information retrieval

[bcm-at-zama]

Work in progress, I've made experiments for PIR with Concrete. I think we can find use-cases for which it's not that bad.

[umut-sahin]

Could you create a .py script as well. It'll be necessary for testing and benchmarks. Maybe even do those directly, wdyt?

Very cool example btw!

[bcm-at-zama]

Let me have a look to make a .py yes. I am currently installing the new computer and I have an issue running the nb: it crashes when doing the

encrypted_y = circuit.run(encrypted_x, database)

I am investigating.

[bcm-at-zama]

OK my issue is some:

OMP: Error #15: Initializing libomp.dylib, but found libomp.dylib already initialized.
OMP: Hint This means that multiple copies of the OpenMP runtime have been linked into the program. That is dangerous, since it can degrade performance or cause incorrect results. The best thing to do is to ensure that only a single OpenMP runtime is linked into the process, e.g. by avoiding static linking of the OpenMP runtime in any library. As an unsafe, unsupported, undocumented workaround you can set the environment variable KMP_DUPLICATE_LIB_OK=TRUE to allow the program to continue to execute, but that may cause crashes or silently produce incorrect results. For more information, please see http://openmp.llvm.org/
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace.
zsh: abort      python pir.py

We have already seen that kind of things but I thought it was just for mac intel. It's not.

[umut-sahin]

Could you share it in #concrete channel? I don't remember the issue, maybe someone can help.

[bcm-at-zama]

I think I fixed that with zama-ai/concrete-ml#870

[bcm-at-zama]

@umut-sahin is it better now? I have adressed your comments I think + added some tests in pytest. I haven't done the benchmark part, I am less sure how to do it but maybe we can do it together?

[umut-sahin]

Tests seem to fail, could you fix it before I give another review?

[bcm-at-zama]

Tests seem to fail, could you fix it before I give another review?

sure my bad, sorry for that, I think they worked locally but let me see

[bcm-at-zama]

@umut-sahin :

• I have fixed the test (but it would be nicer with a way to reset the compilation, ie https://github.com/zama-ai/concrete-internal/issues/845)
• I have added a small thing we have discussed yersterday with @aquint-zama and @BourgerieQuentin : having an hash-based system as well to have eg URL filtering. That's in the end of the nb

[bcm-at-zama]

CC @aquint-zama and @BourgerieQuentin if you want to have a look and review a bit the notebook

[umut-sahin]

You'll have the ability to reset pretty soon 😉
#1048

[bcm-at-zama]

@umut-sahin the tests look good!

[umut-sahin]

I think there shouldn't be a pir_utils.py, there should be pir.py, which should be runable on it's own (e.g., to print an example, or assert some things, show timings).

Within it, there should be a class named PrivateInfomationRetrival, it should accept configuration: Optional[fhe.Configuration] = None and compiled: bool = True for its constructor.

Lastly, the test logic should be defined in tests not in the example itself.

Sorry it's not a small request, but it's for being consistent across the examples!

[bcm-at-zama]

I think there shouldn't be a pir_utils.py, there should be pir.py, which should be runable on it's own (e.g., to print an example, or assert some things, show timings).

Within it, there should be a class named PrivateInfomationRetrival, it should accept configuration: Optional[fhe.Configuration] = None and compiled: bool = True for its constructor.

Lastly, the test logic should be defined in tests not in the example itself.

Sorry it's not a small request, but it's for being consistent across the examples!

Let's discuss it, then. What I want is:

• a runnable .ipynb
• containing the code (ie, not using the code elsewhere that we don't see in the notebook)
• and I want to test my notebook with as less as possible code copy in the test files

That's why I did what I did.

[umut-sahin]

I'm just saying we don't have example_utils.py anywhere else, and we try to provide a simple python script that the users can run directly. I think the structure of an example should be:

• start by having a plain python script to implement the algorithm, show how to use it, and do a few assertions to show how it works (so not for testing)
• then add tests for it using the class defined in the example script
• then add benchmarks for it using the class defined in the example script
• then optionally create a notebook to convert it from an example to a tutorial
  The ordering is important here. We shouldn't neglect the others and directly be a tutorial instead of an example.

[bcm-at-zama]

I'm just saying we don't have example_utils.py anywhere else, and we try to provide a simple python script that the users can run directly. I think the structure of an example should be:

• start by having a plain python script to implement the algorithm, show how to use it, and do a few assertions to show how it works (so not for testing)
• then add tests for it using the class defined in the example script
• then add benchmarks for it using the class defined in the example script
• then optionally create a notebook to convert it from an example to a tutorial
  The ordering is important here. We shouldn't neglect the others and directly be a tutorial instead of an example.

I like it for the first three parts, but then I really think it's at the price of having nice notebook-tutorials. Those notebook-tutorials are really easier for the users, especially the beginners, to follow and appreciate.

[bcm-at-zama]

We're not in the same view, let's see with another person to break the tie. @BourgerieQuentin , what do you think please? I am fine refactoring all of this, but I don't think it goes for the best of the final users.

[bcm-at-zama]

I am starting a make jupyter, see #1056, we really need to test the notebooks (2 of the current 4's are broken!)

[bcm-at-zama]

@BourgerieQuentin , we debated a bit with @umut-sahin and came to the following proposal:

To test the CP examples, we would either do it:

• in pytest for the .py
• or via juputer.sh for the .nb (it's a PR in progress, Adding notebook tests 855 #1056)

What do you think of this proposal?

An alternative approach was:

We write the tutorials in a notebook, and convert it to a plain script, which we can just run, import from tests/benchmarks. CI could check if the conversion is correct. Wdyt?
During development, we can start with the notebook. When it's ready we can do:
TARGET=game_of_life make tutorial-to-example
Which would create:
examples/game_of_life/game_of_life.py
And in the CI, we'd check if this committed file is exactly the same as the notebook

but we thought it would be a bit over engineering to do that.

[BourgerieQuentin]

Ok fine for me 👍

[bcm-at-zama]

Awesome so I'm going to remove my changes in *.py, and just keep the notebook, which will be tested with the other PR.

[bcm-at-zama]

So now, it's just the notebook to review please.

[bcm-at-zama]

Pushed changes requested by @BourgerieQuentin , let me see if the build is still green

[BourgerieQuentin]

I added a very minor comment, so good to me for merging. Thanks for this cool example 👍

[bcm-at-zama]

Thanks a lot @umut-sahin / @BourgerieQuentin , let's merge!