Preserve explicit optimizer partition frontiers for TFHE circuit parametrization

[andidr]

The Concrete Optimizer is invoked on a representation of the program
in the high-level FHELinalg / FHE Dialects and yields a solution with
a one-to-one mapping of operations to keys. However, the abstractions
used by these dialects do not allow for references to keys and the
application of the solution is delayed util the pipeline reaches a
representation of the program in the lower-level TFHE dialect. Various
transformation applied by the pipeline along the way may break the
one-to-one mapping and add indirections into producer-consumer
relationships, resulting in abiguous or partial mappings of TFHE
operations to the keys. In particular, explicit frontiers between
optimizer partitions may not be recovered.

This commit preserves explicit frontiers between optimizer partitions
as optimizer.partition_frontier operations and lowers these to
keyswitch operations before parametrization of TFHE operations.

Resolves Issue #538.

[andidr]

@slab-ci concrete-python-tests-linux

[rudy-6-4]

Please split that in several PR next time, even if all other commits are small.

[andidr]

Please split that in several PR next time, even if all other commits are small.

Agreed that this diverts the focus from the actual commit fixing the parametrization pass by submerging it in CI fixes. The idea was to give the CI a chance to complete for the review and then to rebase the commits to main once the CI fixes are merged.

[BourgerieQuentin]

Follow our online discussion the following case is not handled as you don't find the extra keyswitch as ancestor due of loop nest

func.func @main(%arg0: tensor<2x!FHE.eint<1>>, %arg1: tensor<2x!FHE.eint<7>>) -> (tensor<2x!FHE.eint<8>>, tensor<2x!FHE.eint<5>>) {
  %cst = arith.constant dense<[0, 1]> : tensor<2xi64>
  %0 = "FHELinalg.apply_lookup_table"(%arg0, %cst) : (tensor<2x!FHE.eint<1>>, tensor<2xi64>) -> tensor<2x!FHE.eint<8>>
  %cst_0 = arith.constant dense<0> : tensor<128xi64>
  %1 = "FHELinalg.apply_lookup_table"(%arg1, %cst_0) : (tensor<2x!FHE.eint<7>>, tensor<128xi64>) -> tensor<2x!FHE.eint<8>>
  %2 = "FHELinalg.add_eint"(%0, %1) : (tensor<2x!FHE.eint<8>>, tensor<2x!FHE.eint<8>>) -> tensor<2x!FHE.eint<8>>
  %c4_i4 = arith.constant 4 : i4
  %cst_1 = arith.constant dense<0> : tensor<256xi64>
  %3 = "FHELinalg.apply_lookup_table"(%1, %cst_1) : (tensor<2x!FHE.eint<8>>, tensor<256xi64>) -> tensor<2x!FHE.eint<5>>
  return %2, %3 : tensor<2x!FHE.eint<8>>, tensor<2x!FHE.eint<5>>
}

got assertion

concretecompiler: /home/yundsi/code/concrete/compilers/concrete-compiler/compiler/lib/Dialect/TFHE/Transforms/TFHECircuitSolutionParametrization.cpp:150: const concrete_optimizer::dag::ConversionKeySwitchKey& mlir::concretelang::{anonymous}::CircuitSolutionWrapper::lookupConversionKeyswitchKey(uint64_t, uint64_t) const: Assertion `convKSKIt != solution.circuit_keys.conversion_keyswitch_keys.cend() && "Required conversion key must be available"' failed.
Aborted (core dumped)

[andidr]

Updated this to include explicit optimizer partition frontiers.

[BourgerieQuentin]

@slab-ci concrete-python-tests-linux

[BourgerieQuentin]

Could be great to add the example that I spotted (same as 538 but with linalg ops) and remove skipping of min max test in concrete python test.

        name.endswith("xfail_if_multi")
        and configuration.parameter_selection_strategy == fhe.ParameterSelectionStrategy.MULTI
    )
    if can_be_known_failure:
        pytest.skip(
            reason="Compiler parametrization pass make the compilation fail with an assertion"
        )

[andidr]

@slab-ci concrete-python-tests-linux

[andidr]

@slab-ci concrete-python-tests-linux