Write an enhancement proposal for OVN InterConnect support

[aswinsuryan]

What would you like to be added:

Add support in Submariner to support OVN interconnect

Why is this needed:

The OVN is moving to a new architecture using an OVN interconnect. This would have an impact on Submariner. We need to test Submariner with the new implementation and make the necessary changes.

Enhancement PR: #163

[maayanf24]

@aswinsuryan - Can you add sizing information and an enhancement proposal for this?

[aswinsuryan]

@maayanf24 Yes I will work on it.

[aswinsuryan]

Was able to deploy OVN with IC enabled using this branch. Followed the instructions in the readme and some of the scripts were modified to rectify issues that we face when multiple clusters deployed. @sridhargaddam documented the changes to be made here

A cluster with a single zone was tried by running the above steps with these flags

export KIND_NUM_ADDITIONAL_ZONES=0
export KIND_NUM_WORKER=2
export OVN_INTERCONNECT_ENABLE=true

The submariner e2e failed since the OVN transit switch that connects two different zone uses the same CIDR as the Submariner Router.

After changing the IP in OVN to a different CIDR, 244.254.0.0/16(changed in OVN as it was easier to test), Submariner tests passed

So the Submariner router needs to use a different CIDR.

Now need to verify submariner tests with multiple zones.