[Port] OD layers to Keras 3

[ariG23498]

TODOs:

• roi_align.py
• roi_generator.py
• roi_pool.py
• roi_sampler.py
• rpn_label_encoder.py

CC: @divyashreepathihalli

[ariG23498]

There are certain blockers that I need some guidance on:

• with tf.name_scope("multilevel_crop_and_resize")
• tf.constant
• tf.math.divide_no_nan

[AakashKumarNain]

What's the blocker exactly?

[ariG23498]

I meant -- how should I be porting the above mentioned APIs to Keras3.

[AakashKumarNain]

I don't think you require name scoping. For constant, use KerasTensor? For third one, just write a custom divide function. Though we should add it in the core actually

[ariG23498]

Thanks for the tips @AakashKumarNain

I am documenting what I did:

• Removed scopes
• Used ops.convert_to_tensor or used ops.ones multiplying a scalar to it
• Use ops.cond for the divide no nan

[ariG23498]

@divyashreepathihalli Could you check the port of roi_generator.py? The only thing to note is that the NMS produces -1.0 instead of 0.0 in the bounding boxes (this has been taken care of in the tests).

[divyashreepathihalli]

Thanks @ariG23498! roi_align and roi_generator LGTM!!

[divyashreepathihalli]

LGTM! Thank you.

[divyashreepathihalli]

Thanks for the updates @ariG23498. There is one failing test

FAILED keras_cv/layers/object_detection/rpn_label_encoder_test.py::RpnLabelEncoderTest::test_rpn_label_encoder_multi_level - AssertionError: 
Not equal to tolerance rtol=1e-06, atol=1e-06
Mismatched value: a is different from b. 
not close where = (array([0]), array([0]))
not close lhs = [0.]
not close rhs = [1.]
not close dif = [1.]
not close tol = [2.e-06]
dtype = float64, shape = (2, 1)
Mismatched elements: 1 / 2 (50%)
Max absolute difference: 1.
Max relative difference: 1.
 x: array([[0.],
       [1.]])
 y: array([[1.],
       [1.]], dtype=float32)
       ```

[ariG23498]

Thanks for the updates @ariG23498. There is one failing test

FAILED keras_cv/layers/object_detection/rpn_label_encoder_test.py::RpnLabelEncoderTest::test_rpn_label_encoder_multi_level - AssertionError: 
Not equal to tolerance rtol=1e-06, atol=1e-06
Mismatched value: a is different from b. 
not close where = (array([0]), array([0]))
not close lhs = [0.]
not close rhs = [1.]
not close dif = [1.]
not close tol = [2.e-06]
dtype = float64, shape = (2, 1)
Mismatched elements: 1 / 2 (50%)
Max absolute difference: 1.
Max relative difference: 1.
 x: array([[0.],
       [1.]])
 y: array([[1.],
       [1.]], dtype=float32)
       ```

Hey @divyashreepathihalli do you think this one has to do with the keras.radom API? I am unable to track the source of the failing test.

[ariG23498]

@divyashreepathihalli @AakashKumarNain I have created a gist to explain the failing test test_rpn_label_encoder_multi_level

GitHub Gist

One can notice that the tests are flaky due to the radom API for sampling.py. I am unsure why the tests pass in the first place (maybe a PRNG issue). How do you think we should mitigate this?

[divyashreepathihalli]

@ariG23498 the randomAPI has changed and these tests can be fllaky. Please disable the test and add and issue and add a todo in the code.

[ariG23498]

@ariG23498 the randomAPI has changed and these tests can be fllaky. Please disable the test and add and issue and add a todo in the code.

I have made the changes requested. I am also adding the link of the issue to this thread.

#2336

[divyashreepathihalli]

Please rebase for the fix - #2360 !

[divyashreepathihalli]

Great work on this @ariG23498! this work is very impactful! Thank you!!