[Object_Detection] Add IOUSimilarity layer. #3
Conversation
|
Will this only a layer or used for losses? See https://github.com/tensorflow/addons/blob/v0.10.0/tensorflow_addons/losses/giou_loss.py#L25-L61 |
|
Making it used for loss seems pretty easy -- just inherit from LossFunctionWrapper instead of Base Layer? I'm fine either way, depending on whether many people use iou directly as loss or not. |
@gabrieldemarmiesse what do you think? |
|
I believe both have their use cases and it would be nice to have both. But this isn't blocking this pull request in any way. No need to add everything at once. We can always put it in the backlog. |
|
@gabrieldemarmiesse Ok thanks in the meantime we could use the Layer from keras-cv and the loss from Tensorflow-addons. |
@bhack @gabrieldemarmiesse Feel free to add comments & feedbacks or tag others that you think might be interested in this, always good to improve from the very beginning :-) |
|
In addons was introduced as loss in 2019 by @fsx950223 with tensorflow/addons#477 |
| import tensorflow as tf | ||
|
|
||
|
|
||
| class IOUSimilarity(tf.keras.layers.Layer): |
There was a problem hiding this comment.
Big question here is why a layer -- what workflows will it be used in? We do already have a IoU metric -- would it make sense for IoU to be a metric and a loss instead of a layer?
There was a problem hiding this comment.
The more I think about it, this is actually pretty different from what 1) giou loss in addons, 2) iou in keras/metrics is doing.
Both 1) and 2) require y_true and y_pred, which needs to be the same shape, i.e.,:
y_pred: [n_boxes, 4]
y_true: [n_boxes, 4]
And the metrics/loss from it is [n_boxes] before reduction
However what we are having here is:
box1: [n_boxes_1, 4]
box2: [n_boxes_2, 4]
and we're computing the bipartite similarities, so the result would be [n_boxes_1, n_boxes_2]
More specifically, we will need to call this layer with call(gt_boxes, anchors), where n_boxes_1 is usually 3-10 (which is also why I added ragged support because we don't know how many), but n_boxes_2 is always fixed (e.g., 8732 in SSD300).
This is really used for different things, so I believe it should be a layer.
There was a problem hiding this comment.
Do we expect the output of this layer to be fed to other layers? Or is it a final output of the model? If the latter, then maybe it should be a metric instead (to be used instead another layer, for instance).
There was a problem hiding this comment.
I think that IOU and derivates could be still interesting also as loss:
https://arxiv.org/abs/1908.03851
https://arxiv.org/abs/1911.08287
…d update iou losses (#1296) * first attempt at introducing YoloX * formatted and fixed bugs * cast fix #1 * cast fix #2 * cast fix #3 * cast fix #4 * adding ensure shape for support * reverting and removing ensure_shape * fixed another bug * updated train.py * updated docs, tests and added support for loss strings * first attempt at introducing YoloX * formatted and fixed bugs * adding ensure shape for support * reverting and removing ensure_shape * reformatted by black * fixed a linting issue * finally rebased atop the recent changes * finally rebased atop the new changes * fixed linting issues * reverted rebasing issues with iou loss * fixing rebased errors part 2 * fixed more linting issues * TPU testing changes * linting fixes * updated with implementation details from paper * updated based on review comments and api changes * first attempt at introducing YoloX * updated docs, tests and added support for loss strings * fixed linting issues * reverted rebasing issues with iou loss * review comments * removed examples * linting fix * fixed rebasing error * updated no_reduction warning * review comments * revert version and linting fixes
* Add numpy ops (initial batch) and some config * Add unit test * fix call * Revert "fix call" This reverts commit 6748ad183029ff4b97317b77ceed8661916bb9a0. * full unit test coverage * fix setup.py * more ops and fix the problem of KerasTensor [ops] numerics
…d update iou losses (keras-team#1296) * first attempt at introducing YoloX * formatted and fixed bugs * cast fix keras-team#1 * cast fix keras-team#2 * cast fix keras-team#3 * cast fix keras-team#4 * adding ensure shape for support * reverting and removing ensure_shape * fixed another bug * updated train.py * updated docs, tests and added support for loss strings * first attempt at introducing YoloX * formatted and fixed bugs * adding ensure shape for support * reverting and removing ensure_shape * reformatted by black * fixed a linting issue * finally rebased atop the recent changes * finally rebased atop the new changes * fixed linting issues * reverted rebasing issues with iou loss * fixing rebased errors part 2 * fixed more linting issues * TPU testing changes * linting fixes * updated with implementation details from paper * updated based on review comments and api changes * first attempt at introducing YoloX * updated docs, tests and added support for loss strings * fixed linting issues * reverted rebasing issues with iou loss * review comments * removed examples * linting fix * fixed rebasing error * updated no_reduction warning * review comments * revert version and linting fixes
Todos with ragged tensor support.