[Model conversion tool] Support fusing Conv+Add

[Linchenn]

(Conv means PointwiseConv, DepthwiseConv or generalConv here.)

Problem

The tool currently supports 'Conv + BiasAdd + Activiation' fusing, but this fusing requires the op of 'BiasAdd' to be 'BiasAdd', while 'Conv + AddV2 + Activiation' could not be fused even though they are mathematically same.

Candidate solutions

Assume we could not fix grappler for such missed fusing pattern. There are a couple of candidate solutions:

1. [Current PR] Add one pass to convert all AddV2 ops that could be fused with Conv to BiasAdd op. After the conversion, when the graph is passed to the grappler, the graph could be fused as expected.
2. Add two passes to fuse 'Conv + AddV2 + Activiation' (one for Conv and one for DepthwiseConv). Cons: This would require duplicate logics because we need to re-write a pass to fuse 'Conv + BiasAdd + Activiation' in addition to grappler's implementation.

Either way, the tool has to find out 'AddV2' nodes to fuse, with the following conditions:

1. The ancestor node has to be Conv or DepthwiseConv op.
2. The current node is the only successor of the ancestor.
3. The current node 'AddV2' could be converted to 'BiasAdd'.

Misc.

From TF API doc, the major difference between 'AddV2' and 'Add' is the data type of input. The previous one additionally supports 'uint16, uint32, uint64', while the later one additionally supports 'string'.

To see the logs from the Cloud Build CI, please join either our discussion or announcement mailing list.

[pyu10055]

this data_format info should come from the ancestor node

[Linchenn]

@pyu10055 If bias is a 1D tensor, the AddV2 op always broadcast/adds it to the last dimension of the ancestor's results, so this behavior is always NHWC BiasAdd.

[Linchenn]

Also took a look at the models that do not fuse ops as expected, mainly for conv/depthwise+bias:

• The op name is 'addV2' instead of 'biasAdd': MobileBert.
• The op name is 'add' instead of 'biasAdd': Coco-SSD-MobileNetV2, Coco-SSD-MobileNetV1, DeepLabV3-pascal, DeepLabV3-ade20k, AutoML Object, AutoML Image.
• The conversion of the following models might be too old to apply fusing ops: TextToxicity, posenet-MobileNetV1
  posenet-ResNet50, USE.

cc @qjia7 @mattsoulanille

[Linchenn]

Update: added conversion supports for 'Add' op fusing, in addition to 'AddV2' because of the use cases of the last comment.

[qjia7]

Also took a look at the models that do not fuse ops as expected, mainly for conv/depthwise+bias:

• The op name is 'addV2' instead of 'biasAdd': MobileBert.
• The op name is 'add' instead of 'biasAdd': Coco-SSD-MobileNetV2, Coco-SSD-MobileNetV1, DeepLabV3-pascal, DeepLabV3-ade20k, AutoML Object, AutoML Image.
• The conversion of the following models might be too old to apply fusing ops: TextToxicity, posenet-MobileNetV1
  posenet-ResNet50, USE.

Great work! After this PR is merged, the benchmarks models can be updated with this PR, right? Look forward to great perf improvement with fused models.

[Linchenn]

Great work! After this PR is merged, the benchmarks models can be updated with this PR, right?

Yes

[mattsoulanille]

I'm surprised we have node.input but not node.output.

[Linchenn]

Yes, node.input is the edge information for model topology, node.output would have the duplicate information.