Amazing Performance possible on iOS / Android?

[madhavajay]

I can't believe how fast this runs on my MacBook pro in CPU mode!!?
Why is this not the default method of TensorFlow?

Are you able to explain how you were able to know which nodes to process on CPU and which on GPU for max performance?

Also is it possible to implement this in C++ for iOS / Android?

[gustavz]

@madhavajay thank you :)
Yes it is possible to visualize the time each opearation consumes (for example since v2.0 with test_objectdetection.py) and from this point on it was possible to find the right splitpoints.
But unfortunately this only works for ssd_mobilenet models, for other models it needs to be investigated from scratch.

[madhavajay]

Right but what i'm saying is will the same split points work using the C++ TF API on iOS and Android to drastically improve the performance?

[gustavz]

To be honest I don’t know as i did not use the c++ API yet. But I guess it should, why not.

If you try please let me know how it works :)