TFProfiler – ready to use 🚀

Overview

TFProfiler is an app that aims to profile your TensorFlow Lite and OpenCV DNN supported model right on the smartphone.

Measure model performance with model initialization time, inference time, FPS, memory consumption, etc.

Tweak model runs with different delegates (CPU, GPU, NNAPI, HEXAGON), XNNPACK option, and number of threads.

Apply a batch processing, adjusting the number of input images. TFLite batch inference is partially supported by TensorFlow.

Run neural models (*.caffemodel, *.onnx, *.pb, etc) supported by OpenCV DNN module.

• Android API SDK 21+
• Written in Kotlin
• TFLite 2.5.0 inside
• OpenCV 4.5.2 inside

Features

The app displays a list of models built inside the app that can be used to measure its performance on device. Also, you can upload your *.tflite model and see how it works on the smartphone.

       

Download

Latest apk release can be found on release page.

Or download app via Google Play:

Built-in models

App has built-in models that you can play with to see how inference performance is estimated.

• MobileNetV1
• MobileNetV2
• YOLOV4
• Blazeface
• Facemesh
• TextClassification

Image Dataset

The app has built-in a subset of image dataset Caltech 101. It is used for running model inteferences.

Libraries

• TensorFlow Lite
• Android Architecture Components
• Timber for logging
• Room - Access your app's SQLite database with in-app objects and compile-time checks.

Performance tips

We can boost model performance and energy efficiency using various optimization techniques.

Model optimization aims to create smaller models that are generally faster and more energy efficient, so that they can be deployed on mobile devices.

1. Check ops compatibility for each delegate, CPU, GPU, XNNPACK.

2. Use XNNPACK to boost float-point inference.

3. Use GPU delegate to compare with CPU and other options. If some of the ops are not supported by the GPU delegate, the TF Lite will only run a part of the graph on the GPU and the remaining part on the CPU.

4. Experiment with NNAPI (Android API 27+) delegate to see whether the models works faster on Android.

5. Try to use quantization to optimize model.

6. Accelerate TFLite model on Qualcomm Hexagon DSPs

7. Experiment with version of TensorFlow Lite library

8. Deploy different models for different devices

Mentions

Featured in AppTractor

Credits

Launcher icon was built using this one. Icons made by Becris from https://www.flaticon.com/

Issues

If you find any problems or would like to suggest a feature, please feel free to file an issue