Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add openvino support #149

Open
wants to merge 5 commits into
base: master
Choose a base branch
from
Open

Add openvino support #149

wants to merge 5 commits into from

Conversation

r-or
Copy link

@r-or r-or commented Apr 30, 2019

Hi,
this branch adds support for intel's openvino, so we can run the feature extractor e.g. on a NCS2 compute stick.

@r-or
Add option to bypass preprocessing step.
83fc559 
Reason: Openvino is not currently able to process tf.maps which create a loop in the model.
As all this loop here does is to convert BGR to RGB it can be left out completely: for the
embeddings the ordering should not matter anyway.
@r-or
Add initial support for Openvino.
552944e 
Add a command-line arg 'use_openvino' which expects an openvino device (CPU, GPU, MYRIAD etc).
Add also an FPS counter.
@r-or
Add a FPS counter to the command line
cb8020f
@r-or
Update readme for openvino
2ea4866
nwojke
nwojke reviewed May 2, 2019
View reviewed changes
tools/freeze_model.py
image_var = tf.cast(input_var, tf.float32)
else:
image_var = tf.map_fn(
lambda x: _preprocess(x), tf.cast(input_var, tf.float32),
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The _preprocess() function changes input order (BGR to RGB). Is this handled anywhere in the code?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No... but it seems to not matter for the computation.

I wrote a test script for that:
https://gist.github.com/r-or/e1b85c47e1906763b6e0e7a209812dda

TF: RGB vs BGR
Diff abs (0.0 is exactly same):
 [[0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
  0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
  0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
  0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
  0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
  0. 0. 0. 0. 0. 0. 0. 0.]]
Diff rel (1.0 is exactly same):
 [[1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.
  1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.
  1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.
  1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.
  1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.
  1. 1. 1. 1. 1. 1. 1. 1.]]
Comparison: PASSED

OV: RGB vs BGR
Diff abs (0.0 is exactly same):
 [[-7.09891319e-05 -2.31117010e-05 -9.74312425e-05 -1.97231770e-04
   1.63659453e-04  1.75914727e-04 -2.75910832e-04 -1.31983310e-04
  -5.64876944e-04 -1.39862299e-04  3.16947699e-05  9.46968794e-06
  -1.95568660e-04 -2.75567174e-04  1.23247504e-04 -2.15724111e-04
   4.99635935e-05 -5.26588410e-05 -3.16016376e-04 -1.02594495e-05
  -2.16595829e-04 -6.73830509e-05  1.24670565e-04 -9.22754407e-05
  -4.97810543e-05 -2.27943063e-04 -1.59956515e-04 -1.92910433e-04
  -4.10407782e-04  2.03438103e-05  6.73681498e-05 -2.15888023e-04
   2.65173614e-04  2.46569514e-04 -1.86443329e-04  1.69944018e-04
  -2.33516097e-04  3.09579074e-04 -3.15740705e-04  6.92084432e-05
   1.41568482e-04 -2.09584832e-04 -7.94976950e-05 -2.32093036e-04
  -7.49334693e-04  1.75776891e-04 -2.28881836e-04  1.02892518e-05
   3.79905105e-05  1.49380416e-04 -1.06245279e-05  2.01821327e-04
  -5.26068732e-04  3.78191471e-04  4.58016992e-04 -3.39105725e-04
   2.95236707e-04  1.17262825e-05  9.52333212e-05 -1.28746033e-05
  -3.78444791e-04  2.62483954e-05  4.32081521e-04  5.81145287e-07
  -2.42659822e-04  4.62397933e-04  9.70438123e-05 -1.38171017e-05
   2.75548548e-04 -1.63108110e-04  1.20360419e-04  5.72502613e-05
   1.78739429e-04 -1.81391835e-04 -3.15383077e-04  7.68899918e-05
   2.03009695e-05  6.54584961e-04 -2.84455717e-04 -9.59448516e-05
  -6.94066286e-04 -5.18411398e-05 -5.67063689e-05  1.15118921e-04
  -1.28496438e-04  3.14921141e-04 -2.13776948e-04  1.39519572e-04
  -2.22593546e-04  9.27075744e-05 -5.28363511e-04  1.63061544e-04
   1.62258744e-04  3.24718654e-04  5.15423715e-04  2.54072249e-04
  -2.00938433e-04  9.00402665e-05 -1.41721219e-04  1.78741291e-04
   4.18379903e-04 -3.99368349e-04 -2.76640058e-05 -2.14301050e-04
   5.50225377e-06  4.53330576e-05  4.59015369e-04 -2.36004591e-04
  -3.93390656e-06  2.04637647e-04  1.68181956e-04  6.85453415e-07
  -7.73668289e-05 -8.13156366e-05  8.03060830e-05 -3.39727849e-04
   2.70333141e-04 -2.06910074e-04 -2.38791108e-05 -2.48849392e-06
  -1.09579414e-04  2.18413770e-05 -2.44807452e-04  2.52965838e-04
  -1.14329159e-05  2.36977823e-04  2.89959833e-04  1.11445785e-04]]
Diff rel (1.0 is exactly same):
 [[0.99967253 1.0002847  1.0013707  1.000923   1.0048432  1.0253918
  1.0197521  0.99684745 1.1119983  1.0031053  0.99962395 0.99975747
  1.0616565  1.010521   0.998444   1.0013002  1.0002184  0.9978832
  0.9966819  1.0001087  1.0198045  0.9997458  1.0013885  0.9991812
  0.99891114 1.0071045  1.0025008  0.99804175 0.99195236 0.9995756
  0.9990167  0.9979031  1.0026736  1.0044912  0.99593943 1.0042726
  0.997915   0.9958782  0.997491   0.99848914 1.0017109  1.0035508
  1.0006112  0.9962664  0.98856336 1.0163243  0.99638605 0.9998841
  1.0005648  1.0043436  1.0000782  0.9924133  1.0327789  1.0026767
  1.0035827  0.99799013 0.9976437  1.0007793  1.001019   1.0001298
  0.9959444  1.00028    1.0035039  0.9999948  0.9903539  1.0029724
  0.9961764  1.0004318  0.98782665 0.9987333  0.38110238 0.9992418
  0.9938017  1.0022285  0.9964751  1.0004631  0.9993321  0.7935085
  0.99424076 1.0016934  0.9955157  0.99962413 1.0005273  1.0375509
  1.0025675  0.9942383  1.0960878  1.0009598  0.9956771  0.99913466
  1.0227439  1.0079089  1.0007166  0.99399155 1.007452   1.0029502
  1.0042864  0.99888796 1.0130587  0.9905349  1.0048269  0.86005217
  1.0004     0.9964666  0.99983996 0.99923646 0.97229576 0.8474253
  1.0000467  1.0014483  0.9952575  0.9999925  0.99120194 0.99910206
  1.0022343  1.0710478  1.0043408  1.0027815  1.0002646  1.0000274
  1.0019089  0.9993802  0.9919843  0.99496603 0.99971646 0.98349696
  1.0108225  0.99822414]]
Comparison: FAILED

TF vs OV
Diff abs (0.0 is exactly same):
 [[-3.94286215e-03 -4.32543457e-04  5.99548221e-05 -5.05328178e-04
   9.27921385e-04 -4.12844960e-03  2.05636956e-03  7.24088401e-04
   1.61289563e-03  3.92809510e-03  1.66483223e-04 -6.79396465e-03
  -6.77901274e-03  1.13993883e-04  2.14674324e-03 -4.47931886e-03
  -3.22103500e-04 -2.19973177e-03 -3.49406153e-03 -4.19918448e-03
  -6.94096927e-03  4.93526459e-04  2.00904161e-03 -2.98864394e-03
   2.46691331e-03  1.02963299e-04  5.58376312e-04  7.87702948e-03
   1.99251249e-03  5.59294969e-03 -2.11878866e-03 -4.32344526e-03
   1.49095058e-03  6.78264722e-03 -6.30068034e-03 -6.66405261e-03
  -8.42780620e-03 -8.48181546e-04 -7.32316077e-03  5.94160333e-03
   1.05012730e-02 -6.90795481e-04  5.66825271e-04 -7.10094348e-03
  -2.20379978e-03  7.70511758e-03 -9.60171223e-03  6.82964921e-04
  -8.90789181e-03 -1.15070492e-03  1.48507953e-03 -2.18164176e-03
   1.88763440e-03  5.90819120e-03 -5.21732867e-03 -1.01997554e-02
   1.66078657e-03  2.63114460e-04 -4.62488085e-03  7.21380115e-04
   6.82404637e-03 -1.24016032e-02  4.28882986e-03  9.12263989e-04
  -5.39921224e-03  6.79710507e-03  2.50028446e-04  3.97107191e-03
   6.00080565e-03 -9.64917988e-03 -6.06814865e-09  3.70991230e-03
  -3.86371464e-03  6.10897690e-03 -1.71685219e-03  9.97103751e-03
  -1.18000209e-02 -7.65118748e-05 -4.82698902e-03 -1.86500326e-03
  -3.85785103e-03  2.82938778e-03  1.72593445e-03 -1.04905255e-02
  -4.40701842e-04  2.87934020e-03 -6.38315035e-03  5.70577383e-03
   5.37855923e-03 -1.30973756e-04  2.21801363e-03 -6.44079037e-03
   1.08246654e-02 -4.04008850e-03 -1.19326711e-02 -5.89519739e-04
  -4.79451939e-03 -2.68968195e-03  2.76577100e-03  2.17592716e-03
   1.15049183e-02  4.09299321e-03  1.38435513e-03  1.79736689e-02
   1.96448714e-03  2.19316036e-03 -3.39902006e-03  4.71524755e-03
  -9.59008932e-04 -2.22636759e-03  3.81218269e-03  1.75344944e-03
  -6.80692634e-03  2.39557773e-03  7.68467784e-04  4.61338321e-03
  -1.34492777e-02 -4.74420190e-03  2.16958672e-03 -2.16118991e-03
   5.91486692e-04  1.67673826e-03  1.77322514e-03  2.77697667e-03
   2.28755921e-03  1.66156795e-03 -1.35983313e-02  2.19796225e-03]]
Diff rel (1.0 is exactly same):
 [[ 0.9818045   1.0053259   0.9991577   1.0023628   1.0273277   0.41884965
   0.8556379   1.0173502   0.71241933  0.9130556   0.9980241   1.1740589
   3.0130827   0.9956931   0.9728552   1.0269623   0.9985928   0.91138774
   0.96319115  1.0444852   1.6223245   1.0018622   1.0223445   0.9734597
   1.0540177   0.9968135   0.9912921   1.0801181   1.0393881   0.8832715
   1.0309559   0.9579188   1.0149928   1.1229923   0.86221653  0.83317214
   0.92459404  1.0113395   0.9416606   0.87009555  1.1266977   1.011662
   0.99564457  0.88534284  0.96597564  1.704076    0.84784424  0.9923086
   0.8676459   0.9666845   0.98906654  1.0826377   0.8861161   1.0417054
   0.95933443  0.93942523  0.9867138   1.0174726   0.9505658   0.99272746
   1.0734274   0.86775905  1.0346589   0.99188757  0.7832818   1.043563
   0.99011093  0.87596244  0.7316255   0.924968    1.0000819   0.950832
   1.1348207   0.92511624  0.98074347  1.0600327   1.3884995   1.0304168
   0.9017044   1.0328609   0.9749625   1.0205228   0.9839614  -2.298085
   1.0087833   0.9470151   3.6175644   1.0392108   1.1049081   1.0012236
   0.9066464   0.69005466  1.047769    1.0752078   0.8287526   0.99317485
   1.1018409   1.0332566   0.748437    0.88367426  1.1320976   2.6676648
   0.9799915   1.2974048   0.94286215  0.9630336   1.2109962   4.5972047
   1.011381    0.98426676  0.89198875  0.98080915  0.21905166  1.0264783
   1.0213337   0.09919678  0.78497773  1.0636002   0.97595745  1.0237603
   0.989716    0.9523874   1.0585294   0.9444593   1.0567468   0.88234735
   0.49788728  0.9649135 ]]
Comparison: FAILED

Batch size 1, 300 iterations:
 OV: 1.66366s
 TF: 4.50380s

Apart from tensorflow and openvino not creating exactly the same results (which is to be expected) swapping BGR and RGB seems to not do anything.

Anyway, even though the results are slightly different, the tracker worked perfectly fine

Copy link
Owner

@nwojke nwojke May 7, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But it wouldn't hurt to convert from BGR to RGB before handing the image over from NumPy to TensorFlow, right?

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, thanks for posting test results.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

True, it wouldn't hurt either. I'll add it for consistency.

@nwojke
Copy link
Owner

nwojke commented May 7, 2019 •
edited
Loading

Thanks for your pull request. I think OpenVINO support can be useful for some users. I would like to add the functionality in a way that it introduces minimal changes to the existing code base. Specifically, I would suggest the following:

  • Instead of adding additional switch parameters to tools/generate_detections.py and if-else branches which check that OpenVINO is installed, create a file tools/generate_detections_openvino.py which provides the OpenVINO compatible implementation. If we want to be very clean, we factor out common functionality between both scripts into a utility module, but (in this case) I am ok with duplicated code as long as there is only one copy.
  • Move the OpenVINO related documentation into a separate section at the bottom of the README.md.
  • Remove the debug/timing code fromdeep_sort_app.py.

This would strengthen the separation between standard implementation and OpenVINO related functionality/documentation. What are your thoughts on that?

@r-or
Copy link
Author

r-or commented May 7, 2019

Agreed, this is probably the better approach. When I have time, I'll update the pull request!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@nwojke nwojke nwojke left review comments

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@r-or @nwojke