Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LSTM-Model with RaspberryPi4 and Coral USB-Accelerator #689

Closed
johannesheid opened this issue Dec 7, 2022 · 2 comments
Closed

LSTM-Model with RaspberryPi4 and Coral USB-Accelerator #689

johannesheid opened this issue Dec 7, 2022 · 2 comments
Assignees
Labels
comp:model Model related isssues comp:thirdparty Thirdparty related issues Hardware:USB Accelerator Coral USB Accelerator issues subtype:ubuntu/linux Ubuntu/Linux Build/installation issues type:performance Performance issues

Comments

@johannesheid
Copy link

johannesheid commented Dec 7, 2022

Description

Hello,

I have a model with LSTM layer and I want to run this on a Raspberry4 with Coral USB accelerator connected.
In another post (#156) I read about the problems and with the additional command USB_ENABLE_BULK_DESCRIPTORS_FROM_DEVICE=1 the program runs. However, with the customization, the model runs only half as fast as when I run it through the Raspberry CPU. Consequently, I currently have no advantage through the USB accelerator.
Is there another way to do this in the meantime or is there no other solution yet?

In addition, I would be very interested to know why this command is/was needed, what exactly it does and why it results in this large latency.

Click to expand!

Issue Type

Performance

Operating System

Linux

Coral Device

USB Accelerator

Other Devices

Rapsberry Pi 4

Programming Language

Python 3.7

Relevant Log Output

No response

@google-coral-bot google-coral-bot bot added comp:model Model related isssues comp:thirdparty Thirdparty related issues Hardware:USB Accelerator Coral USB Accelerator issues subtype:ubuntu/linux Ubuntu/Linux Build/installation issues type:performance Performance issues labels Dec 7, 2022
@hjonnala
Copy link
Contributor

Is there another way to do this in the meantime or is there no other solution yet?

As of now there is no solution other than setting USB_ENABLE_BULK_DESCRIPTORS_FROM_DEVICE to 1 with the USB accelerator. For pcie devices(ex: Dev Board), the command is not needed..

The extra latency is to due to the USB DMA drivers. Please check the source code for more details at: https://github.com/google-coral/libedgetpu/blob/master/driver/usb/usb_driver.cc#L324.

On Dev Board:

mendel@bored-calf:~$ ./single_model_benchmark --model weather_forecast_quant.tflite
2022-12-15 06:25:52
Running ./single_model_benchmark
Run on (4 X 1500 MHz CPU s)
Load Average: 0.06, 0.07, 0.03
***WARNING*** CPU scaling is enabled, the benchmark real time measurements may be noisy and will incur extra overhead.
-----------------------------------------------------
Benchmark           Time             CPU   Iterations
-----------------------------------------------------
BM_Model         2.18 ms         2.16 ms          324 weather_forecast_quant.tflite

mendel@bored-calf:~$ ./single_model_benchmark --model weather_forecast_quant_edgetpu.tflite
2022-12-15 06:26:06
Running ./single_model_benchmark
Run on (4 X 1500 MHz CPU s)
Load Average: 0.05, 0.06, 0.03
***WARNING*** CPU scaling is enabled, the benchmark real time measurements may be noisy and will incur extra overhead.
-----------------------------------------------------
Benchmark           Time             CPU   Iterations
-----------------------------------------------------
BM_Model         1.98 ms        0.443 ms         1424 weather_forecast_quant_edgetpu.tflite


@google-coral-bot
Copy link

Are you satisfied with the resolution of your issue?
Yes
No

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
comp:model Model related isssues comp:thirdparty Thirdparty related issues Hardware:USB Accelerator Coral USB Accelerator issues subtype:ubuntu/linux Ubuntu/Linux Build/installation issues type:performance Performance issues
Projects
None yet
Development

No branches or pull requests

2 participants