Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize JPEG scanline copy of yuv format #235

Merged
merged 2 commits into from
Oct 12, 2023
Merged

Conversation

chraac
Copy link
Contributor

@chraac chraac commented Oct 9, 2023

Summary

In our current source, when encoding JPEG with CPU we'll first convert the yuv pixel int rgb, but from jpeg spec, encoder using YCbCr which is yuv internally (see also: JPEG#Encoding), this means we can passthrough the yuv value to encoder directly without converting to RGB.

Changes

  • For V4L2_PIX_FMT_YUYV and V4L2_PIX_FMT_UYVY copy the yuv value directly into scanline buffer instead of convert to rgb

Test

  • Run on my RK3399 board with asan enabled, no error found

  • Performance test

    • compiling flag: -O3 -march='armv8-a' -mtune='cortex-a72.cortex-a53
    • running params: ./ustreamer --device=/dev/video0 -r 1280x720 --quality=80 --encoder=CPU
    \ Before After
    latency 0.231~0.301 0.201
    cpu usage 525300375043aea19a2076d3d7f54ba c16530eb8e29076f75724dd30dffd63

@argakon
Copy link
Contributor

argakon commented Oct 9, 2023

It is actually really good performance optimisation. I have Elgato Camlink 4k stick and with pixel format YUYV and 1920x1080 50fps i have more than twice cpu utilisation difference. It uses only one core right now on RK3588, was 2,5 cores.

@argakon
Copy link
Contributor

argakon commented Oct 9, 2023

I've added NV12 format support in my repo. It also should close #97 issue. But it converts frame into RGB24, i tried to pass interpolated YUV data from NV12 directly into jpeg_write_scanlines with YCbCr, but no success, could i ask you to look in it?

@chraac
Copy link
Contributor Author

chraac commented Oct 10, 2023

I've added NV12 format support in my repo. It also should close #97 issue. But it converts frame into RGB24, i tried to pass interpolated YUV data from NV12 directly into jpeg_write_scanlines with YCbCr, but no success, could i ask you to look in it?

Sure, leave some comment on it, let me known if have any question about

@mdevaev
Copy link
Member

mdevaev commented Oct 10, 2023

Please resolve a conflict.

@chraac
Copy link
Contributor Author

chraac commented Oct 11, 2023

Please resolve a conflict.

Resolved!

@mdevaev mdevaev self-assigned this Oct 12, 2023
@mdevaev mdevaev added the type:feature New feature or request label Oct 12, 2023
@mdevaev mdevaev merged commit 4732c85 into pikvm:mp Oct 12, 2023
@chraac chraac deleted the cpu-jpeg-enc-opt branch October 12, 2023 02:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
resolution:fixed Fixed type:feature New feature or request
Development

Successfully merging this pull request may close these issues.

3 participants