Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Frames decoder gpu #3615

Merged
merged 35 commits into from
Feb 7, 2022
Merged

Frames decoder gpu #3615

merged 35 commits into from
Feb 7, 2022
+617 −11

Conversation

awolant
Copy link
Contributor

@awolant awolant commented Jan 12, 2022 •
edited
Loading

Description

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Refactoring (Redesign of existing code that doesn't affect functionality)
  • Other (e.g. Documentation, Tests, Configuration)

What happened in this PR

Adds new class FramesDecoderGpu capable of decoding video files using NVIDIA Video Codec SDK.

This class derives from FramesDecoder that uses FFMPEG to do the same thing.

Checklist

Tests

  • Existing tests apply
  • New tests added
    • Python tests
    • GTests
    • Benchmark
    • Other
  • N/A

Documentation

  • Existing documentation applies
  • Documentation updated
    • Docstring
    • Doxygen
    • RST
    • Jupyter
    • Other
  • N/A

DALI team only

Requirements

  • Implements new requirements
  • Affects existing requirements
  • N/A

REQ IDs: N/A

JIRA TASK: 2252

Sorry, something went wrong.

awolant added 12 commits October 30, 2021 15:42
@awolant
Add FramesDecoderGpu
82d1e42 
Signed-off-by: Albert Wolant <awolant@nvidia.com>
@awolant
Import more parts of sample
68510a8 
Signed-off-by: Albert Wolant <awolant@nvidia.com>
@awolant
Add frames decoder GPU
61688f8 
Signed-off-by: Albert Wolant <awolant@nvidia.com>
@awolant
Merge remote-tracking branch 'nvidia/main' into frames_decoder_gpu
34d6a8d
@awolant
Working GPU reader
e05e210 
Signed-off-by: Albert Wolant <awolant@nvidia.com>
@awolant
Tmp version
8c78822 
Signed-off-by: Albert Wolant <awolant@nvidia.com>
@awolant
Almost working version
c64a672 
Signed-off-by: Albert Wolant <awolant@nvidia.com>
@awolant
Remove out of scope parts
f0a37a5 
Signed-off-by: Albert Wolant <awolant@nvidia.com>
@awolant
Merge remote-tracking branch 'nvidia/main' into frames_decoder_gpu
ca6329f
@awolant
Tmp disable test
8cfef9a 
Signed-off-by: Albert Wolant <awolant@nvidia.com>
@awolant
Working version
2b8f58a 
Signed-off-by: Albert Wolant <awolant@nvidia.com>
@awolant
Fix issues
6450203 
Signed-off-by: Albert Wolant <awolant@nvidia.com>
@dali-automaton
Copy link
Collaborator

CI MESSAGE: [3785236]: BUILD FAILED

@awolant
Copy link
Contributor Author

awolant commented Jan 19, 2022

!build

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [3785564]: BUILD STARTED

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [3785564]: BUILD FAILED

@awolant
Fix CI builds
ba641df 
Signed-off-by: Albert Wolant <awolant@nvidia.com>
@awolant
Copy link
Contributor Author

awolant commented Jan 19, 2022

!build

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [3787139]: BUILD STARTED

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [3787139]: BUILD FAILED

@awolant
Fix clang build
6ea2c3f 
Signed-off-by: Albert Wolant <awolant@nvidia.com>
@dali-automaton
Copy link
Collaborator

CI MESSAGE: [3792048]: BUILD STARTED

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [3792048]: BUILD FAILED

JanuszL
JanuszL reviewed Jan 20, 2022
View reviewed changes
dali/operators/reader/loader/video/nvdecode/ColorSpace.cu Outdated
Clamp(matYuv2Rgb[2][0] * fy + matYuv2Rgb[2][1] * fu + matYuv2Rgb[2][2] * fv, 0.0f, maxf)};
}

__global__ static void YuvToRgbKernel(uint8_t *pYuv, int nYuvPitch, uint8_t *pRgb, int nRgbPitch, int nWidth, int nHeight) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if this can be moved to color_space_conversion_impl.h.

Copy link
Contributor Author

@awolant awolant Feb 2, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I made this file more readable for now.

In the future, we need to extend this simple code to support more pixel formats. Then I think will be the right time to extract impl. Is that ok?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure

JanuszL
JanuszL reviewed Jan 20, 2022
View reviewed changes
dali/operators/reader/loader/video/frames_decoder_gpu.cc Outdated
FramesDecoderGpu *frames_decoder = static_cast<FramesDecoderGpu*>(user_data);
if (frames_decoder->flush_) {
frames_decoder->flush_ = false;
frames_decoder->decode_success_ = false;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpick - I guess you can move this to L36 and remove L43.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@awolant
Fix for review
3a1c041 
Signed-off-by: Albert Wolant <awolant@nvidia.com>
JanuszL
JanuszL reviewed Feb 4, 2022
View reviewed changes
dali/operators/reader/loader/video/frames_decoder_gpu.cc Outdated
frames_decoder->piped_pts_.pop();

// current_pts is pts of frame that came from the decoder
// CurrentFramePts() is pts of the frame that we want to return
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// CurrentFramePts() is pts of the frame that we want to return
// NextFramePts() is pts of the frame that we want to return

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@awolant
Fix review comments
c1f980d 
Signed-off-by: Albert Wolant <awolant@nvidia.com>
@awolant
Copy link
Contributor Author

awolant commented Feb 7, 2022

!build

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [3909057]: BUILD STARTED

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [3909057]: BUILD FAILED

@awolant
Add assert before last packet send
0da6bd1 
Signed-off-by: Albert Wolant <awolant@nvidia.com>
@awolant
Merge remote-tracking branch 'nvidia/main' into frames_decoder_gpu
874ed98
@awolant
Update for new FFMPEG
e802a6f 
Signed-off-by: Albert Wolant <awolant@nvidia.com>
@awolant
Fix assertion
2f990c3 
Signed-off-by: Albert Wolant <awolant@nvidia.com>
@awolant
Copy link
Contributor Author

awolant commented Feb 7, 2022

!build

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [3909959]: BUILD STARTED

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [3909959]: BUILD FAILED

@awolant
Fix lint
947400a 
Signed-off-by: Albert Wolant <awolant@nvidia.com>
@awolant
Copy link
Contributor Author

awolant commented Feb 7, 2022

!build

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [3910090]: BUILD STARTED

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [3910090]: BUILD FAILED

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [3910090]: BUILD PASSED

@awolant awolant merged commit 766463b into NVIDIA:main Feb 7, 2022
cyyever pushed a commit to cyyever/DALI that referenced this pull request Feb 21, 2022
@awolant @cyyever
Frames decoder gpu (NVIDIA#3615)
e294a16 
* Add FramesDecoderGpu

Signed-off-by: Albert Wolant <awolant@nvidia.com>
@JanuszL JanuszL mentioned this pull request Mar 30, 2022
Closed
cyyever pushed a commit to cyyever/DALI that referenced this pull request May 13, 2022
@awolant @cyyever
Frames decoder gpu (NVIDIA#3615)
ce89d65 
* Add FramesDecoderGpu

Signed-off-by: Albert Wolant <awolant@nvidia.com>
cyyever pushed a commit to cyyever/DALI that referenced this pull request Jun 7, 2022
@awolant @cyyever
Frames decoder gpu (NVIDIA#3615)
f39d3b4 
* Add FramesDecoderGpu

Signed-off-by: Albert Wolant <awolant@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@stiepan stiepan

@JanuszL JanuszL

Assignees

@stiepan stiepan

@JanuszL JanuszL

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@awolant @dali-automaton @stiepan @JanuszL