Imgcodec's decode operator #4223
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
szkarpinski
force-pushed
the
imgcodec-operator
branch
5 times, most recently
from
36b3923 to
9cd6c47
Compare
mzient
reviewed
Sep 26, 2022
szkarpinski
force-pushed
the
imgcodec-operator
branch
4 times, most recently
from
8b6d59e to
8cc5230
Compare
szkarpinski
changed the title
mzient
reviewed
Sep 28, 2022
mzient
reviewed
Sep 28, 2022
mzient
reviewed
Sep 28, 2022
mzient
reviewed
Sep 28, 2022
mzient
reviewed
Sep 28, 2022
mzient
reviewed
Sep 28, 2022
mzient
reviewed
Sep 28, 2022
mzient
reviewed
Sep 29, 2022
mzient
reviewed
Sep 29, 2022
mzient
reviewed
Sep 29, 2022
mzient
reviewed
Sep 29, 2022
author Szymon Karpiński <skarpinski@nvidia.com> 1662129348 +0200 committer Michal Zientkiewicz <michalz@nvidia.com> 1665501980 +0200 Add imgcodec decoder Signed-off-by: Szymon Karpiński <skarpinski@nvidia.com> Tests cleanup Signed-off-by: Szymon Karpiński <skarpinski@nvidia.com> Undo C++17 fix Signed-off-by: Szymon Karpiński <skarpinski@nvidia.com> Change old EXIF note Signed-off-by: Szymon Karpiński <skarpinski@nvidia.com> Add missing decoder-specific arguments Signed-off-by: Szymon Karpiński <skarpinski@nvidia.com> Keep persistent ImageDecoder Signed-off-by: Szymon Karpiński <skarpinski@nvidia.com> Use BatchVector Signed-off-by: Szymon Karpiński <skarpinski@nvidia.com> Use CalculateOutputShape Signed-off-by: Szymon Karpiński <skarpinski@nvidia.com> Fix support for ANY_DATA in nvJPEG2K Signed-off-by: Szymon Karpiński <skarpinski@nvidia.com> Review fixes Signed-off-by: Szymon Karpiński <skarpinski@nvidia.com> Remove unused tests Signed-off-by: Szymon Karpiński <skarpinski@nvidia.com> Revert "Fix support for ANY_DATA in nvJPEG2K" This reverts commit 592bceb. Signed-off-by: Szymon Karpiński <skarpinski@nvidia.com> Add tests for peek_shape Signed-off-by: Szymon Karpiński <skarpinski@nvidia.com> Use fn.experimental.peek_image_shape Signed-off-by: Szymon Karpiński <skarpinski@nvidia.com> Use persistent thread pool Signed-off-by: Szymon Karpiński <skarpinski@nvidia.com> Provide source info to ImageSource Signed-off-by: Szymon Karpiński <skarpinski@nvidia.com> Add device_id_ member Signed-off-by: Szymon Karpiński <skarpinski@nvidia.com> Deprecate memory_stats argument Signed-off-by: Szymon Karpiński <skarpinski@nvidia.com> Remove mentions of memory_stats from the docs Signed-off-by: Szymon Karpiński <skarpinski@nvidia.com> Add adjust_orientation to decoder Signed-off-by: Szymon Karpiński <skarpinski@nvidia.com> Extend args of peek_image_shape Signed-off-by: Szymon Karpiński <skarpinski@nvidia.com> Bulk copy + threading. Signed-off-by: Michal Zientkiewicz <michalz@nvidia.com> Issue immediate copy even without fallback. Signed-off-by: Michal Zientkiewicz <michalz@nvidia.com> Start NVJPEG multiple-buffering. Signed-off-by: Michal Zientkiewicz <michalz@nvidia.com> NVPEG multiple buffering. Signed-off-by: Michal Zientkiewicz <michalz@nvidia.com>
* Python linter. Signed-off-by: Michal Zientkiewicz <michalz@nvidia.com>
Signed-off-by: Michal Zientkiewicz <michalz@nvidia.com>
Signed-off-by: Michal Zientkiewicz <michalz@nvidia.com>
Signed-off-by: Michal Zientkiewicz <michalz@nvidia.com>
Signed-off-by: Michal Zientkiewicz <michalz@nvidia.com>
Signed-off-by: Michal Zientkiewicz <michalz@nvidia.com>
Signed-off-by: Michal Zientkiewicz <michalz@nvidia.com>
Signed-off-by: Michal Zientkiewicz <michalz@nvidia.com>
mzient
force-pushed
the
imgcodec-operator
branch
from
116373d to
156649c
Compare
Signed-off-by: Michal Zientkiewicz <michalz@nvidia.com>
JanuszL
reviewed
Nov 14, 2022
| @@ -21,6 +21,7 @@ | |||
| #include <unordered_map> | |||
| #include "dali/imgcodec/decoders/memory_pool.h" | |||
| #include "dali/core/cuda_error.h" | |||
| #include "dali/core/cuda_stream_pool.h" | |||
JanuszL
approved these changes
Nov 14, 2022
jantonguirao
approved these changes
Nov 14, 2022
|
CI MESSAGE: [6485048]: BUILD FAILED |
Signed-off-by: Michal Zientkiewicz <michalz@nvidia.com>
|
CI MESSAGE: [6499506]: BUILD STARTED |
| ForEachThread(*tp_, [&](int tid) noexcept { | ||
| CUDA_CALL(cudaSetDevice(device_id)); | ||
| per_thread_resources_[tid] = {nvjpeg2k_handle_, device_memory_padding, device_id_}; | ||
| AccessOrder s = per_thread_resources_[tid].cuda_stream; |
| @@ -86,7 +86,7 @@ class DLL_PUBLIC NvJpeg2000DecoderInstance : public BatchParallelDecoderImpl { | |||
| } | |||
| }; | |||
|
|
|||
| static constexpr int kNumParallelTiles = 1; // TODO(michalz): Use a different memory resource | |||
| static constexpr int kNumParallelTiles = 2; // TODO(michalz): Use a different memory resource | |||
There was a problem hiding this comment.
I think more is too much - each tile has its own state and associated memory consumption.
jantonguirao
approved these changes
Nov 15, 2022
JanuszL
approved these changes
Nov 15, 2022
Signed-off-by: Michal Zientkiewicz <michalz@nvidia.com>
|
CI MESSAGE: [6499574]: BUILD STARTED |
|
CI MESSAGE: [6499574]: BUILD FAILED |
Signed-off-by: Michal Zientkiewicz <michalz@nvidia.com>
|
CI MESSAGE: [6501260]: BUILD STARTED |
|
CI MESSAGE: [6501260]: BUILD PASSED |
Closed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Category:
New feature (non-breaking change which adds functionality)
Description:
In this PR I add a set of experimental decoding operators which are analogous to the existing ones, but use the new imgcodec decoder architecture, on which we worked during our internship. The added operators are
experimental.decoders.image,experimental.decoders.image_slice,experimental.decoders.image_crop,experimental.decoders.image_random_cropandpeek_shape.Changes in operator functionality
The new operators attempt to mimic the old ones, but also adds a new feature:
dtypeargument, which allows the user to specify the desired output type.Missing features in the decoders
The individual decoders don't support all required features yet, so some tests in
test/python/decoder/test_imgcodec.pyhad to be disabled. Each disabled test is commented with an appropriateTODO, explaining what's missing.The decoder-specific parameters are passed to the
ImageDecoder, but a significant part of them is not yet used by the decoders. This should be fixed in a follow-up once the decoders are complete.Code walkthrough
decoder.h
DecoderBaseclassIt's the abstract base of all decoders, being responsible for:
GetDecoderSpecificArguments)SetupShapes)It also has two virtual functions for implementing ROI handling in the subclasses (more details below):
SetupRoiGenerator, allowing some initial setup of the ROI-handling code based on operator arguments.GetRoi, which is used to ask for ROI for a particular sampleWithCropAttr/WithSliceAttr/WithRandomCropAttrmixinsAll three of those are used to turn an
imageoperator class (eithercpuormixed) into animage_crop,image_sliceorimage_random_cropoperator, respectively. They provide appropriate overloads forSetupRoiGeneratorandGetRoi.They use existing
SliceAttr,CropAttrandRandomCropAttrclasses, and convert theCropWindowGenerators output by them intoROIusingRoiFromCropWindowGeneratorutility function.ImgcodecHostDecoderandImgcodecMixedDecoderThey simply do the actual decoding on CPU or GPU. There's not much in them, because most of operator-related work is done by
DecoderBase, and most of decoder-related work is done by lower-levelImageDecoderclass (#4224). Anyway, the implementation of those is indecoder.cc.decoder.cc
Most of this of this file are op schemas, with the common part extracted to
ImgcodecDecoderAttrschema. Those are copy-pasted from the old decoder, with some minor changes: thedtypeargument is added and an information is added that EXIF metadata is no longer disregarded. The docs would probably need some polishing once we're done adding the features.peek_image_shape.cc/h
The
peek_shapeoperator is implemented here, with most of the code being stolen from the old operator. The actual shape-peeking is moved tooperator_utils.h, because it's used both bypeek_shapeand the decoder.Affected modules and functionalities:
Apart from the actual decoder, this PR contains minor changes to other files, which were required to make the operator working properly. These changes are:
DALI_FAIL'ed. As we need the EXIF header for applying orientation only, such behaviour was way too aggressive. I made the JPEG parser ignore the unreadable EXIF instead of failingKey points relevant for the review:
Tests:
The tests are copy-pasted from
decoders/test_image.py, adjusted to useexperimental.decoders.imageinstead. The tests for missing features are commented-out.Checklist
Documentation
DALI team only
Requirements
REQ IDs: N/A
JIRA TASK: N/A