Skip to content

Commit

Permalink
rocAL - Video pipeline Support (#640)
Browse files Browse the repository at this point in the history 
* Fix a few bugs

* caffe and caffe2 changes for optimization

* removed commented code

* Fix issue with SSD meta node.

* Add support for original width and height for TF Detection

* Add change required to reflect mean and std values in an images pixel value

* Clean codes

* Hardcoding the key values for tf detection and classification

* Release RingBuffer memory

* Modify RALI API's to return the bbox coords and bbox labels for all images in the output batch

* Update rali_unittest.cpp

* Clean repo

* Add batching support in PYTHON API for labels,bboxes,img_sizes excluding image_names

* Add support for image names for batching support

* Merge RALI_Upgrade

* Update raliunittest.cpp

* Add support for bytes instead of str in rali pybind

* Code clean Up

* Fix codacy issues

* Fix codacy issues

* Fix indendation error

* Fix codacy warnings

* Fix trailing spaces warnings

* Fix scope of the variable can be reduced warning

* Fix errors in the RALI API

* Fix Codacy warnings

* Remove extra empty lines

* Add support for Box Encoder in coco_pipeline.py

* Add codes to retrieve meta data information in loader module

* Add support for One Hot Labels for all classification based Readers

* Add codes to retrieve meta data information in decoder module

* Introduce data loader for coco reader using partail decoder

* Add casting Support for Encoded Labels

* Add support for RandomBBoxCrop augmenation

* Clean codes

* Introduce RandomBBoxCrop_MetaData Reader to store the crop params returned by RandomBBoxCrop function

* Update RandomBBoxCrop_MetaData Reader

* Add meta data update support for both vertical and horizontal flip.

* Add RandomBBox support.
Introduced map to store image name and crop generated by randomBbox.
Look up to fetch CropCordsBatch.
Get functionality to get crop wrt to image_name as key.
Fixed the seg fault.

* Add support for BBFlip

* Add support for Random BBox Crop Reader to be part of load routine.
Fetches the crop of the image to be decodes and does partial decoding for crop part.

* Fix the warnings.

* Fix issue with the meta data updation in master graph.

* Add API changes.
Fix issues with  RandomBBoxCrop algorithm.

* Add support for Random BBox Crop & ImageDecoderSlice

* Small changes in Box Encoder

* Small Change in Anchor boxes input comment

* Add changes for Crop Dim exceding Image Dim

* Minor Changes for RandomBboxCrop

* Fix issues with RandomBboxCrop.

* Fix Reader seg fault issue.

* Add minor Changes for fp16

* Minor Changes

* Clean Codes

* Minor Changes for Random BBox Crop

* Add support for Multi-GPU

* Minor Changes for Multi-GPU support in COCO file souce partial

* Remove unwanted code.

* Minor Changes in RandomBBoxCrop

* Fix Random BBox Crop

* Comment out the print statements

* Minor Changes for RBBOX

* Code clean up.

* Fix warnings wrt Ubuntu 20.04

* Resolve codacy warnings

* Resolve codacy warnings

* Fix PR issues.

* Revert back Slice to Crop

* Resolve Codacy warnings

* Resolve codacy issues

* Resolve Minor codacy issue

* Fix issue to make branch compatible with AMDRPP master TOT.

* Fix the crop_x difference in partial decoding image crop

* Fix issue with crop fixed.
* Crop_x & crop_y value fixed.
* Check bounday conditions and update crop params.

* Fix the crop_width difference in partial decoding image crop

* Change wrt invalid_bboxes.

* Rewrite RandomBBoxCrop Code

* Fix issue with random generation.
Used randomdevice for seed when initializing random param.

* Fix Key value zero error.
Error introduced by partial decoding crop correction.
Fixed by adjusting the calculation of top and right.

* Add codes for Video Reader and Loader module

(cherry picked from commit d97ab927e9a6597fde666f7dec50ba987259dccf)

* Changes in image_augmentation library

(cherry picked from commit 153f59bd377cf7cbd87ecf58d722cd88ba79ff22)

* Fix Build issues

(cherry picked from commit e023b104c9a220e4745b83693b6c38a7411b54fc)

* Fix Build errors

(cherry picked from commit 908c599fe7b300dfd837a87215b33295bc2752a9)

* Add reader type for video reader

(cherry picked from commit 843a6cd6a6436513b4c000813a48b66461768ecc)

* Adding codes to decode video input file

(cherry picked from commit 2c249d7312e0c746fac600b5e38c5b4cb16f1910)

* Introduce Video Decoder module to decode video files

(cherry picked from commit 4d18ea384a2599aed3b0a0c975d7cd0a343720d2)

* Add decoder functions in FFMPEG_VIDEO_DECODER

(cherry picked from commit 02224f9601ee4269c68f6725a19468a171718276)

* Clean Video Decoder codes

(cherry picked from commit 8699179b282aa7870f60ca670a19b512ca45d9ac)

* Clean codes to remove build issues

(cherry picked from commit 39c5f45ff875d111f2977af0487e9cf8d22174c2)

* Clean codes

* Initial changes for video reader pipeline. [NBC]
* To handle sequence length.
* To handle shuffle.

* Temp local changes. [NYC]

* Changes in the video reader pipeline. [NWC]

* Video Reader changes

* Fix the segmentation fault in the video reader pipeline

* Add support to save decoded output frames in video decoder

* Working Pipeline - Single Video file input

Add support to modify internal and user batch size in master graph
Add ffmpeg seek operation

* Minor Changes

* Add support for decoding multiple video files and shuffle

* Add support to initialize ffmpeg context for each video decoder instance

* Code cleanup

* Fix issue in Shuffling the images in video reader

* Add seek_frame function in video decoder

* Code clean up

* Update rali_unittest

* Add folder based label meta data reader for video reader

* Add support for Sequence Reader in RALI

* Fix codacy issue

* Add Sequence Rearrange initial setup.
Works only for sequence length equal to video reader.
Introduce ovx node sequence rearrange to support.
Introduce API in rali_api_augmentations.

* Fix issue with Sequence Rearrange with different sequence length.

* Introduce raliVideoFileResize node in RALI to fuse video decoding and resize

* Add new_sequence_length parameter to sequence rearrange

* Add sequence rearrange algorithm for RGB images

* Add support for Sequence Reader in RALI

* Fix random shuffling of sequences in video reader

* Add support for folder based reader and label support for video decoder and labels.

* Clean codes

* Fix issue in raliVideoReaderResize

* Code clean up.
* set batchsize to internal batch size in video pipeline loaders.
* Add flag in master graph to switch between video and image pipelines.

* Add step and stride parameter to VideoReader and SequenceReader

* Fix issue with the sequence rearrange.

* Adjust remaining image count in master graph wrt sequence rearrange.

* Add meta data support for video reader folder based.

* Update decode image info name according to stride

* Minor bug fix

* Add support for text file input

Add support to fetch video properties from text file
Modify reader to read from the start to end frame specified in text file
Add meta data support for text file input to the video reader

* Add support to process repeated file inputs in text file

* Add meta data reader support to parse timestamps from text file

Introduce enable_timestamps parameter and set_timestamps_bool to the meta data readers

* Add rali_video_unittests

Video Reader
Vidoe Reader Resize
Sequence Reader
Sequence Rearramge

* Code clean up

* Fix maximum limit for decoder instance creation.
Check if instance is there for the video file if not initialize one using previously created instance.

* Fix warnings.

* Minor fix

* Add file_list_frame_num parameter

To switch between timestamp or frame number input passed with text file

* Add data samples for testing

Add video samples
Add coco sample data with 10 images for train and val

* Add support to generate frame number and timestamps output

* Fix multiple video file input to video pipeline

* Add labelled video folder samples

* Modified test suite

Modified rali_video_unittests.cpp
Add testScript.sh to build and execute rali_video_unittests
Remove video pipeline tests from rali_unittests.cpp

* Code clean up

* Modify frame_rate variable

* Add step and stride parameters to SequenceReaderSingleSharded

* Minor fix

* Modify ffmpeg video decoder functions

Initialize the ffmpeg context once for each video file

* Fix ffmpeg deprecation warnings

* Modify ffmpeg video decoder

Add width, height, stride and pixel format paramters to Decode

* Code clean up
Change Video label reader folders to Video label reader

* Remove text file input parameter to dataloader

* Add support to check variable frame rate videos

* Minor changes

* Minor fix

* Code clean up

* Code clean up

* Change rali to rocAL

* Merge branch 'AMD-Master' into video_devel

* Resolve build issues
Code clean up

* Fix bug with Sequence Rearrange

* Add sharding support to Video Reader

* Add sharding support for Sequence Reader

* Introduce decoder mode parameter

* Add U8 support for Sequence Rearrange
Minor changes

* Add SingleShard API for video readers

* Add support to decode more than one sequence

Modify the load routine to decode more than one sequence
Add sequence count parameter to Sequence rearrange

* Merge branch 'video_devel_PR' of https://github.com/MCW-Dev/MIVISION into video_devel_PR

* Fix SequenceReader and SequenceReaderSingleShard

* Resolve merge conflicts

* Minor fix

* Add codes for multithreading

* Fix build isssue with HIP backend

* Fix warnings

* Resolve codacy issues

Remove blank lines
Adjust spacing

* Resolve codacy issues

* Modify the sequence reader arguments of the ImageLoaderNode

* Remove rocAL sample data

* Minor changes

Add RALI_VIDEO flag to few files

* Add seperate VideoReader

Introduce VideoFilesourceReader and VideoReaderConfig

* Introduce SequenceInfo struct

Minor changes

* Fix codacy issues in video unit test testScript.sh

* Minor fix

* Minor bug fix

* Introduce the latest FFmpeg API in ffmpeg_video_decoder.cpp

* Merge branch 'PR_changes' of https://github.com/fiona-gladwin/MIVisionX into video_devel_PR

* Video Pipeline changes

* Batch size changes for Video Reader

* Video Pipeline Meta data reader changes to store the meta data for each sequence and not for every frame in the sequence

* Code cleanup - Video Reader changes

* Batch size variable changes

Change batch size and internal batch size variables to constant.
Introduce batch size and batch ratio variables for the Sequence Reader in master graph.

* Change datatype of frame_rate to float

* Enable HIP Backend support for video pipeline

* Add codes to dump the images in each batch as AVI video file

* Minor change in video unit tests

* Add HIP backend support for sequence rearrange

* Add OpenCL backend support for sequence rearrange

* Add condtion to disable ResizeNode update in raliVideoFileResize if resize width and height is same as the videos

* PR changes

* Fix single folder of images issue in Sequence Reader

* Fix codacy issues

* PR changes

* API changes

* Introduce separate output routine for the video pipeline

* PR changes

* PR changes

* Fix for codacy issues

Co-authored-by: LokeshBonta <lokeswara@multicorewareinc.com>
Co-authored-by: Swetha B S <swetha@multicorewareinc.com>
Co-authored-by: shobana-mcw <shobana@multicorewareinc.com>
Co-authored-by: fionagladwin <fionagladwin@multicorewareinc.com>
Co-authored-by: r-abishekmcw <abishek@multicorewareinc.com>
  • Loading branch information
@Indumathi31 @swetha097
@shobana-mcw @fiona-gladwin @r-abishek
6 people authored Nov 3, 2021
1 parent 2751608 commit a36e62a
Show file tree
Hide file tree
Showing 89 changed files with 5,409 additions and 350 deletions.
1 change: 0 additions & 1 deletion .gitignore
View file
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
build*
docker_build*
hip_build*
.vscode*
opencv-3.4.0/*
openvx_test_results*
Expand Down
1 change: 1 addition & 0 deletions amd_openvx_extensions/amd_rpp/CMakeLists.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,7 @@ list(APPEND SOURCES
source/RotatebatchPD.cpp
source/SaturationbatchPD.cpp
source/ScalebatchPD.cpp
source/SequenceRearrange.cpp
source/SnowbatchPD.cpp
source/SobelbatchPD.cpp
source/SubtractbatchPD.cpp
Expand Down
2 changes: 2 additions & 0 deletions amd_openvx_extensions/amd_rpp/include/internal_publishKernels.h
View file
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,7 @@ vx_status ResizeCropMirrorPD_Register(vx_context);
vx_status RotatebatchPD_Register(vx_context);
vx_status SaturationbatchPD_Register(vx_context);
vx_status ScalebatchPD_Register(vx_context);
vx_status SequenceRearrange_Register(vx_context);
vx_status SnowbatchPD_Register(vx_context);
vx_status SobelbatchPD_Register(vx_context);
vx_status SubtractbatchPD_Register(vx_context);
Expand Down Expand Up @@ -195,5 +196,6 @@ vx_status WarpPerspectivebatchPD_Register(vx_context);
#define VX_KERNEL_RPP_CROPMIRRORNORMALIZEBATCHPD_NAME "org.rpp.CropMirrorNormalizebatchPD"
#define VX_KERNEL_RPP_CROPPD_NAME "org.rpp.CropPD"
#define VX_KERNEL_RPP_RESIZECROPMIRRORPD_NAME "org.rpp.ResizeCropMirrorPD"
#define VX_KERNEL_RPP_SEQUENCEREARRANGE_NAME "org.rpp.SequenceRearrange"

#endif //_AMDVX_EXT__PUBLISH_KERNELS_H_
19 changes: 10 additions & 9 deletions amd_openvx_extensions/amd_rpp/include/kernels_rpp.h
View file
Original file line number Diff line number Diff line change
Expand Up @@ -102,15 +102,16 @@ extern "C"
VX_KERNEL_RPP_SCALEBATCHPD = VX_KERNEL_BASE(VX_ID_AMD, VX_LIBRARY_RPP) + 0x45,
VX_KERNEL_RPP_SATURATIONBATCHPD = VX_KERNEL_BASE(VX_ID_AMD, VX_LIBRARY_RPP) + 0x46,
VX_KERNEL_RPP_SOBELBATCHPD = VX_KERNEL_BASE(VX_ID_AMD, VX_LIBRARY_RPP) + 0x47,
VX_KERNEL_RPP_THRESHOLDINGBATCHPD = VX_KERNEL_BASE(VX_ID_AMD, VX_LIBRARY_RPP) + 0x48,
VX_KERNEL_RPP_TENSORADD = VX_KERNEL_BASE(VX_ID_AMD, VX_LIBRARY_RPP) + 0x49,
VX_KERNEL_RPP_TENSORSUBTRACT = VX_KERNEL_BASE(VX_ID_AMD, VX_LIBRARY_RPP) + 0x4a,
VX_KERNEL_RPP_TENSORMULTIPLY = VX_KERNEL_BASE(VX_ID_AMD, VX_LIBRARY_RPP) + 0x4b,
VX_KERNEL_RPP_TENSORMATRIXMULTIPLY = VX_KERNEL_BASE(VX_ID_AMD, VX_LIBRARY_RPP) + 0x4c,
VX_KERNEL_RPP_TENSORLOOKUP = VX_KERNEL_BASE(VX_ID_AMD, VX_LIBRARY_RPP) + 0x4d,
VX_KERNEL_RPP_VIGNETTEBATCHPD = VX_KERNEL_BASE(VX_ID_AMD, VX_LIBRARY_RPP) + 0x4e,
VX_KERNEL_RPP_WARPAFFINEBATCHPD = VX_KERNEL_BASE(VX_ID_AMD, VX_LIBRARY_RPP) + 0x4f,
VX_KERNEL_RPP_WARPPERSPECTIVEBATCHPD = VX_KERNEL_BASE(VX_ID_AMD, VX_LIBRARY_RPP) + 0x50,
VX_KERNEL_RPP_SEQUENCEREARRANGE = VX_KERNEL_BASE(VX_ID_AMD, VX_LIBRARY_RPP) +0x48,
VX_KERNEL_RPP_THRESHOLDINGBATCHPD = VX_KERNEL_BASE(VX_ID_AMD, VX_LIBRARY_RPP) + 0x49,
VX_KERNEL_RPP_TENSORADD = VX_KERNEL_BASE(VX_ID_AMD, VX_LIBRARY_RPP) + 0x4a,
VX_KERNEL_RPP_TENSORSUBTRACT = VX_KERNEL_BASE(VX_ID_AMD, VX_LIBRARY_RPP) + 0x4b,
VX_KERNEL_RPP_TENSORMULTIPLY = VX_KERNEL_BASE(VX_ID_AMD, VX_LIBRARY_RPP) + 0x4c,
VX_KERNEL_RPP_TENSORMATRIXMULTIPLY = VX_KERNEL_BASE(VX_ID_AMD, VX_LIBRARY_RPP) + 0x4d,
VX_KERNEL_RPP_TENSORLOOKUP = VX_KERNEL_BASE(VX_ID_AMD, VX_LIBRARY_RPP) + 0x4e,
VX_KERNEL_RPP_VIGNETTEBATCHPD = VX_KERNEL_BASE(VX_ID_AMD, VX_LIBRARY_RPP) + 0x4f,
VX_KERNEL_RPP_WARPAFFINEBATCHPD = VX_KERNEL_BASE(VX_ID_AMD, VX_LIBRARY_RPP) + 0x50,
VX_KERNEL_RPP_WARPPERSPECTIVEBATCHPD = VX_KERNEL_BASE(VX_ID_AMD, VX_LIBRARY_RPP) + 0x51
};

#ifdef __cplusplus
Expand Down
2 changes: 1 addition & 1 deletion amd_openvx_extensions/amd_rpp/include/vx_ext_rpp.h
View file
Original file line number Diff line number Diff line change
Expand Up @@ -125,7 +125,7 @@ extern "C" SHARED_PUBLIC vx_node VX_API_CALL vxExtrppNode_ThresholdingbatchPD(v
extern "C" SHARED_PUBLIC vx_node VX_API_CALL vxExtrppNode_VignettebatchPD(vx_graph graph,vx_image pSrc,vx_array srcImgWidth,vx_array srcImgHeight,vx_image pDst,vx_array stdDev,vx_uint32 nbatchSize);
extern "C" SHARED_PUBLIC vx_node VX_API_CALL vxExtrppNode_WarpAffinebatchPD(vx_graph graph,vx_image pSrc,vx_array srcImgWidth,vx_array srcImgHeight,vx_image pDst,vx_array dstImgWidth,vx_array dstImgHeight,vx_array affine,vx_uint32 nbatchSize);
extern "C" SHARED_PUBLIC vx_node VX_API_CALL vxExtrppNode_WarpPerspectivebatchPD(vx_graph graph,vx_image pSrc,vx_array srcImgWidth,vx_array srcImgHeight,vx_image pDst,vx_array dstImgWidth,vx_array dstImgHeight,vx_array perspective,vx_uint32 nbatchSize);

extern "C" SHARED_PUBLIC vx_node VX_API_CALL vxExtrppNode_SequenceRearrange(vx_graph graph,vx_image pSrc,vx_image pDst, vx_array newOrder,vx_uint32 newSequenceLength, vx_uint32 sequenceLength, vx_uint32 sequenceCount);
#ifdef __cplusplus
}
#endif
Expand Down
300 changes: 300 additions & 0 deletions amd_openvx_extensions/amd_rpp/source/SequenceRearrange.cpp
View file

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions amd_openvx_extensions/amd_rpp/source/internal_publishKernels.cpp
View file
Original file line number Diff line number Diff line change
Expand Up @@ -123,6 +123,7 @@ vx_status get_kernels_to_publish()
STATUS_ERROR_CHECK(ADD_KERENEL(ResizeCropMirrorPD_Register));
STATUS_ERROR_CHECK(ADD_KERENEL(Copy_Register));
STATUS_ERROR_CHECK(ADD_KERENEL(Nop_Register));
STATUS_ERROR_CHECK(ADD_KERENEL(SequenceRearrange_Register));
return status;
}

Expand Down
24 changes: 24 additions & 0 deletions amd_openvx_extensions/amd_rpp/source/kernel_rpp.cpp
View file
Original file line number Diff line number Diff line change
Expand Up @@ -1800,6 +1800,30 @@ VX_API_CALL vx_node VX_API_CALL vxExtrppNode_Nop(vx_graph graph, vx_image pSrc,
return node;
}

VX_API_CALL vx_node VX_API_CALL vxExtrppNode_SequenceRearrange(vx_graph graph,vx_image pSrc,vx_image pDst, vx_array newOrder, vx_uint32 newSequenceLength, vx_uint32 sequenceLength, vx_uint32 sequenceCount)
{
vx_node node = NULL;
vx_context context = vxGetContext((vx_reference)graph);
if(vxGetStatus((vx_reference)context) == VX_SUCCESS) {
vx_uint32 dev_type = getGraphAffinity(graph);
vx_scalar DEV_TYPE = vxCreateScalar(vxGetContext((vx_reference)graph), VX_TYPE_UINT32, &dev_type);
vx_scalar NEWSEQUENCELENGTH = vxCreateScalar(vxGetContext((vx_reference)graph), VX_TYPE_UINT32, &newSequenceLength);
vx_scalar SEQUENCELENGTH = vxCreateScalar(vxGetContext((vx_reference)graph), VX_TYPE_UINT32, &sequenceLength);
vx_scalar SEQUENCECOUNT = vxCreateScalar(vxGetContext((vx_reference)graph), VX_TYPE_UINT32, &sequenceCount);
vx_reference params[] = {
(vx_reference) pSrc,
(vx_reference) pDst,
(vx_reference) newOrder,
(vx_reference) NEWSEQUENCELENGTH,
(vx_reference) SEQUENCELENGTH,
(vx_reference) SEQUENCECOUNT,
(vx_reference) DEV_TYPE
};
node = createNode(graph, VX_KERNEL_RPP_SEQUENCEREARRANGE, params, 7);
}
return node;
}

// utility functions
vx_node createNode(vx_graph graph, vx_enum kernelEnum, vx_reference params[], vx_uint32 num)
{
Expand Down
6 changes: 4 additions & 2 deletions apps/image_augmentation/image_augmentation.cpp
View file
Original file line number Diff line number Diff line change
Expand Up @@ -123,13 +123,15 @@ int main(int argc, const char ** argv)

if(video_mode != 0)
{
unsigned sequence_length = 3;
unsigned frame_step = 3;
unsigned frame_stride = 1;
if (decode_height <= 0 || decode_width <= 0)
{
std::cout << "Output width and height is needed for video decode\n";
return -1;
}
input1 = raliVideoFileSource(handle, folderPath1, color_format, ((video_mode == 1) ? RaliDecodeDevice::RALI_HW_DECODE:RaliDecodeDevice::RALI_SW_DECODE)
, true, decode_width, decode_height, false);
input1 = raliVideoFileSource(handle, folderPath1, color_format, ((video_mode == 1) ? RaliDecodeDevice::RALI_HW_DECODE:RaliDecodeDevice::RALI_SW_DECODE), shard_count, sequence_length, frame_step, frame_stride, shuffle, true, false);
}
else
{
Expand Down
21 changes: 16 additions & 5 deletions rocAL/rocAL/CMakeLists.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,6 @@ if(GPU_SUPPORT)
find_package(AMDRPP QUIET)
find_package(TurboJpeg QUIET)
find_package(Protobuf QUIET)
# If FFMPEG is found, vx_amd_media is expected to be built as well
find_package(FFmpeg QUIET)
find_package(OpenCV QUIET)
else()
Expand Down Expand Up @@ -155,7 +154,19 @@ if(${BUILD_RALI})
include/rali_api_meta_data.h
DESTINATION ./include
)
if(FFMPEG_FOUND)
find_path(AVUTIL_INCLUDE_DIR libavutil/avutil.h)
find_path(AVCODEC_INCLUDE_DIR libavcodec/avcodec.h)
find_path(SWSCALE_INCLUDE_DIR libswscale/swscale.h)
find_path(AVFORMAT_INCLUDE_DIR libavformat/avformat.h)
endif()
include_directories(
${OpenCL_INCLUDE_DIRS}
${OpenCL_INCLUDE_DIRS}/Headers
${AVUTIL_INCLUDE_DIR}
${AVCODEC_INCLUDE_DIR}
${SWSCALE_INCLUDE_DIR}
${AVFORMAT_INCLUDE_DIR}
${CMAKE_CURRENT_BINARY_DIR}
${TurboJpeg_INCLUDE_DIRS}
${PROTOBUF_INCLUDE_DIRS}
Expand Down Expand Up @@ -203,17 +214,17 @@ if(${BUILD_RALI})
else()
target_compile_definitions(${PROJECT_NAME} PUBLIC ENABLE_OPENCV=0)
endif()
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -O3 -fopenmp -msse4.2 -mavx2 -Wall -fPIC -pg -pthread -std=c++14")
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -O3 -fopenmp -msse4.2 -mavx2 -Wall -fPIC -pg -pthread -std=c++14 -Wno-deprecated-declarations")
message("-- ${Green}rocAL - CMAKE_CXX_FLAGS:${CMAKE_CXX_FLAGS}")

if(NOT FFMPEG_FOUND)
message("-- ${Yellow}rocAL library is going to be built without video decode functionality ${ColourReset}")
target_link_libraries(${PROJECT_NAME} -fPIC ${PROTOBUF_LIBRARIES} lmdb boost_system boost_filesystem turbojpeg openvx vx_rpp )
target_link_libraries(${PROJECT_NAME} -fPIC ${PROTOBUF_LIBRARIES} lmdb boost_system boost_filesystem turbojpeg openvx vx_rpp)
else()
message("-- ${Green}rocAL library is going to be built with video decode functionality ${ColourReset}")
target_link_libraries(${PROJECT_NAME} -DRALI_VIDEO -fPIC ${PROTOBUF_LIBRARIES} lmdb turbojpeg openvx vx_rpp )
target_compile_definitions(${PROJECT_NAME} PUBLIC -DRALI_VIDEO)
target_link_libraries(${PROJECT_NAME} -fPIC ${PROTOBUF_LIBRARIES} ${FFMPEG_LIBRARIES} lmdb turbojpeg openvx vx_rpp)
endif()

if("${BACKEND}" STREQUAL "HIP" AND HIP_FOUND)
target_link_libraries(${PROJECT_NAME} $<TARGET_OBJECTS:rocAL_hip>)
endif()
Expand Down
4 changes: 2 additions & 2 deletions rocAL/rocAL/include/caffe2_lmdb_record_reader.h
View file
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ class Caffe2LMDBRecordReader : public Reader
\param buf User's provided buffer to receive the loaded images
\return Size of the loaded resource
*/
size_t read(unsigned char* buf, size_t max_size) override;
size_t read_data(unsigned char* buf, size_t max_size) override;
//! Opens the next file in the folder
/*!
\return The size of the next file, 0 if couldn't access it
Expand All @@ -60,7 +60,7 @@ class Caffe2LMDBRecordReader : public Reader
//! Returns the id of the latest file opened
std::string id() override { return _last_id;};

unsigned count() override;
unsigned count_items() override;

~Caffe2LMDBRecordReader() override;

Expand Down
1 change: 1 addition & 0 deletions rocAL/rocAL/include/caffe2_meta_data_reader.h
View file
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ public :
void release(std::string image_name);
void release() override;
void print_map_contents();
bool set_timestamp_mode() override { return false; }
MetaDataBatch * get_output() override { return _output; }
Caffe2MetaDataReader();
~Caffe2MetaDataReader() override { delete _output; }
Expand Down
1 change: 1 addition & 0 deletions rocAL/rocAL/include/caffe2_meta_data_reader_detection.h
View file
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ public :
void release(std::string image_name);
void release() override;
void print_map_contents();
bool set_timestamp_mode() override { return false; }
MetaDataBatch * get_output() override { return _output; }
Caffe2MetaDataReaderDetection();
~Caffe2MetaDataReaderDetection() override { delete _output; }
Expand Down
4 changes: 2 additions & 2 deletions rocAL/rocAL/include/caffe_lmdb_record_reader.h
View file
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ class CaffeLMDBRecordReader : public Reader{
\param buf User's provided buffer to receive the loaded images
\return Size of the loaded resource
*/
size_t read(unsigned char* buf, size_t max_size) override;
size_t read_data(unsigned char* buf, size_t max_size) override;
//! Opens the next file in the folder
/*!
\return The size of the next file, 0 if couldn't access it
Expand All @@ -59,7 +59,7 @@ class CaffeLMDBRecordReader : public Reader{
//! Returns the id of the latest file opened
std::string id() override { return _last_id;};

unsigned count() override;
unsigned count_items() override;

~CaffeLMDBRecordReader() override;

Expand Down
1 change: 1 addition & 0 deletions rocAL/rocAL/include/caffe_meta_data_reader.h
View file
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ public :
void read_all(const std::string& path) override;
void release(std::string image_name);
void release() override;
bool set_timestamp_mode() override { return false; }
void print_map_contents();
MetaDataBatch * get_output() override { return _output; }
CaffeMetaDataReader();
Expand Down
1 change: 1 addition & 0 deletions rocAL/rocAL/include/caffe_meta_data_reader_detection.h
View file
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ public :
void read_all(const std::string& path) override;
void release(std::string image_name);
void release() override;
bool set_timestamp_mode() override { return false; }
void print_map_contents();
MetaDataBatch * get_output() override { return _output; }
CaffeMetaDataReaderDetection();
Expand Down
4 changes: 2 additions & 2 deletions rocAL/rocAL/include/cifar10_data_reader.h
View file
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ class CIFAR10DataReader : public Reader {
\param buf User's provided buffer to receive the loaded images
\return Size of the loaded resource
*/
size_t read(unsigned char* buf, size_t max_size) override;
size_t read_data(unsigned char* buf, size_t max_size) override;
//! Opens the next file in the folder
/*!
\return The size of the next file, 0 if couldn't access it
Expand All @@ -53,7 +53,7 @@ class CIFAR10DataReader : public Reader {
//! Returns the name of the latest data_id opened
std::string id() override { return _last_id;};

unsigned count() override;
unsigned count_items() override;

~CIFAR10DataReader() override;

Expand Down
1 change: 1 addition & 0 deletions rocAL/rocAL/include/cifar10_meta_data_reader.h
View file
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ public :
void release(std::string image_name);
void release() override;
void print_map_contents();
bool set_timestamp_mode() override { return false; }
MetaDataBatch * get_output() override { return _output; }
Cifar10MetaDataReader();
~Cifar10MetaDataReader() override { delete _output; }
Expand Down
4 changes: 2 additions & 2 deletions rocAL/rocAL/include/coco_file_source_reader.h
View file
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ class COCOFileSourceReader : public Reader {
\param buf User's provided buffer to receive the loaded images
\return Size of the loaded resource
*/
size_t read(unsigned char* buf, size_t max_size) override;
size_t read_data(unsigned char* buf, size_t max_size) override;
//! Opens the next file in the folder
/*!
\return The size of the next file, 0 if couldn't access it
Expand All @@ -36,7 +36,7 @@ class COCOFileSourceReader : public Reader {
//! Returns the name of the latest file opened
std::string id() override { return _last_id;};

unsigned count() override;
unsigned count_items() override;
unsigned long long get_shuffle_time() {return _shuffle_time.get_timing();};

~COCOFileSourceReader() override;
Expand Down
1 change: 1 addition & 0 deletions rocAL/rocAL/include/coco_meta_data_reader.h
View file
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ class COCOMetaDataReader: public MetaDataReader
void release(std::string image_name);
void release() override;
void print_map_contents();
bool set_timestamp_mode() override { return false; }
MetaDataBatch * get_output() override { return _output; }
std::map<std::string, std::shared_ptr<BoundingBox>> get_map_content() { return _map_content;}
COCOMetaDataReader();
Expand Down
13 changes: 13 additions & 0 deletions rocAL/rocAL/include/commons.h
View file
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,16 @@ enum class RaliMemType
HIP
};

/*! \brief Decoder mode for Video decoding
*
* Currently supports Software decoding, will support Hardware decoding in future
*/
enum class DecodeMode
{
HW_VAAPI = 0,
CPU = 1
};

struct Timing
{
// The following timings are accumulated timing not just the most recent activity
Expand All @@ -86,4 +96,7 @@ struct Timing
long long unsigned bb_load_time= 0;
long long unsigned mask_load_time = 0;
long long unsigned shuffle_time = 0;
long long unsigned video_read_time= 0;
long long unsigned video_decode_time= 0;
long long unsigned video_process_time= 0;
};
48 changes: 48 additions & 0 deletions rocAL/rocAL/include/ffmpeg_video_decoder.h
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
/*
Copyright (c) 2019 - 2021 Advanced Micro Devices, Inc. All rights reserved.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
*/

#pragma once

#include "video_decoder.h"

#ifdef RALI_VIDEO
class FFmpegVideoDecoder : public VideoDecoder
{
public:
//! Default constructor
FFmpegVideoDecoder();
VideoDecoder::Status Initialize(const char *src_filename) override;
VideoDecoder::Status Decode(unsigned char *output_buffer, unsigned seek_frame_number, size_t sequence_length, size_t stride, int out_width, int out_height, int out_stride, AVPixelFormat out_format) override;
int seek_frame(AVRational avg_frame_rate, AVRational time_base, unsigned frame_number) override;
void release() override;
~FFmpegVideoDecoder() override;
private:
const char *_src_filename = NULL;
AVFormatContext *_fmt_ctx = NULL;
AVCodecContext *_video_dec_ctx = NULL;
AVCodec *_decoder = NULL;
AVStream *_video_stream = NULL;
int _video_stream_idx = -1;
AVPixelFormat _dec_pix_fmt;
int _codec_width, _codec_height;
};
#endif
4 changes: 2 additions & 2 deletions rocAL/rocAL/include/file_source_reader.h
View file
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ class FileSourceReader : public Reader {
\param buf User's provided buffer to receive the loaded images
\return Size of the loaded resource
*/
size_t read(unsigned char* buf, size_t max_size) override;
size_t read_data(unsigned char* buf, size_t max_size) override;
//! Opens the next file in the folder
/*!
\return The size of the next file, 0 if couldn't access it
Expand All @@ -55,7 +55,7 @@ class FileSourceReader : public Reader {
//! Returns the name of the latest file opened
std::string id() override { return _last_id;};

unsigned count() override;
unsigned count_items() override;

~FileSourceReader() override;

Expand Down
1 change: 1 addition & 0 deletions rocAL/rocAL/include/label_reader_folders.h
View file
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ public :
void release(std::string image_name);
void release() override;
void print_map_contents();
bool set_timestamp_mode() override { return false; }
MetaDataBatch * get_output() override { return _output; }
LabelReaderFolders();
~LabelReaderFolders() override { delete _output; }
Expand Down
Loading

0 comments on commit a36e62a

Please sign in to comment.