nvidia.dali.fn.experimental.decoders.video cannot parse raw HEVC

[zeruniverse]

This sample hevc file is raw HEVC without container (I manually appended ODT suffix as otherwise GitHub won't let me upload it) black.hevc

FFMPEG can parse it successfully with

ffmpeg -i black.hevc 123/%2d.jpg

However, DALI cannot parse this (I used following code, and it said Could not seek to the first frame of video memory filedue toOperation not permitted)

import numpy as np
class ExternalInputIterator(object):
    def __init__(self, batch_size):
        pass

    def __iter__(self):
        self.i = 0
        self.n = 1
        return self

    def __next__(self):
        return [np.frombuffer(open('black.hevc', 'rb').read(), dtype=np.uint8)]
import nvidia.dali as dali
from nvidia.dali import fn
import nvidia.dali.types as types
eii = ExternalInputIterator(1)
@dali.pipeline_def(batch_size=256, num_threads=12, device_id=0)
def i_pipeline():
    vid = fn.external_source(source=eii, device="cpu", name="INPUT_0", ndim=1, dtype=dali.types.UINT8)
    seq = fn.experimental.decoders.video(vid, device='mixed')
    return seq
pipe = i_pipeline()
pipe.build()
pipe.run()

Changing black.hevc to other regular HEVC encoded video with MP4 container, the above code runs successfully.

The sample raw HEVC is generated by code:

import av
import numpy as np
c = av.open('black.hevc', 'w', format='hevc')
s = c.add_stream(codec_name='libx265', rate=1)
s.pix_fmt='gray'
s.height, s.width = [300, 300]
for i in range(10):
    f = av.VideoFrame.from_ndarray(np.zeros((300, 300), dtype=np.uint8),format="gray8")
    for p in s.encode(f):
        c.mux(p)
for p in s.encode():
    c.mux(p)
c.close()

[JanuszL]

Hi @zeruniverse,

Thank you for reporting this problem. We are aware of it (that DALI cannot properly handle raw video files as it needs to seek to the beginning) and working hard to resolve it.
We will keep you posted.

[zeruniverse]

Thanks @JanuszL Please let me know when it is resolved.

[JanuszL]

#4480 should improve the decoding of the video files that are raw streams not packed inside any specific container. Still, this particular video will not work as described in #4483 until a CPU fallback is implemented.