Docs recipe: stream media with range request

[belese]

Hello,

i'm tryning to stream mp4 video, with range request support.but i cannot manage to make it work with resp.stream.

This code work :

def on_get(self, req, resp):        
        media = 'test.mp4'
        resp.set_header('Content-Type', 'video/mp4')
        resp.accept_ranges = 'bytes'            
        stream = open(media,'rb')
        size = os.path.getsize(media)            
        if req.range :        
                end = req.range[1]
                if end < 0 :
                    end = size + end
                stream.seek(req.range[0])                
                resp.content_range = (req.range[0],end,size)
                size = end - req.range[0] + 1
                resp.status = falcon.HTTP_206
         resp.content_length = size
         resp.body = stream.read(size)

but this will load all file in memory, which is not an option.

if i change the 2 last line with
resp.set_stream(stream,size),

i've got an error

SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /api/stream/557 (ip 10.0.0.136) !!!
uwsgi_response_sendfile_do(): Broken pipe [core/writer.c line 645] during GET /api/stream/557 (10.0.0.136)
IOError: write error

i'm using uwsgi with nginx as reverse proxy. Not sure it's related to falcon, but i don't have any clue where to look at.

Any idea?

Thanks

Johan

Ps : i know it's not optimal to use falcon for this, but i cannot expose real video path to client (in real in come from a db). and performance are not really a problem in my case.

Edit :
Here's the chrome requests when it don't work.

Name	Url	Method	Status	Protocol	type	initiator	size	time
557	http://10.1.12.2/api/stream/557	GET	206	http/1.1	media	Other	32.6 kB	5.24 s
557	http://10.1.12.2/api/stream/557	GET	206	http/1.1	media	Other	28.1 kB	365 ms
557	http://10.1.12.2/api/stream/557	GET	(canceled)		media	Other	0 B	1 ms

[open-collective-bot]

Hi 👋,

Thanks for using Falcon. The large amount of time and effort needed to
maintain the project and develop new features is not sustainable without
the generous financial support of community members like you.

Please consider helping us secure the future of the Falcon framework with a
one-time or recurring donation.

Thank you for your support!

[vytas7]

Relates to: #349

[vytas7]

Hi Johan,
and thanks for filing this issue.

As already briefly discussed on Gitter, someone needs to dig deeper into this and understand what is going on. It would be great to add recipes/articles/tutorials to our docs illustrating how to perform these tasks, even though doing that directly from a Python app server is probably not the most optimal way.

Maybe something in the DIY spirit along the lines of that blog post mentioned on #349, or uWSGI's Fun with Perl, Eyetoy and RaspberryPi. So that people can start tinkering 🙂

[belese]

i made a few test after, and it look likes there's a problem with size.
If i set size to size-1 in set stream (which is not the right size), chrome will make request in loop for one byte (last one), but don't have the uwsgi write error.
i've to go deeper in falcon code to see how stream is handle. (Maybe my code has problem, but as it work if i write the whole body, i'm a bit lost).

This could be related
python-django-streaming-video-mp4-file-using-httpresponse

But as it's for my work, i'll do more testing monday.

[vytas7]

Hi again,
as I hinted on Gitter, it's of utmost importance to set the outgoing range and length headers correctly, as well as make sure that the stream itself produces exactly the advertised amount of data.

I hacked together a simple proof-of-concept that does play reasonably smoothly in Firefox and Chrome.
Needless to say, the code is not production ready as it lacks proper validation, error and edge case handling. Caveat emptor!

I called my mini-app test.py:

import os

import falcon

CHUNK_SIZE = io.DEFAULT_BUFFER_SIZE * 16


class FileWrapper:
    def __init__(self, filename):
        self._fobj = open(filename, 'rb')
        self._stream_start = None
        self._stream_size = None

    def __iter__(self):
        self._fobj.seek(self._stream_start)

        remaining = self._stream_size
        while remaining > 0:
            chunk_size = min(CHUNK_SIZE, remaining)
            yield self._fobj.read(chunk_size)
            remaining -= chunk_size

    def get_file_size(self):
        return os.fstat(self._fobj.fileno()).st_size

    def stream(self, start, size):
        self._stream_start = start
        self._stream_size = size
        return self

    def close(self):
        self._fobj.close()


class Stream:
    def on_get(self, req, resp):
        media_file = FileWrapper('/tmp' + req.path)
        start = 0
        size = full_size = media_file.get_file_size()

        if req.range:
            start, end = req.range
            if end < 0:
                end = full_size - 1
            size = end - start + 1

            resp.content_range = (start, end, full_size)
            resp.status = falcon.HTTP_206

        resp.content_length = size
        resp.content_type = 'video/webm'
        resp.stream = media_file.stream(start, size)


app = falcon.API()
app.add_route('/tears_of_steel_1080p.webm', Stream())

I ran with gunicorn --workers 4 test:app just to test, then played http://localhost:8000/tears_of_steel_1080p.webm in the browser.

Edit: it works fine using uwsgi --http :8000 --workers 4 --wsgi-file test.py --callable app too. I am seeing some of those uWSGI errors, but I think that's normal for aborted requests, since the WSGI spec doesn't really offer any affordances to handle that part.

[belese]

Many thanks! Will do testing and let you know.

[vytas7]

@belese Slightly off-topic, but if you're interested to dig deeper into how client disconnects are handled across different WSGI app servers, see also this comment (by me): #1483 (comment) . And browsers do disconnect the ongoing streaming request if you seek further away from the current position, and issue a new range request.

[belese]

I've just test your code in my app, and yes it's just working fine!

i still have a canceled request in chrome, but video is playing fine.
First request is all file, second is to read metadata at end, third is canceled, and the fourth request start playing

687 | http://10.1.12.2/api/stream/687 | GET | (canceled) |   | media | Other | 0 B | 46 ms

Edit : About cancelling, it's seems ok for chrome while caching media..

[vytas7]

@belese Adding to my previous responses, if you are interested in further streamlining the browser interaction, you might benefit from generating an ETag, and setting Cache-Control and Last-Modified headers on your responses; as well as support conditional requests via If-None-Match and If-Modified-Since.

[belese]

@vytas7 Thanks for info. In my case, it's circular movie record from a security camera. i guess 99.9% of them will never be seen. But i had to have a preview in browser. And with you snippet, for 5 minutes records length, there's very few bufferring time which is already perfect for my use case. But in future, if could be an enhancement (lot of other work for now), I let you know what i've changed. And again, many thanks for your support.

[vytas7]

Closing this since range support has been added natively to static routes as per #1858.