Support streaming upload of Directories #360
Labels
kind:enhancement
A net-new feature or an improvement to an existing feature
status:blocked
Unable to be worked further until needs are met (see 'need:*' category)
type:issue
Currently, it is possible to upload in streaming way file with the
uploadFiles()
method whenReadable
is passed. This is possible only in NodeJs at the moment as browsers currently does not support streaming requests. This will hopefully change soon as for example, Chrome has already experimental support for this (https://web.dev/fetch-upload-streaming/).But... The current implementation of upload of directories relies on the creation of Tar before the upload. This is done all in the memory, so if for example uploading a large directory to Swarm, then everything needs to be loaded into memory, which might lead to Out of Memory errors for Nodejs. It is possible to create Tar in streaming way with
tar-stream
. We should revisit this to add support for the streaming of directories.The text was updated successfully, but these errors were encountered: