Skip to content

Commit d2b012c

Browse files
committed
enh(chunk): Add docs for chunked upload v2
Signed-off-by: Julius Härtl <jus@bitgrid.net>
1 parent 9cdf815 commit d2b012c

File tree

1 file changed

+103
-3
lines changed

1 file changed

+103
-3
lines changed

developer_manual/client_apis/WebDAV/chunking.rst

+103-3
Original file line numberDiff line numberDiff line change
@@ -10,13 +10,113 @@ Uploading large files is always a bit problematic as your connection can be inte
1010
which will fail your entire upload. Nextcloud has a chunking API where you can
1111
upload smaller chunks which will be assembled on the server once they are all uploaded.
1212

13-
Usage
14-
-----
13+
There are two versions of the chunking API. Version 1 is the original version and version 2 was built as a backward compatible extension to support uploads directly to supporting target storages like S3. Version 2 is the recommended version to use.
14+
15+
Version 2 comes with a few additional requirements and limitations to consider (compared to version 1):
16+
17+
- Every request needs to have a ``Destination`` header present which specifies the target path of the file
18+
- The naming of the individual chunks is limited to be a number between 1 and 10000
19+
- The chunks will be assembled in the order of their names
20+
- The size of chunks must be between 5MB and 5GB (except for the last chunk, which can be smaller)
21+
- Chunks cannot be downloaded from the upload directory
22+
23+
Nextcloud will expire the upload directory after 24 hours of inactivity. This means that if you start an upload and do not finish it within 24 hours, the upload directory will be deleted and the upload will fail.
24+
25+
Chunked upload v2
26+
-----------------
1527

1628
The API is only available for registered users of your instance. And uses the path:
1729
``<server>/remote.php/dav/uploads/<userid>``. For this guide we will assume:
1830
``https://server/remote.php/dav/uploads/roeland``
1931

32+
33+
34+
Starting a chunked upload
35+
^^^^^^^^^^^^^^^^^^^^^^^^^
36+
37+
A chunked upload is handled in 1 folder. This is the location all the chunks
38+
are uploaded to.
39+
40+
Start by creating a folder with a unique name. You can list the current available
41+
folder but if you take a random UUID chances of collision are tiny.
42+
43+
.. code-block:: console
44+
45+
curl -X MKCOL -u roeland:pass \
46+
https://server/remote.php/dav/uploads/roeland/myapp-e1663913-4423-4efe-a9cd-26e7beeca3c0 \
47+
--header 'Destination: Destination https://server/remote.php/dav/files/roeland/dest/file.zip'
48+
49+
Uploading chunks
50+
^^^^^^^^^^^^^^^^
51+
52+
Once a folder for the chunks has been created we can start uploading the chunks.
53+
54+
- The naming of the individual chunks is limited to be a number between 1 and 10000
55+
- The chunks will be assembled in the order of their names
56+
- The size of chunks must be between 5MB and 5GB (except for the last chunk, which can be smaller)
57+
58+
59+
.. code-block:: console
60+
61+
curl -X PUT -u roeland:pass \
62+
https://server/remote.php/dav/uploads/roeland/myapp-e1663913-4423-4efe-a9cd-26e7beeca3c0/00001 \
63+
--data-binary @chunk1 \
64+
--header 'Destination: Destination https://server/remote.php/dav/files/roeland/dest/file.zip'
65+
66+
curl -X PUT -u roeland:pass \
67+
https://server/remote.php/dav/uploads/roeland/myapp-e1663913-4423-4efe-a9cd-26e7beeca3c0/00002 \
68+
--data-binary @chunk2 \
69+
--header 'Destination: Destination https://server/remote.php/dav/files/roeland/dest/file.zip'
70+
71+
This will upload 2 chunks of a file. The first chunk is 10MB in size and the second
72+
chunk is 5MB in size.
73+
74+
Assembling the chunks
75+
^^^^^^^^^^^^^^^^^^^^^
76+
77+
Assembling the chunk on the server is a matter of initiating a move from the client.
78+
79+
.. code-block:: console
80+
81+
curl -X MOVE -u roeland:pass
82+
--header 'Destination:https://server/remote.php/dav/files/roeland/dest/file.zip' \
83+
https://server/remote.php/dav/uploads/roeland/myapp-e1663913-4423-4efe-a9cd-26e7beeca3c0/.file
84+
85+
The server will now assemble the chunks and move the final file to the folder ``dest/file.zip``.
86+
87+
If a modification time should be set, you can by adding it as header with date as unixtime:
88+
89+
.. code-block:: console
90+
91+
curl -X MOVE -u roeland:pass
92+
--header 'Destination: https://server/remote.php/dav/files/roeland/dest/file.zip' \
93+
--header 'X-OC-Mtime: 1547545326' \
94+
--header 'Destination: https://server/remote.php/dav/files/roeland/dest/file.zip' \
95+
--header 'X-OC-Mtime: 1547545326' \
96+
Otherwise the current upload date will be used as modification date.
97+
98+
The chunks and the upload folder will be deleted afterwards.
99+
100+
Aborting the upload
101+
^^^^^^^^^^^^^^^^^^^
102+
103+
If the upload has to be aborted this is a simple matter or deleting the upload folder.
104+
105+
.. code-block::
106+
107+
curl -X DELETE -u roeland:pass \
108+
https://server/remote.php/dav/uploads/roeland/myapp-e1663913-4423-4efe-a9cd-26e7beeca3c0/
109+
110+
111+
Chunked upload v1
112+
-----------------
113+
114+
The API is only available for registered users of your instance. And uses the path:
115+
``<server>/remote.php/dav/uploads/<userid>``. For this guide we will assume:
116+
``https://server/remote.php/dav/uploads/roeland``
117+
118+
119+
20120
Starting a chunked upload
21121
^^^^^^^^^^^^^^^^^^^^^^^^^
22122

@@ -60,7 +160,7 @@ Assembling the chunk on the server is a matter of initiating a move from the cli
60160
The server will now assemble the chunks and move the final file to the folder ``dest/file.zip``.
61161

62162
If a modification time should be set, you can by adding it as header with date as unixtime:
63-
``curl -X MOVE -u roeland:pass --header 'X-OC-Mtime:1547545326' --header 'Destination:https://server/remote.php/dav/files/roeland/dest/file.zip' https://server/remote.php/dav/uploads/roeland/myapp-e1663913-4423-4efe-a9cd-26e7beeca3c0/.file``"
163+
``curl -X MOVE -u roeland:pass --header 'X-OC-Mtime:1547545326' --header 'Destination:https://server/remote.php/dav/files/roeland/dest/file.zip' https://server/remote.php/dav/uploads/roeland/myapp-e1663913-4423-4efe-a9cd-26e7beeca3c0/.file``"
64164
Otherwise the current upload date will be used as modification date.
65165

66166
The chunks and the upload folder will be deleted afterwards.

0 commit comments

Comments
 (0)