Skip to content

Commit e964a96

Browse files
committedJul 13, 2023
enh(chunk): Add docs for chunked upload v2
Signed-off-by: Julius Härtl <jus@bitgrid.net>
1 parent 49177ac commit e964a96

File tree

1 file changed

+101
-3
lines changed

1 file changed

+101
-3
lines changed
 

‎developer_manual/client_apis/WebDAV/chunking.rst

+101-3
Original file line numberDiff line numberDiff line change
@@ -10,13 +10,111 @@ Uploading large files is always a bit problematic as your connection can be inte
1010
which will fail your entire upload. Nextcloud has a chunking API where you can
1111
upload smaller chunks which will be assembled on the server once they are all uploaded.
1212

13-
Usage
14-
-----
13+
There are two versions of the chunking API. Version 1 is the original version and version 2 was built as a backward compatible extension to support uploads directly to supporting target storages like S3. Version 2 is the recommended version to use.
14+
15+
Version 2 comes with a few additional requirements and limitations to consider (compared to version 1):
16+
17+
- Every request needs to have a ``Destination`` header present which specifies the target path of the file
18+
- The naming of the individual chunks is limited to be a number between 1 and 10000
19+
- The chunks will be assembled in the order of their names
20+
- The size of chunks must be between 5MB and 5GB (except for the last chunk, which can be smaller)
21+
- Chunks cannot be downloaded from the upload directory
22+
23+
Chunked upload v2
24+
-----------------
25+
26+
The API is only available for registered users of your instance. And uses the path:
27+
``<server>/remote.php/dav/uploads/<userid>``. For this guide we will assume:
28+
``https://server/remote.php/dav/uploads/roeland``
29+
30+
31+
32+
Starting a chunked upload
33+
^^^^^^^^^^^^^^^^^^^^^^^^^
34+
35+
A chunked upload is handled in 1 folder. This is the location all the chunks
36+
are uploaded to.
37+
38+
Start by creating a folder with a unique name. You can list the current available
39+
folder but if you take a random UUID chances of collision are tiny.
40+
41+
.. code-block:: console
42+
43+
curl -X MKCOL -u roeland:pass \
44+
https://server/remote.php/dav/uploads/roeland/myapp-e1663913-4423-4efe-a9cd-26e7beeca3c0 \
45+
--header 'Destination: Destination https://server/remote.php/dav/files/roeland/dest/file.zip'
46+
47+
Uploading chunks
48+
^^^^^^^^^^^^^^^^
49+
50+
Once a folder for the chunks has been created we can start uploading the chunks.
51+
52+
- The naming of the individual chunks is limited to be a number between 1 and 10000
53+
- The chunks will be assembled in the order of their names
54+
- The size of chunks must be between 5MB and 5GB (except for the last chunk, which can be smaller)
55+
56+
57+
.. code-block:: console
58+
59+
curl -X PUT -u roeland:pass \
60+
https://server/remote.php/dav/uploads/roeland/myapp-e1663913-4423-4efe-a9cd-26e7beeca3c0/00001 \
61+
--data-binary @chunk1 \
62+
--header 'Destination: Destination https://server/remote.php/dav/files/roeland/dest/file.zip'
63+
64+
curl -X PUT -u roeland:pass \
65+
https://server/remote.php/dav/uploads/roeland/myapp-e1663913-4423-4efe-a9cd-26e7beeca3c0/00002 \
66+
--data-binary @chunk2 \
67+
--header 'Destination: Destination https://server/remote.php/dav/files/roeland/dest/file.zip'
68+
69+
This will upload 2 chunks of a file. The first chunk is 10MB in size and the second
70+
chunk is 5MB in size.
71+
72+
Assembling the chunks
73+
^^^^^^^^^^^^^^^^^^^^^
74+
75+
Assembling the chunk on the server is a matter of initiating a move from the client.
76+
77+
.. code-block:: console
78+
79+
curl -X MOVE -u roeland:pass
80+
--header 'Destination:https://server/remote.php/dav/files/roeland/dest/file.zip' \
81+
https://server/remote.php/dav/uploads/roeland/myapp-e1663913-4423-4efe-a9cd-26e7beeca3c0/.file
82+
83+
The server will now assemble the chunks and move the final file to the folder ``dest/file.zip``.
84+
85+
If a modification time should be set, you can by adding it as header with date as unixtime:
86+
87+
.. code-block:: console
88+
89+
curl -X MOVE -u roeland:pass
90+
--header 'Destination: https://server/remote.php/dav/files/roeland/dest/file.zip' \
91+
--header 'X-OC-Mtime: 1547545326' \
92+
--header 'Destination: https://server/remote.php/dav/files/roeland/dest/file.zip' \
93+
--header 'X-OC-Mtime: 1547545326' \
94+
Otherwise the current upload date will be used as modification date.
95+
96+
The chunks and the upload folder will be deleted afterwards.
97+
98+
Aborting the upload
99+
^^^^^^^^^^^^^^^^^^^
100+
101+
If the upload has to be aborted this is a simple matter or deleting the upload folder.
102+
103+
.. code-block::
104+
105+
curl -X DELETE -u roeland:pass \
106+
https://server/remote.php/dav/uploads/roeland/myapp-e1663913-4423-4efe-a9cd-26e7beeca3c0/
107+
108+
109+
Chunked upload v1
110+
-----------------
15111

16112
The API is only available for registered users of your instance. And uses the path:
17113
``<server>/remote.php/dav/uploads/<userid>``. For this guide we will assume:
18114
``https://server/remote.php/dav/uploads/roeland``
19115

116+
117+
20118
Starting a chunked upload
21119
^^^^^^^^^^^^^^^^^^^^^^^^^
22120

@@ -60,7 +158,7 @@ Assembling the chunk on the server is a matter of initiating a move from the cli
60158
The server will now assemble the chunks and move the final file to the folder ``dest/file.zip``.
61159

62160
If a modification time should be set, you can by adding it as header with date as unixtime:
63-
``curl -X MOVE -u roeland:pass --header 'X-OC-Mtime:1547545326' --header 'Destination:https://server/remote.php/dav/files/roeland/dest/file.zip' https://server/remote.php/dav/uploads/roeland/myapp-e1663913-4423-4efe-a9cd-26e7beeca3c0/.file``"
161+
``curl -X MOVE -u roeland:pass --header 'X-OC-Mtime:1547545326' --header 'Destination:https://server/remote.php/dav/files/roeland/dest/file.zip' https://server/remote.php/dav/uploads/roeland/myapp-e1663913-4423-4efe-a9cd-26e7beeca3c0/.file``"
64162
Otherwise the current upload date will be used as modification date.
65163

66164
The chunks and the upload folder will be deleted afterwards.

0 commit comments

Comments
 (0)