Skip to content

Uploading to Transloadit using tus

Jump to bottom
Kevin van Zonneveld edited this page Jul 8, 2016 · 17 revisions

A small note before you start reading: The tus integration is still undergoing some last minute changes. This document reflects the desired (and not actual) state, but even the desired state is still evolving for a few more days.

Overview

Historically, beyond importing files from sources, Transloadit had only one way for getting files to its servers for processing them. This is directly integrated with the creation of an Assembly by sending a POST request to the /assemblies endpoint. Alongside with the access keys and the assembly instructions, the actual files had to be inside this request via multipart, to be parsed by node-formidable.

Using the tus protocol for uploading these files, the approach is a bit different. Basically, it is divided into two steps:

  1. An Assembly is created using the same endpoint as with the "traditional" way but instead of containing the actual files, you only define how many files you want to upload using tus, along with the (a reference to the) encoding instructions you would normally provide. An Assembly URL returned. This can be regarded as the encoding endpoint.
  2. The files themselves are now uploaded using tus. During its file creation you will pass along the Assembly URL that you have already received. The tusd server will 'feed' this encoding endpoint with references to the uploaded files, as they complete. The Assembly will know when it has received all file references from tus, or when it is still pending. The Assembly can however start processing files as soon as the first file upload completes.

This was the short version, now follows the more comprehensive!

Creating a Resumable Assembly

A POST request must be sent to the /assemblies endpoint. As in the traditional approach (see https://transloadit.com/docs/api-docs/#create-a-new-assembly), its body must be multipart-encoded and must contain the params parameter which contains the usual information about authentication and instructions. However, additionally the tus_num_expected_upload_files parameter must be included. Its value is the integer number of how many uploads will be performed using the tus protocol.

Following snippet contains an example HTTP request, which wants to perform two tus uploads:

POST /assemblies HTTP/1.1
Host: api2.transloadit.com
Content-Type:multipart/form-data; boundary=----WebKitFormBoundaryIAWBI8vxocZzsG03

------WebKitFormBoundaryIAWBI8vxocZzsG03
Content-Disposition: form-data; name="params"

{"auth":{"key":"XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"},"steps":{"encode":{"robot":"/image/resize"}}}
------WebKitFormBoundaryIAWBI8vxocZzsG03
Content-Disposition: form-data; name="tus_num_expected_upload_files"

2
------WebKitFormBoundaryIAWBI8vxocZzsG03--

After we have received a successful response from the server, step #1 is completed and we can move on!

@TODO Show the example response, with the Assembly URL that we received

Uploading using tus

Transloadit is running a tus server (using the tusd software) at the /resumable endpoint. To test, you can send an OPTIONS request to https://api2.transloadit.com/resumable/ and inspect the headers in the response:

curl -IXOPTIONS https://api2.transloadit.com/resumable/
HTTP/1.1 200 OK
Content-Type: text/plain; charset=utf-8
Date: Fri, 08 Jul 2016 07:51:20 GMT
Tus-Extension: creation,termination,concatenation
Tus-Max-Size: 42949672960
Tus-Resumable: 1.0.0
Tus-Version: 1.0.0 @<--- @TODO Should this not rather be: Tusd-Version: 1.0.0
X-Content-Type-Options: nosniff
Content-Length: 0
Connection: keep-alive

In order to tell the server to which Assembly a tus upload belongs, you have to add special metadata to the tus creation. In total, there are three values which must be presented in the metadata (if you miss one of them, the upload will silently be dropped without any error but we will probably change this in the future):

  • assembly_url: The Assembly's URL to which the upload belongs. You can obtain the value from the previous POST response
  • filename: The file's name which will be uploaded in this tus procedure.
  • fieldname: The equivalent to input field names in HTML forms.

Here's one example, in which are uploading a file called isaac.png to the Assembly 14b1b490447d11e6aba4756b3e9d3a0d with a field name of file-input:

POST /resumable/ HTTP/1.1
Content-Length:0
Host: api2.transloadit.com
Tus-Resumable: 1.0.0
Upload-Length: 24213
Upload-Metadata: assembly_url aHR0cHM6Ly9hcGkyLnRyYW5zbG9hZGl0LmNvbS9hc3NlbWJsaWVzLzE0YjFiNDkwNDQ3ZDExZTZhYmE0NzU2YjNlOWQzYTBk,filename aXNhYWMucG5n,fieldname ZmlsZS1pbnB1dA==

After that, most of the work is done. What's still to be done is that you have to upload the actual file content to the tus upload URL using the usual procedure using PATCH requests.

If one tus upload if finished, Transloadit will automatically process it using the parameters that you created the Assembly with - without requiring you to do anything special. Until all tus uploads have been finished the Assembly will be in the ASSEMBLY_UPLOADING state (although in reality, the first files may be processed already). If you are trying to add more tus uploads to an Assembly than you specified during the Assembly creation, it will be silently dropped. In the future, we want to error out hard on this.

Clone this wiki locally