4.0 blog post

blog/2024-07-3-uppy-4.0.md

@@ -0,0 +1,364 @@
+---
+title:
+ 'New Uppy 4.0 major: TypeScript rewrite, Google Photos, React hooks, and much
+ more.'
+date: 2024-07-03
+authors: [aduh95, evgenia, mifi, murderlon]
+image: 'https://uppy.io/img/blog/3.13-3.21/dog-coding-laptop-mars-christmas-tree-2.jpg'
+slug: 'uppy-4.0'
+published: false
+toc_max_heading_level: 2
+---
+
+Hi ha, this is some goofy introduction.
+
+## TypeScript rewrite
+
+In the year 2024 people expect excellent types from their libraries. We used to
+author types separately by hand but they were often inconsistent or incomplete.
+Now Uppy has been completely rewritten in TypeScript!
+
+From now on you’ll be in safe hands when working with Uppy, whether it’s setting
+the right options, building plugins, or listening to events.
+
+```ts
+import Uppy from '@uppy/core';
+
+const uppy = new Uppy();
+
+// Event name autocompletion and inferred argument types
+uppy.on('file-added', (file) => {
+ console.log(file);
+});
+```
+
+One important thing to note are the new generics on `@uppy/core`.
+
+```ts
+import Uppy from '@uppy/core';
+// xhr-upload is for uploading to your own backend.
+import XHRUpload from '@uppy/xhr-upload';
+
+// Your own metadata on files
+type Meta = { myCustomMetadata: string };
+// The response from your server
+type Body = { someThingMyBackendReturns: string };
+
+const uppy = new Uppy<Meta, Body>().use(XHRUpload, {
+ endpoint: '/upload',
+});
+
+const id = uppy.addFile({
+ name: 'example.jpg',
+ data: new Blob(),
+ meta: { myCustomMetadata: 'foo' },
+});
+
+// This is now typed
+const { myCustomMetadata } = uppy.getFile(id).meta;
+
+await uppy.upload();
+
+// This is strictly typed too
+const { someThingMyBackendReturns } = uppy.getFile(id).response;
+```
+
+Happy inferring!
+
+## Merging the two AWS S3 plugins
+
+We used to two separate plugins for uploading to S3 (and S3-compatible
+services): `@uppy/aws-s3` and `@uppy/aws-s3-multpart`. They have different use
+cases. The advantages of multipart uploads are:
+
+- Improved throughput – You can upload parts in parallel to improve throughput.
+- Quick recovery from any network issues – Smaller part size minimizes the
+ impact of restarting a failed upload due to a network error.
+- Pause and resume object uploads – You can upload object parts over time. After
+ you initiate a multipart upload, there is no expiry; you must explicitly
+ complete or stop the multipart upload.
+- Begin an upload before you know the final object size – You can upload an
+ object as you are creating it.
+
+However, the downside is request overhead, as it needs to do creation, signing,
+and completion requests besides the upload requests. For example, if you are
+uploading files that are only a couple kilobytes with a 100ms roundtrip latency,
+you are spending 400ms on overhead and only a few milliseconds on uploading.
+This really adds up if you upload a lot of small files.
+
+AWS, and generally the internet from what we found, tend to agree that **you
+don’t want to use multipart uploads for files under 100MB**. But this sometimes
+puts users of our libraries in an awkward position, as their end users may not
+only upload very large files, or only small files. In this case a portion of
+their users get a subpar experience.
+
+---
+
+<!-- TODO: /aws-s3-multipart link should be /aws-s3 (needs site changes) -->
+
+We’ve merged the two plugins into `@uppy/aws-s3` with a new
+[`shouldUseMultipart`](/docs/aws-s3-multipart/#shouldusemultipartfile) option!
+By default it switches to multipart uploads if the file is larger than 100MB.
+You can pass a `boolean` or a function to determine it per file.
+
+## React hooks
+
+People working with React are more likely to create their own user interface on
+top of Uppy than those working with “vanilla” setups. Working with our pre-build
+UI components is a plug-and-play experience, but building on top of Uppy’s state
+with React primitives has been tedious.
+
+To address this we’re introducing to new hooks: `useUppyState` and
+`useUppyEvent`. Thanks to the TypeScript rewrite, we can now do powerful
+inference in hooks as well.
+
+### `useUppyState(uppy, selector)`
+
+Use this hook when you need to access Uppy’s state reactively.
+
+```js
+import Uppy from '@uppy/core';
+import { useUppyState } from '@uppy/react';
+
+// IMPORTANT: passing an initializer function to prevent Uppy from being recreated on every render.
+const [uppy] = useState(() => new Uppy());
+
+const files = useUppyState(uppy, (state) => state.files);
+const totalProgress = useUppyState(uppy, (state) => state.totalProgress);
+// We can also get specific plugin state.
+// Note that the value on `plugins` depends on the `id` of the plugin.
+const metaFields = useUppyState(
+ uppy,
+ (state) => state.plugins?.Dashboard?.metaFields,
+);
+```
+
+<!-- TODO: this permalink to State will get outdated. Maybe put it in a file we can link to? -->
+
+You can see all the values you can access on the
+[`State`](https://github.com/transloadit/uppy/blob/dab8082a4e67c3e7f109eacfbd6c3185f117dc60/packages/%40uppy/core/src/Uppy.ts#L156)
+type. If you are accessing plugin state, you would have to look at the types of
+the plugin.
+
+### `useUppyEvent(uppy, event, callback)`
+
+Listen to Uppy [events](/docs/uppy/#events) in a React component.
+
+Returns an array of which the first item is an array of results from the event.
+Depending on the event, that can be empty or have up to three values. The second
+item is a function to clear the results.
+
+Values remain in state until the next event (if that ever comes). Depending on
+your use case, you may want to keep the values in state or clear the state after
+something else happened.
+
+```ts
+import Uppy from '@uppy/core';
+import { useUppyEvent } from '@uppy/react';
+
+// IMPORTANT: passing an initializer function
+// to prevent Uppy from being recreated on every render.
+const [uppy] = useState(() => new Uppy());
+
+const [results, clearResults] = useUppyEvent(uppy, 'transloadit:result');
+const [stepName, result, assembly] = results; // strongly typed
+
+useUppyEvent(uppy, 'cancel-all', doSomethingElse);
+```
+
+## Google Photos
+
+A long requested feature is finally here: Google Photos support!
+
+:::info
+
+Uppy can bring in files from the cloud with [Companion](/docs/companion/).
+
+Companion is a hosted, standalone, or middleware server to take away the
+complexity of authentication and the cost of downloading files from remote
+sources, such as Instagram, Google Drive, and others.
+
+This means a 5GB video isn’t eating into your users’ data plans and you don’t
+have to worry about OAuth.
+
+:::
+
+<!-- TODO: video of the Google Photos plugin -->
+
+[`@uppy/google-photos`](/docs/google-photos/) is a new plugin so you can use it
+next to your existing [`@uppy/google-drive`](/docs/google-drive/) plugin.
+
+## UX improvements for viewing remote files
+
+When using [Dashboard](/docs/dashboard) with any of our remote sources (such as
+Google Drive) you use our internal `@uppy/provider-views` plugin to navigate and
+select files.
+
+We now made a handful of quality of life improvements for users.
+
+<!-- TODO: video of the improvements-->
+
+- **Folder caching**. When naviging in and out of folders, you now no longer
+ have to wait for the same API call — you’ll see the results instantly.
+- **Indeterminate checkmark states**. Before going into a folder we can’t know
+ how many files it contains so checking it immediately will show a traditional
+ checkmark. But when you go into a folder and you only select a subset of the
+ files, we’ll now show the indeterminate checkmark for the folder when you
+ navigate back out. Making it more clear you’re only uploading some of the
+ files in that folder.
+- **Reworked restrictions**. Uppy supports file
+ [restrictions](/docs/uppy/#restrictions), such as max number of files and max
+ file size. It’s now immediately clear in the UI when you are exceeding the max
+ number of files you can select. The error notifications are now also more
+ clear.
+- **Shift-click multi-select fixes**. You can shift-click the checkboxes to
+ select many files at once. This did not always work correctly and it also
+ highlighted the files names, which we now improved.
+
+We’re confident this turns our interface for remote sources into the most
+advanced one out there. We’ve seen some competing libraries not even aggregating
+results beyond the first page API limit of providers.
+
+## Revamped options for `@uppy/xhr-upload`
+
+Before the plugin had the options `getResponseData`, `getResponseError`,
+`validateStatus` and `responseUrlFieldName`. These were inflexible and too
+specific. Now we have hooks similar to `@uppy/tus`:
+
+- `onBeforeRequest` to manipulate the request before it is sent.
+- `shouldRetry` to determine if a request should be retried. By default 3
+ retries with exponential backoff. After three attempts it will throw an error,
+ regardless of whether you returned `true`.
+- `onAfterResponse` called after a successful response but before Uppy resolves
+ the upload.
+
+You could for instance use them to refresh your auth token when it expires:
+
+```js
+import Uppy from '@uppy/core';
+import XHR from '@uppy/xhr-upload';
+
+let token = null;
+
+async function getAuthToken() {
+ const res = await fetch('/auth/token');
+ const json = await res.json();
+ return json.token;
+}
+
+new Uppy().use(XHR, {
+ endpoint: '<your-endpoint>',
+ // Called again for every retry too.
+ async onBeforeRequest(xhr) {
+ if (!token) {
+ token = await getAuthToken();
+ }
+ xhr.setRequestHeader('Authorization', `Bearer ${token}`);
+ },
+ async onAfterResponse(xhr) {
+ if (xhr.status === 401) {
+ token = await getAuthToken();
+ }
+ },
+});
+```
+
+Checkout the `@uppy/xhr-upload` [docs](/docs/xhr-upload/) for more info.
+
+## Simpler configuration for `@uppy/transloadit`
+
+To get started with `@uppy/transloadit` you would configure
+[`assemblyOptions`](/docs/transloadit/#assemblyoptions) with your auth key,
+template ID, and other optional values. `assemblyOptions` can be an object or a
+function, which is called per file, which returns an object:
+
+```json
+{
+ "params": {
+ "auth": { "key": "key-from-transloadit" },
+ "template_id": "id-from-transloadit",
+ "steps": {
+ // Overruling Template at runtime
+ },
+ "notify_url": "https://your-domain.com/assembly-status"
+ },
+ "signature": "generated-signature",
+ "fields": {
+ // Dynamic or static fields to send along
+ }
+}
+```
+
+When you go to production you always want to make sure to set the `signature`.
+**Not using
+[Signature Authentication](https://transloadit.com/docs/topics/signature-authentication/)
+can be a security risk**. Signature Authentication is a security measure that
+can prevent outsiders from tampering with your Assembly Instructions.
+
+This means the majority of implementers will write something like this, as
+recommended:
+
+```ts
+new Uppy().use(Transloadit, {
+ async assemblyOptions(file) {
+ const res = await fetch('/transloadit-params');
+ return response.json();
+ },
+});
+```
+
+But now you are making a request to your backend for _every file_, while the
+response likely remains the same, unless you are setting dynamic `fields` per
+file.
+
+**This has now been improved to**:
+
+- only call `assemblyOptions()` once.
+- `fields` is for global variables in your
+ [template](https://transloadit.com/docs/topics/templates/).
+- all metadata on your files is automatically sent along to Transloadit. This
+ will end up in `file.user_meta` for you to dynamically access in your template
+ _per file_.
+
+You can read more about Assembly Variables in the
+[docs](https://transloadit.com/docs/topics/assembly-variables/).
+
+## Companion
+
+### Streaming uploads by default
+
+Streaming uploads are now the default in in Companion. This comes with greatly
+improved upload speeds and allows uploading up to hundreds of gigabytes without
+needing a large server storage. We found that this improves speeds by about 37%
+for a Google Drive upload of a 1 GB file
+([source](https://github.com/transloadit/uppy/pull/4046#issuecomment-1235697937)).
+This feature was also available before but we wanted to have more real world
+usage before setting it as the default.
+
+With streaming upload disabled, the whole file will be downloaded first. The
+upload will then start when the download has completely finished.
+
+When streaming upload is enabled, Companion will start downloading the file from
+the provider (such as Google Drive), while at the same time starting the upload
+to the destination (such as Tus), and sending every chunk of data consecutively.
+
+For more information, see the [Companion docs](/docs/companion/).
+
+### New required option `corsOrigins`
+
+As a security measure, we now require the
+[`corsOrigins`](/docs/companion/#corsorigins) option to be set.
+
+It serves two purposes, it sets the `Access-Control-Allow-Origin` header and it
+sets the origin for `window.postMessage()`, which is needed to communicate the
+OAuth token from the new tab you used to log-in to a provider back to Companion.
+
+## And more
+
+The 4.0 release contained over 170 contributions, many too small to mention, but
+together resulting in Uppy continuing to grow and improve. We closely listen to
+the community and are always looking for ways to improve the experience.
+
+Since our last blog post other (non-breaking) changes have also been released.
+
+<!-- TODO: add some minor but interesting changes -->