Payload v3 accepts only single file uploads, export fetchAPIFileUpload as a workaround

[max-degterev]

Describe the Bug

Payload v2 used to be able to handle any shape of multipart data and parse files accordingly. v3 is only able to find a single file named file.

Ideally as a long term solution it would be great to have payload automatically figure out data types passed in a multipart body and create matching data structure. So just req.files = files and done I guess...

As a quick and dirty fix it would be also great to just expose fetchAPIFileUpload so that end users can easily create custom body parsers while still using global payload upload configuration.

Link to the code that reproduces this issue

https://github.com/payloadcms/payload/blob/main/packages/next/src/fetchAPI-multipart/index.ts

Reproduction Steps

upgrade to v3
make sad noises attempting to migrate file uploads

Which area(s) are affected? (Select all that apply)

area: core

Environment Info

v3

[max-degterev]

Temporary hackfix:

import fs from 'node:fs/promises';

const packages = {
  "@payloadcms/next": {
    "exports": {
      "./fetchAPI-multipart": {
        "import": "./dist/fetchAPI-multipart/index.js",
        "types": "./dist/fetchAPI-multipart/index.d.ts",
        "default": "./dist/fetchAPI-multipart/index.js"
      },
    },
  },
};

const deepMerge = (target, overrides) => {
  for (const [key, value] of Object.entries(overrides)) {
    if (!target[key]) target[key] = {};
    if (typeof value === 'object') return deepMerge(target[key], value);
    target[key] = value;
  }
};

const postinstall = async () => Object.entries(packages).forEach(async ([name, patches]) => {
  const packageJsonPath = `node_modules/${name}/package.json`;

  const packageString = await fs.readFile(packageJsonPath, 'utf8');
  const packageJson = JSON.parse(packageString);

  deepMerge(packageJson, patches);

  await fs.writeFile(packageJsonPath, JSON.stringify(packageJson, null, 2));
});

postinstall();
console.log(`Applied patches to ${Object.keys(packages).length} packages`);

run this in "postinstall"

[max-degterev]

And this brings back the old style of body parsing, automatically finding data field and safely parsing as JSON as a bonus

import { set } from 'lodash-es';
import { APIError, type PayloadRequest } from 'payload';
import { fetchAPIFileUpload, type FetchAPIFileUploadResponse } from '@payloadcms/next/fetchAPI-multipart';
import { v4 as uuidv4 } from 'uuid';

type UploadedFile = FetchAPIFileUploadResponse['files'][string];
type NestedFiles = {
  // Not using Record because TS fails to handle circular references
  [key: string]: UploadedFile | NestedFiles,
};

export type PayloadRequestWithFiles = Omit<PayloadRequest, 'file'> & {
  files?: NestedFiles,
};

const createFilesStructure = (files: FetchAPIFileUploadResponse['files']) => {
  const result: Record<string, UploadedFile> = {};
  Object.entries(files).forEach(([key, file]) => {
    if (!file || !file.size || !file.data?.length) return; // Uploading empty file

    const parts = (file.name || 'file').split('.');
    const random = uuidv4().slice(0, 8);
    const extension = parts.pop();

    // Patch file name to include random string, avoiding collisions
    const mutable = file;
    mutable.name = `${parts.join('.')}-${random}.${extension}`;

    set(result, key, mutable);
  });
  return result;
};

export const addDataAndFilesToRequest = async (req: PayloadRequest) => {
  const { headers, method, payload } = req;
  const mutableReq = req as PayloadRequestWithFiles;
  const bodyByteSize = parseInt(req.headers.get('Content-Length') || '0', 10);

  if (method && ['PATCH', 'POST', 'PUT', 'DELETE'].includes(method.toUpperCase()) && bodyByteSize) {
    const [contentType] = (headers.get('Content-Type') || '').split(';');

    if (contentType === 'application/json') {
      const bodyText = await req.text?.();
      const data = tryParse<JsonObject>(bodyText, {})!;
      mutableReq.data = data;
      mutableReq.json = () => Promise.resolve(data);
    } else if (contentType.includes('multipart/')) {
      const { error, fields, files } = await fetchAPIFileUpload({
        options: payload.config.upload,
        request: req as Request,
      });

      if (error) throw new APIError(error.message);
      if (files) mutableReq.files = createFilesStructure(files);

      // Guessing that the string field is JSON compatible, not throwing an error if it's not
      const maybeJSON = Object.values(fields).find((value) => typeof value === 'string');
      const data = tryParse<JsonObject>(maybeJSON, {})!;
      mutableReq.data = data;
      mutableReq.json = () => Promise.resolve(data);
    }
  }
};

[max-degterev]

Additional issue: if filename contains a # it will not be processed by s3 correctly resulting in a failed upload. Try uploading attached file: Trollface # 31337.png.zip

To summarize all issues with addDataAndFileToRequest:

1. Only accepts 1 file
2. Doesn't find JSON data unless it's _payload
3. Only allows 'PATCH'/'POST'/'PUT' although it's also possible to attach body to 'DELETE' (See https://stackoverflow.com/questions/299628/is-an-entity-body-allowed-for-an-http-delete-request) (tbh I'd just remove the check entirely)
4. Explodes if 'application/json' is sent with empty body
5. Doesn't catch if multipart is sent with 0 byte file
6. Doesn't sanitize filenames resulting in failed uploads to s3
7. (?) Not sure if a bug, but collisions are not checked here, perhaps it's also better to check in the cloud plugin so just FYI