Programmatic form validation

[Rich-Harris]

Describe the problem

We now have declarative form validation...

export const myform = form(schema, (data) => {
  // `data` satisfies `schema`
});

...but there are some situations where validity can't be determined without attempting some action — e.g. if you check remaining stock inside the schema, the stock could change before the callback runs. It's unlikely, but possible.

Describe the proposed solution

Pass an invalid function as the second argument to the callback. This function receives an array of issues, and throws an error so that no further processing takes place (and, for example, the ongoing database transaction is rolled back).

The issues array can be an array of Standard Schema issues (i.e. produced with a validation library), or an array of { message: string, name?: string } objects. name must correspond to one of the form fields (TypeScript will enforce this); if omitted, the resulting issue will only exist in the top-level issues.$ array.

Whichever type of issue is used, they are all normalized to RemoteFormIssue.

[jdgamble555]

I'm getting an issue where this works:

export const createTodo = form(
    v.object({
        completed: v.optional(v.string())
    }),
    async (data) => {
        // ...

        return { success: true };
    }
);

but this doesn't

export const createTodo = form(
    v.object({
        completed: v.optional(v.boolean())
    }),
    async (data) => {
        // ...

        return { success: true };
    }
);

It seems something with v.boolean() does not work. To be fair, this could just be VSCode, but it is inferring from TS.

No overload matches this call.
  Overload 1 of 3, '(validate: "unchecked", fn: (data: RemoteFormInput) => unknown): RemoteForm<RemoteFormInput, unknown>', gave the following error.
    Argument of type 'ObjectSchema<{ readonly title: SchemaWithPipe<readonly [StringSchema<undefined>, MinLengthAction<string, 2, undefined>, MaxLengthAction<string, 100, undefined>]>; readonly completed: OptionalSchema<...>; }, undefined>' is not assignable to parameter of type '"unchecked"'.
  Overload 2 of 3, '(validate: StandardSchemaV1<RemoteFormInput, Record<string, any>>, fn: (data: Record<string, any>) => unknown): RemoteForm<RemoteFormInput, unknown>', gave the following error.
    Argument of type 'ObjectSchema<{ readonly title: SchemaWithPipe<readonly [StringSchema<undefined>, MinLengthAction<string, 2, undefined>, MaxLengthAction<string, 100, undefined>]>; readonly completed: OptionalSchema<...>; }, undefined>' is not assignable to parameter of type 'StandardSchemaV1<RemoteFormInput, Record<string, any>>'.
      The types of '"~standard".types' are incompatible between these types.
        Type 'StandardTypes<{ title: string; completed?: boolean | undefined; }, { title: string; completed?: boolean | undefined; }> | undefined' is not assignable to type 'Types<RemoteFormInput, Record<string, any>> | undefined'.
          Type 'StandardTypes<{ title: string; completed?: boolean | undefined; }, { title: string; completed?: boolean | undefined; }>' is not assignable to type 'Types<RemoteFormInput, Record<string, any>>'.
            Types of property 'input' are incompatible.
              Type '{ title: string; completed?: boolean | undefined; }' is not assignable to type 'RemoteFormInput'.
                Property 'completed' is incompatible with index signature.
                  Type 'boolean' is not assignable to type 'RemoteFormInput | FormDataEntryValue | FormDataEntryValue[] | RemoteFormInput[]'.ts(2769)

J

[Rich-Harris]

Inputs must be string or File, because that's what exists on FormData. If you want the validated data to have a number, the validator should coerce it

[jdgamble555]

I don't understand the point of having a validator in form if you have to coerce or transform it. As you said, formdata already validates it for string or File. If you can't verify the actual data type, no reason to put the validator in the input at all like the other remote functions.

    v.object({
        title: v.pipe(v.string(), v.minLength(2), v.maxLength(100)),
        completed: v.optional(
            v.pipe(
                v.string(),
                v.transform((val) => {
                    const s = val.toLowerCase();
                    if (["true", "1", "on", "yes"].includes(s)) return true;
                    if (["false", "0", "off", "no"].includes(s)) return false;
                    return false; 
                })
            )
        )
    }),

This is insanity.

This why Valibot, for example, has decode-formdata, so you can actually accept the proper schema.

J

[dummdidumm]

How would SvelteKit know whether your string "on" is meant as the string "on" or the Boolean value true

[Rich-Harris]

This why Valibot, for example, has decode-formdata

Yes and you have to use it like this:

const formValues = decode(formData, {
  arrays: ['tags', 'images'],
  booleans: ['active'],
  dates: ['created', 'images.$.created'],
  files: ['images.$.file'],
  numbers: ['price'],
});

As @dummdidumm says, it has no way of knowing, other than you manually specifying it per field, what types you're dealing with. So we would need to add an equivalent options object to form, which would be annoying, or you can just use the validator. This is sufficient for an <input type="checkbox">:

const checkbox = v.pipe(
  v.optional(v.string(), ''),
  v.transform((str) => str === 'on')
);

const schema = v.object({
  mystring: v.string(),
  myboolean: checkbox
});

[jdgamble555]

So I just used GPT to create that, as I wouldn't necessarily use it like that. But it does raise the important question for those use cases.

Preferable, yes, SvelteKit would have something built in like decode, or whatever superforms does. The on/off uses cases are more specific and would require transformations like that, which is fine (unless that was the norm).

At the very lease, it needs to be very clear that the validators will NOT work as expected in the docs with form. I don't think many people understand this limitation at this point.

How would you even run the validators after the fact. If I have to transform the data, then I should be able to run the validator later (not in the first parameter) to work the same way.

Perhaps a third parameter decodeFn(formData) => formData?

export const createTodo = form(
    v.object({
        completed: v.optional(v.string())
    }),
    async (data) => {
        // ...

        return { success: true };
    },
    (formData) => {
        return decode(formData, {
          arrays: ['tags', 'images'],
          booleans: ['active'],
          dates: ['created', 'images.$.created'],
          files: ['images.$.file'],
          numbers: ['price'],
        });
    },
    }
);

The goal here is to be able to transform the data without having to run manual validation since form automates this and adds issues etc.

J

[cdcarson]

SvelteKit has overreached with this notion that it will be responsible for

1. parsing the FormData entries to a POJO (enumeration/traversal of form names, casting to js types, etc.) and
2. validating the POJO against a schema (no matter how "standardized" schema libraries become).

First (agreeing with Rich above) business logic usually involves something more that the validity of the form data. For example, displaying this error...

Woops! Your todo list item limit has been reached! Upgrade to our Junior Executive Plan in order to add up to 10 items to each list!

... is not possible in the current implementation of form.

Second, I disagree entirely with the current approach.

I appreciate what the current implementation is trying to do, but in the end it is imposing a layer of constraints and complexity on userland logic, for very little value compared to what utilities (or a focused library) could do in ~1000 lines, on the server, on the client and for both "actions" and "remote functions."

WIP library, currently broken for the remote functions case.

To be clear, I'm not complaining that recent changes broke my library. I'm just showing an example of how I think it could be done, relatively simply, outside of SvelteKit.

Also to be clear, I'm aware that my library is zod-only and has more restrictive types. That's my point. It can be restrictive in a way that SvelteKit should not be. Presumably, if someone hates the way my library reduces errors or handles form names, they can choose another, better library. They cannot do the same swap as blithely for SvelteKit.

[jdgamble555]

SvelteKit has overreached with this notion that it will be responsible for

1. parsing the FormData entries to a POJO (enumeration/traversal of form names, casting to js types, etc.) and
2. validating the POJO against a schema (no matter how "standardized" schema libraries become).

First (agreeing with Rich above) business logic usually involves something more that the validity of the form data. For example, displaying this error...

Woops! Your todo list item limit has been reached! Upgrade to our Junior Executive Plan in order to add up to 10 items to each list!

... is not possible in the current implementation of form.

Second, I disagree entirely with the current approach.

I appreciate what the current implementation is trying to do, but in the end it is imposing a layer of constraints and complexity on userland logic, for very little value compared to what utilities (or a focused library) could do in ~1000 lines, on the server, on the client and for both "actions" and "remote functions."

WIP library, currently broken for the remote functions case.

To be clear, I'm not complaining that recent changes broke my library. I'm just showing an example of how I think it could be done, relatively simply, outside of SvelteKit.

Also to be clear, I'm aware that my library is zod-only and has more restrictive types. That's my point. It can be restrictive in a way that SvelteKit should not be. Presumably, if someone hates the way my library reduces errors or handles form names, they can choose another, better library. They cannot do the same swap as blithely for SvelteKit.

I disagree that it is overreaching. There are things we keep doing over and over and over again, that is the same ol things. Why can't we have are framework save us hours of time, which this does.

Parsing form data is huge for any server or client environment. There is a reason Angular Forms exists (although Angular devs don't care about the server). I just wish all other frameworks handled this internally and it worked well with signals.

It is not imposing any constraints or complexity, it is getting rid of constraints AND complexity. Have you looked superforms? It is incredibly complicated because it is impossible to simplify it without changing the way SvelteKit works.

Woops! Your todo list item limit has been reached! Upgrade to our Junior Executive Plan in order to add up to 10 items to each list!

This absolutely can be done in any schema. For example.

const schema = z.object({
  tags: z
    .array(z.string())
    .min(1, { message: "You must provide at least 1 tag." })
    .max(5, { message: "You can only provide up to 5 tags." })
});

Also, there is NO REASON why you can't use your own logic if you don't want to use this logic. As long as it is optional, don't ruin good code for the rest of us.

I, personally, do not want to write my own schema parser like superforms, nor do I want to rely on third party packages that may never get updated. Superforms, again, will never be updated to use signals, as said by the owner. Putting something like this inside the framework is nothing short of magic. Despite what some people say, magic is amazing and saves us HOURS of time.

J

[sillvva]

There is a reason why sveltekit-superforms takes a POJO input for initial data and outputs a POJO store that you bind fields to — because a POJO is easier to validate than FormData. Formisch is also similar in that it is based on POJO data rather than FormData. Another benefit is that you don't need to provide a field (hidden or otherwise) for every POJO leaf, since the data being submitted is the whole POJO even if you only have fields to change one or two properties.

The tradeoff of doing it that way is that you lose the benefit of progressive enhancement.

If Svelte decides to go with the "nothing in, FormData out" route, then I'd make it abundantly clear in the docs the limitations of validating FormData and possibly list alternatives.

[jdgamble555]

@sillvva - That is not true. Svelte accepts POJO as the initial values, but definitely wants you to input the data.

 // page load
const courseForm = await superValidate({
    id: data.id,
    slug: data.slug,
    title: data.title,
    description: data.description
}, valibot(courseSchema));

or

// form action
const form = await superValidate(request, valibot(lessonListSchema));

Here it takes the request itself as input.

J

[GauBen]

Hi there, sorry if I start to sound spammy, but I feel like I still have some value to add to this exchange

How would SvelteKit know whether your string "on" is meant as the string "on" or the Boolean value true

SvelteKit has no way to know that unless the user tells it.

Instead, I'd say Kit should offer an API to tell it that some field is a checkbox. And once Kit knows that some field is a checkbox, it can do both validation and bring back the value to the realm of consistent JS values:

null -> valid -> false
"on" -> valid -> true
"b̷͙̞̙̘̮̠͉̲̈͐̈́̕͘͜r̴̢̖̩̰̯̬̻͇̂̍̆͐̚u̶̙̟̗̇͒h̴͖̰̦̣̱̭̬̲̹̍̈́̆́͆͛̀̓͘͠" -> invalid
File -> invalid

Because SvelteKit has exposed FormData objects for a long time, I have built formgator as a seamless validation library for it, with an interface pretty close to the current form, except that it understands FormData objects. I will let you compare the DX of the two implementations:

<!-- Input form -->
<input name="email" type="email" required />
<input name="age" type="number" required min="18" />
<input name="weekly" type="checkbox" />

// formgator, on top of actions
import * as fg from 'formgator'
import { formgate, formfail } from 'formgator/sveltekit'

export const actions = {
  subscribeToNewsletter: formgate(
    {
      email: fg.email({ required: true }),
      age: fg.number({ required: true, min: 18 }),
      weekly: fg.checkbox(),
    },
    async (data, event) => {
      // data is typed { email: string; age: number; weekly: boolean }
      if (await registered(data.email)) {
        formfail({ email: 'This email is already registered' })
      }
    }
  )
}

// kit current/suggested implementation
import * as v from 'valibot'
import { form } from '$app/server'

export const subscribeToNewsletter = form(
  v.object({
    email: v.pipe(v.string(), v.email()),
    age: v.pipe(v.string(), v.transform(Number), v.number(), v.minValue(18)),
    weekly: v.pipe(v.optional(v.string(), ''), v.transform((str) => str === 'on'))
  }),
  async (data, invalid) => {
    // data is typed { email: string; age: number; weekly: boolean }
    if (await registered(data.email)) {
      invalid([{ name: 'email', message: 'This email is already registered' }])
    }
  }
)

I specifically targeted weak spots of the standard schema approach, that's true, but formgator offers the best DX to my eyes

Note that formgator is MIT licensed and I'm 100% ok with it being integrated in some form or another into Kit

[Rich-Harris]

Just for clarity, that is not the suggested implementation in SvelteKit — #14481 doesn't quite articulate it, but this is what we're experimenting with:

export const subscribeToNewsletter = form(
  v.object({
    email: v.pipe(v.string(), v.email()),
-   age: v.pipe(v.string(), v.transform(Number), v.number(), v.minValue(18)),
-   weekly: v.pipe(v.optional(v.string(), ''), v.transform((str) => str === 'on'))
+   age: v.pipe(v.number(), v.minValue(18)),
+   weekly: v.optional(v.boolean(), false)
  }),
  async (data, invalid) => {
    // data is typed { email: string; age: number; weekly: boolean }
    if (await registered(data.email)) {
-     invalid([{ name: 'email', message: 'This email is already registered' }])
+     invalid(/* err... we haven't actually figured this part out yet */)
    }
  }
)

The 'unless the user tells it' part is handled via the only channel we have for metadata — abuse of the name attribute. Specifically, if you do this...

<label>
  Age
  <input {...subscribeToNewsletter.fields.age.as('number')} />
</label>

<label>
  Weekly
  <input {...subscribeToNewsletter.fields.weekly.as('checkbox')} />
</label>

...then the name would be something like number:age on the first input and checkbox:weekly on the second, which would be enough for SvelteKit to know that those fields should be coerced accordingly. (The other attributes that get spread onto the element are the type, the value in the case of a form that's being repopulated after an invalid submission, and aria-invalid.)

It's not perfect, of course. Having to do v.optional(v.boolean(), false) is definitely less nice than fg.checkbox(). But I think that's the only thing that needs special handling, except for possibly-empty arrays? The question is whether it's imperfect enough to justify another API. I do agree that validation libraries generally don't think about forms enough, and it would be nice if an alias for v.optional(v.boolean(), false) was commonly provided.

[GauBen]

This does indeed look much nicer (let 'em cook!), although since the .as(...) lives far away from the schema definition, how can we prevent a distracted user from writing ...?

<label>
  Age
  <input {...subscribeToNewsletter.fields.age.as('checkbox')} />
</label>

<label>
  Weekly
  <input {...subscribeToNewsletter.fields.weekly.as('number')} />
</label>

I was intending to send this messages as is, but it turns out I have an idea: since we know the expected output type of the field, we can restrict to a consistent list of input types

It seems that some validation libraries (e.g. valibot and zod) support metadata attached to validators, it would be great that .as(...) spreads these metadata to the input field