Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Overhaul validator packages #3508

Merged
merged 72 commits into from
Dec 6, 2024

Conversation

L-Mario564
Copy link
Collaborator

@L-Mario564 L-Mario564 commented Nov 7, 2024

Addresses #2247, #573, #1458, #503, #1110, #1327, #1345, #1458, #1609, #1810, #2245, #2358, #2424, #2516, #2521, #2524, #2550, #2737, #2755, #2957, #3398, #3621, #3645 & #3511.

This PR completely rewrites all validator packages from the ground up to the previous implemention of each library being severely outdated with either Drizzle ORM, the validator library or both.

Notable things

  • Output for all packages are now unminified, makes exploring the compiled code easier when published to npm.
  • Smaller footprint. Previously, we imported the column types at runtime for each dialect, meaning that for example, if you're just using Postgres then you'd likely only have drizzle-orm and drizzle-orm/pg-core in the build output of your app; however, these packages imported all dialects which could lead to mysql-core and sqlite-core being bundled as well even if they're unused in your app. This is now fixed.
  • Slight performance gain. To determine the column data type we used the is function which performs a few checks to ensure the column data type matches. This was slow, as these checks would pile up every quickly when comparing all data types for many fields in a table/view. The easier and faster alternative is to simply go off of the column's columnType property.
  • This PR depends on a feature and bugfix from Add some view utils #3553 to make the implementation for views possible.
  • Some changes had to be made at the type level in the ORM package for better compatibility with drizzle-valibot.

Breaking changes

  • When refining a field, if a schema is provided instead of a callback function, it will ignore the field's nullability and optional status.
  • Some data types have more specific schemas for improved validation. For example, the 32-bit integer types in each dialect are now represented as z.number().min(-2147483648).max(2147483647).int() in drizzle-zod, ensuring it's within the 32-bit integer limit and that it's an integer rather than a floating point number.

New features

  • createSelectSchema function now also accepts views and enums.
  • New function: createUpdateSchema, for use in updating queries.
  • New function: createSchemaFactory, to provide more advanced options and to avoid bloating the parameters of the other schema functions. This has only been implemented for drizzle-zod and drizzle-typebox since Valibot is modular in design rather than being one large class like the other validators, and as of right now, the only additional option is to pass a custom Zod and Typebox instance respectively.

Bugfixes

  • PG arrays are now fully supported.
  • All data types from all dialects are tested to work.

I've updated the docs for these packages here: drizzle-team/drizzle-orm-docs#439

Credits
Took some notes from @fabian-hiller's implementation in #2481 for drizzle-valibot.

@L-Mario564 L-Mario564 changed the title Update validator packages [WIP] Overhaul validator packages Nov 27, 2024
@L-Mario564 L-Mario564 marked this pull request as ready for review November 27, 2024 18:23
@AndriiSherman
Copy link
Member

@L-Mario564 Awesome job! Thanks so much for your efforts!

Initially, I wasn't sure about the type changes, but after reviewing them further, I see that nothing old was modified and only new types were added. It's important for us to know if anything in the types has been changed, so we can mark it as a breaking change in the release notes.

For now, everything looks good, so we can proceed with merging it. However, it would be great if you could create a PR to the drizzle-orm-docs repository with all the changes related to the validator packages.

Here are the relevant links for reference:

https://orm.drizzle.team/docs/zod
https://orm.drizzle.team/docs/typebox
https://orm.drizzle.team/docs/valibot

To run the docs repository locally, you can simply use pnpm i followed by pnpm run dev to access the local version of the docs

@L-Mario564
Copy link
Collaborator Author

@AndriiSherman Most altered types affect lower level functionality that the end-user usually doesn't interact with. Here are the types that have breaking changes made to them:

  • MySqlCharBuilderInitial
  • MySqlVarCharBuilderInitial
  • PgCharBuilderInitial
  • PgArrayBuilder
  • PgArray
  • PgVarcharBuilderInitial
  • PgBinaryVectorBuilderInitial
  • PgBinaryVectorBuilder
  • PgBinaryVector
  • PgHalfVectorBuilderInitial
  • PgHalfVectorBuilder
  • PgHalfVector
  • PgVectorBuilderInitial
  • PgVectorBuilder
  • PgVector
  • SQLiteTextBuilderInitial

PR for updated docs already exists here: drizzle-team/drizzle-orm-docs#439.

@AndriiSherman AndriiSherman merged commit 7525e49 into drizzle-team:beta Dec 6, 2024
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants