Skip to content

Commit 23f1ed4

Browse files
r1tsuur1tsuu
authored
feat(db-postgres, db-sqlite): drizzle schema generation (#9953)
This PR allows to have full type safety on `payload.drizzle` with a single command ```sh pnpm payload generate:db-schema ``` Which generates TypeScript code with Drizzle declarations based on the current database schema. Example of generated file with the website template: https://gist.github.com/r1tsuu/b8687f211b51d9a3a7e78ba41e8fbf03 Video that shows the power: https://github.com/user-attachments/assets/3ced958b-ec1d-49f5-9f51-d859d5fae236 We also now proxy drizzle package the same way we do for Lexical so you don't have to install it (and you shouldn't because you may have version mismatch). Instead, you can import from Drizzle like this: ```ts import { pgTable, index, foreignKey, integer, text, varchar, jsonb, boolean, numeric, serial, timestamp, uniqueIndex, pgEnum, } from '@payloadcms/db-postgres/drizzle/pg-core' import { sql } from '@payloadcms/db-postgres/drizzle' import { relations } from '@payloadcms/db-postgres/drizzle/relations' ``` Fixes #4318 In the future we can also support types generation for mongoose / raw mongodb results.
1 parent ba0e7ae commit 23f1ed4

39 files changed

+4519
-32
lines changed

‎docs/database/postgres.mdx

Lines changed: 58 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -65,21 +65,33 @@ export default buildConfig({
6565
| `schemaName` (experimental) | A string for the postgres schema to use, defaults to 'public'. |
6666
| `idType` | A string of 'serial', or 'uuid' that is used for the data type given to id columns. |
6767
| `transactionOptions` | A PgTransactionConfig object for transactions, or set to `false` to disable using transactions. [More details](https://orm.drizzle.team/docs/transactions) |
68-
| `disableCreateDatabase` | Pass `true` to disable auto database creation if it doesn't exist. Defaults to `false`. |
68+
| `disableCreateDatabase` | Pass `true` to disable auto database creation if it doesn't exist. Defaults to `false`. |
6969
| `localesSuffix` | A string appended to the end of table names for storing localized fields. Default is '_locales'. |
7070
| `relationshipsSuffix` | A string appended to the end of table names for storing relationships. Default is '_rels'. |
7171
| `versionsSuffix` | A string appended to the end of table names for storing versions. Defaults to '_v'. |
7272
| `beforeSchemaInit` | Drizzle schema hook. Runs before the schema is built. [More Details](#beforeschemainit) |
7373
| `afterSchemaInit` | Drizzle schema hook. Runs after the schema is built. [More Details](#afterschemainit) |
74+
| `generateSchemaOutputFile` | Override generated schema from `payload generate:db-schema` file path. Defaults to `{CWD}/src/payload-generated.schema.ts` |
7475

7576
## Access to Drizzle
7677

7778
After Payload is initialized, this adapter will expose the full power of Drizzle to you for use if you need it.
7879

79-
You can access Drizzle as follows:
80+
To ensure type-safety, you need to generate Drizzle schema first with:
81+
```sh
82+
npx payload generate:db-schema
83+
```
8084

81-
```text
82-
payload.db.drizzle
85+
Then, you can access Drizzle as follows:
86+
```ts
87+
import { posts } from './payload-generated-schema'
88+
// To avoid installing Drizzle, you can import everything that drizzle has from our re-export path.
89+
import { eq, sql, and } from '@payloadcms/db-postgres/drizzle'
90+
91+
// Drizzle's Querying API: https://orm.drizzle.team/docs/rqb
92+
const posts = await payload.db.drizzle.query.posts.findMany()
93+
// Drizzle's Select API https://orm.drizzle.team/docs/select
94+
const result = await payload.db.drizzle.select().from(posts).where(and(eq(posts.id, 50), sql`lower(${posts.title}) = 'example post title'`))
8395
```
8496

8597
## Tables, relations, and enums
@@ -114,7 +126,7 @@ Runs before the schema is built. You can use this hook to extend your database s
114126

115127
```ts
116128
import { postgresAdapter } from '@payloadcms/db-postgres'
117-
import { integer, pgTable, serial } from 'drizzle-orm/pg-core'
129+
import { integer, pgTable, serial } from '@payloadcms/db-postgres/drizzle/pg-core'
118130

119131
postgresAdapter({
120132
beforeSchemaInit: [
@@ -194,7 +206,7 @@ The following example adds the `extra_integer_column` column and a composite ind
194206

195207
```ts
196208
import { postgresAdapter } from '@payloadcms/db-postgres'
197-
import { index, integer } from 'drizzle-orm/pg-core'
209+
import { index, integer } from '@payloadcms/db-postgres/drizzle/pg-core'
198210
import { buildConfig } from 'payload'
199211

200212
export default buildConfig({
@@ -236,3 +248,43 @@ export default buildConfig({
236248
})
237249

238250
```
251+
252+
### Note for generated schema:
253+
Columns and tables, added in schema hooks won't be added to the generated via `payload generate:db-schema` Drizzle schema.
254+
If you want them to be there, you either have to edit this file manually or mutate the internal Payload "raw" SQL schema in the `beforeSchemaInit`:
255+
256+
```ts
257+
import { postgresAdapter } from '@payloadcms/db-postgres'
258+
259+
postgresAdapter({
260+
beforeSchemaInit: [
261+
({ schema, adapter }) => {
262+
// Add a new table
263+
schema.rawTables.myTable = {
264+
name: 'my_table',
265+
columns: [{
266+
name: 'my_id',
267+
type: 'serial',
268+
primaryKey: true
269+
}],
270+
}
271+
272+
// Add a new column to generated by Payload table:
273+
schema.rawTables.posts.columns.customColumn = {
274+
name: 'custom_column',
275+
// Note that Payload SQL doesn't support everything that Drizzle does.
276+
type: 'integer',
277+
notNull: true
278+
}
279+
// Add a new index to generated by Payload table:
280+
schema.rawTables.posts.indexes.customColumnIdx = {
281+
name: 'custom_column_idx',
282+
unique: true,
283+
on: ['custom_column']
284+
}
285+
286+
return schema
287+
},
288+
],
289+
})
290+
```

‎docs/database/sqlite.mdx

Lines changed: 71 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -34,28 +34,41 @@ export default buildConfig({
3434

3535
## Options
3636

37-
| Option | Description |
38-
| --------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
39-
| `client` \* | [Client connection options](https://orm.drizzle.team/docs/get-started-sqlite#turso) that will be passed to `createClient` from `@libsql/client`. |
40-
| `push` | Disable Drizzle's [`db push`](https://orm.drizzle.team/kit-docs/overview#prototyping-with-db-push) in development mode. By default, `push` is enabled for development mode only. |
41-
| `migrationDir` | Customize the directory that migrations are stored. |
42-
| `logger` | The instance of the logger to be passed to drizzle. By default Payload's will be used. |
43-
| `idType` | A string of 'number', or 'uuid' that is used for the data type given to id columns. |
44-
| `transactionOptions` | A SQLiteTransactionConfig object for transactions, or set to `false` to disable using transactions. [More details](https://orm.drizzle.team/docs/transactions) |
45-
| `localesSuffix` | A string appended to the end of table names for storing localized fields. Default is '_locales'. |
46-
| `relationshipsSuffix` | A string appended to the end of table names for storing relationships. Default is '_rels'. |
47-
| `versionsSuffix` | A string appended to the end of table names for storing versions. Defaults to '_v'. |
48-
| `beforeSchemaInit` | Drizzle schema hook. Runs before the schema is built. [More Details](#beforeschemainit) |
49-
| `afterSchemaInit` | Drizzle schema hook. Runs after the schema is built. [More Details](#afterschemainit) |
37+
| Option | Description |
38+
| -------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
39+
| `client` \* | [Client connection options](https://orm.drizzle.team/docs/get-started-sqlite#turso) that will be passed to `createClient` from `@libsql/client`. |
40+
| `push` | Disable Drizzle's [`db push`](https://orm.drizzle.team/kit-docs/overview#prototyping-with-db-push) in development mode. By default, `push` is enabled for development mode only. |
41+
| `migrationDir` | Customize the directory that migrations are stored. |
42+
| `logger` | The instance of the logger to be passed to drizzle. By default Payload's will be used. |
43+
| `idType` | A string of 'number', or 'uuid' that is used for the data type given to id columns. |
44+
| `transactionOptions` | A SQLiteTransactionConfig object for transactions, or set to `false` to disable using transactions. [More details](https://orm.drizzle.team/docs/transactions) |
45+
| `localesSuffix` | A string appended to the end of table names for storing localized fields. Default is '_locales'. |
46+
| `relationshipsSuffix` | A string appended to the end of table names for storing relationships. Default is '_rels'. |
47+
| `versionsSuffix` | A string appended to the end of table names for storing versions. Defaults to '_v'. |
48+
| `beforeSchemaInit` | Drizzle schema hook. Runs before the schema is built. [More Details](#beforeschemainit) |
49+
| `afterSchemaInit` | Drizzle schema hook. Runs after the schema is built. [More Details](#afterschemainit) |
50+
| `generateSchemaOutputFile` | Override generated schema from `payload generate:db-schema` file path. Defaults to `{CWD}/src/payload-generated.schema.ts` |
5051

5152
## Access to Drizzle
5253

5354
After Payload is initialized, this adapter will expose the full power of Drizzle to you for use if you need it.
5455

55-
You can access Drizzle as follows:
56+
To ensure type-safety, you need to generate Drizzle schema first with:
57+
```sh
58+
npx payload generate:db-schema
59+
```
5660

57-
```text
58-
payload.db.drizzle
61+
Then, you can access Drizzle as follows:
62+
```ts
63+
// Import table from the generated file
64+
import { posts } from './payload-generated-schema'
65+
// To avoid installing Drizzle, you can import everything that drizzle has from our re-export path.
66+
import { eq, sql, and } from '@payloadcms/db-sqlite/drizzle'
67+
68+
// Drizzle's Querying API: https://orm.drizzle.team/docs/rqb
69+
const posts = await payload.db.drizzle.query.posts.findMany()
70+
// Drizzle's Select API https://orm.drizzle.team/docs/select
71+
const result = await payload.db.drizzle.select().from(posts).where(and(eq(posts.id, 50), sql`lower(${posts.title}) = 'example post title'`))
5972
```
6073

6174
## Tables and relations
@@ -89,7 +102,7 @@ Runs before the schema is built. You can use this hook to extend your database s
89102

90103
```ts
91104
import { sqliteAdapter } from '@payloadcms/db-sqlite'
92-
import { integer, sqliteTable } from 'drizzle-orm/sqlite-core'
105+
import { integer, sqliteTable } from '@payloadcms/db-sqlite/drizzle/sqlite-core'
93106

94107
sqliteAdapter({
95108
beforeSchemaInit: [
@@ -169,7 +182,7 @@ The following example adds the `extra_integer_column` column and a composite ind
169182

170183
```ts
171184
import { sqliteAdapter } from '@payloadcms/db-sqlite'
172-
import { index, integer } from 'drizzle-orm/sqlite-core'
185+
import { index, integer } from '@payloadcms/db-sqlite/drizzle/sqlite-core'
173186
import { buildConfig } from 'payload'
174187

175188
export default buildConfig({
@@ -211,3 +224,43 @@ export default buildConfig({
211224
})
212225

213226
```
227+
228+
### Note for generated schema:
229+
Columns and tables, added in schema hooks won't be added to the generated via `payload generate:db-schema` Drizzle schema.
230+
If you want them to be there, you either have to edit this file manually or mutate the internal Payload "raw" SQL schema in the `beforeSchemaInit`:
231+
232+
```ts
233+
import { sqliteAdapter } from '@payloadcms/db-sqlite'
234+
235+
sqliteAdapter({
236+
beforeSchemaInit: [
237+
({ schema, adapter }) => {
238+
// Add a new table
239+
schema.rawTables.myTable = {
240+
name: 'my_table',
241+
columns: [{
242+
name: 'my_id',
243+
type: 'integer',
244+
primaryKey: true
245+
}],
246+
}
247+
248+
// Add a new column to generated by Payload table:
249+
schema.rawTables.posts.columns.customColumn = {
250+
name: 'custom_column',
251+
// Note that Payload SQL doesn't support everything that Drizzle does.
252+
type: 'integer',
253+
notNull: true
254+
}
255+
// Add a new index to generated by Payload table:
256+
schema.rawTables.posts.indexes.customColumnIdx = {
257+
name: 'custom_column_idx',
258+
unique: true,
259+
on: ['custom_column']
260+
}
261+
262+
return schema
263+
},
264+
],
265+
})
266+
```

‎package.json

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -54,6 +54,7 @@
5454
"clean:build": "node ./scripts/delete-recursively.js 'media/' '**/dist/' '**/.cache/' '**/.next/' '**/.turbo/' '**/tsconfig.tsbuildinfo' '**/payload*.tgz' '**/meta_*.json'",
5555
"clean:cache": "node ./scripts/delete-recursively.js node_modules/.cache! packages/payload/node_modules/.cache! .next/*",
5656
"dev": "cross-env NODE_OPTIONS=--no-deprecation tsx ./test/dev.ts",
57+
"dev:generate-db-schema": "pnpm runts ./test/generateDatabaseSchema.ts",
5758
"dev:generate-graphql-schema": "pnpm runts ./test/generateGraphQLSchema.ts",
5859
"dev:generate-importmap": "pnpm runts ./test/generateImportMap.ts",
5960
"dev:generate-types": "pnpm runts ./test/generateTypes.ts",

‎packages/db-postgres/package.json

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,26 @@
3333
"import": "./src/exports/migration-utils.ts",
3434
"types": "./src/exports/migration-utils.ts",
3535
"default": "./src/exports/migration-utils.ts"
36+
},
37+
"./drizzle": {
38+
"import": "./src/drizzle-proxy/index.ts",
39+
"types": "./src/drizzle-proxy/index.ts",
40+
"default": "./src/drizzle-proxy/index.ts"
41+
},
42+
"./drizzle/pg-core": {
43+
"import": "./src/drizzle-proxy/pg-core.ts",
44+
"types": "./src/drizzle-proxy/pg-core.ts",
45+
"default": "./src/drizzle-proxy/pg-core.ts"
46+
},
47+
"./drizzle/node-postgres": {
48+
"import": "./src/drizzle-proxy/node-postgres.ts",
49+
"types": "./src/drizzle-proxy/node-postgres.ts",
50+
"default": "./src/drizzle-proxy/node-postgres.ts"
51+
},
52+
"./drizzle/relations": {
53+
"import": "./src/drizzle-proxy/relations.ts",
54+
"types": "./src/drizzle-proxy/relations.ts",
55+
"default": "./src/drizzle-proxy/relations.ts"
3656
}
3757
},
3858
"main": "./src/index.ts",
@@ -90,6 +110,26 @@
90110
"import": "./dist/exports/migration-utils.js",
91111
"types": "./dist/exports/migration-utils.d.ts",
92112
"default": "./dist/exports/migration-utils.js"
113+
},
114+
"./drizzle": {
115+
"import": "./dist/drizzle-proxy/index.js",
116+
"types": "./dist/drizzle-proxy/index.d.ts",
117+
"default": "./dist/drizzle-proxy/index.js"
118+
},
119+
"./drizzle/pg-core": {
120+
"import": "./dist/drizzle-proxy/pg-core.js",
121+
"types": "./dist/drizzle-proxy/pg-core.d.ts",
122+
"default": "./dist/drizzle-proxy/pg-core.js"
123+
},
124+
"./drizzle/node-postgres": {
125+
"import": "./dist/drizzle-proxy/node-postgres.js",
126+
"types": "./dist/drizzle-proxy/node-postgres.d.ts",
127+
"default": "./dist/drizzle-proxy/node-postgres.js"
128+
},
129+
"./drizzle/relations": {
130+
"import": "./dist/drizzle-proxy/relations.js",
131+
"types": "./dist/drizzle-proxy/relations.d.ts",
132+
"default": "./dist/drizzle-proxy/relations.js"
93133
}
94134
},
95135
"main": "./dist/index.js",

‎packages/db-postgres/src/drizzle-proxy/index.ts

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
export * from 'drizzle-orm'

‎packages/db-postgres/src/drizzle-proxy/node-postgres.ts

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
export * from 'drizzle-orm/node-postgres'

‎packages/db-postgres/src/drizzle-proxy/pg-core.ts

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
export * from 'drizzle-orm/pg-core'

‎packages/db-postgres/src/drizzle-proxy/relations.ts

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
export * from 'drizzle-orm/relations'

‎packages/db-postgres/src/index.ts

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ import {
1010
create,
1111
createGlobal,
1212
createGlobalVersion,
13+
createSchemaGenerator,
1314
createVersion,
1415
deleteMany,
1516
deleteOne,
@@ -36,6 +37,7 @@ import {
3637
updateVersion,
3738
} from '@payloadcms/drizzle'
3839
import {
40+
columnToCodeConverter,
3941
countDistinct,
4042
createDatabase,
4143
createExtensions,
@@ -106,6 +108,14 @@ export function postgresAdapter(args: Args): DatabaseAdapterObj<PostgresAdapter>
106108
json: true,
107109
},
108110
fieldConstraints: {},
111+
generateSchema: createSchemaGenerator({
112+
columnToCodeConverter,
113+
corePackageSuffix: 'pg-core',
114+
defaultOutputFile: args.generateSchemaOutputFile,
115+
enumImport: 'pgEnum',
116+
schemaImport: 'pgSchema',
117+
tableImport: 'pgTable',
118+
}),
109119
idType: postgresIDType,
110120
initializing,
111121
localesSuffix: args.localesSuffix || '_locales',
@@ -188,4 +198,5 @@ export function postgresAdapter(args: Args): DatabaseAdapterObj<PostgresAdapter>
188198
}
189199

190200
export type { MigrateDownArgs, MigrateUpArgs } from '@payloadcms/drizzle/postgres'
201+
export { geometryColumn } from '@payloadcms/drizzle/postgres'
191202
export { sql } from 'drizzle-orm'

‎packages/db-postgres/src/types.ts

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@ import type {
88
} from '@payloadcms/drizzle/postgres'
99
import type { DrizzleAdapter } from '@payloadcms/drizzle/types'
1010
import type { DrizzleConfig } from 'drizzle-orm'
11+
import type { NodePgDatabase } from 'drizzle-orm/node-postgres'
1112
import type { PgSchema, PgTableFn, PgTransactionConfig } from 'drizzle-orm/pg-core'
1213
import type { Pool, PoolConfig } from 'pg'
1314

@@ -30,6 +31,8 @@ export type Args = {
3031
*/
3132
disableCreateDatabase?: boolean
3233
extensions?: string[]
34+
/** Generated schema from payload generate:db-schema file path */
35+
generateSchemaOutputFile?: string
3336
idType?: 'serial' | 'uuid'
3437
localesSuffix?: string
3538
logger?: DrizzleConfig['logger']
@@ -52,7 +55,17 @@ export type Args = {
5255
versionsSuffix?: string
5356
}
5457

58+
export interface GeneratedDatabaseSchema {
59+
schemaUntyped: Record<string, unknown>
60+
}
61+
62+
type ResolveSchemaType<T> = 'schema' extends keyof T
63+
? T['schema']
64+
: GeneratedDatabaseSchema['schemaUntyped']
65+
66+
type Drizzle = NodePgDatabase<ResolveSchemaType<GeneratedDatabaseSchema>>
5567
export type PostgresAdapter = {
68+
drizzle: Drizzle
5669
pool: Pool
5770
poolOptions: PoolConfig
5871
} & BasePostgresAdapter
@@ -65,7 +78,7 @@ declare module 'payload' {
6578

6679
beforeSchemaInit: PostgresSchemaHook[]
6780
beginTransaction: (options?: PgTransactionConfig) => Promise<null | number | string>
68-
drizzle: PostgresDB
81+
drizzle: Drizzle
6982
enums: Record<string, GenericEnum>
7083
extensions: Record<string, boolean>
7184
/**

0 commit comments

Comments
 (0)