Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(db-postgres, db-sqlite): drizzle schema generation #9953

Merged
merged 2 commits into from
Dec 19, 2024
Merged

feat(db-postgres, db-sqlite): drizzle schema generation #9953

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
ea6e6f3
feat(db-postgres, db-sqlite): drizzle schema generation, commit squas…
r1tsuu Dec 15, 2024
8f3221f
1: write docs, 2: write int test and improve conditoning in tests, wi…
r1tsuu Dec 19, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 58 additions & 6 deletions docs/database/postgres.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,21 +65,33 @@ export default buildConfig({
| `schemaName` (experimental) | A string for the postgres schema to use, defaults to 'public'. |
| `idType` | A string of 'serial', or 'uuid' that is used for the data type given to id columns. |
| `transactionOptions` | A PgTransactionConfig object for transactions, or set to `false` to disable using transactions. [More details](https://orm.drizzle.team/docs/transactions) |
| `disableCreateDatabase` | Pass `true` to disable auto database creation if it doesn't exist. Defaults to `false`. |
| `disableCreateDatabase` | Pass `true` to disable auto database creation if it doesn't exist. Defaults to `false`. |
| `localesSuffix` | A string appended to the end of table names for storing localized fields. Default is '_locales'. |
| `relationshipsSuffix` | A string appended to the end of table names for storing relationships. Default is '_rels'. |
| `versionsSuffix` | A string appended to the end of table names for storing versions. Defaults to '_v'. |
| `beforeSchemaInit` | Drizzle schema hook. Runs before the schema is built. [More Details](#beforeschemainit) |
| `afterSchemaInit` | Drizzle schema hook. Runs after the schema is built. [More Details](#afterschemainit) |
| `generateSchemaOutputFile` | Override generated schema from `payload generate:db-schema` file path. Defaults to `{CWD}/src/payload-generated.schema.ts` |

## Access to Drizzle

After Payload is initialized, this adapter will expose the full power of Drizzle to you for use if you need it.

You can access Drizzle as follows:
To ensure type-safety, you need to generate Drizzle schema first with:
```sh
npx payload generate:db-schema
```

```text
payload.db.drizzle
Then, you can access Drizzle as follows:
```ts
import { posts } from './payload-generated-schema'
// To avoid installing Drizzle, you can import everything that drizzle has from our re-export path.
import { eq, sql, and } from '@payloadcms/db-postgres/drizzle'

// Drizzle's Querying API: https://orm.drizzle.team/docs/rqb
const posts = await payload.db.drizzle.query.posts.findMany()
// Drizzle's Select API https://orm.drizzle.team/docs/select
const result = await payload.db.drizzle.select().from(posts).where(and(eq(posts.id, 50), sql`lower(${posts.title}) = 'example post title'`))
```

## Tables, relations, and enums
Expand Down Expand Up @@ -114,7 +126,7 @@ Runs before the schema is built. You can use this hook to extend your database s

```ts
import { postgresAdapter } from '@payloadcms/db-postgres'
import { integer, pgTable, serial } from 'drizzle-orm/pg-core'
import { integer, pgTable, serial } from '@payloadcms/db-postgres/drizzle/pg-core'

postgresAdapter({
beforeSchemaInit: [
Expand Down Expand Up @@ -194,7 +206,7 @@ The following example adds the `extra_integer_column` column and a composite ind

```ts
import { postgresAdapter } from '@payloadcms/db-postgres'
import { index, integer } from 'drizzle-orm/pg-core'
import { index, integer } from '@payloadcms/db-postgres/drizzle/pg-core'
import { buildConfig } from 'payload'

export default buildConfig({
Expand Down Expand Up @@ -236,3 +248,43 @@ export default buildConfig({
})

```

### Note for generated schema:
Columns and tables, added in schema hooks won't be added to the generated via `payload generate:db-schema` Drizzle schema.
If you want them to be there, you either have to edit this file manually or mutate the internal Payload "raw" SQL schema in the `beforeSchemaInit`:

```ts
import { postgresAdapter } from '@payloadcms/db-postgres'

postgresAdapter({
beforeSchemaInit: [
({ schema, adapter }) => {
// Add a new table
schema.rawTables.myTable = {
name: 'my_table',
columns: [{
name: 'my_id',
type: 'serial',
primaryKey: true
}],
}

// Add a new column to generated by Payload table:
schema.rawTables.posts.columns.customColumn = {
name: 'custom_column',
// Note that Payload SQL doesn't support everything that Drizzle does.
type: 'integer',
notNull: true
}
// Add a new index to generated by Payload table:
schema.rawTables.posts.indexes.customColumnIdx = {
name: 'custom_column_idx',
unique: true,
on: ['custom_column']
}

return schema
},
],
})
```
89 changes: 71 additions & 18 deletions docs/database/sqlite.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,28 +34,41 @@ export default buildConfig({

## Options

| Option | Description |
| --------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `client` \* | [Client connection options](https://orm.drizzle.team/docs/get-started-sqlite#turso) that will be passed to `createClient` from `@libsql/client`. |
| `push` | Disable Drizzle's [`db push`](https://orm.drizzle.team/kit-docs/overview#prototyping-with-db-push) in development mode. By default, `push` is enabled for development mode only. |
| `migrationDir` | Customize the directory that migrations are stored. |
| `logger` | The instance of the logger to be passed to drizzle. By default Payload's will be used. |
| `idType` | A string of 'number', or 'uuid' that is used for the data type given to id columns. |
| `transactionOptions` | A SQLiteTransactionConfig object for transactions, or set to `false` to disable using transactions. [More details](https://orm.drizzle.team/docs/transactions) |
| `localesSuffix` | A string appended to the end of table names for storing localized fields. Default is '_locales'. |
| `relationshipsSuffix` | A string appended to the end of table names for storing relationships. Default is '_rels'. |
| `versionsSuffix` | A string appended to the end of table names for storing versions. Defaults to '_v'. |
| `beforeSchemaInit` | Drizzle schema hook. Runs before the schema is built. [More Details](#beforeschemainit) |
| `afterSchemaInit` | Drizzle schema hook. Runs after the schema is built. [More Details](#afterschemainit) |
| Option | Description |
| -------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `client` \* | [Client connection options](https://orm.drizzle.team/docs/get-started-sqlite#turso) that will be passed to `createClient` from `@libsql/client`. |
| `push` | Disable Drizzle's [`db push`](https://orm.drizzle.team/kit-docs/overview#prototyping-with-db-push) in development mode. By default, `push` is enabled for development mode only. |
| `migrationDir` | Customize the directory that migrations are stored. |
| `logger` | The instance of the logger to be passed to drizzle. By default Payload's will be used. |
| `idType` | A string of 'number', or 'uuid' that is used for the data type given to id columns. |
| `transactionOptions` | A SQLiteTransactionConfig object for transactions, or set to `false` to disable using transactions. [More details](https://orm.drizzle.team/docs/transactions) |
| `localesSuffix` | A string appended to the end of table names for storing localized fields. Default is '_locales'. |
| `relationshipsSuffix` | A string appended to the end of table names for storing relationships. Default is '_rels'. |
| `versionsSuffix` | A string appended to the end of table names for storing versions. Defaults to '_v'. |
| `beforeSchemaInit` | Drizzle schema hook. Runs before the schema is built. [More Details](#beforeschemainit) |
| `afterSchemaInit` | Drizzle schema hook. Runs after the schema is built. [More Details](#afterschemainit) |
| `generateSchemaOutputFile` | Override generated schema from `payload generate:db-schema` file path. Defaults to `{CWD}/src/payload-generated.schema.ts` |

## Access to Drizzle

After Payload is initialized, this adapter will expose the full power of Drizzle to you for use if you need it.

You can access Drizzle as follows:
To ensure type-safety, you need to generate Drizzle schema first with:
```sh
npx payload generate:db-schema
```

```text
payload.db.drizzle
Then, you can access Drizzle as follows:
```ts
// Import table from the generated file
import { posts } from './payload-generated-schema'
// To avoid installing Drizzle, you can import everything that drizzle has from our re-export path.
import { eq, sql, and } from '@payloadcms/db-sqlite/drizzle'

// Drizzle's Querying API: https://orm.drizzle.team/docs/rqb
const posts = await payload.db.drizzle.query.posts.findMany()
// Drizzle's Select API https://orm.drizzle.team/docs/select
const result = await payload.db.drizzle.select().from(posts).where(and(eq(posts.id, 50), sql`lower(${posts.title}) = 'example post title'`))
```

## Tables and relations
Expand Down Expand Up @@ -89,7 +102,7 @@ Runs before the schema is built. You can use this hook to extend your database s

```ts
import { sqliteAdapter } from '@payloadcms/db-sqlite'
import { integer, sqliteTable } from 'drizzle-orm/sqlite-core'
import { integer, sqliteTable } from '@payloadcms/db-sqlite/drizzle/sqlite-core'

sqliteAdapter({
beforeSchemaInit: [
Expand Down Expand Up @@ -169,7 +182,7 @@ The following example adds the `extra_integer_column` column and a composite ind

```ts
import { sqliteAdapter } from '@payloadcms/db-sqlite'
import { index, integer } from 'drizzle-orm/sqlite-core'
import { index, integer } from '@payloadcms/db-sqlite/drizzle/sqlite-core'
import { buildConfig } from 'payload'

export default buildConfig({
Expand Down Expand Up @@ -211,3 +224,43 @@ export default buildConfig({
})

```

### Note for generated schema:
Columns and tables, added in schema hooks won't be added to the generated via `payload generate:db-schema` Drizzle schema.
If you want them to be there, you either have to edit this file manually or mutate the internal Payload "raw" SQL schema in the `beforeSchemaInit`:

```ts
import { sqliteAdapter } from '@payloadcms/db-sqlite'

sqliteAdapter({
beforeSchemaInit: [
({ schema, adapter }) => {
// Add a new table
schema.rawTables.myTable = {
name: 'my_table',
columns: [{
name: 'my_id',
type: 'integer',
primaryKey: true
}],
}

// Add a new column to generated by Payload table:
schema.rawTables.posts.columns.customColumn = {
name: 'custom_column',
// Note that Payload SQL doesn't support everything that Drizzle does.
type: 'integer',
notNull: true
}
// Add a new index to generated by Payload table:
schema.rawTables.posts.indexes.customColumnIdx = {
name: 'custom_column_idx',
unique: true,
on: ['custom_column']
}

return schema
},
],
})
```
1 change: 1 addition & 0 deletions package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@
"clean:build": "node ./scripts/delete-recursively.js 'media/' '**/dist/' '**/.cache/' '**/.next/' '**/.turbo/' '**/tsconfig.tsbuildinfo' '**/payload*.tgz' '**/meta_*.json'",
"clean:cache": "node ./scripts/delete-recursively.js node_modules/.cache! packages/payload/node_modules/.cache! .next/*",
"dev": "cross-env NODE_OPTIONS=--no-deprecation tsx ./test/dev.ts",
"dev:generate-db-schema": "pnpm runts ./test/generateDatabaseSchema.ts",
"dev:generate-graphql-schema": "pnpm runts ./test/generateGraphQLSchema.ts",
"dev:generate-importmap": "pnpm runts ./test/generateImportMap.ts",
"dev:generate-types": "pnpm runts ./test/generateTypes.ts",
Expand Down
40 changes: 40 additions & 0 deletions packages/db-postgres/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,26 @@
"import": "./src/exports/migration-utils.ts",
"types": "./src/exports/migration-utils.ts",
"default": "./src/exports/migration-utils.ts"
},
"./drizzle": {
"import": "./src/drizzle-proxy/index.ts",
"types": "./src/drizzle-proxy/index.ts",
"default": "./src/drizzle-proxy/index.ts"
},
"./drizzle/pg-core": {
"import": "./src/drizzle-proxy/pg-core.ts",
"types": "./src/drizzle-proxy/pg-core.ts",
"default": "./src/drizzle-proxy/pg-core.ts"
},
"./drizzle/node-postgres": {
"import": "./src/drizzle-proxy/node-postgres.ts",
"types": "./src/drizzle-proxy/node-postgres.ts",
"default": "./src/drizzle-proxy/node-postgres.ts"
},
"./drizzle/relations": {
"import": "./src/drizzle-proxy/relations.ts",
"types": "./src/drizzle-proxy/relations.ts",
"default": "./src/drizzle-proxy/relations.ts"
}
},
"main": "./src/index.ts",
Expand Down Expand Up @@ -90,6 +110,26 @@
"import": "./dist/exports/migration-utils.js",
"types": "./dist/exports/migration-utils.d.ts",
"default": "./dist/exports/migration-utils.js"
},
"./drizzle": {
"import": "./dist/drizzle-proxy/index.js",
"types": "./dist/drizzle-proxy/index.d.ts",
"default": "./dist/drizzle-proxy/index.js"
},
"./drizzle/pg-core": {
"import": "./dist/drizzle-proxy/pg-core.js",
"types": "./dist/drizzle-proxy/pg-core.d.ts",
"default": "./dist/drizzle-proxy/pg-core.js"
},
"./drizzle/node-postgres": {
"import": "./dist/drizzle-proxy/node-postgres.js",
"types": "./dist/drizzle-proxy/node-postgres.d.ts",
"default": "./dist/drizzle-proxy/node-postgres.js"
},
"./drizzle/relations": {
"import": "./dist/drizzle-proxy/relations.js",
"types": "./dist/drizzle-proxy/relations.d.ts",
"default": "./dist/drizzle-proxy/relations.js"
}
},
"main": "./dist/index.js",
Expand Down
1 change: 1 addition & 0 deletions packages/db-postgres/src/drizzle-proxy/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
export * from 'drizzle-orm'
1 change: 1 addition & 0 deletions packages/db-postgres/src/drizzle-proxy/node-postgres.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
export * from 'drizzle-orm/node-postgres'
1 change: 1 addition & 0 deletions packages/db-postgres/src/drizzle-proxy/pg-core.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
export * from 'drizzle-orm/pg-core'
1 change: 1 addition & 0 deletions packages/db-postgres/src/drizzle-proxy/relations.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
export * from 'drizzle-orm/relations'
11 changes: 11 additions & 0 deletions packages/db-postgres/src/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import {
create,
createGlobal,
createGlobalVersion,
createSchemaGenerator,
createVersion,
deleteMany,
deleteOne,
Expand All @@ -36,6 +37,7 @@ import {
updateVersion,
} from '@payloadcms/drizzle'
import {
columnToCodeConverter,
countDistinct,
createDatabase,
createExtensions,
Expand Down Expand Up @@ -106,6 +108,14 @@ export function postgresAdapter(args: Args): DatabaseAdapterObj<PostgresAdapter>
json: true,
},
fieldConstraints: {},
generateSchema: createSchemaGenerator({
columnToCodeConverter,
corePackageSuffix: 'pg-core',
defaultOutputFile: args.generateSchemaOutputFile,
enumImport: 'pgEnum',
schemaImport: 'pgSchema',
tableImport: 'pgTable',
}),
idType: postgresIDType,
initializing,
localesSuffix: args.localesSuffix || '_locales',
Expand Down Expand Up @@ -188,4 +198,5 @@ export function postgresAdapter(args: Args): DatabaseAdapterObj<PostgresAdapter>
}

export type { MigrateDownArgs, MigrateUpArgs } from '@payloadcms/drizzle/postgres'
export { geometryColumn } from '@payloadcms/drizzle/postgres'
export { sql } from 'drizzle-orm'
15 changes: 14 additions & 1 deletion packages/db-postgres/src/types.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ import type {
} from '@payloadcms/drizzle/postgres'
import type { DrizzleAdapter } from '@payloadcms/drizzle/types'
import type { DrizzleConfig } from 'drizzle-orm'
import type { NodePgDatabase } from 'drizzle-orm/node-postgres'
import type { PgSchema, PgTableFn, PgTransactionConfig } from 'drizzle-orm/pg-core'
import type { Pool, PoolConfig } from 'pg'

Expand All @@ -30,6 +31,8 @@ export type Args = {
*/
disableCreateDatabase?: boolean
extensions?: string[]
/** Generated schema from payload generate:db-schema file path */
generateSchemaOutputFile?: string
idType?: 'serial' | 'uuid'
localesSuffix?: string
logger?: DrizzleConfig['logger']
Expand All @@ -52,7 +55,17 @@ export type Args = {
versionsSuffix?: string
}

export interface GeneratedDatabaseSchema {
schemaUntyped: Record<string, unknown>
}

type ResolveSchemaType<T> = 'schema' extends keyof T
? T['schema']
: GeneratedDatabaseSchema['schemaUntyped']

type Drizzle = NodePgDatabase<ResolveSchemaType<GeneratedDatabaseSchema>>
export type PostgresAdapter = {
drizzle: Drizzle
pool: Pool
poolOptions: PoolConfig
} & BasePostgresAdapter
Expand All @@ -65,7 +78,7 @@ declare module 'payload' {

beforeSchemaInit: PostgresSchemaHook[]
beginTransaction: (options?: PgTransactionConfig) => Promise<null | number | string>
drizzle: PostgresDB
drizzle: Drizzle
enums: Record<string, GenericEnum>
extensions: Record<string, boolean>
/**
Expand Down
Loading
Loading