Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hub DB; turbo generator #3686

Merged
merged 6 commits into from
Mar 5, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 2 additions & 33 deletions apps/hub/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,40 +16,9 @@ EMAIL_FROM=dev@squigglehub.org # Doesn't matter too much
ROOT_EMAILS=[your-email] # Email of the root user on Squiggle Hub, to get extra permissions
```

## Migrating the development database
## Managing the database

Database migrations are created with a usual [Prisma](https://www.prisma.io/) workflow.

Run `pnpm run db:migrate:dev` once to create the necessary tables in the database.

When you pull new changes from GitHub that include more migrations, you'll have to run `pnpm run db:migrate:dev` again.

## Changing the database

You should usually prototype schema changes with `prisma db push`, as described in [Prisma docs](https://www.prisma.io/docs/orm/prisma-migrate/workflows/prototyping-your-schema).

The basic loop is:

1. Edit `schema.prisma`
2. Run `npx prisma db push`
3. Check the results; repeat from step 1 until your branch is ready.
4. Run `pnpm prisma migrate dev --name [migration-name-here]` to create a migration when your schema changes have stabilized.
5. (Optional) Update the migration file if necessary; one common scenario is if you want to add a new non-nullable column, then you'll have to do first `ALTER TABLE` to a nullable column, then `UPDATE` it with the default value, and then `ALTER TABLE` to non-nullable value again.

## Notes on changing the schema

`pnpm run gen` generates `@prisma/client` from `prisma/schema.prisma`.

If it looks like VS Code doesn't see your latest changes, try this:

1. Run `pnpm run gen`.
2. Invoke `> TypeScript: Restart TS Server` in VS Code (you should be editing any `.ts` or `.tsx` file for this command to show)

Note: the "Restart TS Server" step is necessary because `@prisma/client` code is out of the main source tree, and VS Code won't notice that it has updated. But restarting TS Server is slow, so a better solution is to keep `@prisma/client` source code open (open `src/prisma.ts`, then "Go to definition" on `PrismaClient`). Then VS Code will watch it for changes.

## Other notes

[Common workflow for updating Prisma schema](https://www.prisma.io/docs/orm/prisma-migrate/workflows/prototyping-your-schema)
Database layer is in the `internal-packages/hub-db` package. See [hub-db/README.md](./internal-packages/hub-db/README.md) for details.

# Deployment

Expand Down
9 changes: 9 additions & 0 deletions apps/hub/next.config.mjs
Original file line number Diff line number Diff line change
@@ -1,9 +1,18 @@
import { PrismaPlugin } from "@prisma/nextjs-monorepo-workaround-plugin";

/** @type {import('next').NextConfig} */
const nextConfig = {
reactStrictMode: true,
experimental: {
runtime: "nodejs",
},
webpack: (config, { isServer }) => {
if (isServer) {
config.plugins = [...config.plugins, new PrismaPlugin()];
}

return config;
},
redirects: async () => [
{
source: "/playground",
Expand Down
16 changes: 5 additions & 11 deletions apps/hub/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -6,18 +6,12 @@
"node": "22.x"
},
"scripts": {
"db:migrate:dev": "PRISMA_HIDE_UPDATE_MESSAGE=1 prisma migrate dev",
"db:migrate": "PRISMA_HIDE_UPDATE_MESSAGE=1 prisma migrate deploy",
"db:reset": "PRISMA_HIDE_UPDATE_MESSAGE=1 prisma migrate reset",
"dev": "next dev -p 3001 --turbo",
"start": "__NEXT_PRIVATE_PREBUNDLED_REACT=next next start",
"gen:prisma": "PRISMA_HIDE_UPDATE_MESSAGE=1 prisma generate --no-hints",
"gen": "pnpm gen:prisma",
"gen:watch": "nodemon --watch src --ext ts,tsx,prisma --exec 'pnpm run gen'",
"start": "next start",
"build:esbuild": "node ./esbuild.cjs",
"build:ts": "pnpm gen && tsc",
"build:ts": "tsc",
"bundle": "pnpm build:esbuild && cp ../../node_modules/.pnpm/@prisma+client*/node_modules/.prisma/client/*.node ./dist",
"build": "pnpm gen && __NEXT_PRIVATE_PREBUNDLED_REACT=next next build",
"build": "next build",
"lint": "pnpm lint:prettier && pnpm eslint",
"lint:prettier": "prettier --check .",
"eslint": "ESLINT_USE_FLAT_CONFIG=false NODE_NO_WARNINGS=1 eslint src",
Expand All @@ -27,12 +21,12 @@
},
"dependencies": {
"@auth/prisma-adapter": "^2.7.4",
"@prisma/client": "6.3.1",
"@quri/squiggle-ai": "workspace:*",
"@quri/squiggle-components": "workspace:*",
"@quri/squiggle-lang": "workspace:*",
"@quri/ui": "workspace:*",
"@quri/versioned-squiggle-components": "workspace:*",
"@quri/hub-db": "workspace:*",
"@vercel/analytics": "^1.3.1",
"base64-js": "^1.5.1",
"clsx": "^2.1.1",
Expand All @@ -58,6 +52,7 @@
"zod": "^3.24.1"
},
"devDependencies": {
"@prisma/nextjs-monorepo-workaround-plugin": "^6.3.1",
"@quri/configs": "workspace:*",
"@types/d3": "^7.4.3",
"@types/invariant": "^2.2.37",
Expand All @@ -75,7 +70,6 @@
"nodemon": "^3.1.9",
"postcss": "^8.4.38",
"prettier": "^3.4.2",
"prisma": "^6.3.1",
"tailwindcss": "^3.4.3",
"tsx": "^4.19.1",
"typescript": "^5.7.3"
Expand Down
3 changes: 1 addition & 2 deletions apps/hub/src/ai/data/analytics.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
import * as Prisma from "@prisma/client";

import * as Prisma from "@quri/hub-db";
import { CodeArtifact, Workflow } from "@quri/squiggle-ai/server";

import { getAiCodec } from "@/ai/data/utils";
Expand Down
3 changes: 1 addition & 2 deletions apps/hub/src/ai/data/loadWorkflows.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
import { Prisma } from "@prisma/client";

import { Prisma } from "@quri/hub-db";
import { ClientWorkflow } from "@quri/squiggle-ai";

import { prisma } from "@/lib/server/prisma";
Expand Down
3 changes: 1 addition & 2 deletions apps/hub/src/ai/data/storage.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
import "server-only";

import { AiWorkflow as PrismaAiWorkflow } from "@prisma/client";

import { AiWorkflow as PrismaAiWorkflow } from "@quri/hub-db";
import { ClientWorkflow } from "@quri/squiggle-ai";

import { decodeV1_0JsonToClientWorkflow } from "@/ai/data/v1_0";
Expand Down
2 changes: 1 addition & 1 deletion apps/hub/src/ai/data/v1_0.ts
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
import "server-only";

import { Prisma } from "@prisma/client";
import { z } from "zod";

import { Prisma } from "@quri/hub-db";
import { ClientWorkflow } from "@quri/squiggle-ai";

// Snapshot of ClientWorkflow schemas as it was at the time we upgraded to V2
Expand Down
2 changes: 1 addition & 1 deletion apps/hub/src/ai/data/v2_0.ts
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
import "server-only";

import { Prisma } from "@prisma/client";
import { z } from "zod";

import { Prisma } from "@quri/hub-db";
import { ClientWorkflow } from "@quri/squiggle-ai";
import { Workflow } from "@quri/squiggle-ai/server";

Expand Down
3 changes: 2 additions & 1 deletion apps/hub/src/app/api/search/route.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
import { Searchable as PrismaSearchable } from "@prisma/client";
import { NextRequest } from "next/server";
import { z } from "zod";

import { Searchable as PrismaSearchable } from "@quri/hub-db";

import {
groupRoute,
modelRoute,
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import { MembershipRole } from "@prisma/client";
import { FC } from "react";

import { MembershipRole } from "@quri/hub-db";
import {
DropdownMenuModalActionItem,
PlusIcon,
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import { type MembershipRole } from "@prisma/client";
import { FC } from "react";

import { type MembershipRole } from "@quri/hub-db";
import { Button, Dropdown, DropdownMenu } from "@quri/ui";

import { GroupMemberDTO } from "@/groups/data/members";
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
import { MembershipRole } from "@prisma/client";
import { FC } from "react";

import { MembershipRole } from "@quri/hub-db";

import { SafeActionDropdownAction } from "@/components/ui/SafeActionDropdownAction";
import { updateMembershipRoleAction } from "@/groups/actions/updateMembershipRoleAction";
import { GroupMemberDTO } from "@/groups/data/members";
Expand Down
3 changes: 2 additions & 1 deletion apps/hub/src/groups/actions/addUserToGroupAction.ts
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
"use server";

import { MembershipRole } from "@prisma/client";
import { z } from "zod";

import { MembershipRole } from "@quri/hub-db";

import { actionClient, ActionError } from "@/lib/server/actionClient";
import { prisma } from "@/lib/server/prisma";
import { zSlug } from "@/lib/zodUtils";
Expand Down
3 changes: 2 additions & 1 deletion apps/hub/src/groups/actions/updateMembershipRoleAction.ts
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
"use server";

import { MembershipRole } from "@prisma/client";
import { z } from "zod";

import { MembershipRole } from "@quri/hub-db";

import { actionClient, ActionError } from "@/lib/server/actionClient";
import { prisma } from "@/lib/server/prisma";
import { zSlug } from "@/lib/zodUtils";
Expand Down
2 changes: 1 addition & 1 deletion apps/hub/src/groups/data/groupCards.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { Prisma } from "@prisma/client";
import { Prisma } from "@quri/hub-db";

import { auth } from "@/lib/server/auth";
import { prisma } from "@/lib/server/prisma";
Expand Down
2 changes: 1 addition & 1 deletion apps/hub/src/groups/data/members.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { MembershipRole, Prisma } from "@prisma/client";
import { MembershipRole, Prisma } from "@quri/hub-db";

import { auth } from "@/lib/server/auth";
import { prisma } from "@/lib/server/prisma";
Expand Down
3 changes: 2 additions & 1 deletion apps/hub/src/lib/server/actionClient.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
import { Prisma } from "@prisma/client";
import {
createSafeActionClient,
DEFAULT_SERVER_ERROR_MESSAGE,
Expand All @@ -7,6 +6,8 @@ import {
} from "next-safe-action";
import { z } from "zod";

import { Prisma } from "@quri/hub-db";

export class ActionError extends Error {}

export const actionClient = createSafeActionClient({
Expand Down
4 changes: 3 additions & 1 deletion apps/hub/src/lib/server/auth.ts
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,9 @@ function buildAuthConfig(): NextAuthConfig {
}

const config: NextAuthConfig = {
adapter: PrismaAdapter(prisma),
adapter: PrismaAdapter(
prisma as any // @auth/prisma-adapter doesn't support our PrismaClient from @quri/hub-db
),
providers,
callbacks: {
async session({ session, user }) {
Expand Down
18 changes: 9 additions & 9 deletions apps/hub/src/lib/server/prisma.ts
Original file line number Diff line number Diff line change
@@ -1,24 +1,24 @@
import "server-only";

import { PrismaClient } from "@prisma/client";
import { PrismaClient } from "@quri/hub-db";

declare global {
// allow global `var` declarations
// eslint-disable-next-line no-var
var _prismaConfig: PrismaConfig;
var _prisma: PrismaClient | undefined;
var _hubPrismaConfig: PrismaConfig;
var _hubPrisma: PrismaClient | undefined;
}

type PrismaConfig = {
logs: "none" | "query" | "query-with-params";
};

global._prismaConfig = {
global._hubPrismaConfig = {
logs: process.env.NODE_ENV === "test" ? "none" : "query",
};

function makePrisma() {
const config = global._prismaConfig;
const config = global._hubPrismaConfig;
const prisma = new PrismaClient({
log:
config.logs === "none"
Expand All @@ -43,17 +43,17 @@ function makePrisma() {
return prisma;
}

export let prisma = global._prisma || makePrisma();
export let prisma = global._hubPrisma || makePrisma();

// Single prisma instance for the entire app, in dev mode.
// This helps with connection leaks during hot reloads.
if (process.env.NODE_ENV !== "production") global._prisma = prisma;
if (process.env.NODE_ENV !== "production") global._hubPrisma = prisma;

// This will work only in dev mode, and will be invoked only in dev mode.
export async function resetPrisma(config: PrismaConfig) {
if (process.env.NODE_ENV === "production") return;
global._prismaConfig = config;
global._hubPrismaConfig = config;
await prisma.$disconnect();
prisma = makePrisma();
global._prisma = prisma;
global._hubPrisma = prisma;
}
2 changes: 1 addition & 1 deletion apps/hub/src/lib/server/runSquiggle.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import { Prisma } from "@prisma/client";
import crypto from "crypto";

import { Prisma } from "@quri/hub-db";
import {
SqLinker,
SqModule,
Expand Down
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
"use server";
import { RelativeValuesDefinition } from "@prisma/client";
import { revalidatePath } from "next/cache";
import { z } from "zod";

import { RelativeValuesDefinition } from "@quri/hub-db";
import { squiggleVersions } from "@quri/versioned-squiggle-components";

import { modelRoute } from "@/lib/routes";
Expand Down
2 changes: 1 addition & 1 deletion apps/hub/src/models/data/authHelpers.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { Prisma } from "@prisma/client";
import { Prisma } from "@quri/hub-db";

import { auth } from "@/lib/server/auth";

Expand Down
2 changes: 1 addition & 1 deletion apps/hub/src/models/data/cards.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { Prisma } from "@prisma/client";
import { Prisma } from "@quri/hub-db";

import { prisma } from "@/lib/server/prisma";
import { Paginated } from "@/lib/types";
Expand Down
2 changes: 1 addition & 1 deletion apps/hub/src/models/data/full.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { Prisma } from "@prisma/client";
import { Prisma } from "@quri/hub-db";

import { prisma } from "@/lib/server/prisma";
import { controlsOwnerId } from "@/owners/data/auth";
Expand Down
2 changes: 1 addition & 1 deletion apps/hub/src/models/data/fullRevision.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { Prisma } from "@prisma/client";
import { Prisma } from "@quri/hub-db";

import { prisma } from "@/lib/server/prisma";

Expand Down
2 changes: 1 addition & 1 deletion apps/hub/src/models/data/revisions.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { Prisma } from "@prisma/client";
import { Prisma } from "@quri/hub-db";

import { prisma } from "@/lib/server/prisma";
import { Paginated } from "@/lib/types";
Expand Down
2 changes: 1 addition & 1 deletion apps/hub/src/models/utils.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { Model, Prisma } from "@prisma/client";
import { Model, Prisma } from "@quri/hub-db";

import { ActionError } from "@/lib/server/actionClient";
import { prisma } from "@/lib/server/prisma";
Expand Down
2 changes: 1 addition & 1 deletion apps/hub/src/owners/data/typedOwner.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { Prisma } from "@prisma/client";
import { Prisma } from "@quri/hub-db";

import { prisma } from "@/lib/server/prisma";

Expand Down
2 changes: 1 addition & 1 deletion apps/hub/src/relative-values/data/cards.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { Prisma } from "@prisma/client";
import { Prisma } from "@quri/hub-db";

import { prisma } from "@/lib/server/prisma";
import { Paginated } from "@/lib/types";
Expand Down
2 changes: 1 addition & 1 deletion apps/hub/src/relative-values/data/exports.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { Prisma } from "@prisma/client";
import { Prisma } from "@quri/hub-db";

import { prisma } from "@/lib/server/prisma";
import { modelWhereHasAccess } from "@/models/data/authHelpers";
Expand Down
3 changes: 2 additions & 1 deletion apps/hub/src/relative-values/data/full.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
import { Prisma } from "@prisma/client";
import { z } from "zod";

import { Prisma } from "@quri/hub-db";

import { prisma } from "@/lib/server/prisma";
import {
relativeValuesClustersSchema,
Expand Down
2 changes: 1 addition & 1 deletion apps/hub/src/relative-values/data/fullExport.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { Prisma } from "@prisma/client";
import { Prisma } from "@quri/hub-db";

import { prisma } from "@/lib/server/prisma";

Expand Down
3 changes: 2 additions & 1 deletion apps/hub/src/scripts/buildRecentModelRevision/main.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
import { PrismaClient } from "@prisma/client";
import { spawn } from "node:child_process";

import { PrismaClient } from "@quri/hub-db";

import { createVariableRevision } from "./createVariableRevision";
import { WorkerOutput, WorkerRunMessage } from "./worker";

Expand Down
Loading
Loading