Supabase
Branching - Early access | Feedback discussion #18937
Replies: 96 comments 217 replies
-
What is Supabase Branching? |
Beta Was this translation helpful? Give feedback.
-
|
In the most concise sense, you can use branching to safely and easily experiment with changes to your Supabase projects. |
Beta Was this translation helpful? Give feedback.
-
|
Is branching expected to be available only from |
Beta Was this translation helpful? Give feedback.
-
|
@KajSzy We've currently closed the Early Access programme for Branching. If you have already upgraded expecting access, if you could send in a support request and we'll look into it 😄. I think some of our docs have not been updated to make it clear that it's in private early access, and we've now closed accepting any more requests. I'll update the docs today to be more clear. |
Beta Was this translation helpful? Give feedback.
-
|
Hope you're all doing great! I've been working with supabase on a project for 2 months and noticed something I wanted to share with you that would cause issues for supabase branching. As our projects grow, our seed file (you know, the trusty I've been sidestepping this by ditching the massive seed file and scripting my local database seeding instead. It's a lot like how migrations work: each file is timestamped and has a quick description, and they run in a set order. This means I have separate files for different seeding needs, like permission tables and location table inserts broken down into 1K-insert chunks – it’s pretty neat. Currently, Supabase runs whatever is in I reckon this could be something worth considering for your roadmap for seeding a branch. Thanks for all the hard work you've been putting into Supabase, and I'm excited to see how branching evolves! |
Beta Was this translation helpful? Give feedback.
-
|
Neat idea. I also mentioned this to @sweatybridge who is one of the developers managing our migration/seed runner. I suppose it would have to be timestamped for it to work correctly. Do we know of any other tools doing this? We can't be the first ones to think of this? 😊 |
Beta Was this translation helpful? Give feedback.
-
|
To be honest I took the idea directly from my experience of working with the ORM Sequelize. Here is a relevant documentation that mention both migrations and seed files: https://sequelize.org/docs/v6/other-topics/migrations/ One thing to note is that their migrations have an up (which is covered by seed.sql) and a down that is not covered by supabase. The down function is if you are to roll back a migration including any data manipulation that was required to complete the migration. It's very useful when you can't predict how a project will evolve once it's in front of users. Sequelize also handle transactions in their migrations, which makes migrating production databases a lot less intimidating. |
Beta Was this translation helpful? Give feedback.
-
|
There is a similar discussion to this here as well now 🙌 |
Beta Was this translation helpful? Give feedback.
-
|
Hasura uses timestamped seed files. https://hasura.io/docs/2.0/migrations-metadata-seeds/manage-seeds/ |
Beta Was this translation helpful? Give feedback.
-
|
if new branch is created, new Supabase url for client is reserved on new branch?? I maen if I use following url on main branch Can I use a different url for a new branch like staging? For example, |
Beta Was this translation helpful? Give feedback.
-
|
How it works today is a bit like that but it's the id string in the middle that changes. main branch (production) const supabase = createClient('https://spztrrugbkgdayfejlvi.supabase.co/', 'public-anon-key')the feature branch your working on will then have a completely different id in the url const supabase = createClient('https://qrelykancvxikefwrefd.supabase.co/', 'public-anon-key')I could understand if this might not be useful in some circumstances, as it makes it harder to build urls based on git branch name in maybe your client or server app. The problem extends to API keys though as well, since every branch in Supabase has it's own unique set of API keys, db password and so on. What do you think @chaos128 ? |
Beta Was this translation helpful? Give feedback.
-
|
It would be okay but I think it would be a little bit inconvenient. It would be better if domain and key are not changed but only pathname is changed based on branch names like I said on question. main branch staging branch feature/test branch |
Beta Was this translation helpful? Give feedback.
-
|
That's interesting. Can see that being a lot more useful and easier to use. Also the keys not changing between Preview Branches is interesting as well - we were mostly concerned with Production and Preview being isolated and concentrated efforts on that and hadn't really investigated on making it easy to switch between Preview branches with predictable URLs and consistent keys. Great suggestion. |
Beta Was this translation helpful? Give feedback.
-
|
@chaos128 how are you using Supabase Branching? |
Beta Was this translation helpful? Give feedback.
-
|
Chiming in here because I would also love to see predictable URLs and consistent keys. We have a Next.js app on Vercel, using Prisma. Our current workflow, without Supabase branching: Our branch That means preview/feature branches create Vercel preview deployments, but have to stay unmigrated (except for staging itself), since we don't want to mess up the staging db before merging to If we turned on Supabase branching on our staging db, I'm picturing that it could work for preview deployments if the app and, separately, the GH action, can both get or construct the branch db's URL/creds (actually we use the full pooler URL) from env variables that are available. And then we'd have the GH action migrate the designated branch db upon branch creation. So a completely new set of URL/creds seems hard to deal with in this scenario, whereas some combination of base URL (already set in Vercel and GH) plus the branch name (already easily available in Vercel and GH) would be very doable. I see someone else who posted has a similar setup with some steps to make it theoretically work in the GH action.. but it seems to rely on CLI commands, and I'm not sure if that would work for getting the Vercel preview app to also point to the branch db. (Update: I also just realized Vercel support is in development which would solve the Vercel half.) |
Beta Was this translation helpful? Give feedback.
-
|
Can branching be used as a "dev" environment? |
Beta Was this translation helpful? Give feedback.
-
|
Yep. What's your use case? Can you tell us more? |
Beta Was this translation helpful? Give feedback.
-
|
According to the docs: "The database is only seeded once, when the preview branch is created. To rerun seeding, delete the preview branch and recreate it." But when I deleted and recreated the branch (
|
Beta Was this translation helpful? Give feedback.
-
|
EDIT: |
Beta Was this translation helpful? Give feedback.
-
|
That's probably a bug. We are using github webhooks and I think it only adds the PR metadata to the Preview Branch when a PR is opened (there's a webhook event for it) - but that makes it impossible for a new Preview Branch to then be associated with a PR later; like when you've made a new Preview Branch. I think we could fix that. Everything should work as normal though, it's only the UI here that's been impacted. |
Beta Was this translation helpful? Give feedback.
-
|
So far pretty good with supabase branching. Should I disable my existing github actions that migrate the production database when merging to the production branch? Is it expected that the migrations should be applied via this new Supabase branching integration? |
Beta Was this translation helpful? Give feedback.
-
|
It's worth stressing first that Branching is not suitable for production based apps right now. With that in mind, and if you want to continue; yes, you should be able to merge your feature git branch into your 'production' git branch and it will run the new migration files against the Production Supabase branch. There is currently no way to turn this off, so it's probably advised to disable your github actions. |
Beta Was this translation helpful? Give feedback.
-
|
@MildTomato are there known issues we should watch out for w.r.t. production? Or is it just that more time is needed to bash the bugs out that in theory could harm a prod instance? |
Beta Was this translation helpful? Give feedback.
-
|
@MildTomato does supabase branching support monorepos? Our |
Beta Was this translation helpful? Give feedback.
-
|
Changing a path which |
Beta Was this translation helpful? Give feedback.
-
|
Yeah would love to see this sort of mono-repo support! Very excited for you all to start using branching on internally too! |
Beta Was this translation helpful? Give feedback.
-
|
I would also require the mono-repo support for my project with a custom path to my migrations folder. Big fan of this branching feature and how it is progressing! |
Beta Was this translation helpful? Give feedback.
-
|
The code for it is already written and we're going to test everything in upcoming week :) |
Beta Was this translation helpful? Give feedback.
-
|
Monorepo support has been added and is already live if you have an access to branching 🎉 |
Beta Was this translation helpful? Give feedback.
-
|
any advice on how to handle issues like this that require setting up things in the database (eg. webhooks)? 
|
Beta Was this translation helpful? Give feedback.
-
|
Any updates here? This is preventing us from using branching |
Beta Was this translation helpful? Give feedback.
-
Beta Was this translation helpful? Give feedback.
-
|
For what it's worth, I have been running into so many issues with branching that the best solution for us was to manage the full lifecycle using a custom github action leveraging the management API. |
Beta Was this translation helpful? Give feedback.
-
|
@blechatellier that may be what we have to end up doing—would be a shame |
Beta Was this translation helpful? Give feedback.
-
|
To be fair, hard for them to come up with something that will work for everyone - it's been the best decision and took a couple of days to get it to work. |
Beta Was this translation helpful? Give feedback.
-
|
Hello! Loving the concept so far. After spending some time working around issues (seeding not being run and functions not being deployed), I realised you have a CLI for managing branches (which looks great btw!), and I think it could be highlighted more, as I was under the impression that only the "managed" workflows were supported (i.e only Github + Vercel), but atleast for me the feature set is complete enough that I could run this anywhere (and if i missed that that was possible, many others probably have too) |
Beta Was this translation helpful? Give feedback.
-
|
@hyldmo How did you manage to work around the issue of webhooks not being able to deployed in preview? I'm experiencing a similar issue, when migrating everything migrates succesfully except for the existing database webhooks: |
Beta Was this translation helpful? Give feedback.
-
|
+1 I am also running into this issue. I went to |
Beta Was this translation helpful? Give feedback.
-
If you are willing to manage your own GHA for setting up branches, all of the above are currently available through the terraform provider. We have an example repo here for those who want to tweak and use https://github.com/supabase/supabase-action-example/blob/main/.github/workflows/preview.yaml
For webhooks, we will look into enabling by default for all new projects and branches. |
Beta Was this translation helpful? Give feedback.
-
|
@sweatybridge any updates on Webhooks? |
Beta Was this translation helpful? Give feedback.
-
|
Yup, we've finally shipped a secure way to use secrets from webhooks supabase/cli#1190 (comment) |
Beta Was this translation helpful? Give feedback.
-
|
We are in dire need of selecting non root directories as the Supabase root. We are using a monorepo with TurboRepo. |
Beta Was this translation helpful? Give feedback.
-
|
It's coming. Bear with us some more time, we're getting there :) |
Beta Was this translation helpful? Give feedback.
-
|
That's great news! We just finished building our own internal psuedo-branching by creating a new project on every PR. It was quite difficult to implement correctly since there was a silent failure when connecting via IPv4. Do you have any rough estimation on timelines? |
Beta Was this translation helpful? Give feedback.
-
|
We're hoping to release this month, but I don't want to be held accountable for any specific date just yet. |
Beta Was this translation helpful? Give feedback.
-
|
Monorepo support has been added and is already live if you have an access to branching 🎉 |
Beta Was this translation helpful? Give feedback.
-
|
Currently, we're testing each branch by setting up a local supabase instance in GH Actions for every run. But that setup takes a significant amount of time compared to the actual test run. The flow that we have in mind is:
Are there any blockers that we should be aware of? Moreover, once this is set up and the Vercel app is used, I'm assuming the preview instances will use the new branched supabase. We also don't want any changes in the preview branches going back into main (which I assume will not happen as we are not using supabase migrations in the preview branches). Can you please confirm this? |
Beta Was this translation helpful? Give feedback.
-
Yes that is correct. The Supabase GitHub integration only look for files in Overall your approach sounds sane to me. We will need to make some changes for step 2 and 3 so that you can fetch a preview branch by its git branch name. Otherwise the current workaround is to parse the output from You might also need to handle retries in step 4 until the status of your branch database is ACTIVE_HEALTHY. I think step 7 is not required if you use the Supabase GitHub integration. Because we delete the branch on PR close. |
Beta Was this translation helpful? Give feedback.
-
|
Feedback/idea: It'd be nice (or even crucial) for the preview branch to copy most local and/or production branch settings. Examples:
|
Beta Was this translation helpful? Give feedback.
-
|
Any updates on this? Without the Auth provider settings this is unusable as we have to go over and manually enable providers for each preview branch to be usable. Because auth provider settings are not stored in the database, or are they? I mean, we couldn't set them in a migration. |
Beta Was this translation helpful? Give feedback.
-
|
Any updates ? This idea is really necessary |
Beta Was this translation helpful? Give feedback.
-
|
Agreed, this is making branches unusable for us, not being able to automatically configure SMTP for example it will not allow authentication to work properly on these branches. |
Beta Was this translation helpful? Give feedback.
-
|
+1 on the edge function secrets, as I ran into this today, but I would love the rest on this list as well - I'd also be happy to pay a little more if this increases costs. Branches are cheap, IMO. |
Beta Was this translation helpful? Give feedback.
-
|
Any updates? I just enabled the branching feature and found it was totally unusable as my customized stmp configurations were not synchronized. I tried to configre in the |
Beta Was this translation helpful? Give feedback.
-
|
For some reason I'm not seeing migrations running or seed data populated for a preview branch (feature or development). When I create a PR, it's spinning up the branch, just no tables or seed data. |
Beta Was this translation helpful? Give feedback.
-
Done, you can use it today.
There is basic seeding capability already with |
Beta Was this translation helpful? Give feedback.
-
+1(00) on this point! |
Beta Was this translation helpful? Give feedback.
-
|
They say the best way now is to create a separate Supabase project for the staging branch. |
Beta Was this translation helpful? Give feedback.
-
|
Q: We have few ideas how to go around it, but just to get a feel for it in public, wdyt is your preference in regards to long-lived branches?
|
Beta Was this translation helpful? Give feedback.
-
|
Adding a data point: we are using a separate project for Staging + Preview branches, while keeping our original project for Production only (same as @RobSchilderr's post above). This seems to work pretty well for persisting Staging so far, though it's not 1:1 with the Vercel style of branching. |
Beta Was this translation helpful? Give feedback.
-
|
We handle our migrations via Prisma migrate. Are there any plans to support the |
Beta Was this translation helpful? Give feedback.
-
|
Currently you'd have to move file to |
Beta Was this translation helpful? Give feedback.
-
|
Would just moving the folder work? The migration files/formats are directly compatible with supabase? |
Beta Was this translation helpful? Give feedback.
-
|
Yes, the migration file is always just a smallest sql snippet that will result in the desired schema shape at the end of execution. find . -type d -depth 1 | xargs -I{} cp {}/migration.sql migrations/{}.sql; |
Beta Was this translation helpful? Give feedback.
-
|
Got it. A cleaner way built in to supabase would be nice! Especially since there's a pretty large audience (I'm assuming) of supabase TS users using prisma. |
Beta Was this translation helpful? Give feedback.
-
|
Here's a bash script for those that want to copy Prisma migrations to a #!/bin/bash
# Define paths
PRISMA_MIGRATIONS_DIR="./prisma/migrations" # Replace with your Prisma migrations directory
SUPABASE_MIGRATIONS_DIR="./supabase/migrations" # Replace with your desired Supabase migrations directory
# Create the Supabase migrations directory if it doesn't exist
mkdir -p "$SUPABASE_MIGRATIONS_DIR"
# Function to convert Prisma migration files to Supabase format
convert_prisma_to_supabase() {
for prisma_migration_dir in "$PRISMA_MIGRATIONS_DIR"/*/; do
if [[ -d "$prisma_migration_dir" ]]; then
migration_file="$prisma_migration_dir/migration.sql"
if [[ -f "$migration_file" ]]; then
migration_name=$(basename "$prisma_migration_dir")
supabase_migration_file="$SUPABASE_MIGRATIONS_DIR/$migration_name.sql"
cp "$migration_file" "$supabase_migration_file"
echo "Converted $migration_file to $supabase_migration_file"
fi
fi
done
}
# Run the conversion function
convert_prisma_to_supabase
echo "Migration conversion completed."
|
Beta Was this translation helpful? Give feedback.
-
|
Hey Folks I want to Setup a Supa Base Branching for the Automation Testing so the scenario is Below So how i can do it can anyone help me out in this Thanks in Advance. |
Beta Was this translation helpful? Give feedback.
-
|
It seems like whatever the branching is doing to handle force pushes from Github doesn't play well with views that don't fall into very specific patterns. With a simple view like this: any force pushes at all result in this branch error: |
Beta Was this translation helpful? Give feedback.
-
|
A workaround for anyone else running into the same issue: CREATE OR REPLACE VIEW "public"."objects"
WITH
(security_invoker = ON) AS
WITH
"query_that_does_nothing" AS (
SELECT
"any_public_table_name"
FROM
"public"."any_public_table_name"
LIMIT
0
)
SELECT
*
FROM
"storage"."objects"; |
Beta Was this translation helpful? Give feedback.
-
|
Does anyone know why running |
Beta Was this translation helpful? Give feedback.
-
|
Hello! I am currently using branching on a project, where I have a persistent 'staging' branch. The issue I am facing is system emails aren't sent when, say, the user signs up. I set up the SMTP settings correctly (copied everything from the production branch). I also checked the auth logs and found no errors, everything seems like it works correctly but the email wasn't sent. Anyone have any ideas about why that is? Thanks! |
Beta Was this translation helpful? Give feedback.
-
|
Hey all, one issue I'm facing is that when I create a persistent branch manually, Supabase does not populate the environment variables for that branch. I've tried:
I have the github + vercel integration on and I would have expected that creating a branch in Supabase would keep the env variables in Vercel for that same branch in sync. Any ideas? EDIT: it seems the only way to trigger Supabase adding env vars in Vercel is if a PR is opened which triggers the Supabase job, however there are scenarios where we simply want a persistent branch that was created manually to still have their env variables synced. |
Beta Was this translation helpful? Give feedback.
-
|
I am having such a hard time setting up branching. I had a repo I was pushing to and the database wouldn't migrate up properly. I created a new Supabase project, pulled code down and linked new project, ran successful migration. Try to enable branching and the Github connection says it connected to the private repo in the settings but then I cant select ANY repos in Supabase. Refreshing the screen I temporarily see a flash screen error stating: You need additional permissions to view this organization's GitHub connections, Contact your organization owner or administrator for assistance. BUT IM THE OWNER! I've set and reset this connection 10 times and it still doesn't work despite a new project, new linking, new migrations, and checking all my ownership permissions. Support just told me to make sure my repo was selected which I already told them in the ticket it was. This many issues at the start of a project do not give a reassuring feeling about the DX and reliability of Supabase. Isn't branching our of testing now? Am I missing something glaringly obvious? Its been two days on this now... |
Beta Was this translation helpful? Give feedback.
-
|
I finally figured out the issue...apparently if you have 2FA enabled on your Github this all seems to fail silently. The UX for error recovery needs improvement because I only caught that error state by recording a video just to capture a still image of what the error was, and eventually trying the 2FA setting. Once I disabled that it connected immediately. Now the question is, does it have to stay disabled to persist the connection or can I rescue my org with 2FA? |
Beta Was this translation helpful? Give feedback.
-
|
Would be nice if we can set the default compute size used by the branch, my seed.sql takes a while to be inserted so I usually manually update the compute size for the preview db. |
Beta Was this translation helpful? Give feedback.
-
|
Using the combination of branching, database functions, and edge functions currently seems to be entirely unconsidered by Supabase. Every example of making database functions work with edge functions (such as with cron triggers) involves hardcoding of the project ref, even the example in the official documentation, which obviously doesn't work at all for branching. To make these features actually work together, the current project ref needs to be exposed to the database in an automatic way, such as through a |
Beta Was this translation helpful? Give feedback.
-
|
+1 to this, we're currently having to move away from database triggers invoking edge functions that send emails etc. as there doesn't seem to be a way to get this working with branching without hardcoding a value for a persistent branch. Either we need to be able to call edge functions with branch relative paths or we need some way of accessing the project_id (provided it's just not hidden in the docs somewhere). |
Beta Was this translation helpful? Give feedback.
-
|
You are both right about this DX problem with mixing dynamic branch url and anon key + webhooks. We will work on some potential solutions next week. |
Beta Was this translation helpful? Give feedback.
-
|
I'm stuck, edge functions can't deploy because the private repo can't be accessed. The npmrc file needs an env variable replaced (because I'm not going to commit that to source), but the name of the env is not being replaced. It ends up being |
Beta Was this translation helpful? Give feedback.
-
|
If you were running this locally or in CI, you would do I don't know if branching has a concept of build-time variables, I'll bring it up internally. |
Beta Was this translation helpful? Give feedback.
-
|
Hello, I've had a bit of a strange problem today, my permanent ‘dev’ branch has been deleted for no apparent reason. I hadn't pushed on my dev branch for 3/4 days, and I had the bad experience today of realising that the branch had disappeared. Given that it's my persistent branch that acts as my staging database, I'm a bit stuck because I no longer have a database for my staging environment. Could you give me some clues as to why this branch was deleted? |
Beta Was this translation helpful? Give feedback.
-
|
Hello, have you filed a support ticket for this? We treat all potential data loss seriously so we can definitely help you trace back why it happened. Are you using seed.sql by any chance? If so, recreating the staging branch should be quite straightforward. |
Beta Was this translation helpful? Give feedback.
-
|
We've encountered some weird behaviour with a persistent branch. Under URL Configuration, both the Site URL and the list of Redirect URLs are getting reset to the default value of I've noticed that this corresponds to the defaults in We can define these values in directly in the config file, or by using a Has anyone else run into this or developed a good flow to handle defining auth URLs? |
Beta Was this translation helpful? Give feedback.
-
|
If you use vercel integration, we will automatically set the For persistent branch, you can define them in This config will only apply to the specific staging branch. |
Beta Was this translation helpful? Give feedback.
-
|
Hey there, does anyone know how to configure the customized smtp for branch project? I wrote configuration into the |
Beta Was this translation helpful? Give feedback.
-
|
Sorry about that. We have just released a fix for the smtp problem. You can enable it in config.toml like this # Use a production-ready SMTP server
[auth.email.smtp]
enabled = true
host = "smtp.sendgrid.net"
port = 587
user = "apikey"
pass = "env(SENDGRID_API_KEY)"
admin_email = "admin@email.com"
sender_name = "Admin"The sensitive value |
Beta Was this translation helpful? Give feedback.
-
|
it works for me, thank you! |
Beta Was this translation helpful? Give feedback.
-
|
@sweatybridge a great fix, thanks! Minor issue, though: parser can't evaluate Also, a question: do I understand correctly, that only certain fields may reference environment variables and
|
Beta Was this translation helpful? Give feedback.
-
|
I have the preview branch working properly, but there are some settings that reset every time there's a commit to the branch. So far I've identified these that I have to keep resetting: Under Authentication -> URL Configuration, Site URL always resets to http://127.0.0.1:3000/ and Redirect URLs resets to https://127.0.0.1:3000/ I intentionally omitted site URL and redirect URL's from the config.toml because I want to set them once and not overwrite them. I'm guessing that these are defaults that are being reset every time there's a push to the branch, these defaults are certainly nowhere in the code base. Any help would be greatly appreciated! |
Beta Was this translation helpful? Give feedback.
-
|
i'm getting this one too. it's frustrating to setting up all settings (auth, db functions&trigger, storage, etc) on every push code. |
Beta Was this translation helpful? Give feedback.
-
|
If you want consistent configs for all development branches, I'd recommend adding them to [auth]
site_url = "https://my-frontend.com"
[api]
schemas = ["public", "graphql_public", "customer_support"]If you need to set sensitive fields like auth provider secrets, you can also encrypt them in config using dotenvx. |
Beta Was this translation helpful? Give feedback.
-
|
we are enjoying branching but i'd like to point out an edge case: if i merge my github branch before the supabase branch is finished building, the supabase branch hangs open with an error. this is notable because these branches do cost money. my team would prefer that the supabase branch just closed itself out no matter what if the github branch is closed. |
Beta Was this translation helpful? Give feedback.
-
|
Hello again. |
Beta Was this translation helpful? Give feedback.
-
|
Rate limit is not currently supported in config.toml but we can definitely add it next week. |
Beta Was this translation helpful? Give feedback.
-
|
That would be great! |
Beta Was this translation helpful? Give feedback.
-
|
Is there any kind of best practices to follow RE seeding object storage - especially for branching. I want to potentially be able to seed brand images and content on branches or is this something we should be handling ourselves? Use case is for having static content get served via the Supabase CDN for things like logos etc. |
Beta Was this translation helpful? Give feedback.
and others
-
This discussion is for tracking feedback and possible issues for Supabase Branching.
This will help us plan a roadmap for the feature going forward.
Vote for hosting provider integrations here and Git providers here.
Beta Was this translation helpful? Give feedback.
All reactions