Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support sqlite in production. #3366

Open
shanna opened this issue Sep 12, 2022 · 2 comments
Open

Support sqlite in production. #3366

shanna opened this issue Sep 12, 2022 · 2 comments
Labels
enhancement New feature or request

Comments

@shanna
Copy link

shanna commented Sep 12, 2022

Is your feature request related to a problem? Please describe.
I'm a fan of self hosting early stage and startup projects on micro budgets (think 5USD to 100USD a month at most). I've been using Temporal on work projects but more recently I've been playing with Temporalite using sqlite replicated with https://litestream.io to https://min.io. It has worked out great in development but I'd like to take Temporal (or Temporalite) to production without clusting, postgres, elasticsearch etc which all blow out small budgets in a heartbeat.

Describe the solution you'd like
Officially support sqlite in production.

I think this would involve (at least):

  • Add sqlite as an option to the sql tool.
  • Add sqlite support to the auto-setup shell scripts.
  • Add sqlite visibility support? (perhaps requiring the fulltext search extension?).

Describe alternatives you've considered
I have been tempted to use temporalite in production despite all the risks and downsides.

Additional context
I don't fully utilise Temporal. Generally my workflows are for service to service coordination and short lived so I'm OK with a single nodes and short stretches of downtime which I understand probably isn't typical for Temporal users.

I asked about future plans for sqlite support here https://community.temporal.io/t/future-sqlite3-support/5915?u=shane

@shanna shanna added the enhancement New feature or request label Sep 12, 2022
@jlegrone
Copy link
Contributor

jlegrone commented Oct 11, 2022

FYI, the vision is for Temporalite to be used in production contexts. I might make a milestone to make tracking easier, but this depends on temporalio/temporalite-archived#23, temporalio/temporalite-archived#110, and signoff from more Temporal server maintainers. I'd love to get published benchmarks, which are in the works.

Of course you are free to use the regular Temporal server binary with the sqlite driver as well! Any improvements to performance and stability will also be available in Temporalite.


Also, 👍 for litestream. I haven't gotten around to playing with it yet, but glad to know it's working for you with Temporalite!

@sync-by-unito sync-by-unito bot closed this as completed Mar 3, 2023
@yiminc yiminc reopened this Mar 3, 2023
@evgenii-moriakhin
Copy link

Hello

There is no progress or discussion on this issue? I see the problem has been reopened

I'm interested because I too want to use temporal with sqlite on a virtualization product where thousands of hosts may have agents that don't have broad access to the enterprise network, but I'd like to be able to execute temporal workflows on them without having to deploy a "serious" DBMS like PostgreSQL on each host, but take advantage of temporal.

I read this topic https://community.temporal.io/t/future-sqlite3-support/5915/6?u=shane
and find the use cases completely similar to mine

and I had the question after reading this topic, if these points

One reason is it starts all server roles in the same container and in a single process (similar to auto-setup) which makes it hard to scale and tune among other things

are not critical for me, can I use temporal with sqlite in production?

@shanna , did you finally get it to work?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

4 participants