Skip to content
This repository has been archived by the owner on Apr 3, 2024. It is now read-only.

SQLite settings/WAL support #26

Closed
plaisted opened this issue Dec 15, 2021 · 4 comments
Closed

SQLite settings/WAL support #26

plaisted opened this issue Dec 15, 2021 · 4 comments
Labels
enhancement New feature or request

Comments

@plaisted
Copy link
Contributor

Great project! Really makes temporal much more accessible and opens up some cool edge use cases (eg. raspberry pi, etc). While playing around for a low resource use case I noticed performance was lower than expected.

Is your feature request related to a problem? Please describe.
Using the Maru benchmarking tool I was getting ~1 workflows/s (3 activities) with persistence turned on. When using sqlite in memory, this increased to ~20 workflows/s. Sqlite has additional pragma settings that could improve the persisted perf.

Describe the solution you'd like
Add ability to CLI to adjusting sqlite connection attributes.

Tinkering around and hardcoding the following significantly improved performance to ~10 workflows/s including disk persistence:

sqliteConfig.ConnectAttributes["_journal"] = "WAL"
sqliteConfig.ConnectAttributes["_sync"] = "NORMAL"

It would be beneficial to have the ability to adjust the sqlite connection attributes. This could be done using a flag (eq --wal) to enable WAL support, or just allow arbitrary sqlite connect attributes (eg. --sqlite-attrib _journal=WAL --sqlite-attrib _sync=NORMAL) to be set.

@plaisted plaisted added the enhancement New feature or request label Dec 15, 2021
@jlegrone
Copy link
Collaborator

Thanks! We'd be happy to support this.

Right now Temporalite uses the github.com/mattn/go-sqlite3 driver but this could change in the future so I'd like to keep the actual connection attributes which can be driver specific abstracted from the CLI and Go library interfaces. How about something like --sqlite-pragma journal=WAL? We could add an additional flag to support DSN query parameters like mode or cache later if we need to.

I'd also like to ship with a more performant default config as long as it doesn't compromise correctness or cross-platform support. We can do that in a separate changeset though.

@plaisted is this something you're interested in contributing? A PR would be welcome. 😄

@plaisted
Copy link
Contributor Author

Haven't touched Go in a few years but I'll give it a shot this weekend if I get some time.

@plaisted
Copy link
Contributor Author

Opened #30. I'm not familiar with golang but core functionality should be there.

@jlegrone
Copy link
Collaborator

Closed by #30, thanks @plaisted!

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants