Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Shadow string timestamps with epoch seconds in database #58

Open
matthiasemde opened this issue Mar 16, 2024 · 0 comments
Open

Shadow string timestamps with epoch seconds in database #58

matthiasemde opened this issue Mar 16, 2024 · 0 comments
Labels
performance This should be fixed to improve performance

Comments

@matthiasemde
Copy link
Owner

matthiasemde commented Mar 16, 2024

Since SQLite does not support a native date format, we use ISO encoded strings to save timestamps in the following format:

2024-03-17T14:00:01.053+01:00[Europe/Berlin]

This has multiple drawbacks:

  1. since the datatime() function of SQLite only understands 2024-03-17T14:00:01.053+01:00 but not the [TZone] suffix, we have to manually strip this suffix away on every query before feeding the ISO-string into datetime().

  2. Database queries that compare timestamps first need to convert these strings back to datetime using datetime which costs a lot of time. If we shadow the string encoded timestamps with epoch seconds and use these internally for comparisons, we can save a lot of computation.

@matthiasemde matthiasemde added the performance This should be fixed to improve performance label Mar 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
performance This should be fixed to improve performance
Projects
None yet
Development

No branches or pull requests

1 participant