Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

overhaul projectionist #503

Merged
merged 13 commits into from
Feb 20, 2024
Merged

overhaul projectionist #503

merged 13 commits into from
Feb 20, 2024

Conversation

DavidBadura
Copy link
Member

@DavidBadura DavidBadura commented Feb 18, 2024

I rewrote the Projectionist core so that it no longer needs an external lock. The "FOR UPDATE" statement is used to lock the rows when reading as long as the transaction exists. If the row is locked, no other process can read it. This means there is no risk of events being processed twice. In addition, only the rows that were affected by the “fetch” are locked. This means that the projections and thus Projectionist can also be run in groups.

Features:

  • Implement row lock
  • Allow grouping of projections
  • Add new run mode named "once"
  • Cleanup code
  • Improve performance

resolve: #321

@DavidBadura DavidBadura added enhancement New feature or request BC-Break labels Feb 18, 2024
@DavidBadura DavidBadura added this to the 3.0.0 milestone Feb 18, 2024
Copy link

github-actions bot commented Feb 18, 2024

Hello 👋

here is the most recent benchmark result:

SimpleSetupBench
================

+----------------------------------------+--------------------+--------------------+-----------------+------------+
|                                        | time (kde mode)                         | memory                       |
+----------------------------------------+--------------------+--------------------+-----------------+------------+
| subject                                | Tag: <current>     | Tag: base          | Tag: <current>  | Tag: base  |
+----------------------------------------+--------------------+--------------------+-----------------+------------+
| benchLoad10000Events ()                | 158.710ms (±0.00%) | 157.623ms (±0.00%) | 41.097mb        | 41.096mb   |
| benchSave1Event ()                     | 1.191ms (±0.00%)   | 1.174ms (±0.00%)   | 41.097mb        | 41.097mb   |
| benchSave10000Events ()                | 333.787ms (±0.00%) | 333.175ms (±0.00%) | 41.364mb        | 41.364mb   |
| benchSave10000Aggregates ()            | 12.816s (±0.00%)   | 15.830s (±0.00%)   | 41.097mb        | 41.097mb   |
| benchSave10000AggregatesTransaction () | 931.593ms (±0.00%) | 921.199ms (±0.00%) | 41.097mb        | 41.097mb   |
+----------------------------------------+--------------------+--------------------+-----------------+------------+

SnapshotsBench
==============

+----------------------------------------+--------------------+--------------------+-----------------+------------+
|                                        | time (kde mode)                         | memory                       |
+----------------------------------------+--------------------+--------------------+-----------------+------------+
| subject                                | Tag: <current>     | Tag: base          | Tag: <current>  | Tag: base  |
+----------------------------------------+--------------------+--------------------+-----------------+------------+
| benchLoad10000EventsMissingSnapshot () | 154.069ms (±0.00%) | 154.657ms (±0.00%) | 41.114mb        | 41.114mb   |
| benchLoad10000Events ()                | 211.550μs (±0.00%) | 200.700μs (±0.00%) | 41.113mb        | 41.113mb   |
+----------------------------------------+--------------------+--------------------+-----------------+------------+

SplitStreamBench
================

+-------------------------+--------------------+--------------------+-----------------+------------+
|                         | time (kde mode)                         | memory                       |
+-------------------------+--------------------+--------------------+-----------------+------------+
| subject                 | Tag: <current>     | Tag: base          | Tag: <current>  | Tag: base  |
+-------------------------+--------------------+--------------------+-----------------+------------+
| benchLoad10000Events () | 2.689ms (±0.00%)   | 2.793ms (±0.00%)   | 44.544mb        | 44.544mb   |
| benchSave10000Events () | 352.185ms (±0.00%) | 354.091ms (±0.00%) | 44.842mb        | 44.842mb   |
+-------------------------+--------------------+--------------------+-----------------+------------+

This comment gets update everytime a new commit comes in!

@DavidBadura DavidBadura force-pushed the overhaul-projectionist branch from 87b9976 to e96cd7a Compare February 19, 2024 10:10
@DavidBadura DavidBadura mentioned this pull request Feb 19, 2024
19 tasks
@DavidBadura DavidBadura marked this pull request as ready for review February 19, 2024 17:55
@DavidBadura DavidBadura merged commit b4ab768 into 3.0.x Feb 20, 2024
32 of 33 checks passed
@DavidBadura DavidBadura deleted the overhaul-projectionist branch February 20, 2024 11:56
@DavidBadura DavidBadura linked an issue Mar 7, 2024 that may be closed by this pull request
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
BC-Break enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

projector tag filter option for all projectionist command
2 participants