[Utility][Persistence][Savepoints/Rollbacks] Refactor UtilityContext into UtilityUnitOfWork (Issue #563) #577

deblasis · 2023-03-12T23:04:30Z

Description

This PR creates the logical abstraction that would represent the "Unit Of Work" (inspired by Martin Fowler's work in his "Patterns of Enterprise Application Architecture")

Essentially what previously was known as UtilityContext becomes LeaderUtilityUnitOfWork or ReplicaUtilityUnitOfWork depending on the fact that the current node is the Leader or not (in Consensus).

It touches many files because it moves everything that is interacting with Persistence into the utility/unit_of_work folder.
This is to facilitate subsequent refactorings (see related PRs under #562) into more modular components.

This PR also sets the ground up for the completion of #508

Issue

Fixes #563

Type of change

Please mark the relevant option(s):

List of changes

Refactored UtilityContext to be a polymorphic UtilityOfWork
Introduced LeaderUtilityUnitOfWork and ReplicaUtilityUnitOfWork that expose the methods that are going be to used depending on the "role" of the node in the consensus process.
Moved utilityContext related code under utility/unit_of_work and renamed all references accordingly

Testing

make develop_test
LocalNet w/ all of the steps outlined in the README

Required Checklist

I have performed a self-review of my own code
I have commented my code, particularly in hard-to-understand areas
I have tested my changes using the available tooling
I have updated the corresponding CHANGELOG

If Applicable Checklist

I have updated the corresponding README(s); local and/or global
I have added tests that prove my fix is effective or that my feature works
I have added, or updated, mermaid.js diagrams in the corresponding README(s)
I have added, or updated, documentation and mermaid.js diagrams in shared/docs/* if I updated shared/*README(s)

…#596) ## Description This PR improves DevX by outputting the result of a changelog validation into the PR and requesting changes: ![image](https://user-images.githubusercontent.com/29378614/226392565-f20c77c2-19c7-42f7-8305-c11592f95f09.png) Once the check is successful, the automatically added review comments are dismissed The commits of this PR are also included in #577 ## Issue None because developed in < 4h ## Type of change Please mark the relevant option(s): - [x] New feature, functionality or library - [ ] Bug fix - [ ] Code health or cleanup - [ ] Major breaking change - [ ] Documentation - [ ] Other  ## List of changes - Updated changelog-verify.yaml to automatically review / dismiss changelog specific reviews ## Testing - [ ] `make develop_test` - [ ] [LocalNet](https://github.com/pokt-network/pocket/blob/main/docs/development/README.md) w/ all of the steps outlined in the `README` ## Required Checklist - [ ] I have performed a self-review of my own code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have added, or updated, [`godoc` format comments](https://go.dev/blog/godoc) on touched members (see: [tip.golang.org/doc/comment](https://tip.golang.org/doc/comment)) - [ ] I have tested my changes using the available tooling - [ ] I have updated the corresponding CHANGELOG ### If Applicable Checklist - [ ] I have updated the corresponding README(s); local and/or global - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] I have added, or updated, [mermaid.js](https://mermaid-js.github.io) diagrams in the corresponding README(s) - [ ] I have added, or updated, documentation and [mermaid.js](https://mermaid-js.github.io) diagrams in `shared/docs/*` if I updated `shared/*`README(s) --------- Signed-off-by: Alessandro De Blasis <alex@deblasis.net>

…or-utilitycontext-to-utilityunitofwork

github-actions

The changelog validation failed with the following output:

"Missing changelog in module: consensus/

Missing changelog in module: persistence/

Missing changelog in module: runtime/

Missing changelog in module: shared/

Missing changelog in module: utility/

Changelog verification failed. See error messages for more detail.
"

Please update the relevant CHANGELOG.md files and ensure they follow the correct format.

github-actions

The changelog validation failed with the following output:

Missing changelog in module: consensus/

Missing changelog in module: persistence/

Missing changelog in module: runtime/

Missing changelog in module: shared/

Missing changelog in module: utility/

Changelog verification failed. See error messages for more detail.

Please update the relevant CHANGELOG.md files and ensure they follow the correct format.

github-actions

The changelog validation failed with the following output:

Missing changelog in module: shared/

Changelog verification failed. See error messages for more detail.

Please update the relevant CHANGELOG.md files and ensure they follow the correct format.

The check succeeded, dismissing the review comment.

deblasis · 2023-03-21T01:04:01Z

Live test for #596

changelog_automation_ok.mp4

Validation transitions from ❌to✅ and the request changes is dismissed successfully

Olshansk · 2023-03-24T01:59:36Z

Live test for #596

Validation transitions from ❌to✅ and the request changes is dismissed successfully

…#596) ## Description This PR improves DevX by outputting the result of a changelog validation into the PR and requesting changes: ![image](https://user-images.githubusercontent.com/29378614/226392565-f20c77c2-19c7-42f7-8305-c11592f95f09.png) Once the check is successful, the automatically added review comments are dismissed The commits of this PR are also included in #577 ## Issue None because developed in < 4h ## Type of change Please mark the relevant option(s): - [x] New feature, functionality or library - [ ] Bug fix - [ ] Code health or cleanup - [ ] Major breaking change - [ ] Documentation - [ ] Other  ## List of changes - Updated changelog-verify.yaml to automatically review / dismiss changelog specific reviews ## Testing - [ ] `make develop_test` - [ ] [LocalNet](https://github.com/pokt-network/pocket/blob/main/docs/development/README.md) w/ all of the steps outlined in the `README` ## Required Checklist - [ ] I have performed a self-review of my own code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have added, or updated, [`godoc` format comments](https://go.dev/blog/godoc) on touched members (see: [tip.golang.org/doc/comment](https://tip.golang.org/doc/comment)) - [ ] I have tested my changes using the available tooling - [ ] I have updated the corresponding CHANGELOG ### If Applicable Checklist - [ ] I have updated the corresponding README(s); local and/or global - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] I have added, or updated, [mermaid.js](https://mermaid-js.github.io) diagrams in the corresponding README(s) - [ ] I have added, or updated, documentation and [mermaid.js](https://mermaid-js.github.io) diagrams in `shared/docs/*` if I updated `shared/*`README(s) --------- Signed-off-by: Alessandro De Blasis <alex@deblasis.net>

Olshansk

Really big fan of this change. W/e approach we end up going with, this will make it easier (e.g. storing the key-value pairs we need to roll back to/from in my design approach).

In either case, let's and see how next week goes. I'll try to think of 40 reasons not to go with the JSON approach ;)

Olshansk · 2023-03-24T03:29:48Z

utility/unit_of_work/module.go

+
+	height int64
+
+	persistenceReadContext modules.PersistenceReadContext


Can you please add a comment explaining why we have both with a potential example use case.

…pokt-network#596) ## Description This PR improves DevX by outputting the result of a changelog validation into the PR and requesting changes: ![image](https://user-images.githubusercontent.com/29378614/226392565-f20c77c2-19c7-42f7-8305-c11592f95f09.png) Once the check is successful, the automatically added review comments are dismissed The commits of this PR are also included in pokt-network#577 ## Issue None because developed in < 4h ## Type of change Please mark the relevant option(s): - [x] New feature, functionality or library - [ ] Bug fix - [ ] Code health or cleanup - [ ] Major breaking change - [ ] Documentation - [ ] Other  ## List of changes - Updated changelog-verify.yaml to automatically review / dismiss changelog specific reviews ## Testing - [ ] `make develop_test` - [ ] [LocalNet](https://github.com/pokt-network/pocket/blob/main/docs/development/README.md) w/ all of the steps outlined in the `README` ## Required Checklist - [ ] I have performed a self-review of my own code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have added, or updated, [`godoc` format comments](https://go.dev/blog/godoc) on touched members (see: [tip.golang.org/doc/comment](https://tip.golang.org/doc/comment)) - [ ] I have tested my changes using the available tooling - [ ] I have updated the corresponding CHANGELOG ### If Applicable Checklist - [ ] I have updated the corresponding README(s); local and/or global - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] I have added, or updated, [mermaid.js](https://mermaid-js.github.io) diagrams in the corresponding README(s) - [ ] I have added, or updated, documentation and [mermaid.js](https://mermaid-js.github.io) diagrams in `shared/docs/*` if I updated `shared/*`README(s) --------- Signed-off-by: Alessandro De Blasis <alex@deblasis.net>

…or-utilitycontext-to-utilityunitofwork Signed-off-by: Alessandro De Blasis <alex@deblasis.net>

github-actions

The changelog validation failed with the following output:

Latest date in persistence/docs/CHANGELOG.md is incorrect.
Latest: 2023-03-21, Current: 2023-03-26

Latest date in runtime/docs/CHANGELOG.md is incorrect.
Latest: 2023-03-21, Current: 2023-03-26

Latest date in utility/doc/CHANGELOG.md is incorrect.
Latest: 2023-03-21, Current: 2023-03-26

Changelog verification failed. See error messages for more detail.

Please update the relevant CHANGELOG.md files and ensure they follow the correct format.

The check succeeded, dismissing the review comment.

…p-modules * pokt/main: [Utility][Persistence][Savepoints/Rollbacks] Refactor UtilityContext into UtilityUnitOfWork (Issue #563) (#577) [Consensus] Add Quorum Certificate to the block (#593) docs(wiki): ValueError: Duplicate wiki path: devlog/template (#620) Remove unused import [Infra][P2P] Bug: "address not found in peerstore" error in localnet_debug_client when scaling up Localnet - Issue #604 (#611) [CLI][Localnet][Bug] - [localnet_client_debug] Error 137 - Issue #607 (#608) [Documentation] Fix links to RPC module docs in README (#603)

deblasis added 30 commits March 9, 2023 21:11

chore(utility): normalize imports

1762efb

chore(utility): setBus (from embedded base_module)

305e96a

refactor(shared): utility module interface changes

cc9f7db

chore(shared): utilitycontext marked for deprecation

c859dbb

refactor(shared): utility unitofwork, catering for #508 as well

771b95e

feat(shared): utility unitofwork for leader and replica for #508

a1d7aab

refactor(shared): mv ReleaseUtilityContext->ReleaseUtilityUnitOfWork

06a5694

chore(shared): TODO in consensus (ReleaseUtilityUnitOfWork)

f938274

refactor(shared): refactored ConsensusDebugModule for uow

2b2469b

feat(utility): base utilityUnitOfWork

324ae5e

feat(utility): leader utilityUnitOfWork

69a8fe1

feat(utility): replicaUtilityUnitOfWork

c151bbf

feat(utility): unitOfWork constructor

d28d2e7

refactor(consensus): utilityUnitOfWork

a8d7792

refactor(consensus): utilityUnitOfWork in pacemaker

2b93b37

chore(consensus): TODO in tests

54ca863

refactor(consensus): refactored block application to use uow

0d6de1b

chore(consensus): removed already handled TODO

1092e6b

chore(consensus): removed already handled TODO

4692611

feat(consensus): techdebt (maxmempoolbytes from consensus config)

676e710

feat(consensus): using refactored uow for leader logic

68cb452

fix(consensus): consolidated maxTxBytes

46f9181

refactor(consensus): pacemaker_test uow

4f86ceb

refactor(shared): updated type for maxTxBytes

b589396

refactor(consensus): tests are ✅

097787a

refactor(consensus): simplified create / apply

5e2dca4

fix(persistence): techdebt: log statement

7ab68c5

refactor(utility): updated create / apply

15365fe

refactor(utility): deprecating utilityContext

bdf0a5c

refactor(utility): WIP separation of concerns module/uow

43294c3

Merge remote-tracking branch 'upstream/main' into issue/563-A1-refact…

7e4e368

…or-utilitycontext-to-utilityunitofwork

github-actions bot previously requested changes Mar 21, 2023

View reviewed changes

chore(tooling): fromJSON trying to remove

8e55768

github-actions bot previously requested changes Mar 21, 2023

View reviewed changes

deblasis added 4 commits March 21, 2023 00:51

docs(consensus): CHANGELOG

ca33ccd

docs(persistence): CHANGELOG

21ba80c

docs(runtime): CHANGELOG

9aa51a9

docs(utility): CHANGELOG

fd8d5ad

github-actions bot previously requested changes Mar 21, 2023

View reviewed changes

docs(shared): CHANGELOG

e37fd47

Olshansk approved these changes Mar 24, 2023

View reviewed changes

deblasis added 2 commits March 26, 2023 21:32

Merge remote-tracking branch 'upstream/main' into issue/563-A1-refact…

3fb4210

…or-utilitycontext-to-utilityunitofwork Signed-off-by: Alessandro De Blasis <alex@deblasis.net>

docs(utility): persistence context comment

a7444a1

github-actions bot previously requested changes Mar 26, 2023

View reviewed changes

deblasis added 3 commits March 26, 2023 21:35

docs(persistence): CHANGELOG

27ab264

docs(runtime): CHANGELOG

36c4268

docs(utility): CHANGELOG

0b1c7cd

deblasis merged commit 19bf4d3 into main Mar 26, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Utility][Persistence][Savepoints/Rollbacks] Refactor UtilityContext into UtilityUnitOfWork (Issue #563) #577

[Utility][Persistence][Savepoints/Rollbacks] Refactor UtilityContext into UtilityUnitOfWork (Issue #563) #577

deblasis commented Mar 12, 2023 •

edited

Loading

github-actions bot left a comment

github-actions bot left a comment

github-actions bot left a comment

deblasis commented Mar 21, 2023 •

edited

Loading

Olshansk commented Mar 24, 2023

Olshansk left a comment

Olshansk Mar 24, 2023

github-actions bot left a comment


		height int64

		persistenceReadContext modules.PersistenceReadContext

[Utility][Persistence][Savepoints/Rollbacks] Refactor UtilityContext into UtilityUnitOfWork (Issue #563) #577

[Utility][Persistence][Savepoints/Rollbacks] Refactor UtilityContext into UtilityUnitOfWork (Issue #563) #577

Conversation

deblasis commented Mar 12, 2023 • edited Loading

Description

Issue

Type of change

List of changes

Testing

Required Checklist

If Applicable Checklist

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

deblasis commented Mar 21, 2023 • edited Loading

Olshansk commented Mar 24, 2023

Olshansk left a comment

Choose a reason for hiding this comment

Olshansk Mar 24, 2023

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

deblasis commented Mar 12, 2023 •

edited

Loading

deblasis commented Mar 21, 2023 •

edited

Loading