Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Neo Store Exception] Add exception to Use after Commit operation to Snapshots and Cache. #3405

Closed
wants to merge 4 commits into from

Conversation

Jim8y
Copy link
Contributor

@Jim8y Jim8y commented Jul 6, 2024

Description

Store snapshot and datacache are not supposed to be used after they are commited since there will be data inconsistancy and very hard to figure out the exact value. Thus we need to ban any attacmpt of using the snapshot or data cache after they are committed.

Fixes # (issue)

Type of change

  • Optimization (the change is only an optimization)
  • Style (the change is only a code style for better maintenance or standard purpose)
  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

How Has This Been Tested?

Test Configuration:

Checklist:

  • My code follows the style guidelines of this project
  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • Any dependent changes have been merged and published in downstream modules

@Jim8y Jim8y requested a review from cschuchardt88 July 6, 2024 16:00
@Jim8y Jim8y requested a review from shargon July 6, 2024 16:00
Copy link
Member

@cschuchardt88 cschuchardt88 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this is a good idea. If snapshot isn't disposed You should still be able to use it. I think it does in ApplicationLogs

@Jim8y
Copy link
Contributor Author

Jim8y commented Jul 7, 2024

I don't think this is a good idea. If snapshot isn't disposed You should still be able to use it. I think it does in ApplicationLogs

you can not use it anymore, cause the value was there before the commit, but will be gone after the commit. you will get unexpected result. if applicationengine or something is using a committed cache, we should fix it. and if you use your experience with snapshot on neo, you should rethink it, cause its different here.

@Jim8y
Copy link
Contributor Author

Jim8y commented Jul 7, 2024

the problem is, you had newest state, but after you commit it, you lose the latest state but fallback to the point when the snapshot is created, making the snapshot unable to process with correct data anymore. thus you should never use a clmmitted snapahot or datacache.

@cschuchardt88
Copy link
Member

cschuchardt88 commented Jul 7, 2024

the problem is, you had newest state, but after you commit it, you lose the latest state but fallback to the point when the snapshot is created, making the snapshot unable to process with correct data anymore. thus you should never use a clmmitted snapahot or datacache.

Thats not true. Because the Read methods get results from the db. So after you commit data, data isn't lost. If you were to read data snapshot, it would come from the db not the snapshot anyways. So if you put new data in and commit it, snapshot will be committed to db. Data isn't visible to snapshot until you commit.

Hints my comment for: #3399 (comment)

@@ -146,6 +148,8 @@ public virtual void Commit()
dictionary.Remove(key);
}
changeSet.Clear();

isCommitted = true;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this flag must be changed to false when write or delete, isn't it?

@Jim8y
Copy link
Contributor Author

Jim8y commented Jul 8, 2024

it would come from the db not the snapshot anyways

Please check my pr on memorystore UTs.

Copy link
Member

@AnnaShaleva AnnaShaleva left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please, see the #3404 (comment). How this do-not-use-after-commit rule is going to work with blocks processing? I'm probably missing something.

@cschuchardt88
Copy link
Member

Why can't you on commit. Refresh the snapshot? Would be much simpler.

@Jim8y
Copy link
Contributor Author

Jim8y commented Jul 11, 2024

the problem is, you had newest state, but after you commit it, you lose the latest state but fallback to the point when the snapshot is created, making the snapshot unable to process with correct data anymore. thus you should never use a clmmitted snapahot or datacache.

Thats not true. Because the Read methods get results from the db. So after you commit data, data isn't lost. If you were to read data snapshot, it would come from the db not the snapshot anyways. So if you put new data in and commit it, snapshot will be committed to db. Data isn't visible to snapshot until you commit.

Hints my comment for: #3399 (comment)

read my test. You can not assume where the data comes from, when you read the data, its from the snapshot of the db, not the db latest state.

@Jim8y Jim8y closed this Jul 11, 2024
@Jim8y Jim8y deleted the snapshot-exception branch July 11, 2024 15:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants