Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CHIA-1427]: Limit full file creation when processing subscription generations #18612

Merged
merged 5 commits into from
Sep 30, 2024

Conversation

emlowe
Copy link
Contributor

@emlowe emlowe commented Sep 19, 2024

When syncing a subscription, DL downloads the delta files for each generation as it walks all the singleton generations. Currently each time it will also write out a full file - however, that full file isn't used and will be eventually deleted as the default is to only keep on full file.

This PR is an attempt to limit creating the full files except when we are within the range where we should generate the full files.

It will also create a full file if a delta file fails to download entirely - this makes sure that if in the middle of processing the deltas the code happens to be unable to move forward, a full file will get generated at that point.

@emlowe emlowe changed the title EL.limit-full-files [CHIA-1427]: Limit full file creation when processing subscription generations Sep 19, 2024
@emlowe emlowe added the Changed Required label for PR that categorizes merge commit message as "Changed" for changelog label Sep 19, 2024
Copy link

coveralls-official bot commented Sep 20, 2024

Pull Request Test Coverage Report for Build 10966220826

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

Details

  • 23 of 23 (100.0%) changed or added relevant lines in 3 files are covered.
  • 25 unchanged lines in 8 files lost coverage.
  • Overall coverage decreased (-0.003%) to 90.97%

Files with Coverage Reduction New Missed Lines %
chia/full_node/full_node_api.py 1 82.36%
chia/wallet/wallet_node.py 1 88.37%
chia/daemon/client.py 1 73.94%
chia/_tests/core/util/test_file_keyring_synchronization.py 1 97.14%
chia/full_node/full_node.py 4 86.7%
chia/server/server.py 4 82.21%
chia/server/node_discovery.py 6 79.26%
chia/server/address_manager.py 7 90.48%
Totals Coverage Status
Change from base Build 10949495080: -0.003%
Covered Lines: 102032
Relevant Lines: 112126

💛 - Coveralls

@emlowe emlowe marked this pull request as ready for review September 23, 2024 13:51
@emlowe emlowe requested a review from a team as a code owner September 23, 2024 13:51
@emlowe emlowe requested review from fchirica and altendky September 23, 2024 20:53
Copy link
Contributor

@altendky altendky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like a good change to me. I put in a few code form comments, but I'm not strongly attached to them.

chia/data_layer/download_data.py Show resolved Hide resolved
chia/data_layer/download_data.py Show resolved Hide resolved
chia/data_layer/data_layer.py Show resolved Hide resolved
chia/data_layer/data_layer.py Show resolved Hide resolved
Copy link
Contributor

@altendky altendky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

comments are being considered separately for potential refactoring later

@pmaslana pmaslana merged commit 7759619 into main Sep 30, 2024
374 checks passed
@pmaslana pmaslana deleted the EL.limit-full-files branch September 30, 2024 16:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Changed Required label for PR that categorizes merge commit message as "Changed" for changelog
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants