Skip to content
This repository has been archived by the owner on Jan 13, 2025. It is now read-only.

refactor untar_snapshot_in to push parallelism deeper for further refactoring #18310

Merged
merged 1 commit into from
Jun 29, 2021
Merged

refactor untar_snapshot_in to push parallelism deeper for further refactoring #18310

merged 1 commit into from
Jun 29, 2021

Conversation

jeffwashington
Copy link
Contributor

@jeffwashington jeffwashington commented Jun 29, 2021 •
edited
Loading

Problem

Startup takes a long time. Untar of snapshot takes a long time. There is time to be saved by improving parallelism and shared data across parallel untar paths. Related to #18205

Summary of Changes

Refactor to push paralleism deeper.
Fixes #

@jeffwashington jeffwashington changed the title refactor untar_snapshot_in to push parallelism deeper for further ref… refactor untar_snapshot_in to push parallelism deeper for further refactoring Jun 29, 2021
@jeffwashington
Copy link
Contributor Author

This change doesn't save time by itself. It reworks where we do parallelism so that Archive can be created with a shared, buffered reader from unzip or whatever format the snapshot is in.

@jeffwashington jeffwashington marked this pull request as ready for review June 29, 2021 23:13
brooksprumo
brooksprumo approved these changes Jun 29, 2021
View reviewed changes
Copy link
Contributor

@brooksprumo brooksprumo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me

@jeffwashington jeffwashington merged commit ce53b84 into solana-labs:master Jun 29, 2021
mergify bot pushed a commit that referenced this pull request Jun 29, 2021
@jeffwashington
refactor untar_snapshot_in to push parallelism deeper for further ref…
2cfb7a1 
…actoring (#18310)

(cherry picked from commit ce53b84)
@mergify mergify bot mentioned this pull request Jun 29, 2021
Merged
mergify bot added a commit that referenced this pull request Jun 30, 2021
@mergify @jeffwashington
refactor untar_snapshot_in to push parallelism deeper for further ref…
a10a082 
…actoring (#18310) (#18312)

(cherry picked from commit ce53b84)

Co-authored-by: Jeff Washington (jwash) <75863576+jeffwashington@users.noreply.github.com>
jeffwashington added a commit to jeffwashington/solana that referenced this pull request Jun 30, 2021
@jeffwashington
refactor untar_snapshot_in to push parallelism deeper for further ref…
06cd477 
…actoring (solana-labs#18310)
@brooksprumo brooksprumo mentioned this pull request Aug 23, 2021
Closed
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@brooksprumo brooksprumo brooksprumo approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@jeffwashington @brooksprumo