refactor: prune everything

[alexanderbez]

Description

Closes: #10352

• Tweak the everything pruning setting to keep the previous state in addition to the current state.
• Disable (return an error) if pruning=everything AND state sync is enabled

---

Author Checklist

All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.

I have...

• included the correct type prefix in the PR title
• added ! to the type prefix if API or client breaking change
• targeted the correct branch (see PR Targeting)
• provided a link to the relevant issue or specification
• followed the guidelines for building modules
• included the necessary unit and integration tests
• added a changelog entry to CHANGELOG.md
• included comments for documenting Go code
• updated the relevant documentation or specification
• reviewed "Files changed" and left comments if necessary
• confirmed all CI checks have passed

Reviewers Checklist

All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.

I have...

• confirmed the correct type prefix in the PR title
• confirmed ! in the type prefix if API or client breaking change
• confirmed all author checklist items have been addressed
• reviewed state machine logic
• reviewed API design and naming
• reviewed documentation is accurate
• reviewed tests and test coverage
• manually tested (if applicable)

[tac0turtle]

I don't think this solves the issue. If I understand it correctly you are assuming it takes less time to snapshot than it does to proceed two blocks. I think in some cases this would work but for chains with large amounts of current state you would still get data base corruption.

If this is the case I would suggest using snapshot-interval the same way we had keep-every, except after the snapshot is saved we prune that height.

[alexanderbez]

@marbar3778 this PR has nothing to do with snapshotting. It has to do with fixing the off by on error, i.e. the failed to load at height X error you see when using pruning=everything.

[tac0turtle]

Yes I know. But only keeping the last two blocks will only fix some on the issues is what I'm saying. If the snapshot takes longer than two blocks the app will try to prune a version that has an active reader on it.

[alexanderbez]

Yes I know. But only keeping the last two blocks will only fix some on the issues is what I'm saying. If the snapshot takes longer than two blocks the app will try to prune a version that has an active reader on it.

I see. I mean it would be ideal to not have to conflate the two things. The issue you state is currently also an issue. What do you propose exactly?

[tac0turtle]

LGTM

[alexanderbez]

@marbar3778 @AmauryM wdyt about backporting this? There technically aren't any API breaking changes. The only thing that would happen now is app.toml would fail validation if a user enables state sync snapshotting and has pruning=everything, which is a no no anyway (hence the change).

[ValarDragon]

Nice change! Im in favor of backporting this