Fix the Rich header analysis algorithm #973
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR aims to fix #960 and to fix #965
The Rich header end is marked by the "Rich" marker somewhere between DOS header/stub and PE header. After the "Rich" follows the XOR key, when XORing the Rich header data, the start can be identified by "DanS" in ASCII and 3 null bytes for padding into a 16-byte paragraph.
Current analysis decrypts the whole thing as it assumes there is no other data than Rich header there and then checks if there is "DanS" at the start. This doesn't behave correctly if there is any other "junk" data before the Rich header as it will decrypt it as well as is the case in #960 and just reports warning about wrong key because it didn't find the "DanS" at the start. I've changed the algorithm so it starts decrypting from the end ("Rich") to the top until it finds the "DanS" that marks the start of the Rich header, this seems to work well for both samples
But this means that the Rich header offset can vary based on the junk data, but this isn't really supported by the current RetDec code structure, so I'll need to edit that so we return the correct offset.
Checklist for PR: