Object stream fix #1
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Commit 8cc27b6 broke object stream parsing by resetting the content cursor PDFParser.charCounter to zero on every invocation. This broke object stream parsing. Reproducer: $ echo -e "create pdf\ncreate object_stream\nall\nsave /tmp/foo.pdf" | \ peepdf -i Without fix: $ peepdf -j /tmp/foo.pdf Error: An error has occurred while parsing an indirect object!! With this change: JSON output as expected (same for other outputs). $ peepdf -j /tmp/foo.pdf { "peepdf_analysis": { [...] "version": "0.3" } } } Signed-off-by: Michael Weiser <michael.weiser@gmx.de>
For unclear reasons, PDFObjectStream.update() delays decoding of the modified raw stream until all references can be resolved. It does however then go on to always try to extract objects from the still empty decoded stream. This produces an error from peepdf cli: $ peepdf image.php Error: An error has occurred while parsing an indirect object!! The error from PDFObjectStream.update() is "Missing offsets in object stream" because self.decodedStream is still empty at that point, making offsetsSection and eventually the numbers list empty, causing the abort. This is triggered by /Length being a reference and setting updateNeeded to True. Sample: https://www.infotek.co.jp/pdflib/demo/sample/image.php. Relevant PDF structure: 32 0 obj <</Length 43 0 R/Filter/FlateDecode/Type/ObjStm/N 7/First 47>> stream [...] endstream endobj 43 0 obj 461 endobj (Length in dict of object 32 R-eferences object 43 which contains 461 what presumably is the length of the stream - which does not seem to be used or checked for consistency by peepdf atm, btw.) This resolves the first half of jesparza#70 in that force mode is no longer necessary to parse such files at all. Signed-off-by: Michael Weiser <michael.weiser@gmx.de>
With the previous change deferring reading of objects from the decoded stream until references can be resolved, it now runs into jesparza#70. This change provides a different approach in fixing it to hatching#6 by syncing it with the other locations where the identical code is in use: 1. Force the numbers extracted by re.findall to int() as before, avoiding the TypeError exception: Traceback (most recent call last): File "peepdf-venv2/lib64/python2.7/site-packages/peepdf/main.py", line 409, in main ret, pdf = pdfParser.parse(fileName, options.isForceMode, options.isLooseMode, options.isManualAnalysis) File "peepdf-venv2/lib64/python2.7/site-packages/peepdf/PDFCore.py", line 7117, in parse ret = body.updateObjects() File "peepdf-venv2/lib64/python2.7/site-packages/peepdf/PDFCore.py", line 4291, in updateObjects object.resolveReferences() File "peepdf-venv2/lib64/python2.7/site-packages/peepdf/PDFCore.py", line 3256, in resolveReferences ret = PDFParser.readObject(objectsSection[offset:]) TypeError: slice indices must be integers or None or have an __index__ method 2. Instantiate a new PDFParser object by adding the missing braces, avoiding another TypeError because readObject is no class method: Traceback (most recent call last): File "peepdf-venv2/lib64/python2.7/site-packages/peepdf/main.py", line 409, in main ret, pdf = pdfParser.parse(fileName, options.isForceMode, options.isLooseMode, options.isManualAnalysis) File "peepdf-venv2/lib64/python2.7/site-packages/peepdf/PDFCore.py", line 7118, in parse ret = body.updateObjects() File "peepdf-venv2/lib64/python2.7/site-packages/peepdf/PDFCore.py", line 4292, in updateObjects object.resolveReferences() File "peepdf-venv2/lib64/python2.7/site-packages/peepdf/PDFCore.py", line 3256, in resolveReferences ret = PDFParser.readObject(objectsSection[offset:]) TypeError: unbound method readObject() must be called with PDFParser instance as first argument (got str instance instead) 3. Explicitly force the id to be an int() as well and append it do the list of indices as at the other callsites of this code. This solves no issue I have run into but seems sensible to avoid other potential TypeErrors and keep internal bookkeeping of the object consistent. This should conclusively resolve jesparza#70 and supersedes hatching#6. Signed-off-by: Michael Weiser <michael.weiser@gmx.de>
In PDF files the Cross-Reference Table or a Cross-Reference Stream contain byte-offsets for the start of objects within the file or the uncompressed stream. Such an offset does not always point the first byte of the initial token (see ISO 32000-2008 section 7.2.2) of the referenced object. The object may be preceded by white-space characters and comments. Without this commit PDFParser.readSymbol() fails to read a symbol, if the first character to be processed is a white-space character. This commit changes PDFParser.readSymbol() to skip leading white-space characters. (PDFParser.readSymbol() already skips any number of leading comments followed by white-space characters.) This enables passing of PDF-files with sloppy cross reference offsets.
An object of class PDFArray can contain JS-code, if one or more array-elements contain JS-code. The getter method was simply missing.
A previous commit adjusted readSymbol() to skip leading whitespace in order to avoid errors with sloppy cross references. This did not fix handling of literals such as numbers and booleans in readObject() because they're not accessed using readSymbol(). Also, adjusting the very low-level readSymbol() function might generate fallout. So instead, this change moves the skipping of leading whitespace into readObject() so that it affects all types of referenced objects equally but not all symbol lookups altogether. Signed-off-by: Michael Weiser <michael.weiser@gmx.de>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.