gh-102950: Implement PEP 706 – Filter for tarfile.extractall #102953
Merged
+1,786
−99
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
1 task
ethanfurman
requested changes
Mar 23, 2023
|
When you're done making the requested changes, leave the comment: |
eryksun
reviewed
Mar 23, 2023
|
@gpshead, did you want to take a look? |
|
I have made the requested changes; please review again |
|
Thanks for making the requested changes! @ethanfurman: please review the changes made to this pull request. |
hroncok
reviewed
Apr 4, 2023
Doc/library/tarfile.rst
Outdated
|
|
||
| .. versionadded:: 3.12 | ||
|
|
||
| The *tar* format is designed to capture all details of a UNIX-like ecosystem, |
There was a problem hiding this comment.
Suggested change
| The *tar* format is designed to capture all details of a UNIX-like ecosystem, | |
| The *tar* format is designed to capture all details of a UNIX-like filesystem, |
There was a problem hiding this comment.
I also tend to say POSIX rather than UNIX at this point.
There was a problem hiding this comment.
POSIX is more precise, but this intentionally hand-wavy terminology: I think UNIX-like is more understandable to more people than POSIX-like.
gpshead
reviewed
Apr 5, 2023
|
If there are no objections, I plan to merge this around Wednesday. Please let me know if you're working on a review. |
hroncok
reviewed
Apr 17, 2023
Lib/tarfile.py
Outdated
Comment on lines
2309
to
2311
| except TarError as e: | ||
| if self.errorlevel > 0: | ||
| raise |
There was a problem hiding this comment.
After reading the docs, I thought only OSErrors will be raised with errorlevel == 1. This seems to raise even TarErrors with errorlevel 1 or 2. What do I miss?
There was a problem hiding this comment.
Oh dear. What a rabbit hole. Thanks for raising it, and for a very helpful private discussion.
This warrants an addition to the PEP: https://discuss.python.org/t/23149/26
Please voice your concerns there!
I've updated this PR with my preferred solution, so you can see the areas that need to change and so I get a CI check. It doesn't mean the discussion is over, of course.
encukou
added
the
🔨 test-with-buildbots
bedevere-bot
removed
the
🔨 test-with-buildbots
|
The remaining buildbot failures are unrelated and happened on |
|
Hi, with this integrated, I see one new test failure on Solaris: The issue seems to be the fact that setting sticky bit on files as regular user doesn't do anything on Solaris, and from my limited testing, the same seems to be true on some other systems as well (I tried FreeBSD and |
|
Thanks for the report! The tests are intentionally pretty strict. I'll relax this one. (Edit: see #103831) |
encukou
added a commit
to encukou/cpython
that referenced
this pull request
Apr 25, 2023
encukou
added a commit
that referenced
this pull request
Apr 28, 2023
…H-102953) (GH-103832) See [Backporting & Forward Compatibility in PEP 706](https://peps.python.org/pep-0706/#backporting-forward-compatibility). - Backport b52ad18 - Backport c8c3956 - Remove the DeprecationWarning - Adjust docs - Remove new `__all__` entries
|
GH-104128 is a backport of this pull request to the 3.10 branch. |
|
GH-104327 is a backport of this pull request to the 3.10 branch. |
encukou
added a commit
that referenced
this pull request
May 10, 2023
|
GH-104382 is a backport of this pull request to the 3.9 branch. |
ambv
pushed a commit
that referenced
this pull request
May 15, 2023
|
|
|
A |
|
GH-104548 is a backport of this pull request to the 3.8 branch. |
ambv
pushed a commit
that referenced
this pull request
May 17, 2023
|
GH-104583 is a backport of this pull request to the 3.7 branch. |
carlosroman
added a commit
to DataDog/cpython
that referenced
this pull request
Jun 22, 2023
* Post 3.8.16 * [3.8] Update copyright years to 2023. (pythongh-100852) * [3.8] Update copyright years to 2023. (pythongh-100848). (cherry picked from commit 11f9932) Co-authored-by: Benjamin Peterson <benjamin@python.org> * Update additional copyright years to 2023. Co-authored-by: Ned Deily <nad@python.org> * [3.8] Update copyright year in README (pythonGH-100863) (pythonGH-100867) (cherry picked from commit 30a6cc4) Co-authored-by: Ned Deily <nad@python.org> Co-authored-by: HARSHA VARDHAN <75431678+Thunder-007@users.noreply.github.com> * [3.8] Correct CVE-2020-10735 documentation (pythonGH-100306) (python#100698) (cherry picked from commit 1cf3d78) (cherry picked from commit 88fe8d7) Co-authored-by: Jeremy Paige <ucodery@gmail.com> Co-authored-by: Gregory P. Smith <greg@krypto.org> * [3.8] Bump Azure Pipelines to ubuntu-22.04 (pythonGH-101089) (python#101215) (cherry picked from commit c22a55c) Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com> * [3.8] pythongh-100180: Update Windows installer to OpenSSL 1.1.1s (pythonGH-100903) (python#101258) * pythongh-101422: (docs) TarFile default errorlevel argument is 1, not 0 (pythonGH-101424) (cherry picked from commit ea23271) Co-authored-by: Owain Davies <116417456+OTheDev@users.noreply.github.com> * [3.8] pythongh-95778: add doc missing in some places (pythonGH-100627) (python#101630) (cherry picked from commit 4652182) * [3.8] pythongh-101283: Improved fallback logic for subprocess with shell=True on Windows (pythonGH-101286) (python#101710) Co-authored-by: Oleg Iarygin <oleg@arhadthedev.net> Co-authored-by: Steve Dower <steve.dower@microsoft.com> * [3.8] pythongh-101981: Fix Ubuntu SSL tests with OpenSSL (3.1.0-beta1) CI i… (python#102095) [3.8] pythongh-101981: Fix Ubuntu SSL tests with OpenSSL (3.1.0-beta1) CI issue (pythongh-102079) * [3.8] pythonGH-102306 Avoid GHA CI macOS test_posix failure by using the appropriate macOS SDK (pythonGH-102307) [3.8] Avoid GHA CI macOS test_posix failure by using the appropriate macOS SDK. * [3.8] pythongh-101726: Update the OpenSSL version to 1.1.1t (pythonGH-101727) (pythonGH-101752) Fixes CVE-2023-0286 (High) and a couple of Medium security issues. https://www.openssl.org/news/secadv/20230207.txt Co-authored-by: Gregory P. Smith <greg@krypto.org> Co-authored-by: Ned Deily <nad@python.org> * [3.8] pythongh-102627: Replace address pointing toward malicious web page (pythonGH-102630) (pythonGH-102667) (cherry picked from commit 61479d4) Co-authored-by: Blind4Basics <32236948+Blind4Basics@users.noreply.github.com> Co-authored-by: C.A.M. Gerlach <CAM.Gerlach@Gerlach.CAM> Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com> * [3.8] pythongh-101997: Update bundled pip version to 23.0.1 (pythonGH-101998). (python#102244) (cherry picked from commit 89d9ff0) * [3.8] pythongh-102950: Implement PEP 706 – Filter for tarfile.extractall (pythonGH-102953) (python#104548) Backport of c8c3956 * [3.8] pythongh-99889: Fix directory traversal security flaw in uu.decode() (pythonGH-104096) (python#104332) (cherry picked from commit 0aeda29) Co-authored-by: Sam Carroll <70000253+samcarroll42@users.noreply.github.com> * [3.8] pythongh-104049: do not expose on-disk location from SimpleHTTPRequestHandler (pythonGH-104067) (python#104121) Do not expose the local server's on-disk location from `SimpleHTTPRequestHandler` when generating a directory index. (unnecessary information disclosure) (cherry picked from commit c7c3a60) Co-authored-by: Ethan Furman <ethan@stoneleaf.us> Co-authored-by: Gregory P. Smith <greg@krypto.org> Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com> * [3.8] pythongh-103935: Use `io.open_code()` when executing code in trace and profile modules (pythonGH-103947) (python#103954) Co-authored-by: Tian Gao <gaogaotiantian@hotmail.com> * [3.8] pythongh-68966: fix versionchanged in docs (pythonGH-105299) * [3.8] Update GitHub CI workflow for macOS. (pythonGH-105302) * [3.8] pythongh-105184: document that marshal functions can fail and need to be checked with PyErr_Occurred (pythonGH-105185) (python#105222) (cherry picked from commit ee26ca1) Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com> * [3.8] pythongh-102153: Start stripping C0 control and space chars in `urlsplit` (pythonGH-102508) (pythonGH-104575) (pythonGH-104592) (python#104593) (python#104895) `urllib.parse.urlsplit` has already been respecting the WHATWG spec a bit pythonGH-25595. This adds more sanitizing to respect the "Remove any leading C0 control or space from input" [rule](https://url.spec.whatwg.org/GH-url-parsing:~:text=Remove%20any%20leading%20and%20trailing%20C0%20control%20or%20space%20from%20input.) in response to [CVE-2023-24329](https://nvd.nist.gov/vuln/detail/CVE-2023-24329). I simplified the docs by eliding the state of the world explanatory paragraph in this security release only backport. (people will see that in the mainline /3/ docs) (cherry picked from commit d7f8a5f) (cherry picked from commit 2f630e1) (cherry picked from commit 610cc0a) (cherry picked from commit f48a96a) Co-authored-by: Miss Islington (bot) <31488909+miss-islington@users.noreply.github.com> Co-authored-by: Illia Volochii <illia.volochii@gmail.com> Co-authored-by: Gregory P. Smith [Google] <greg@krypto.org> * [3.8] pythongh-103142: Upgrade binary builds and CI to OpenSSL 1.1.1u (pythonGH-105174) (pythonGH-105200) (pythonGH-105205) (python#105370) Upgrade builds to OpenSSL 1.1.1u. Also updates _ssl_data_111.h from OpenSSL 1.1.1u, _ssl_data_300.h from 3.0.9. Manual edits to the _ssl_data_300.h file prevent it from removing any existing definitions in case those exist in some peoples builds and were important (avoiding regressions during backporting). (cherry picked from commit ede89af) (cherry picked from commit e15de14) Co-authored-by: Gregory P. Smith <greg@krypto.org> Co-authored-by: Ned Deily <nad@python.org> * Python 3.8.17 * Post 3.8.17 * Updated CI to build 3.8.17 --------- Co-authored-by: Łukasz Langa <lukasz@langa.pl> Co-authored-by: Benjamin Peterson <benjamin@python.org> Co-authored-by: Ned Deily <nad@python.org> Co-authored-by: Miss Islington (bot) <31488909+miss-islington@users.noreply.github.com> Co-authored-by: HARSHA VARDHAN <75431678+Thunder-007@users.noreply.github.com> Co-authored-by: Gregory P. Smith <greg@krypto.org> Co-authored-by: Jeremy Paige <ucodery@gmail.com> Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com> Co-authored-by: Steve Dower <steve.dower@python.org> Co-authored-by: Owain Davies <116417456+OTheDev@users.noreply.github.com> Co-authored-by: Éric <earaujo@caravan.coop> Co-authored-by: Oleg Iarygin <oleg@arhadthedev.net> Co-authored-by: Steve Dower <steve.dower@microsoft.com> Co-authored-by: Dong-hee Na <donghee.na@python.org> Co-authored-by: Blind4Basics <32236948+Blind4Basics@users.noreply.github.com> Co-authored-by: C.A.M. Gerlach <CAM.Gerlach@Gerlach.CAM> Co-authored-by: Pradyun Gedam <pradyunsg@gmail.com> Co-authored-by: Petr Viktorin <encukou@gmail.com> Co-authored-by: Sam Carroll <70000253+samcarroll42@users.noreply.github.com> Co-authored-by: Ethan Furman <ethan@stoneleaf.us> Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com> Co-authored-by: Tian Gao <gaogaotiantian@hotmail.com> Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com> Co-authored-by: stratakis <cstratak@redhat.com> Co-authored-by: Illia Volochii <illia.volochii@gmail.com>
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
maxwell-k
added a commit
to maxwell-k/dotlocalslashbin
that referenced
this pull request
Jul 16, 2024
So that the script can be used on Debian 12. > Changed in version 3.11.4: Added the filter parameter. -- https://docs.python.org/3.11/library/tarfile.html The implementation was in python/cpython#102953 Before this change, Debian 12 has Python 3.11.2, so this script errors on a call with `filter=` After this change the script does not error.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
See PEP-706 for details.
I might have overengineered the tests a bit, moving boilerplate to a set of helpers. Hopefully the tests themselves are clear enough to make it worth it.