[CVE-2023-27043] gh-102988: Reject malformed addresses in email.parseaddr() #111116

vstinner · 2023-10-20T14:20:59Z

Detect email address parsing errors and return empty tuple to indicate the parsing error (old API). Add an optional 'strict' parameter to getaddresses() and parseaddr() functions. Patch by Thomas Dwyer.

Issue: [CVE-2023-27043] Parsing errors in email/_parseaddr.py lead to incorrect value in email address part of tuple #102988

📚 Documentation preview 📚: https://cpython-previews--111116.org.readthedocs.build/

vstinner · 2023-10-20T14:25:12Z

@gpshead @serhiy-storchaka @bitdancer @warsaw: Would you mind to review this security fix?

See issue gh-102988 for the context.

This PR is a copy of PR #108250 but I added strict=True parameter, so it's possible to get the old behavior. I added tests on both modes, strict=True and strict=False.

vstinner · 2023-10-20T15:27:59Z

This PR is a copy of PR #108250 but I added strict=True parameter

My colleague Lumir Balhar @frenzymadness ran an impact check of PR #108250 on Fedora: in short, there is no impact, the test suite of all Python packages (in Fedora) pass with the change. While there were some build errors, they were unrelated to the email issue. For details, see https://copr.fedorainfracloud.org/coprs/lbalhar/email-CVE/builds/ COPR which as more than 4300 builds.

Now with an additional strict parameter, if there is any impacted project, at least there is a way to "opt out".

vstinner · 2023-10-20T15:29:04Z

@tdwyer: Would you mind to review my change, to see if I preserved your work correctly? (code and tests)

vstinner · 2023-10-27T14:04:13Z

I think that we should backport the change to all branches accepting security fixes. Problem: the change refer to version numbers, which as .. versionchanged:: 3.13. I suppose that if the change is backported, we should compute the next version of each branch, so backport manually.

vstinner · 2023-10-27T14:04:50Z

@ambv @SethMichaelLarson: Would you mind to review this PR?

ambv · 2023-10-27T15:31:40Z

Why is this a separate PR from #108250?

serhiy-storchaka · 2023-10-27T16:34:38Z

Doc/whatsnew/3.13.rst

@@ -165,7 +165,7 @@ email
 encountered instead of potentially inaccurate values. Add optional *strict*
 parameter to these two functions: use ``strict=False`` to get the old
 behavior, accept malformed inputs.
- (Contributed by Thomas Dwyer for :gh:`102988` to ameliorate CVE-2023-27043
+ (Contributed by Thomas Dwyer for :gh:`102988` to improve the CVE-2023-27043


TIL a new word.

serhiy-storchaka · 2023-10-27T16:42:14Z

Lib/email/utils.py

@@ -42,6 +42,8 @@

 specialsre = re.compile(r'[][\\()<>@,:;".]')
 escapesre = re.compile(r'[\\"]')
+realname_comma_re = re.compile(r'"[^"]*,[^"]*"')


Suggested change

realname_comma_re = re.compile(r'"[^"]*,[^"]*"')

realname_comma_re = re.compile(r'"[^",]*+,[^"]*+"')

It is faster. But I am not sure that the use of such regex is correct.

serhiy-storchaka · 2023-10-27T16:58:26Z

Lib/email/utils.py

+def _pre_parse_validation(email_header_fields):
+ accepted_values = []
+ for v in email_header_fields:
+ s = v.replace('\\(', '').replace('\\)', '')


But what if that backslash was already escaped with a backslash? For example \\) or \\\\).

Lib/email/utils.py

vstinner · 2023-10-27T20:25:47Z

Why is this a separate PR from #108250?

I'm not the author of the other PR. I copied the other PR and added strict parameter.

ambv · 2023-10-27T20:54:44Z

I'm not the author of this PR and I was able to make commits to it.

vstinner · 2023-10-28T22:37:18Z

I'm not the author of this PR and I was able to make commits to it.

I don't feel comfortable to make significant change of a PR without asking the author. I prefer to create a separated PR and ask for review.

vstinner · 2023-10-30T13:11:53Z

Is this behavior a bug or a feature? I don't know how ; is supposed to behave.

$ python
Python 3.11.6 (main, Oct  3 2023, 00:00:00) [GCC 13.2.1 20230728 (Red Hat 13.2.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from email.utils import getaddresses
>>> from pprint import pprint
>>> pprint(getaddresses('<bob@example.org>; <alice@example.org>'))
[('', ''),
 ('', 'b'),
 ('', 'o'),
 ('', 'b'),
 ('', ''),
 ('', 'e'),
 ('', 'x'),
 ('', 'a'),
 ('', 'm'),
 ('', 'p'),
 ('', 'l'),
 ('', 'e'),
 ('', '.'),
 ('', 'o'),
 ('', 'r'),
 ('', 'g'),
 ('', ''),
 ('', ''),
 ('', ''),
 ('', ''),
 ('', 'a'),
 ('', 'l'),
 ('', 'i'),
 ('', 'c'),
 ('', 'e'),
 ('', ''),
 ('', 'e'),
 ('', 'x'),
 ('', 'a'),
 ('', 'm'),
 ('', 'p'),
 ('', 'l'),
 ('', 'e'),
 ('', '.'),
 ('', 'o'),
 ('', 'r'),
 ('', 'g'),
 ('', '')]

vstinner · 2023-10-30T13:13:45Z

Is this behavior a bug or a feature? I don't know how ; is supposed to behave.

Oh. getaddresses() expects a sequence, not a string :-)

vstinner · 2023-10-30T13:46:33Z

Except of parsedate_tz() function, Lib/email/_parseaddr.py file didn't evolve much it was added in 2002 by commit 030ddf7. The file was created from Lib/rfc822.py which was added in 1992 (commit 01ca336).

The latest major change was done in... 1997 with commit be7c45e

New address parser by Ben Escoto replaces Sjoerd Mullender's parseaddr()

The latest minor change was done in 2019 to fix CVE-2019-16056: commit 8cb65d1 of issue #78336.

vstinner · 2023-10-30T13:48:56Z

What if the input is '"Jane Doe" jane@example.net, "John Doe" john@example.net'?

Oh, realname_comma_re replaces "Jane Doe" <jane@example.net>, "John Doe" <john@example.net> with "Jane Doe <john@example.net> which is invalid...

vstinner · 2023-10-30T14:03:05Z

Email addresses have multiple standards:

1982: https://datatracker.ietf.org/doc/html/rfc822
2001: https://datatracker.ietf.org/doc/html/rfc2822 (replace 822)
2008: https://datatracker.ietf.org/doc/html/rfc5322 (replace 2822)
2012: https://datatracker.ietf.org/doc/html/rfc6532

miss-islington-app · 2024-09-06T09:34:10Z

Thanks @vstinner for the PR 🌮🎉.. I'm working now to backport this PR to: 3.9.
🐍🍒⛏🤖

miss-islington-app · 2024-09-06T09:34:11Z

Thanks @vstinner for the PR 🌮🎉.. I'm working now to backport this PR to: 3.8.
🐍🍒⛏🤖

miss-islington-app · 2024-09-06T09:34:14Z

Sorry, @vstinner, I could not cleanly backport this to 3.12 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 4a153a1d3b18803a684cd1bcc2cdf3ede3dbae19 3.12

miss-islington-app · 2024-09-06T09:34:34Z

Sorry, @vstinner, I could not cleanly backport this to 3.10 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 4a153a1d3b18803a684cd1bcc2cdf3ede3dbae19 3.10

miss-islington-app · 2024-09-06T09:34:48Z

Sorry, @vstinner, I could not cleanly backport this to 3.11 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 4a153a1d3b18803a684cd1bcc2cdf3ede3dbae19 3.11

miss-islington-app · 2024-09-06T09:35:01Z

Sorry, @vstinner, I could not cleanly backport this to 3.9 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 4a153a1d3b18803a684cd1bcc2cdf3ede3dbae19 3.9

miss-islington-app · 2024-09-06T09:35:15Z

Sorry, @vstinner, I could not cleanly backport this to 3.8 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 4a153a1d3b18803a684cd1bcc2cdf3ede3dbae19 3.8

…n email.parseaddr() (pythonGH-111116) Detect email address parsing errors and return empty tuple to indicate the parsing error (old API). Add an optional 'strict' parameter to getaddresses() and parseaddr() functions. Patch by Thomas Dwyer. Co-Authored-By: Thomas Dwyer <github@tomd.tel> (cherry picked from commit 4a153a1)

…n email.parseaddr() (pythonGH-111116) Detect email address parsing errors and return empty tuple to indicate the parsing error (old API). Add an optional 'strict' parameter to getaddresses() and parseaddr() functions. Patch by Thomas Dwyer. (cherry picked from commit 4a153a1) Co-authored-by: Victor Stinner <vstinner@python.org> Co-Authored-By: Thomas Dwyer <github@tomd.tel>

…n email.parseaddr() (pythonGH-111116) Detect email address parsing errors and return empty tuple to indicate the parsing error (old API). Add an optional 'strict' parameter to getaddresses() and parseaddr() functions. Patch by Thomas Dwyer. Co-Authored-By: Thomas Dwyer <github@tomd.tel> (cherry picked from commit 4a153a1)

bedevere-app · 2024-09-06T09:45:28Z

GH-123766 is a backport of this pull request to the 3.12 branch.

…n email.parseaddr() (pythonGH-111116) Detect email address parsing errors and return empty tuple to indicate the parsing error (old API). Add an optional 'strict' parameter to getaddresses() and parseaddr() functions. Patch by Thomas Dwyer. (cherry picked from commit 4a153a1) Co-authored-by: Victor Stinner <vstinner@python.org> Co-Authored-By: Thomas Dwyer <github@tomd.tel>

bedevere-app · 2024-09-06T09:55:01Z

GH-123767 is a backport of this pull request to the 3.11 branch.

…n email.parseaddr() (pythonGH-111116) Detect email address parsing errors and return empty tuple to indicate the parsing error (old API). Add an optional 'strict' parameter to getaddresses() and parseaddr() functions. Patch by Thomas Dwyer. (cherry picked from commit 4a153a1) Co-authored-by: Victor Stinner <vstinner@python.org> Co-Authored-By: Thomas Dwyer <github@tomd.tel>

bedevere-app · 2024-09-06T10:03:03Z

GH-123768 is a backport of this pull request to the 3.10 branch.

…l.parseaddr() (GH-111116) (#123766) Detect email address parsing errors and return empty tuple to indicate the parsing error (old API). Add an optional 'strict' parameter to getaddresses() and parseaddr() functions. Patch by Thomas Dwyer. Co-Authored-By: Thomas Dwyer <github@tomd.tel> (cherry picked from commit 4a153a1) Co-authored-by: Victor Stinner <vstinner@python.org>

…l.parseaddr() (GH-111116) (#123767) Detect email address parsing errors and return empty tuple to indicate the parsing error (old API). Add an optional 'strict' parameter to getaddresses() and parseaddr() functions. Patch by Thomas Dwyer. (cherry picked from commit 4a153a1) Co-authored-by: Victor Stinner <vstinner@python.org> Co-authored-by: Thomas Dwyer <github@tomd.tel>

… email.parseaddr() (pythonGH-111116) Detect email address parsing errors and return empty tuple to indicate the parsing error (old API). Add an optional 'strict' parameter to getaddresses() and parseaddr() functions. Patch by Thomas Dwyer. (cherry picked from commit 4a153a1) Co-authored-by: Victor Stinner <vstinner@python.org> Co-Authored-By: Thomas Dwyer <github@tomd.tel>

bedevere-app · 2024-09-06T10:50:33Z

GH-123769 is a backport of this pull request to the 3.9 branch.

… email.parseaddr() (pythonGH-111116) Detect email address parsing errors and return empty tuple to indicate the parsing error (old API). Add an optional 'strict' parameter to getaddresses() and parseaddr() functions. Patch by Thomas Dwyer. (cherry picked from commit 4a153a1) Co-authored-by: Victor Stinner <vstinner@python.org> Co-Authored-By: Thomas Dwyer <github@tomd.tel>

bedevere-app · 2024-09-06T10:56:31Z

GH-123770 is a backport of this pull request to the 3.8 branch.

….parseaddr() (GH-111116) (#123769) Detect email address parsing errors and return empty tuple to indicate the parsing error (old API). Add an optional 'strict' parameter to getaddresses() and parseaddr() functions. Patch by Thomas Dwyer. (cherry picked from commit 4a153a1) Co-authored-by: Victor Stinner <vstinner@python.org> Co-Authored-By: Thomas Dwyer <github@tomd.tel>

….parseaddr() (GH-111116) (#123770) Detect email address parsing errors and return empty tuple to indicate the parsing error (old API). Add an optional 'strict' parameter to getaddresses() and parseaddr() functions. Patch by Thomas Dwyer. (cherry picked from commit 4a153a1) Co-authored-by: Victor Stinner <vstinner@python.org> Co-Authored-By: Thomas Dwyer <github@tomd.tel>

…l.parseaddr() (GH-111116) (#123768) Detect email address parsing errors and return empty tuple to indicate the parsing error (old API). Add an optional 'strict' parameter to getaddresses() and parseaddr() functions. Patch by Thomas Dwyer. (cherry picked from commit 4a153a1) Co-authored-by: Victor Stinner <vstinner@python.org> Co-Authored-By: Thomas Dwyer <github@tomd.tel>

vstinner requested a review from a team as a code owner October 20, 2023 14:21

bedevere-app bot mentioned this pull request Oct 20, 2023

[CVE-2023-27043] Parsing errors in email/_parseaddr.py lead to incorrect value in email address part of tuple #102988

Closed

bedevere-app bot added the awaiting core review label Oct 20, 2023

vstinner force-pushed the parse_email branch from 8fdc842 to c67d8f9 Compare October 20, 2023 14:23

vstinner mentioned this pull request Oct 20, 2023

gh-102988: Detect email address parsing errors and return empty tuple to indicate the parsing error (old API) #108250

Closed

vstinner added type-security A security issue needs backport to 3.8 needs backport to 3.9 only security fixes needs backport to 3.10 only security fixes needs backport to 3.11 only security fixes needs backport to 3.12 bug and security fixes labels Oct 27, 2023

ambv changed the title ~~gh-102988: email parseaddr() now rejects malformed address~~ gh-102988: Reject malformed addresses in email.parseaddr() Oct 27, 2023

serhiy-storchaka reviewed Oct 27, 2023

View reviewed changes

serhiy-storchaka self-requested a review October 27, 2023 16:39

serhiy-storchaka reviewed Oct 27, 2023

View reviewed changes

vstinner marked this pull request as draft October 30, 2023 14:20

miss-islington-app bot assigned vstinner Sep 6, 2024

bedevere-app bot removed the needs backport to 3.12 bug and security fixes label Sep 6, 2024

bedevere-app bot removed the needs backport to 3.11 only security fixes label Sep 6, 2024

bedevere-app bot removed the needs backport to 3.10 only security fixes label Sep 6, 2024

bedevere-app bot removed the needs backport to 3.9 only security fixes label Sep 6, 2024

bedevere-app bot removed the needs backport to 3.8 label Sep 6, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[CVE-2023-27043] gh-102988: Reject malformed addresses in email.parseaddr() #111116

[CVE-2023-27043] gh-102988: Reject malformed addresses in email.parseaddr() #111116

vstinner commented Oct 20, 2023 •

edited by github-actions bot

Loading

vstinner commented Oct 20, 2023

vstinner commented Oct 20, 2023

vstinner commented Oct 20, 2023

vstinner commented Oct 27, 2023

vstinner commented Oct 27, 2023

ambv commented Oct 27, 2023

serhiy-storchaka Oct 27, 2023

serhiy-storchaka Oct 27, 2023

serhiy-storchaka Oct 27, 2023

vstinner commented Oct 27, 2023

ambv commented Oct 27, 2023

vstinner commented Oct 28, 2023

vstinner commented Oct 30, 2023 •

edited

Loading

vstinner commented Oct 30, 2023

vstinner commented Oct 30, 2023

vstinner commented Oct 30, 2023

vstinner commented Oct 30, 2023

miss-islington-app bot commented Sep 6, 2024

miss-islington-app bot commented Sep 6, 2024

miss-islington-app bot commented Sep 6, 2024

miss-islington-app bot commented Sep 6, 2024

miss-islington-app bot commented Sep 6, 2024

miss-islington-app bot commented Sep 6, 2024

miss-islington-app bot commented Sep 6, 2024

bedevere-app bot commented Sep 6, 2024

bedevere-app bot commented Sep 6, 2024

bedevere-app bot commented Sep 6, 2024

bedevere-app bot commented Sep 6, 2024

bedevere-app bot commented Sep 6, 2024

	realname_comma_re = re.compile(r'"[^"],[^"]"')
	realname_comma_re = re.compile(r'"[^",]+,[^"]+"')

[CVE-2023-27043] gh-102988: Reject malformed addresses in email.parseaddr() #111116

[CVE-2023-27043] gh-102988: Reject malformed addresses in email.parseaddr() #111116

Conversation

vstinner commented Oct 20, 2023 • edited by github-actions bot Loading

vstinner commented Oct 20, 2023

vstinner commented Oct 20, 2023

vstinner commented Oct 20, 2023

vstinner commented Oct 27, 2023

vstinner commented Oct 27, 2023

ambv commented Oct 27, 2023

serhiy-storchaka Oct 27, 2023

Choose a reason for hiding this comment

serhiy-storchaka Oct 27, 2023

Choose a reason for hiding this comment

serhiy-storchaka Oct 27, 2023

Choose a reason for hiding this comment

vstinner commented Oct 27, 2023

ambv commented Oct 27, 2023

vstinner commented Oct 28, 2023

vstinner commented Oct 30, 2023 • edited Loading

vstinner commented Oct 30, 2023

vstinner commented Oct 30, 2023

vstinner commented Oct 30, 2023

vstinner commented Oct 30, 2023

miss-islington-app bot commented Sep 6, 2024

miss-islington-app bot commented Sep 6, 2024

miss-islington-app bot commented Sep 6, 2024

miss-islington-app bot commented Sep 6, 2024

miss-islington-app bot commented Sep 6, 2024

miss-islington-app bot commented Sep 6, 2024

miss-islington-app bot commented Sep 6, 2024

bedevere-app bot commented Sep 6, 2024

bedevere-app bot commented Sep 6, 2024

bedevere-app bot commented Sep 6, 2024

bedevere-app bot commented Sep 6, 2024

bedevere-app bot commented Sep 6, 2024

vstinner commented Oct 20, 2023 •

edited by github-actions bot

Loading

vstinner commented Oct 30, 2023 •

edited

Loading