Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-124452: Fix header mismatches when folding/unfolding with email message #125919

Merged
merged 15 commits into from
Nov 16, 2024

Conversation

RanKKI
Copy link
Contributor

@RanKKI RanKKI commented Oct 24, 2024

@RanKKI RanKKI requested a review from a team as a code owner October 24, 2024 10:45
Copy link

cpython-cla-bot bot commented Oct 24, 2024

All commit authors signed the Contributor License Agreement.
CLA signed

@bedevere-app
Copy link

bedevere-app bot commented Oct 24, 2024

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

Lib/email/policy.py Outdated Show resolved Hide resolved
Lib/test/test_email/test_message.py Outdated Show resolved Hide resolved
Lib/test/test_email/test_message.py Outdated Show resolved Hide resolved
Lib/test/test_email/test_message.py Outdated Show resolved Hide resolved
Lib/test/test_email/test_message.py Outdated Show resolved Hide resolved
Lib/test/test_email/test_message.py Outdated Show resolved Hide resolved
Lib/email/_policybase.py Outdated Show resolved Hide resolved
Lib/email/_policybase.py Outdated Show resolved Hide resolved
Lib/email/_policybase.py Outdated Show resolved Hide resolved
Lib/email/_policybase.py Outdated Show resolved Hide resolved
Lib/test/test_email/test_message.py Show resolved Hide resolved
@bedevere-app
Copy link

bedevere-app bot commented Oct 24, 2024

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

@RanKKI
Copy link
Contributor Author

RanKKI commented Oct 24, 2024

I have made the requested changes; please review again

@bedevere-app
Copy link

bedevere-app bot commented Oct 24, 2024

Thanks for making the requested changes!

@bitdancer: please review the changes made to this pull request.

@bedevere-app bedevere-app bot requested a review from bitdancer October 24, 2024 22:28
@rruuaanng
Copy link
Contributor

Please specify the method. It seems that you haven't read this link carefully :)

Lib/email/_policybase.py Show resolved Hide resolved
Lib/email/policy.py Show resolved Hide resolved
RanKKI and others added 3 commits October 27, 2024 10:13
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
…qTRgx.rst

Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
@bitdancer
Copy link
Member

Thanks, RanKII. Sorry, I missed the update notification, and I only just checked back on the PR.

@bitdancer bitdancer merged commit ed81971 into python:main Nov 16, 2024
38 of 39 checks passed
@bitdancer bitdancer added needs backport to 3.12 bug and security fixes needs backport to 3.13 bugs and security fixes labels Nov 16, 2024
@miss-islington-app
Copy link

Thanks @RanKKI for the PR, and @bitdancer for merging it 🌮🎉.. I'm working now to backport this PR to: 3.12.
🐍🍒⛏🤖

@miss-islington-app
Copy link

Thanks @RanKKI for the PR, and @bitdancer for merging it 🌮🎉.. I'm working now to backport this PR to: 3.13.
🐍🍒⛏🤖

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Nov 16, 2024
…ail message (pythonGH-125919)

The header-folder of the new email API has a long standing known buglet where
if the first token is longer than max_line_length, it puts that token on the next
line.  It turns out there is also a *parsing* bug when parsing such a header:
the space prefixing that first, non-empty line gets preserved and tacked on to
the start of the header value, which is not the expected behavior per the RFCs.
The bug arises from the fact that the parser assumed that there would be at
least one token on the line with the header, which is going to be true for
probably every email producer other than the python email library with its
folding buglet.  Clearly, though, this is a case that needs to be handled
correctly.  The fix is simple: strip the blanks off the start of the whole
value, not just the first physical line of the value.

(cherry picked from commit ed81971)

Co-authored-by: RanKKI <hliu86.me@gmail.com>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
@bedevere-app
Copy link

bedevere-app bot commented Nov 16, 2024

GH-126916 is a backport of this pull request to the 3.12 branch.

@bedevere-app bedevere-app bot removed the needs backport to 3.12 bug and security fixes label Nov 16, 2024
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Nov 16, 2024
…ail message (pythonGH-125919)

The header-folder of the new email API has a long standing known buglet where
if the first token is longer than max_line_length, it puts that token on the next
line.  It turns out there is also a *parsing* bug when parsing such a header:
the space prefixing that first, non-empty line gets preserved and tacked on to
the start of the header value, which is not the expected behavior per the RFCs.
The bug arises from the fact that the parser assumed that there would be at
least one token on the line with the header, which is going to be true for
probably every email producer other than the python email library with its
folding buglet.  Clearly, though, this is a case that needs to be handled
correctly.  The fix is simple: strip the blanks off the start of the whole
value, not just the first physical line of the value.

(cherry picked from commit ed81971)

Co-authored-by: RanKKI <hliu86.me@gmail.com>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
@bedevere-app
Copy link

bedevere-app bot commented Nov 16, 2024

GH-126917 is a backport of this pull request to the 3.13 branch.

@bedevere-app bedevere-app bot removed the needs backport to 3.13 bugs and security fixes label Nov 16, 2024
bitdancer pushed a commit that referenced this pull request Nov 17, 2024
…mail message (GH-125919) (#126917)

gh-124452: Fix header mismatches when folding/unfolding with email message (GH-125919)

The header-folder of the new email API has a long standing known buglet where
if the first token is longer than max_line_length, it puts that token on the next
line.  It turns out there is also a *parsing* bug when parsing such a header:
the space prefixing that first, non-empty line gets preserved and tacked on to
the start of the header value, which is not the expected behavior per the RFCs.
The bug arises from the fact that the parser assumed that there would be at
least one token on the line with the header, which is going to be true for
probably every email producer other than the python email library with its
folding buglet.  Clearly, though, this is a case that needs to be handled
correctly.  The fix is simple: strip the blanks off the start of the whole
value, not just the first physical line of the value.

(cherry picked from commit ed81971)

Co-authored-by: RanKKI <hliu86.me@gmail.com>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
bitdancer pushed a commit that referenced this pull request Nov 17, 2024
…mail message (GH-125919) (#126916)

gh-124452: Fix header mismatches when folding/unfolding with email message (GH-125919)

The header-folder of the new email API has a long standing known buglet where
if the first token is longer than max_line_length, it puts that token on the next
line.  It turns out there is also a *parsing* bug when parsing such a header:
the space prefixing that first, non-empty line gets preserved and tacked on to
the start of the header value, which is not the expected behavior per the RFCs.
The bug arises from the fact that the parser assumed that there would be at
least one token on the line with the header, which is going to be true for
probably every email producer other than the python email library with its
folding buglet.  Clearly, though, this is a case that needs to be handled
correctly.  The fix is simple: strip the blanks off the start of the whole
value, not just the first physical line of the value.

(cherry picked from commit ed81971)

Co-authored-by: RanKKI <hliu86.me@gmail.com>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
ebonnal pushed a commit to ebonnal/cpython that referenced this pull request Jan 12, 2025
…ail message (python#125919)

The header-folder of the new email API has a long standing known buglet where
if the first token is longer than max_line_length, it puts that token on the next
line.  It turns out there is also a *parsing* bug when parsing such a header:
the space prefixing that first, non-empty line gets preserved and tacked on to
the start of the header value, which is not the expected behavior per the RFCs.
The bug arises from the fact that the parser assumed that there would be at
least one token on the line with the header, which is going to be true for
probably every email producer other than the python email library with its
folding buglet.  Clearly, though, this is a case that needs to be handled
correctly.  The fix is simple: strip the blanks off the start of the whole
value, not just the first physical line of the value.

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Parsing adds whitespace to the front of long headers
5 participants