Make PythonParser resumable (alternative)

[kristjanvalur]

Pull Request check-list

Please make sure to review and check all of these items:

• Does $ tox pass with this change (including linting)?
• Do the CI tests pass with this change (enable it first in your forked repo and wait for the github action build to finish)?
• Is the new or changed code fully tested?
• Is a documentation update included (if this change modifies existing APIs, or introduces new ones)?
• Is there an example added to the examples folder (if applicable)?
• Was the change added to CHANGES file?

NOTE: these things are not required to open a PR and can be done
afterwards / while the PR is open.

Description of change

See issue #2513

Recent changes in redis.asyncio changed the way timeouts are handled, by catching timeouts on a higher level and not leave a Connection object in a failed state. Timeouts are now received directly when doing IO, instead of running a preliminary can_read() test with timeout. This resulted in significant simplification of the logic and consolidated different timeout handling.

Unfortunately, the PythonParser, unlike the HiredisParser was not able to deal with IO being interrupted as it didn't maintain internal state. A TimeoutException caught while parsing a response would cause data to be lost. The HiredisParser, however, cleanly separates parsing and IO, and maintains its own internal parse buffer which is fed by IO separately.

This PR fixes PythonParser to keep a buffer of incomplete responses, so that if IO is interrupted, the parser's read_response() method can be called again, if it is interrupted during IO.

Note: This is an alternative change to pr #2510

See issue #2499 where this problem was pointed out.

[codecov-commenter]

Codecov Report

Base: 92.23% // Head: 92.20% // Decreases project coverage by -0.02% ⚠️

Coverage data is based on head (addd59c) compared to base (ed38e77).
Patch coverage: 89.50% of modified lines in pull request are covered.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #2512      +/-   ##
==========================================
- Coverage   92.23%   92.20%   -0.03%     
==========================================
  Files         113      115       +2     
  Lines       29388    29530     +142     
==========================================
+ Hits        27106    27229     +123     
- Misses       2282     2301      +19     

Impacted Files	Coverage Δ
tests/test_asyncio/mocks.py	63.88% <63.88%> (ø)
redis/connection.py	86.60% <91.66%> (+<0.01%)	⬆️
tests/test_asyncio/test_connection.py	97.70% <95.83%> (-0.88%)	⬇️
tests/test_connection.py	98.24% <96.15%> (-0.64%)	⬇️
tests/mocks.py	96.55% <96.55%> (ø)
redis/asyncio/connection.py	87.39% <97.61%> (+0.81%)	⬆️
tests/test_asyncio/test_search.py	98.29% <0.00%> (-0.35%)	⬇️
tests/test_cluster.py	96.79% <0.00%> (-0.23%)	⬇️
tests/test_asyncio/test_pubsub.py	99.53% <0.00%> (+0.15%)	⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

[dvora-h]

@kristjanvalur Do you have any idea why the tests are canceling? It only happens in these 2 PR's...

[kristjanvalur]

@kristjanvalur Do you have any idea why the tests are canceling? It only happens in these 2 PR's...

No. I'll investigate.

[kristjanvalur]

No. I'll investigate.

Turned out it was the complementary enhancement to the syncronous PythonParser causing issues. Fixed now. Both sync and async PythonParser are now restartable in case of error, same as the HiredisParser.

[kristjanvalur]

current failure is with sync hiredis parser in 3.11 cluster mode, code which is not changed by this.

[kristjanvalur]

Closed, since #2510 was merged (an alternative implementation)