-
Notifications
You must be signed in to change notification settings - Fork 14.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Close open connections for deferrable SFTPSensor #38881
Close open connections for deferrable SFTPSensor #38881
Conversation
6e39800
to
662d61b
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The change looks good, I think it is possible to test it by checking if the method __aexit__
(or close if it has one) is called.
However, I wonder why you need # type: ignore[return]
? 🤔
5c0bd70
to
119fd3b
Compare
Yep, this is what we should check and is part of the reason why the tests failed originally. Just fixed it 🔧
Otherwise, we'll encounter |
I also test it locally to verify the functionality. 👍 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
It's observed that running the SFTPSensor in deferrable mode
fails to close opened SSH connections to the server, resulting
in a continuous increase of open connections over time.
This pull request addresses this issue by ensuring that
connections are opened using a context manager, allowing
them to be closed once the relevant data has been read
from the SSH server during file sensing.
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rst
or{issue_number}.significant.rst
, in newsfragments.