soft-pool: lock channel queue when trying to open new channels

[pmrowla]

In SFTPSoftChannelPool we try to open as many SFTP channels as the server will allow (up to a default max of 10), and then cap our max if/when the server rejects a request to open an additional channel.

Before this PR there was a bug where we could await multiple client.start_sftp_client() calls at once. If any of the calls which were rejected by the server, we would not account for the case that any of the other calls which were being awaited at the same time eventually succeeded (i.e. we could request something like 10 channels from the server at once, but the server would only allow us to open 1 and reject the other 9 requests). In this situation we would just fail as if the server had not allowed us to open any channels at all.

In the soft channel pool we should only try to open a single channel at a time. (And then SFTP requests using those open channels can be done in parallel)

See iterative/dvc-ssh#16

[codecov-commenter]

Codecov Report

❗ No coverage uploaded for pull request base (main@b62b9a3). Click here to learn what that means.
Patch has no changes to coverable lines.

❗ Current head fe35699 differs from pull request most recent head 9e2e305. Consider uploading reports for the commit 9e2e305 to get more accurate results

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

Additional details and impacted files

@@           Coverage Diff           @@
##             main      #31   +/-   ##
=======================================
  Coverage        ?   93.80%           
=======================================
  Files           ?       13           
  Lines           ?      678           
  Branches        ?        0           
=======================================
  Hits            ?      636           
  Misses          ?       42           
  Partials        ?        0           

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.