Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lock behavior differs between asyncio and trio after the lock changes in 4.5.0 #798

Closed
2 tasks done
gruuuve opened this issue Oct 2, 2024 · 4 comments · Fixed by #799
Closed
2 tasks done

Lock behavior differs between asyncio and trio after the lock changes in 4.5.0 #798

gruuuve opened this issue Oct 2, 2024 · 4 comments · Fixed by #799
Labels
bug Something isn't working

Comments

@gruuuve
Copy link

gruuuve commented Oct 2, 2024

Things to check first

  • I have searched the existing issues and didn't find my bug already reported there

  • I have checked that my bug is still present in the latest release

AnyIO version

AnyIO 4.6.0

Python version

3.12.5

What happened?

Prior to AnyIO 4.5.0, an exception would be raised when a caller attempted to acquire a lock that was already claimed on both the asyncio and trio backends.
Exception output: RuntimeError: attempt to re-acquire an already held Lock

Now, in both 4.5.0 and 4.6.0, asyncio will hang indefinitely. In the context manager example, RuntimeError: The current task is not holding this lock is output when the process is interrupted with ctrl-c . Trio still behaves the same as it did prior to 4.5.0, so it appears this issue is only with the asyncio changes.

How can we reproduce the bug?

Simplest case to demonstrate the error:

async def main() -> None:
    _lock = anyio.Lock()

    await _lock.acquire()


    await _lock.acquire()

if __name__ == "__main__":
    anyio.run(main)#, backend="trio")

Similar behavior with context managers:

async def main() -> None:
    _lock = anyio.Lock()

    async def get_lock() -> None:
        async with _lock:
            print("Error should be raised here")

    async with _lock:
        await get_lock()

if __name__ == "__main__":
    anyio.run(main)#, backend="trio")
@gruuuve gruuuve added the bug Something isn't working label Oct 2, 2024
@gruuuve
Copy link
Author

gruuuve commented Oct 2, 2024

I tested this on python 3.11.9 as well to check for any variance with the standard library, and the same behavior occurred.

@agronholm
Copy link
Owner

An unfortunate regression. I'll see this fixed in v4.6.1. I may also need to release v4.5.1 for Python 3.8 compatibility.

@vfazio
Copy link

vfazio commented Oct 2, 2024

Thanks @agronholm, we're coming off 3.7.1 so for now we plan to pin to 4.4.0 to avoid the regression and will attempt taking 4.6.1 when released.

agronholm added a commit that referenced this issue Oct 13, 2024
…#799)

Fixes #798.

---------

Co-authored-by: Thomas Grainger <tagrain@gmail.com>

(cherry picked from commit 5c60291)
mkjpryor pushed a commit to azimuth-cloud/cluster-api-addon-provider that referenced this issue Oct 14, 2024
Bumps [anyio](https://github.com/agronholm/anyio) from 4.6.0 to 4.6.2.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/agronholm/anyio/releases">anyio's
releases</a>.</em></p>
<blockquote>
<h2>4.6.2</h2>
<ul>
<li>Fixed regression caused by (<a
href="https://redirect.github.com/agronholm/anyio/pull/807">#807</a>)
that prevented the use of parametrized async fixtures</li>
</ul>
<h2>4.6.1</h2>
<p>This release contains all the changes from both v4.5.1 and v4.6.0,
plus:</p>
<ul>
<li>Fixed TaskGroup and CancelScope producing cyclic references in
tracebacks when raising exceptions (<a
href="https://redirect.github.com/agronholm/anyio/pull/806">#806</a>)
(PR by <a
href="https://github.com/graingert"><code>@​graingert</code></a>)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/agronholm/anyio/blob/master/docs/versionhistory.rst">anyio's
changelog</a>.</em></p>
<blockquote>
<h1>Version history</h1>
<p>This library adheres to <code>Semantic Versioning 2.0
&lt;http://semver.org/&gt;</code>_.</p>
<p><strong>4.6.2</strong></p>
<ul>
<li>Fixed regression caused by
(<code>[#807](agronholm/anyio#807)
&lt;https://github.com/agronholm/anyio/pull/807&gt;</code>_)
that prevented the use of parametrized async fixtures</li>
</ul>
<p><strong>4.6.1</strong></p>
<p>This release contains all the changes from both v4.5.1 and v4.6.0,
plus:</p>
<ul>
<li>Fixed TaskGroup and CancelScope producing cyclic references in
tracebacks
when raising exceptions
(<code>[#806](agronholm/anyio#806)
&lt;https://github.com/agronholm/anyio/pull/806&gt;</code>_)
(PR by <a
href="https://github.com/graingert"><code>@​graingert</code></a>)</li>
</ul>
<p><strong>4.6.0</strong></p>
<p>This release is the successor to v4.5.0 with Python 3.8 support
dropped, and does not
contain the changes from v4.5.1.</p>
<ul>
<li>Dropped support for Python 3.8
(as <code>[#698](agronholm/anyio#698)
&lt;https://github.com/agronholm/anyio/issues/698&gt;</code>_ cannot be
resolved
without cancel message support)</li>
<li>Fixed 100% CPU use on asyncio while waiting for an exiting task
group to finish while
said task group is within a cancelled cancel scope
(<code>[#695](agronholm/anyio#695)
&lt;https://github.com/agronholm/anyio/issues/695&gt;</code>_)</li>
<li>Fixed cancel scopes on asyncio not propagating
<code>CancelledError</code> on exit when the
enclosing cancel scope has been effectively cancelled
(<code>[#698](agronholm/anyio#698)
&lt;https://github.com/agronholm/anyio/issues/698&gt;</code>_)</li>
<li>Fixed asyncio task groups not yielding control to the event loop at
exit if there were
no child tasks to wait on</li>
<li>Fixed inconsistent task uncancellation with asyncio cancel scopes
belonging to a
task group when said task group has child tasks running</li>
</ul>
<p><strong>4.5.1</strong></p>
<p>As Python 3.8 support was dropped in v4.6.0, this interim release was
created to bring a
regression fix to Python 3.8, and adds a few other fixes also present in
v4.6.1.</p>
<ul>
<li>Fixed acquring a lock twice in the same task on asyncio hanging
instead of raising a
<code>RuntimeError</code>
(<code>[#798](agronholm/anyio#798)
&lt;https://github.com/agronholm/anyio/issues/798&gt;</code>_)</li>
<li>Fixed an async fixture's <code>self</code> being different than the
test's <code>self</code> in
class-based tests
(<code>[#633](agronholm/anyio#633)
&lt;https://github.com/agronholm/anyio/issues/633&gt;</code>_)
(PR by <a
href="https://github.com/agronholm"><code>@​agronholm</code></a> and <a
href="https://github.com/graingert"><code>@​graingert</code></a>)</li>
<li>Fixed <code>TypeError</code> with <code>TLSStream</code> on Windows
when a certificate verification
error occurs when using a <code>truststore
&lt;https://github.com/sethmlarson/truststore&gt;</code>_
SSL certificate
(<code>[#795](agronholm/anyio#795)
&lt;https://github.com/agronholm/anyio/issues/795&gt;</code>_)</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/agronholm/anyio/commit/5861355e2a882e0e33c209fcab287569c8405189"><code>5861355</code></a>
Bumped up the version</li>
<li><a
href="https://github.com/agronholm/anyio/commit/f6ddfc803e2bc2fb79cd0556091b3b98e94a1de1"><code>f6ddfc8</code></a>
Fixed a regression in the pytest plugin that broke parametrized async
fixtures</li>
<li><a
href="https://github.com/agronholm/anyio/commit/4ecc96330c0dd8789db964a5569bff7f09ca083b"><code>4ecc963</code></a>
Bumped up the version</li>
<li><a
href="https://github.com/agronholm/anyio/commit/5489fbd47148207fa830e1f33e751f9361c25713"><code>5489fbd</code></a>
Fixed tox configuration</li>
<li><a
href="https://github.com/agronholm/anyio/commit/77b94df83e0c48264ef1ad3ba2fa58a28f292887"><code>77b94df</code></a>
Updated Path documentation and made is_junction() conditional (<a
href="https://redirect.github.com/agronholm/anyio/issues/800">#800</a>)</li>
<li><a
href="https://github.com/agronholm/anyio/commit/bf130dc793bcb52d65000e714d1e98b8ab243786"><code>bf130dc</code></a>
Fixed TypeError when TLS handshake fails with truststore SSLContext (<a
href="https://redirect.github.com/agronholm/anyio/issues/801">#801</a>)</li>
<li><a
href="https://github.com/agronholm/anyio/commit/4cb89a525cfaf2f02d0f761dc31185afb7b5fbd1"><code>4cb89a5</code></a>
Migrated to native TOML configuration for Tox</li>
<li><a
href="https://github.com/agronholm/anyio/commit/6bebf18279924f7a5178ef3c9ac926768b9240f9"><code>6bebf18</code></a>
Made test_start_task_soon_cancel_immediately() less flaky</li>
<li><a
href="https://github.com/agronholm/anyio/commit/e8546bd1cdac8922d5d43efd6e0be334488f9244"><code>e8546bd</code></a>
Rebind instance method fixtures to the same instance as the test (<a
href="https://redirect.github.com/agronholm/anyio/issues/807">#807</a>)</li>
<li><a
href="https://github.com/agronholm/anyio/commit/57bcbc9c5674fd6fc40077c2ad3810f84d9399ae"><code>57bcbc9</code></a>
Updated macOS and Windows to test on Python 3.13 by default</li>
<li>Additional commits viewable in <a
href="https://github.com/agronholm/anyio/compare/4.6.0...4.6.2">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=anyio&package-manager=pip&previous-version=4.6.0&new-version=4.6.2)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@gruuuve
Copy link
Author

gruuuve commented Oct 16, 2024

Awesome, thank you so much @agronholm!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants