Competing refresh token requests can cause concurrent update issues on PostgreSQL

[aeneasr]

Preflight checklist

• I could not find a solution in the existing issues, docs, nor discussions.
• I agree to follow this project's Code of Conduct.
• I have read and am following this repository's Contribution Guidelines.
• I have joined the Ory Community Slack.
• I am signed up to the Ory Security Patch Newsletter.

Ory Network Project

No response

Describe the bug

When executing two token refresh requests in parallel, with the same OAuth2 Client ID and OAuth2 Refresh Token, a situation may arise where:

1. First request succeeds, second request fails with 400
2. First request fails with 500, second succeeds with 200
3. First request fails with 400, second with 401

Error payloads are as follows:

    "payload": {
      "error": {
        "debug": "Unable to serialize access due to a concurrent update in another session: The request could not be completed due to concurrent access",
        "message": "invalid_request",
        "reason": "Failed to refresh token because of multiple concurrent requests using the same token which is not allowed.",
        "status": "Bad Request",
        "status_code": 400
      },

    "payload": {
      "error": {
        "debug": "Unable to serialize access due to a concurrent update in another session",
        "message": "server_error",
        "reason": "",
        "status": "Internal Server Error",
        "status_code": 500
      },

Reproducing the bug

1. Create a client
2. Issue access + refresh token
3. Try to refresh the same token multiple times in parallel

Relevant log output

No response

Relevant configuration

No response

Version

master

On which operating system are you observing this issue?

None

In which environment are you deploying?

None

Additional Context

I believe the issue is that we have one big transaction that is used for refresh tokens. It:

1. Begin
2. Deletes old access token
3. Deletes the old refresh token
4. Writes new access token
5. Writes new refresh token
6. Commit

The code for this is here:

https://github.com/ory/fosite/blob/1df109bb45fadd90b34c3d15cbc7431426cc9853/handler/oauth2/flow_refresh.go#L136-L167

Kind of related: #1831

There are two ways to fix this:

1. Do not return a 500 error when this happens but instead a human-readable error message like "You tried to refresh the same token twice, at the same time. Please ensure your clients are refresh tokens only once".
2. Solve Refresh tokens need a grace period to deal with network errors and similar issues #1831 (this needs a very clear design doc as it can lead to several access+refresh token chains being valid, which may be a security issue).

[struhtanov]

I'm having similar issue. I use hydra for oauth2 flow. Third-party service that I don't control (make.com in my case) seems to send 2 parallel token refresh requests. I see it from proxy access logs. Then I see error in hydra logs same as described in an issue. End user loses his auth in make.com due to that error and complains to me about it.

Looking for any possible workaround for this issue 🙏

[nyhu]

Same issue on our side using react-oidc-context.
The issue results in random disconnections for our users, I would say it's a critical problem.

[aeneasr]

#3895