-
Notifications
You must be signed in to change notification settings - Fork 6
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
BUG: Waiting on wrong key #24
Conversation
return value if value.present? | ||
|
||
# wait for the other process if a last known value isn't there | ||
if key.present? | ||
return time('wait.run', tags: tags) do | ||
wait_for_new_value(key, options, tags) | ||
wait_for_new_value(keyspace, options, tags) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
quick_retry
and wait_for_new_value
changed their approach here a bit:
quick_retry
now uses the initial key obtained on line 40, as very little time has passed, and it's more efficient for all the locations that have quick_retry disabled.wait_for_new_value
no longer takes the key, as time will have elapsed between attempts, such that it should make sure it's looking at the most recent key for a value.
@@ -162,6 +159,8 @@ def wait_for_new_value(key, options, tags) | |||
backoff_duration_ms = option(:backoff_duration_ms, options, backoff_duration_ms) | |||
sleep((backoff_duration_ms.to_f / 1000) * attempt) | |||
|
|||
# re-fetch the key each time, to make sure we're actually getting the latest key with the correct LMT | |||
key = @timestamp_manager.current_key(keyspace) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This line here is effectively the bug fix. Each attempt it will make sure to get the most current key.
How this is actually working in a cold start scenario is that because LMT is empty, the initial key is just prefix.foobar
. The value that actually gets written by the generating process however is prefix.foobar.timestamp
. The old code would continue to try and read prefix.foobar
then exhaust it's attempts. This fix causes it to always grab the most current key, which will cause it to see prefix.foobar.timestamp
as written by another generating process, and use that key for a value.
end | ||
end | ||
|
||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
be warned: there be dragons below 🐉
🙈
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The crux of this update generally makes sense - the integration test is really cool, but noting I'm not 100% about threadsafe-ness and how all the process queue management is working.
Background
A bug was identified earlier today where the
wait_for_new_value
method is incorrectly waiting on the wrong key. This bugfix updates the code to re-fetch the latest key value whenever it attempts to see if another thread has written the value. This makes sure that re-attempts are always attempting to get the latest value.Tasks
./lib/atomic_cache/version.rb
) when applicable