Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net/redisclient: fix nil pointer dereference #2939

Merged
merged 1 commit into from
Feb 19, 2024

Conversation

AlexanderYastrebov
Copy link
Member

Fix use of logger before initialization.

Followup on #2261

Fix use of logger before initialization.

Followup on #2261

Signed-off-by: Alexander Yastrebov <alexander.yastrebov@zalando.de>
if ro.Log == nil {
ro.Log = &logging.DefaultLog{}
}

if ro.AddrUpdater != nil {
address, err := ro.AddrUpdater()
for i := 0; i < retryCount; i++ {
Copy link
Member Author

@AlexanderYastrebov AlexanderYastrebov Feb 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should remove this probe from the client - it simply delays creation of the client and client will recover after redis is up. Also client provides RingAvailable method that could be used to check if ring is up.

We also check AddrUpdater once at startup

skipper/skipper.go

Lines 1717 to 1730 in 37c474a

_, err = redisOptions.AddrUpdater()
if err != nil {
log.Errorf("Failed to update redis addresses from kubernetes: %v", err)
return err
}
} else if redisOptions != nil && o.SwarmRedisEndpointsRemoteURL != "" {
log.Infof("Use remote address %s to fetch updates redis shards", o.SwarmRedisEndpointsRemoteURL)
redisOptions.AddrUpdater = getRemoteURLRedisAddrUpdater(o.SwarmRedisEndpointsRemoteURL)
_, err = redisOptions.AddrUpdater()
if err != nil {
log.Errorf("Failed to update redis addresses from URL: %v", err)
return err
}

which might be a bigger problem - imagine permanent redis failure prevents new pods startup leading to cascading failure. I think its better to fail open/close with ratelimits then fail startup when redis is not avaialble.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I get your point but 10 sec extra startup time is not that much and it will take longer with container restart. Maybe we don't need to fail startup but it's still good to have the retries, wdyt?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

which might be a bigger problem - imagine permanent redis failure prevents new pods startup leading to cascading failure. I think its better to fail open/close with ratelimits then fail startup when redis is not avaialble.

I disagree here and you should page if this is a permanent issue. A new bad deployment will block and you get a page.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I get your point but 10 sec extra startup time is not that much and it will take longer with container restart.

Check the code, its just a delay, it only logs an error.

@AlexanderYastrebov AlexanderYastrebov added the minor no risk changes, for example new filters label Feb 16, 2024
@szuecs
Copy link
Member

szuecs commented Feb 19, 2024

👍

1 similar comment
@AlexanderYastrebov
Copy link
Member Author

👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
minor no risk changes, for example new filters
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants