Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cache the result of is_sharded? #311

Merged
merged 2 commits into from
Apr 6, 2023

Conversation

KJTsanaktsidis
Copy link
Contributor

@KJTsanaktsidis KJTsanaktsidis commented Mar 23, 2023

It can't possibly change.... and it's eating ~2% of our app's CPU?????

image

Depending on connection parameters, it seems that the Ticket#show endpoint in Classic is spending ~2% of total CPU time inside this check for is_sharded?

Here's a flamegraph snippet that shows where some of the time is going:

image

Basically, is_sharded? calls shard_names which winds up validating the config by checking e.g. that all the shard numbers are integers every time it's called. That obviously only needs to happen once.

I'm open to caching this at a different level than what I went for, but it seems this might move the needle a bit.

Copy link
Member

@bquorning bquorning left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ActiveRecordShards::Model module is being extended onto ActiveRecord::Base, not included. So it’s already working at the class level, and self.class.foo is just calling methods on the Class class. 😬

You’re probably more looking for a diff like this:

diff --git a/lib/active_record_shards/model.rb b/lib/active_record_shards/model.rb
index 7d3c658..2dadd33 100644
--- a/lib/active_record_shards/model.rb
+++ b/lib/active_record_shards/model.rb
@@ -11,7 +11,9 @@ module ActiveRecordShards
     end
 
     def is_sharded? # rubocop:disable Naming/PredicateName
-      if self == ActiveRecord::Base
+      return @_ars_model_class_is_sharded if defined?(@_ars_model_class_is_sharded)
+
+      @_ars_model_class_is_sharded = if self == ActiveRecord::Base
         sharded != false && supports_sharding?
       elsif self == base_class
         if sharded.nil?

or even…

diff --git a/lib/active_record_shards/model.rb b/lib/active_record_shards/model.rb
index 7d3c658..a9944bd 100644
--- a/lib/active_record_shards/model.rb
+++ b/lib/active_record_shards/model.rb
@@ -11,7 +11,9 @@ module ActiveRecordShards
     end
 
     def is_sharded? # rubocop:disable Naming/PredicateName
-      if self == ActiveRecord::Base
+      return sharded unless sharded.nil?
+
+      self.sharded = if self == ActiveRecord::Base
         sharded != false && supports_sharding?
       elsif self == base_class
         if sharded.nil?

Copy link
Member

@bquorning bquorning left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The method chain supports_sharding?shard_namesconfig_for_envshard_envapp_env which can change during execution (even if it seems unlikely).

I am curious to hear your thoughts on #312 as an alternative (but not quite as optimized) solution.

@KJTsanaktsidis KJTsanaktsidis force-pushed the ktsanaktsidis/optimise_is_sharded branch from c528413 to f66d884 Compare April 5, 2023 05:14
@KJTsanaktsidis
Copy link
Contributor Author

I think your suggestion works @bquorning.

I checked that caching the is_sharded? status & environment doesn't cause any funny load order issues - when I ran classic with eager loading on, I got the same result both with and without this patch:

[1] [zendesk][development] pry(main)> ActiveRecord::Base.descendants.partition(&:is_sharded?).map(&:count)
=> [429, 54]

@KJTsanaktsidis KJTsanaktsidis force-pushed the ktsanaktsidis/optimise_is_sharded branch 2 times, most recently from 51640c4 to fe8627a Compare April 5, 2023 05:34
It can't change (except during tests), so cache it.
@KJTsanaktsidis KJTsanaktsidis force-pushed the ktsanaktsidis/optimise_is_sharded branch from fe8627a to f6cd9ce Compare April 5, 2023 05:37
Sets self.sharded depending on the superclass, if it is not otherwise
set.
@KJTsanaktsidis KJTsanaktsidis force-pushed the ktsanaktsidis/optimise_is_sharded branch from f6cd9ce to bfc99f0 Compare April 5, 2023 06:06
@KJTsanaktsidis KJTsanaktsidis merged commit 5669646 into master Apr 6, 2023
@KJTsanaktsidis KJTsanaktsidis deleted the ktsanaktsidis/optimise_is_sharded branch April 6, 2023 01:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants