Skip to content

Commit

Permalink
One semaphore to load them all
Browse files Browse the repository at this point in the history
From MartinH's findings in #544, we had one thread at the
`cspec[:klass].constantize` line while another thread was trying to run
`klass.columns_hash`, causing a
`Circular dependency detected while autoloading...` error.

I then tried a bunch of things until I found a way to reliably recreate
this error:

```ruby
require_relative 'config/environment'

threads = []
4.times do
  threads << Thread.new { Api::Environment.time_attributes }
end
threads.collect(&:join)
```

I could then run the above script several times in my shell and get the
`circular dependency` error most of the time.

```
for x in `seq 1 10`; do; bundle exec ruby test.rb; done
```

With this test in place, I then tried a few solutions:

1) Move the `klass.columns_hash` block into the permit_concurrent_loads
block:

```
diff --git a/lib/api/environment.rb b/lib/api/environment.rb
index 87b34f99..f4b6554a 100644
--- a/lib/api/environment.rb
+++ b/lib/api/environment.rb
@@ -22,9 +22,10 @@ module Api
         # Temporary measure to avoid thread race condition which could lead to a deadlock
         ActiveSupport::Dependencies.interlock.permit_concurrent_loads do
           klass = cspec[:klass].constantize
-        end
-        klass.columns_hash.each do |name, typeobj|
-          result << name if %w(date datetime).include?(typeobj.type.to_s)
+
+          klass.columns_hash.each do |name, typeobj|
+            result << name if %w(date datetime).include?(typeobj.type.to_s)
+          end
         end
       end
     end
```

This did not fix the `circular dependency` error. Perhaps
`permit_concurrent_loads` doesn't handle arbitrarily deep nested autoloads
cross threads?

2) I tried the same solution as what's in this commit (Mutex#synchronize)
but with a Sync.new object and either a `synchronize(:SH)` or EX block and instead
received a thread killed error.

3) Finally, tried Mutex#synchronize and this worked.

I then verified that this last Mutex#synchronize change also fixed the
original problem and it did.
  • Loading branch information
jrafanie committed Jan 28, 2019
1 parent 3d33b03 commit 76a2d68
Showing 1 changed file with 7 additions and 5 deletions.
12 changes: 7 additions & 5 deletions lib/api/environment.rb
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
module Api
class Environment
ONE_AUTOLOADER_LOCK = Mutex.new
def self.url_attributes
@url_attributes ||= Set.new(%w(href))
end
Expand All @@ -19,12 +20,13 @@ def self.time_attributes
next if cspec[:klass].blank?
klass = nil

# Temporary measure to avoid thread race condition which could lead to a deadlock
ActiveSupport::Dependencies.interlock.permit_concurrent_loads do
# Ensure we're the only thread trying to autoload classes and their columns
ONE_AUTOLOADER_LOCK.synchronize do
klass = cspec[:klass].constantize
end
klass.columns_hash.each do |name, typeobj|
result << name if %w(date datetime).include?(typeobj.type.to_s)

klass.columns_hash.each do |name, typeobj|
result << name if %w(date datetime).include?(typeobj.type.to_s)
end
end
end
end
Expand Down

0 comments on commit 76a2d68

Please sign in to comment.