Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Actor class that is exported from driver will not be exported from workers in next Ray session. #4843

Closed
robertnishihara opened this issue May 23, 2019 · 0 comments · Fixed by #4844
Labels
bug Something that is supposed to be working; but isn't

Comments

@robertnishihara
Copy link
Collaborator

robertnishihara commented May 23, 2019

The following script will hang.

import ray

@ray.remote
class Actor(object):
    def method(self):
        pass

@ray.remote
def f(actor_class):
    handle = actor_class.remote()
    ray.get(handle.method.remote())

ray.init(num_cpus=2)

a = Actor.remote()
ray.get(a.method.remote())

ray.shutdown()

ray.init(num_cpus=2)

ray.get(f.remote(Actor))  # This line hangs.

ray.shutdown()

Right now, the Actor actor class has a field _last_export_session (which is a counter), which keeps track of when the actor class definition was last exported (to workers through the GCS). If it was exported in a given Ray session (that is, the span between a call to ray.init and ray.shutdown), then in a subsequent Ray session, the definition will still be exported again from the driver, but not from workers, so in situations where only the workers are in a position to export the definition (as in the example above), it won't get exported.

The trouble with the counter is that the counter on the workers is not set correctly. This can be fixed by using a driver ID instead of a counter.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something that is supposed to be working; but isn't
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant