Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Different CachingFileSystem instances compare equal #577

Closed
gerritholl opened this issue Mar 19, 2021 · 2 comments · Fixed by #578
Closed

Different CachingFileSystem instances compare equal #577

gerritholl opened this issue Mar 19, 2021 · 2 comments · Fixed by #578

Comments

@gerritholl
Copy link
Contributor

Different CachingFileSystem instances compare equal, even if they have different properties:

from fsspec.implementations.cached import CachingFileSystem
from fsspec.implementations.local import LocalFileSystem
lfs = LocalFileSystem()
cfs = CachingFileSystem(
        fs=LocalFileSystem(),
        cache_storage="/data/gholl/cache/dummy",
        cache_check=600,
        check_files=False,
        expiry_times=False,
        same_names=False)
cfs2 = CachingFileSystem(
        fs=LocalFileSystem(),
        cache_storage="/data/gholl/cache/dummy2",
        cache_check=900,
        check_files=False,
        expiry_times=True,
        same_names=True)
print(cfs == cfs2)

Running this gives True. I would expect it to give False, because those CachingFileSystems are clearly not equal.

@martindurant
Copy link
Member

It seems that cfs._fs_token is picking up lfs's token. Caching file systems have a custom getattribute for looking up methods/attributed in the cacher as opposed to the target, that's where this will be fixed.

Would you like to work on this?

@gerritholl
Copy link
Contributor Author

Possibly related to this and #576:

In [15]: CachingFileSystem(fs=LocalFileSystem()).to_json()
Out[15]: '{"cls": "fsspec.implementations.local.LocalFileSystem", "protocol": "file", "args": []}'

I will have a look if I can figure out how this should be fixed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants