Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

repo: index: don't set cache storage if there is no out.cache #10275

Merged
merged 1 commit into from
Feb 1, 2024

Conversation

efiop
Copy link
Contributor

@efiop efiop commented Feb 1, 2024

To avoid situations like

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[5], line 1
----> 1 fs.download(cfg["dataset"]["dvc_config"]["path_in_repo"], "d.parquet")

File ~/.../.venv/lib/python3.11/site-packages/fsspec/spec.py:1546, in AbstractFileSystem.download(self, rpath, lpath, recursive, **kwargs)
   1544 def download(self, rpath, lpath, recursive=False, **kwargs):
   1545     """Alias of `AbstractFileSystem.get`."""
-> 1546     return self.get(rpath, lpath, recursive=recursive, **kwargs)

File ~/.../.venv/lib/python3.11/site-packages/fsspec/spec.py:983, in AbstractFileSystem.get(self, rpath, lpath, recursive, callback, maxdepth, **kwargs)
    981 for lpath, rpath in callback.wrap(zip(lpaths, rpaths)):
    982     callback.branch(rpath, lpath, kwargs)
--> 983     self.get_file(rpath, lpath, **kwargs)

File ~/.../.venv/lib/python3.11/site-packages/dvc/fs/dvc.py:479, in _DVCFileSystem.get_file(self, rpath, lpath, **kwargs)
    476         raise
    478 dvc_path = _get_dvc_path(dvc_fs, subkey)
--> 479 return dvc_fs.get_file(dvc_path, lpath, **kwargs)

File ~/.../.venv/lib/python3.11/site-packages/dvc_objects/fs/base.py:653, in FileSystem.get_file(self, from_info, to_info, callback, **kwargs)
    646 def get_file(
    647     self,
    648     from_info: AnyFSPath,
   (...)
    651     **kwargs,
    652 ) -> None:
--> 653     self.fs.get_file(from_info, to_info, callback=callback, **kwargs)

File ~/.../.venv/lib/python3.11/site-packages/dvc_data/fs.py:201, in DataFileSystem.get_file(self, rpath, lpath, callback, **kwargs)
    198 from dvc_data.index import ObjectStorage
    200 try:
--> 201     typ, storage, cache_storage, hi, fs, path = self._get_fs_path(rpath)
    202 except IsADirectoryError:
    203     os.makedirs(lpath, exist_ok=True)

File ~/.../.venv/lib/python3.11/site-packages/dvc_data/fs.py:112, in DataFileSystem._get_fs_path(self, path)
    110     if not storage:
    111         continue
--> 112     data = storage.get(entry)
    113 except (ValueError, StorageKeyError):
    114     continue

File ~/.../.venv/lib/python3.11/site-packages/dvc_data/index/index.py:214, in ObjectStorage.get(self, entry)
    211 if not entry.hash_info:
    212     raise ValueError
--> 214 return self.odb.fs, self.odb.oid_to_path(entry.hash_info.value)

AttributeError: 'NoneType' object has no attribute 'fs'

Though similar to #10273 it is quite challenging to reproduce in a test, so also waiting for #9296 to use bare repos everywhere and expose similar bugs.

@efiop efiop added the bug Did we break something? label Feb 1, 2024
@efiop efiop self-assigned this Feb 1, 2024
Copy link

codecov bot commented Feb 1, 2024

Codecov Report

Attention: 1 lines in your changes are missing coverage. Please review.

Comparison is base (0bb2f3d) 90.41% compared to head (3a32d90) 90.17%.

Files Patch % Lines
dvc/repo/index.py 50.00% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #10275      +/-   ##
==========================================
- Coverage   90.41%   90.17%   -0.25%     
==========================================
  Files         494      494              
  Lines       37769    37770       +1     
  Branches     5460     5461       +1     
==========================================
- Hits        34150    34059      -91     
- Misses       2987     3056      +69     
- Partials      632      655      +23     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@efiop efiop merged commit 84b2e68 into main Feb 1, 2024
19 checks passed
@efiop efiop deleted the efiop-patch-3 branch February 1, 2024 13:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Did we break something?
Projects
No open projects
Archived in project
Development

Successfully merging this pull request may close these issues.

1 participant