-
Notifications
You must be signed in to change notification settings - Fork 3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
BUG: abspath apparently prepends the drive letter of the current working directory on Windows #23
Conversation
…ing directory on Windows
@jotelha thanks for raising this. I seem to remember seeing something like this when I was working on this a long time ago. Can you show me how the issue manifests when using dtool code rather than urlparse? I guess I'm wondering what the URIs you are passing to dtool look like? |
Here as simple and direct as possible,
With the proper URI scheme The issue then manifests wherever we use The testing venv, fresh today:
|
@jotelha thanks that helped jog my memory. Currently I can't do these tests because I don't have access to a Windows machine with multiple drives. I'm not sure how well your fix would be able to deal with relative paths in Windows. But maybe that is not an issue in that those URI's would not have a drive letter in them? However, I would probably still prefer a different approach to tackling this problem by adding some custom logic at the end to check the drive letter of the generated URI is the same as that of the drive letter, if the platform is Windows. Take lines 86-87 from master:
and update them to be:
What do you think? Would my suggested fix work? |
Ok, that works pretty generously for many cases, even for a few malformed URIs (such as file:C:/Users/admin/dtool) here a thorough testing session C:\Users\admin>venv\2022-03-07-python-3.10\Scripts\activate
(2022-03-07-python-3.10) C:\Users\admin>pip install git+https://github.com/jotelha/dtoolcore
Collecting git+https://github.com/jotelha/dtoolcore
Cloning https://github.com/jotelha/dtoolcore to c:\users\admin\appdata\local\temp\pip-req-build-sjmetgec
Running command git clone --filter=blob:none --quiet https://github.com/jotelha/dtoolcore 'C:\Users\admin\AppData\Local\Temp\pip-req-build-sjmetgec'
Resolved https://github.com/jotelha/dtoolcore to commit 6aff99531d1192f86512f662caf22a6ecd2198a5
Preparing metadata (setup.py) ... done
Using legacy 'setup.py install' for dtoolcore, since package 'wheel' is not installed.
Installing collected packages: dtoolcore
Attempting uninstall: dtoolcore
Found existing installation: dtoolcore 3.18.0
Uninstalling dtoolcore-3.18.0:
Successfully uninstalled dtoolcore-3.18.0
Running setup.py install for dtoolcore ... done
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
dtool 3.26.1 requires dtoolcore==3.18.0, but you have dtoolcore 3.17.0 which is incompatible.
Successfully installed dtoolcore-3.17.0
(2022-03-07-python-3.10) C:\Users\admin>pip install git+https://github.com/jotelha/dtoolcore@2022-02-22-geneours_parse_uri_windows_drive_letter_fix
Collecting git+https://github.com/jotelha/dtoolcore@2022-02-22-geneours_parse_uri_windows_drive_letter_fix
Cloning https://github.com/jotelha/dtoolcore (to revision 2022-02-22-geneours_parse_uri_windows_drive_letter_fix) to c:\users\admin\appdata\local\temp\pip-req-build-nsq7ji7e
Running command git clone --filter=blob:none --quiet https://github.com/jotelha/dtoolcore 'C:\Users\admin\AppData\Local\Temp\pip-req-build-nsq7ji7e'
Running command git checkout -b 2022-02-22-geneours_parse_uri_windows_drive_letter_fix --track origin/2022-02-22-geneours_parse_uri_windows_drive_letter_fix
branch '2022-02-22-geneours_parse_uri_windows_drive_letter_fix' set up to track 'origin/2022-02-22-geneours_parse_uri_windows_drive_letter_fix'.
Switched to a new branch '2022-02-22-geneours_parse_uri_windows_drive_letter_fix'
Resolved https://github.com/jotelha/dtoolcore to commit 757487cd54f367ced292462b7928ac60dbe66e73
Preparing metadata (setup.py) ... done
Using legacy 'setup.py install' for dtoolcore, since package 'wheel' is not installed.
Installing collected packages: dtoolcore
Attempting uninstall: dtoolcore
Found existing installation: dtoolcore 3.17.0
Uninstalling dtoolcore-3.17.0:
Successfully uninstalled dtoolcore-3.17.0
Running setup.py install for dtoolcore ... done
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
dtool 3.26.1 requires dtoolcore==3.18.0, but you have dtoolcore 3.18.1 which is incompatible.
Successfully installed dtoolcore-3.18.1
(2022-03-07-python-3.10) C:\Users\admin>cd dtool
(2022-03-07-python-3.10) C:\Users\admin\dtool>dtool create test-dataset
Created proto dataset file:///C:/Users/admin/dtool/test-dataset
Next steps:
1. Add raw data, eg:
dtool add item my_file.txt file:///C:/Users/admin/dtool/test-dataset
Or use your system commands, e.g:
mv my_data_directory C:\Users\admin\dtool\test-dataset\data/
2. Add descriptive metadata, e.g:
dtool readme interactive file:///C:/Users/admin/dtool/test-dataset
3. Convert the proto dataset into a dataset:
dtool freeze file:///C:/Users/admin/dtool/test-dataset
(2022-03-07-python-3.10) C:\Users\admin\dtool>cd ..
(2022-03-07-python-3.10) C:\Users\admin>dtool ls dtool
*test-dataset
file:///C:/Users/admin/dtool/test-dataset
(2022-03-07-python-3.10) C:\Users\admin>dtool ls ./dtool
*test-dataset
file:///C:/Users/admin/dtool/test-dataset
(2022-03-07-python-3.10) C:\Users\admin>dtool ls C:dtool
*test-dataset
file:///C:/Users/admin/dtool/test-dataset
(2022-03-07-python-3.10) C:\Users\admin>dtool ls C:./dtool
*test-dataset
file:///C:/Users/admin/dtool/test-dataset
(2022-03-07-python-3.10) C:\Users\admin>dtool ls C:.\dtool
*test-dataset
file:///C:/Users/admin/dtool/test-dataset
(2022-03-07-python-3.10) C:\Users\admin>dtool ls .\dtool
*test-dataset
file:///C:/Users/admin/dtool/test-dataset
(2022-03-07-python-3.10) C:\Users\admin>dtool ls C:/Users/admin/dtool
*test-dataset
file:///C:/Users/admin/dtool/test-dataset
(2022-03-07-python-3.10) C:\Users\admin>dtool ls C:\Users\admin\dtool
*test-dataset
file:///C:/Users/admin/dtool/test-dataset
(2022-03-07-python-3.10) C:\Users\admin>dtool ls file:///C:/Users/admin/dtool
*test-dataset
file:///C:/Users/admin/dtool/test-dataset
(2022-03-07-python-3.10) C:\Users\admin>dtool ls file://host/C:/Users/admin/dtool
*test-dataset
file:///C:/Users/admin/dtool/test-dataset
(2022-03-07-python-3.10) C:\Users\admin>dtool ls file:/C:/Users/admin/dtool
*test-dataset
file:///C:/Users/admin/dtool/test-dataset
(2022-03-07-python-3.10) C:\Users\admin>dtool ls file:C:/Users/admin/dtool
*test-dataset
file:///C:/Users/admin/dtool/test-dataset
(2022-03-07-python-3.10) C:\Users\admin>dtool ls bla:C:/Users/admin/dtool
Traceback (most recent call last):
File "C:\Users\admin\venv\2022-03-07-python-3.10\Scripts\dtool-script.py", line 33, in <module>
sys.exit(load_entry_point('dtool-cli==0.7.1', 'console_scripts', 'dtool')())
File "C:\Users\admin\venv\2022-03-07-python-3.10\lib\site-packages\click\core.py", line 1128, in __call__
return self.main(*args, **kwargs)
File "C:\Users\admin\venv\2022-03-07-python-3.10\lib\site-packages\click\core.py", line 1053, in main
rv = self.invoke(ctx)
File "C:\Users\admin\venv\2022-03-07-python-3.10\lib\site-packages\click\core.py", line 1659, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "C:\Users\admin\venv\2022-03-07-python-3.10\lib\site-packages\click\core.py", line 1395, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "C:\Users\admin\venv\2022-03-07-python-3.10\lib\site-packages\click\core.py", line 754, in invoke
return __callback(*args, **kwargs)
File "C:\Users\admin\venv\2022-03-07-python-3.10\lib\site-packages\dtool_info\dataset.py", line 182, in ls
if dtoolcore._is_dataset(uri, CONFIG_PATH):
File "C:\Users\admin\venv\2022-03-07-python-3.10\lib\site-packages\dtoolcore\__init__.py", line 84, in _is_dataset
storage_broker = _get_storage_broker(uri, config_path)
File "C:\Users\admin\venv\2022-03-07-python-3.10\lib\site-packages\dtoolcore\__init__.py", line 62, in _get_storage_broker
StorageBroker = storage_broker_lookup[parsed_uri.scheme]
KeyError: 'bla'
(2022-03-07-python-3.10) C:\Users\admin>W:
(2022-03-07-python-3.10) W:\>dtool ls C:/Users/admin/dtool
*test-dataset
file:///C:/Users/admin/dtool/test-dataset
(2022-03-07-python-3.10) W:\>dtool ls C:\Users\admin\dtool
*test-dataset
file:///C:/Users/admin/dtool/test-dataset
(2022-03-07-python-3.10) W:\>dtool ls file:///C:/Users/admin/dtool
*test-dataset
file:///C:/Users/admin/dtool/test-dataset
(2022-03-07-python-3.10) W:\>dtool ls file:C:/Users/admin/dtool
*test-dataset
file:///C:/Users/admin/dtool/test-dataset
(2022-03-07-python-3.10) W:\>dtool ls bla:C:/Users/admin/dtool
Traceback (most recent call last):
File "C:\Users\admin\venv\2022-03-07-python-3.10\Scripts\dtool-script.py", line 33, in <module>
sys.exit(load_entry_point('dtool-cli==0.7.1', 'console_scripts', 'dtool')())
File "C:\Users\admin\venv\2022-03-07-python-3.10\lib\site-packages\click\core.py", line 1128, in __call__
return self.main(*args, **kwargs)
File "C:\Users\admin\venv\2022-03-07-python-3.10\lib\site-packages\click\core.py", line 1053, in main
rv = self.invoke(ctx)
File "C:\Users\admin\venv\2022-03-07-python-3.10\lib\site-packages\click\core.py", line 1659, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "C:\Users\admin\venv\2022-03-07-python-3.10\lib\site-packages\click\core.py", line 1395, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "C:\Users\admin\venv\2022-03-07-python-3.10\lib\site-packages\click\core.py", line 754, in invoke
return __callback(*args, **kwargs)
File "C:\Users\admin\venv\2022-03-07-python-3.10\lib\site-packages\dtool_info\dataset.py", line 182, in ls
if dtoolcore._is_dataset(uri, CONFIG_PATH):
File "C:\Users\admin\venv\2022-03-07-python-3.10\lib\site-packages\dtoolcore\__init__.py", line 84, in _is_dataset
storage_broker = _get_storage_broker(uri, config_path)
File "C:\Users\admin\venv\2022-03-07-python-3.10\lib\site-packages\dtoolcore\__init__.py", line 62, in _get_storage_broker
StorageBroker = storage_broker_lookup[parsed_uri.scheme]
KeyError: 'bla'
(2022-03-07-python-3.10) W:\>dtool ls C:/Users/admin/dtool
*test-dataset
file:///C:/Users/admin/dtool/test-dataset What won't work is this likely rarely encountered Windows/DOS case of relative paths on another drive, i.e. on C: we are still within
but
What's coincidentally working when on the same drive, but not from another, is this malformed URI,
but we probably do not want to bother about those special cases. |
Fixed in https://pypi.org/project/dtoolcore/3.18.2/, see 90470b2, closing this issue |
I have encountered some unexpected behavior when testing our GUI on Windows, see https://github.com/livMatS/dtool-lookup-gui/blob/master/dtool_lookup_gui/utils/patch.py.
On Windows, urlparse will remove the drive letter and abspath will prepend the drive letter of the current working directory. See for example within a mingw64/msys2 environment:
or just directly Python 3.10 on Windows,
Best