Template not found / permission denied on Windows

[pawamoy]

Trying to generate a project from a template that works fine on Linux, but fails on Windows:

Something went wrong. Removing destination folder.
Traceback (most recent call last):
  File "c:\users\user\.local\pipx\venvs\copier\lib\site-packages\copier\main.py", line 142, in copy
    copy_local(conf=conf)
  File "c:\users\user\.local\pipx\venvs\copier\lib\site-packages\copier\main.py", line 190, in copy_local
    render_file(conf, rel_path, source_path, render, must_skip)
  File "c:\users\user\.local\pipx\venvs\copier\lib\site-packages\copier\main.py", line 325, in render_file
    content = render(src_path)
  File "c:\users\user\.local\pipx\venvs\copier\lib\site-packages\copier\tools.py", line 158, in __call__
    tmpl = self.env.get_template(relpath)
  File "c:\users\user\.local\pipx\venvs\copier\lib\site-packages\jinja2\environment.py", line 883, in get_template
    return self._load_template(name, self.make_globals(globals))
  File "c:\users\user\.local\pipx\venvs\copier\lib\site-packages\jinja2\environment.py", line 857, in _load_template
    template = self.loader.load(self, name, globals)
  File "c:\users\user\.local\pipx\venvs\copier\lib\site-packages\jinja2\loaders.py", line 115, in load
    source, filename, uptodate = self.get_source(environment, name)
  File "c:\users\user\.local\pipx\venvs\copier\lib\site-packages\jinja2\loaders.py", line 177, in get_source
    pieces = split_template_path(template)
  File "c:\users\user\.local\pipx\venvs\copier\lib\site-packages\jinja2\loaders.py", line 32, in split_template_path
    raise TemplateNotFound(template)
jinja2.exceptions.TemplateNotFound: project\.gitignore.jinja

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\users\user\appdata\local\programs\python\python36\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "c:\users\user\appdata\local\programs\python\python36\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\user\.local\bin\copier.exe\__main__.py", line 7, in <module>
  File "c:\users\user\.local\pipx\venvs\copier\lib\site-packages\plumbum\cli\application.py", line 577, in run
    inst, retcode = subapp.run(argv, exit=False)
  File "c:\users\user\.local\pipx\venvs\copier\lib\site-packages\plumbum\cli\application.py", line 572, in run
    retcode = inst.main(*tailargs)
  File "c:\users\user\.local\pipx\venvs\copier\lib\site-packages\copier\cli.py", line 17, in _wrapper
    return method(*args, **kwargs)
  File "c:\users\user\.local\pipx\venvs\copier\lib\site-packages\copier\cli.py", line 158, in main
    self.parent._copy(template_src, destination_path)
  File "c:\users\user\.local\pipx\venvs\copier\lib\site-packages\copier\cli.py", line 121, in _copy
    **kwargs,
  File "c:\users\user\.local\pipx\venvs\copier\lib\site-packages\copier\main.py", line 150, in copy
    shutil.rmtree(conf.src_path)
  File "c:\users\user\appdata\local\programs\python\python36\lib\shutil.py", line 500, in rmtree
    return _rmtree_unsafe(path, onerror)
  File "c:\users\user\appdata\local\programs\python\python36\lib\shutil.py", line 390, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  File "c:\users\user\appdata\local\programs\python\python36\lib\shutil.py", line 390, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  File "c:\users\user\appdata\local\programs\python\python36\lib\shutil.py", line 390, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  File "c:\users\user\appdata\local\programs\python\python36\lib\shutil.py", line 395, in _rmtree_unsafe
    onerror(os.unlink, fullname, sys.exc_info())
  File "c:\users\user\appdata\local\programs\python\python36\lib\shutil.py", line 393, in _rmtree_unsafe
    os.unlink(fullname)
PermissionError: [WinError 5] Accès refusé: 'C:\\Users\\user\\AppData\\Local\\Temp\\1\\copier.vcs.clone.dmtng7gz\\.git\\objects\\pack\\pack-764d5af3bf8b77b96e71aba712eca189b4cc8c51.idx'

I'll try to give more details later.

[pawamoy]

Hi again, so, I don't understand why the Jinja env cannot find the template. I don't see any hardcoded forward slash in the code (Linux path separators), so it should handle Windows paths correctly. In the same way, we don't pass any Path object to Jinja (which caused problems in the past on Windows), only str, and any way this was fixed in later version of Jinja, and Copier depends on a recent (if not the latest) version of Jinja.

About the permisison error, I saw that happen in other projects on Windows, such as Cruft, which also does git clone then deletes the temporary cloned repositories. I would say that there is a discrepancy between the git subprocess permission mask and the Python/Copier one, causing a permission error when trying to shutil.rmtree the directory created by git.

Here is the cruft issue: cruft/cruft#13
EDIT: ah, there's actually a PR and an explanation here: https://github.com/timothycrosley/cruft/pull/20/files#diff-d423f4246f0ee9d4dccb189702a1f0d3R35

[yajo]

Hi! I've #224 open which aims to make the test suite pass on windows amd macosx, but given I don't use them, all I have to debug is the GH actions machine and it's evolving slowly (also because I don't really need that TBH 😅).

But yes, I really want Copier to work reliably across platforms.

On windows, have you tried running it under WSL? Do you think you could lend me a hand in that PR?

[pawamoy]

Of course, I don't use Windows myself personally, only at work. I can test out things for you if you want. Unfortunately I don't have access/permission to use WSL on my workstation 😕

[yajo]

Well, if you can help that'd be great. The sad truth is that until we fix #224 windows/mac support is kinda none... 😢

[pawamoy]

I wanted to run the Copier's test suite on my workstation but it has trouble installing flake8-debugger. This plugin tries to install pytest-runner by downloading it itself, and the URL is blocked by the proxy. I'll remove it from the deps and try again.

EDIT: I'll comment on #224 instead

[yajo]

Fixed in #224. TL;DR: use python 3.8+ on windows.