installation and building issues

[Goldziher]

Hi there,

I tried to install this library as a python dependency - buts its not possible because the parser is not in place:

pdm add "git+https://github.com/latex-lsp/tree-sitter-latex.git@master"
Adding packages to default dependencies: git+https://github.com/latex-lsp/tree-sitter-latex.git@master
🔒 Lock successful
Changes are written to pyproject.toml.
Synchronizing working set with resolved packages: 1 to add, 0 to update, 0 to remove

  ✖ Install tree-sitter-latex 0.3.0 failed
Retry failed jobs
  ✖ Install tree-sitter-latex 0.3.0 failed

ERRORS:
add tree-sitter-latex failed:
Traceback (most recent call last):
  File
"/opt/homebrew/Cellar/python@3.12/3.12.4/Frameworks/Python.framework/Versions/3.12/lib/python3.12/concurrent/futures/thread
.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/pdm/2.16.1/libexec/lib/python3.12/site-packages/pdm/installers/synchronizers.py", line 282, in
install_candidate
    self.manager.install(can)
  File "/opt/homebrew/Cellar/pdm/2.16.1/libexec/lib/python3.12/site-packages/pdm/installers/manager.py", line 33, in
install
    prepared.build(),
    ^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/pdm/2.16.1/libexec/lib/python3.12/site-packages/pdm/models/candidates.py", line 418, in build
    self._cached = Path(builder.build(build_dir, metadata_directory=self._metadata_dir))
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/pdm/2.16.1/libexec/lib/python3.12/site-packages/pdm/builders/wheel.py", line 24, in build
    filename = self._hook.build_wheel(out_dir, self.config_settings, metadata_directory)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/pdm/2.16.1/libexec/lib/python3.12/site-packages/pyproject_hooks/_impl.py", line 256, in
build_wheel
    return self._call_hook(
           ^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/pdm/2.16.1/libexec/lib/python3.12/site-packages/pyproject_hooks/_impl.py", line 392, in
_call_hook
    self._subprocess_runner(
  File "/opt/homebrew/Cellar/pdm/2.16.1/libexec/lib/python3.12/site-packages/pdm/builders/base.py", line 255, in
subprocess_runner
    return log_subprocessor(cmd, cwd, extra_environ=env)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/pdm/2.16.1/libexec/lib/python3.12/site-packages/pdm/builders/base.py", line 106, in
log_subprocessor
    raise build_error(e) from None
pdm.exceptions.BuildError: Build backend raised error: Showing the last 10 lines of the build output:
copying bindings/python/tree_sitter_latex/__init__.pyi -> build/lib.macosx-14.2-arm64-cpython-312/tree_sitter_latex
copying bindings/python/tree_sitter_latex/binding.c -> build/lib.macosx-14.2-arm64-cpython-312/tree_sitter_latex
copying bindings/python/tree_sitter_latex/py.typed -> build/lib.macosx-14.2-arm64-cpython-312/tree_sitter_latex
running build_ext
building '_binding' extension
clang -fno-strict-overflow -Wsign-compare -Wunreachable-code -DNDEBUG -g -O3 -Wall -DPy_LIMITED_API=0x03080000
-DPY_SSIZE_T_CLEAN -Isrc -I/Users/naamanhirschfeld/workspace/py-tree-sitter-languages-fork/.venv/include
-I/Users/naamanhirschfeld/.pyenv/versions/3.12.2/include/python3.12 -c bindings/python/tree_sitter_latex/binding.c -o
build/temp.macosx-14.2-arm64-cpython-312/bindings/python/tree_sitter_latex/binding.o -std=c11
clang -fno-strict-overflow -Wsign-compare -Wunreachable-code -DNDEBUG -g -O3 -Wall -DPy_LIMITED_API=0x03080000
-DPY_SSIZE_T_CLEAN -Isrc -I/Users/naamanhirschfeld/workspace/py-tree-sitter-languages-fork/.venv/include
-I/Users/naamanhirschfeld/.pyenv/versions/3.12.2/include/python3.12 -c src/parser.c -o
build/temp.macosx-14.2-arm64-cpython-312/src/parser.o -std=c11
clang: error: no such file or directory: 'src/parser.c'
clang: error: no input files
error: command '/usr/bin/clang' failed with exit code 1

I can clone the repo locally, generate the parser using the tree-sitter-cli and build the bindings from a vendor directory, but this in turn creates a failure on windows machines - i am myself building a library with linux/macos/windows wheels. The tree-sitter-cli fails on windows with the following:

pdm.termui: git clone --depth=1 --branch=master https://github.com/latex-lsp/tree-sitter-latex vendor\latex
pdm.termui: Cloning into 'vendor\latex'...
pdm.termui: tree-sitter generate
pdm.termui: Traceback (most recent call last):
pdm.termui:   File "C:\Users\RUNNER~1\AppData\Local\Temp\pdm-build-env-136vjg1i-shared\Lib\site-packages\setuptools\_distutils\spawn.py", line 57, in spawn
pdm.termui:     proc = subprocess.Popen(cmd, env=env)
pdm.termui:   File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\subprocess.py", line 951, in __init__
pdm.termui:     self._execute_child(args, executable, preexec_fn, close_fds,
pdm.termui:   File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\subprocess.py", line 1420, in _execute_child
pdm.termui:     hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
pdm.termui: FileNotFoundError: [WinError 2] The system cannot find the file specified
pdm.termui: 
pdm.termui: The above exception was the direct cause of the following exception:
pdm.termui: 
pdm.termui: Traceback (most recent call last):
pdm.termui:   File "C:\Users\RUNNER~1\AppData\Local\Temp\pdm-build-env-136vjg1i-shared\Lib\site-packages\setuptools\command\editable_wheel.py", line 153, in run
pdm.termui:     self._create_wheel_file(bdist_wheel)
pdm.termui:   File "C:\Users\RUNNER~1\AppData\Local\Temp\pdm-build-env-136vjg1i-shared\Lib\site-packages\setuptools\command\editable_wheel.py", line 355, in _create_wheel_file
pdm.termui:     files, mapping = self._run_build_commands(dist_name, unpacked, lib, tmp)
pdm.termui:   File "C:\Users\RUNNER~1\AppData\Local\Temp\pdm-build-env-136vjg1i-shared\Lib\site-packages\setuptools\command\editable_wheel.py", line 278, in _run_build_commands
pdm.termui:     self._run_build_subcommands()
pdm.termui:   File "C:\Users\RUNNER~1\AppData\Local\Temp\pdm-build-env-136vjg1i-shared\Lib\site-packages\setuptools\command\editable_wheel.py", line 305, in _run_build_subcommands
pdm.termui:     self.run_command(name)
pdm.termui:   File "C:\Users\RUNNER~1\AppData\Local\Temp\pdm-build-env-136vjg1i-shared\Lib\site-packages\setuptools\_distutils\cmd.py", line 316, in run_command
pdm.termui:     self.distribution.run_command(command)
pdm.termui:   File "C:\Users\RUNNER~1\AppData\Local\Temp\pdm-build-env-136vjg1i-shared\Lib\site-packages\setuptools\dist.py", line 976, in run_command
pdm.termui:     super().run_command(command)
pdm.termui:   File "C:\Users\RUNNER~1\AppData\Local\Temp\pdm-build-env-136vjg1i-shared\Lib\site-packages\setuptools\_distutils\dist.py", line 988, in run_command
pdm.termui:     cmd_obj.run()
pdm.termui:   File "C:\Users\RUNNER~1\AppData\Local\Temp\pdm-build-env-136vjg1i-shared\Lib\site-packages\setuptools\command\build_ext.py", line 93, in run
pdm.termui:     _build_ext.run(self)
pdm.termui:   File "C:\Users\RUNNER~1\AppData\Local\Temp\pdm-build-env-136vjg1i-shared\Lib\site-packages\setuptools\_distutils\command\build_ext.py", line 359, in run
pdm.termui:     self.build_extensions()
pdm.termui:   File "C:\Users\RUNNER~1\AppData\Local\Temp\pdm-build-env-136vjg1i-shared\Lib\site-packages\setuptools\_distutils\command\build_ext.py", line 479, in build_extensions
pdm.termui:     self._build_extensions_serial()
pdm.termui:   File "C:\Users\RUNNER~1\AppData\Local\Temp\pdm-build-env-136vjg1i-shared\Lib\site-packages\setuptools\_distutils\command\build_ext.py", line 505, in _build_extensions_serial
pdm.termui:     self.build_extension(ext)
pdm.termui:   File "<string>", line 105, in build_extension
pdm.termui:   File "C:\Users\RUNNER~1\AppData\Local\Temp\pdm-build-env-136vjg1i-shared\Lib\site-packages\setuptools\_distutils\cmd.py", line 388, in spawn
pdm.termui:     spawn(cmd, search_path, dry_run=self.dry_run)
pdm.termui:   File "C:\Users\RUNNER~1\AppData\Local\Temp\pdm-build-env-136vjg1i-shared\Lib\site-packages\setuptools\_distutils\spawn.py", line 63, in spawn
pdm.termui:     raise DistutilsExecError(f"command {cmd!r} failed: {exc.args[-1]}") from exc
pdm.termui: distutils.errors.DistutilsExecError: command 'tree-sitter' failed: None
pdm.termui: C:\Users\RUNNER~1\AppData\Local\Temp\pdm-build-env-136vjg1i-shared\Lib\site-packages\setuptools\_distutils\dist.py:988: _DebuggingTips: Problem in editable installation.

It would be great if you guys could include generated output in src. Alternatively, it would be great if you can publish the bindings to pypi and allow installation from there.

Thanks