Directory for tool is present, but tool was not found. NixOS linker path issue. Documentation or warning needed (IDFGH-12839)

[ZXTube]

Is your feature request related to a problem?

WARNING: directory for tool xtensa-esp-elf-gdb version 12.1_20221002 is present, but tool was not found
ERROR: tool xtensa-esp-elf-gdb has no installed versions. Please run '/nix/store/8dzgagiwp6xi6hahsdgq6y4kb5v8kn4j-python3-3.11.8/bin/python3 /home/ziyad/esp/esp-idf/tools/idf_tools.py install' to install it.
WARNING: directory for tool riscv32-esp-elf-gdb version 12.1_20221002 is present, but tool was not found
ERROR: tool riscv32-esp-elf-gdb has no installed versions. Please run '/nix/store/8dzgagiwp6xi6hahsdgq6y4kb5v8kn4j-python3-3.11.8/bin/python3 /home/ziyad/esp/esp-idf/tools/idf_tools.py install' to install it.
WARNING: directory for tool xtensa-esp32-elf version esp-12.2.0_20230208 is present, but tool was not found
ERROR: tool xtensa-esp32-elf has no installed versions. Please run '/nix/store/8dzgagiwp6xi6hahsdgq6y4kb5v8kn4j-python3-3.11.8/bin/python3 /home/ziyad/esp/esp-idf/tools/idf_tools.py install' to install it.
WARNING: directory for tool xtensa-esp32s2-elf version esp-12.2.0_20230208 is present, but tool was not found
ERROR: tool xtensa-esp32s2-elf has no installed versions. Please run '/nix/store/8dzgagiwp6xi6hahsdgq6y4kb5v8kn4j-python3-3.11.8/bin/python3 /home/ziyad/esp/esp-idf/tools/idf_tools.py install' to install it.
WARNING: directory for tool xtensa-esp32s3-elf version esp-12.2.0_20230208 is present, but tool was not found
ERROR: tool xtensa-esp32s3-elf has no installed versions. Please run '/nix/store/8dzgagiwp6xi6hahsdgq6y4kb5v8kn4j-python3-3.11.8/bin/python3 /home/ziyad/esp/esp-idf/tools/idf_tools.py install' to install it.
WARNING: directory for tool riscv32-esp-elf version esp-12.2.0_20230208 is present, but tool was not found
ERROR: tool riscv32-esp-elf has no installed versions. Please run '/nix/store/8dzgagiwp6xi6hahsdgq6y4kb5v8kn4j-python3-3.11.8/bin/python3 /home/ziyad/esp/esp-idf/tools/idf_tools.py install' to install it.
WARNING: directory for tool esp32ulp-elf version 2.35_20220830 is present, but tool was not found
ERROR: tool esp32ulp-elf has no installed versions. Please run '/nix/store/8dzgagiwp6xi6hahsdgq6y4kb5v8kn4j-python3-3.11.8/bin/python3 /home/ziyad/esp/esp-idf/tools/idf_tools.py install' to install it.
WARNING: directory for tool openocd-esp32 version v0.12.0-esp32-20230921 is present, but tool was not found
ERROR: tool openocd-esp32 has no installed versions. Please run '/nix/store/8dzgagiwp6xi6hahsdgq6y4kb5v8kn4j-python3-3.11.8/bin/python3 home/username/esp/esp-idf/tools/idf_tools.py install' to install it.

This is the output I get after running idf_tools.py install then export.sh.
This is happening mainly because I'm on NixOS and files aren't in their usual location. I don't know what else could cause this
And trying to run the binary files for example openocd only outputs cannot execute: required file not found which isn't at all helpful
And it took me way to much time to find out how to fix this.

Describe the solution you'd like.

So it would be very helpful if maybe documentation was added for that, or a warning is given when running idf_tools.py install in a nix environment or on NixOS

Currently how I solved this was just running patchelf on each binary file in these tools and setting the interpreter then adding into rpath any error it gives me for a not found library

e.g.
patchelf --set-interpreter /nix/store/qn3ggz5sf3hkjs2c797xf7nan3amdxmp-glibc-2.38-27/lib/ld-linux-x86-64.so.2 openocd
./openocd --> ./openocd: error while loading shared libraries: libudev.so.1: cannot open shared object file: No such file or directory
find -iname 'libudev.so.1' /nix/store pick anything that is found
patchelf --set-rpath "/nix/store/2m7fcpn68c5pq6zby22xddsv5hwx1x67-steam-fhs/usr/lib64/" openocd
./openocd --> output indicating it's working Open On-Chip Debu...

I have no idea if there are better ways to solve this and I have little experience so anything that would at least inform people if they have the same problem I have would be very appreciated.

Thanks in advance.

Describe alternatives you've considered.

No response

Additional context.

No response

[dobairoland]

I'm not entirely sure but maybe https://github.com/mirrexagon/nixpkgs-esp-dev exist for this. Does this help you?

Also there is this Wiki page: https://nixos.wiki/wiki/ESP-IDF

As for ESP-IDF, we don't support oficially NixOS .

[ZXTube]

Yes this github repository worked like a charm I don't know how I didn't find it, But thank you very much. It fixed other problems I was having.

Would it be possible to add a warning in maybe the installation part of the documentation for NixOS?

[dobairoland]

Would it be possible to add a warning in maybe the installation part of the documentation for NixOS?

We try to keep it as simple as possible for new users. But this might change if more of our users will start to use NixOS.

[dobairoland]

@ZXTube Could you help me a little, please?

This is the output I get after running idf_tools.py install then export.sh.

Did you run export.sh already in nix-shell? I'm wondering how could we detect that the script is run in NixOS. Checking the presence of the IN_NIX_SHELL environment variable is my current idea.

[ZXTube]

No, I didn't need to enter any nix shell while installing it.

Do you want to detect if the person is specially using NixOS or also if using nix on another OS?
These are the environment variables I have in NixOS outside of a nix-shell:
$NIX_GSETTINGS_OVERRIDES_DIR, $NIX_PATH, $NIX_PROFILES, $NIXOS_XDG_DESKTOP_PORTAL_CONFIG_DIR, $NIXPKGS_CONFIG, $NIX_USER_PROFILE_DIR

I think $NIX_PATH is there in NixOS and nix-shell (on a different OS)

[fhrbata]

Hi,

Yes, it seems that NIX_PATH could be a way.

docker run --rm -it  nixos/nix bash -c 'set | grep NIX'
BASH_EXECUTION_STRING='set | grep NIX'
NIX_PATH=/nix/var/nix/profiles/per-user/root/channels:/root/.nix-defexpr/channels
NIX_SSL_CERT_FILE=/nix/var/nix/profiles/default/etc/ssl/certs/ca-bundle.crt

Or maybe just check if nix command is available or /nix/store directory.
https://nix.dev/manual/nix/2.22/store/types/local-store

It is also possible, but not recommended, to change the "logical" location of the Nix store from its default of /nix/store.

I have no hands on experience with the nix package manager though.

[dobairoland]

@ZXTube @fhrbata Thank you both for your help!