LSP completions, signatures and definitions missing in shells using flakes

[erikkrieg]

Just tried making a nix shell using rust-overlay as a flake and encountered some issues that impact LSP integrations. In particular, I am using Neovim and not getting LSP completions (except for some imports), signatures or definitions. Going back to using my previous nix shell config that doesn't use flakes and all the mentioned LSP features work again as expected.

This is the flake.nix I used to create the shell that has LSP issues:

{
  description = "Development shell";
  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
    rust-overlay.url = "github:oxalica/rust-overlay";
    flake-utils.url = "github:numtide/flake-utils";
  };
  outputs = { nixpkgs, rust-overlay, flake-utils, ... }:
    flake-utils.lib.eachDefaultSystem (system:
      let
        overlays = [ (import rust-overlay) ];
        pkgs = import nixpkgs {
          inherit system overlays;
        };
      in
      with pkgs;
      {
        devShells.default = mkShell {
          nativeBuildInputs = [
            (rust-bin.selectLatestNightlyWith (toolchain: toolchain.default.override {
              extensions = [ "rust-src" ];
            }))
            rust-analyzer
          ];
        };
      }
    );
}

This is the shell.nix config that does work as expected.

let
  rust_overlay = import (builtins.fetchTarball https://github.com/oxalica/rust-overlay/archive/master.tar.gz);
  pkgs = import <nixpkgs> { overlays = [ rust_overlay ]; };
  rust = pkgs.rust-bin.selectLatestNightlyWith (toolchain: toolchain.default.override {
    extensions = [ "rust-src" ];
  });
in
with pkgs;
mkShell {
  nativeBuildInputs = [
    rust
    rust-analyzer
  ];
}

For some additional info, I am on aarch64-darwin and using nix-direnv to create the shell.

[shekohex]

try export the following:

{
  # ...
  # Environment variables
  RUST_SRC_PATH = "${rust}/lib/rustlib/src/rust/library";
  # ...
}

This should help rust-analyzer to work.

Also, you should use rust-analyzer-unwrapped pkg instead.

[erikkrieg]

Tried switching to rust-analyzer-unwrapped and setting the rust source path.

{
    devShells.default = mkShell {
      nativeBuildInputs = [
        rust
        rust-analyzer-unwrapped
      ];
      shellHook = ''
        export RUST_SRC_PATH="${rust}/lib/rustlib/src/rust/library";
      '';
    };
  }

Getting errors like this in my LSP:

[error][2023-06-04 01:27:32] .../vim/lsp/rpc.lua:734	"rpc"	"/nix/store/m2irjryix3caczyqpk804mdrd0idaslr-rust-analyzer-2023-04-17/bin/rust-analyzer"	"stderr"	"[error rust_analyzer::main_loop] flycheck 0: file with cargo diagnostic not found in vfs: file not found: /nix/store/1jgsai0yh6yawk7jnb07aj18xj8h2gd6-rust-default-1.71.0-nightly-2023-05-27/lib/rustlib/src/rust/library/core/src/iter/traits/iterator.rs\n"

This is just one line from the logs. Similar errors for a number of other files, too.

Something I noticed is that the file that isn't being imported is symlinked. Not sure, but wondering if rust-analyzer is not able to follow to symlinks.

$ readlink /nix/store/1jgsai0yh6yawk7jnb07aj18xj8h2gd6-rust-default-1.71.0-nightly-2023-05-27/lib/rustlib/src/rust/library/core/src/iter/traits/iterator.rs
/nix/store/vp1fqn7kj6gd3znvasdvzvhn7aghnpkh-rust-src-1.71.0-nightly-2023-05-27-aarch64-apple-darwin/lib/rustlib/src/rust/library/core/src/iter/traits/iterator.rs

[erikkrieg]

Not sure if VFS error is a red herring or the actual root problem. I also found this issue on rust-analyzer that seems to be related to the VFS error I see: rust-lang/rust-analyzer#12534

[erikkrieg]

Also seeing some errors like this in the LspLog:

[ERROR][2023-06-04 02:29:41] .../vim/lsp/rpc.lua:734	"rpc"	"/nix/store/m2irjryix3caczyqpk804mdrd0idaslr-rust-analyzer-2023-04-17/bin/rust-analyzer"	"stderr"	'[ERROR rust_analyzer::dispatch] unknown request: Request { id: RequestId(I32(12209)), method: "workspace/executeCommand", params: Object {"command": String("rust-analyzer.triggerParameterHints"), "title": String("triggerParameterHints")} }\n'

[shekohex]

Here is my basic flake.nix file from a project that I'm working on:

{
  description = "<PROJECT_DISC>";
  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";
    flake-utils.url = "github:numtide/flake-utils";
    # Rust
    rust-overlay = {
      url = "github:oxalica/rust-overlay";
      inputs = {
        nixpkgs.follows = "nixpkgs";
        flake-utils.follows = "flake-utils";
      };
    };
  };

  outputs = { self, nixpkgs, rust-overlay, flake-utils }:
    flake-utils.lib.eachDefaultSystem (system:
      let
        overlays = [ (import rust-overlay) ];
        pkgs = import nixpkgs {
          inherit system overlays;
        };
        lib = pkgs.lib;
        toolchain = pkgs.rust-bin.fromRustupToolchainFile ./rust-toolchain.toml;
      in
      {
        devShells.default = pkgs.mkShell {
          name = "<PROJECT_NAME>";
          nativeBuildInputs = [
            pkgs.pkg-config
            pkgs.clang
            # Mold Linker for faster builds (only on Linux)
            (lib.optionals pkgs.stdenv.isLinux pkgs.mold)
          ];
          buildInputs = [
            pkgs.openssl
            # We want the unwrapped version, wrapped comes with nixpkgs' toolchain
            pkgs.rust-analyzer-unwrapped
            # Finally the toolchain
            toolchain
          ];
          packages = [ ];

          # Environment variables
          RUST_SRC_PATH = "${toolchain}/lib/rustlib/src/rust/library";
          LD_LIBRARY_PATH = lib.makeLibraryPath [ pkgs.openssl pkgs.gmp ];
        };
      });
}

• rust-toolchain.toml

[toolchain]
channel = "stable"
components = ["rustfmt", "clippy", "rust-src"]
targets = []

That works very well with my Neovim Setup here, which I'm using rust-tools.nvim

[erikkrieg]

Thanks for sharing this. I can see a few things for myself to try.

[erikkrieg]

Have played around with something very similar but no luck so far. This was the flake at time of writing.. For nvim I didn't make any changes. This is what I'm doing with rust-tools atm.

Are you using your flake-based nix shell on NixOS (or some other distro)? Wondering if there could be something to that.

[shekohex]

Are you using your flake-based nix shell on NixOS (or some other distro)? Wondering if there could be something to that.

Yes, I do most of my development on NixOS, but sometimes on macOS too, However I haven't faced any similar issues. Maybe try on a new project?

[erikkrieg]

I have some NixOS hosts setup with my same nvim configuration. I'll give them a try with the exact nix shell you are using. If the problem still occurs then maybe there is something about the combination of the flake shell and my nvim configuration that isn't playing well together.

[erikkrieg]

New clue: I was able to use the flake successfully on my NixOS intel machine. So it appears as though the problem is Darwin-specific.

[Avimitin]

I got the same problem on my machine, an Arch Linux with Nix standalong installation. I've tried to export the RUST_SRC_PATH, but I found it is not working at all. And I try to list file in the $RUST_SRC_PATH, found that this path is not exist.

Here is my nix flake

{
  description = "Dev shell for setup build environment";

  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";

    flake-utils.url = "github:numtide/flake-utils";
    rust-overlay.url = "github:oxalica/rust-overlay";
  };

  outputs = { self, nixpkgs, flake-utils, rust-overlay }:
    flake-utils.lib.eachDefaultSystem (system:
      let
        overlays = [ (import rust-overlay) ];
        pkgs = import nixpkgs { inherit system overlays; };

        toolchain = pkgs.rust-bin.stable.latest.default;
      in with pkgs; {
        devShells.default = mkShell {
          nativeBuildInputs = [
            clang
            # Use mold when we are runnning in Linux
            (lib.optionals stdenv.isLinux mold)
          ];
          buildInputs = [
            # Including cargo,clippy,cargo-fmt
            toolchain
            # rust-analyzer comes from nixpkgs toolchain, I want the unwrapped version
            rust-analyzer-unwrapped
          ];

          # Some environment to make rust-analyzer work correctly (Still the path prefix issue)
          RUST_SRC_PATH = "${toolchain}/lib/rustlib/src/rust/library";
        };
      });
}

And here is what I got:

$ ls $RUST_SRC_PATH
ls: cannot access '/nix/store/j3d1lcyjy4kp1rj7qwskw1hxjqnbz2yp-rust-default-1.70.0/lib/rustlib/src/rust/library': No such file or directory

And under the ${rust-bin.stable.latest.default}/lib/rustlib, there are only these folders

$ ls /nix/store/j3d1lcyjy4kp1rj7qwskw1hxjqnbz2yp-rust-default-1.70.0/lib/rustlib
etc  x86_64-unknown-linux-gnu

[Avimitin]

Oh I know why I can't get the std completion and why the path is missing, this is because there is no rust-src component by default. To resolve the issue, I have to override the extensions field:

-toolchain = pkgs.rust-bin.stable.latest.default;
+toolchain = pkgs.rust-bin.stable.latest.default.override {
+    extensions = [ "rust-src" ];
+};

And now it works fine.

[erikkrieg]

My problem is something different from that. The rust src is present, but I'm not getting definitions and signatures etc.

[thecaralice]

This works fine for me:

{
  inputs = {
    naersk.url = "github:nix-community/naersk/master";
    nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";
    utils.url = "github:numtide/flake-utils";
    rust-overlay.url = "github:oxalica/rust-overlay";
  };

  outputs = { self, nixpkgs, utils, naersk, rust-overlay }:
    utils.lib.eachDefaultSystem (system:
      let
        pkgs = (import nixpkgs) {
          inherit system;
          overlays = [
            (import rust-overlay)
          ];
        };
        toolchain = (pkgs.rust-bin.fromRustupToolchainFile ./rust-toolchain.toml).override {
          extensions = [ "rust-src" ];
        };
        naersk' = pkgs.callPackage naersk {
          cargo = toolchain;
          rustc = toolchain;
        };
      in
      {
        defaultPackage = naersk'.buildPackage ./.;
        devShell = with pkgs; mkShell {
          nativeBuildInputs = [ rust-analyzer-unwrapped ];
          buildInputs = [ toolchain ];
          RUST_SRC_PATH = "${toolchain}/lib/rustlib/src/rust/library";
        };
      });
}

Maybe it's because rust-analyzer-unwrapped is in nativeBuildInputs

[erikkrieg]

It seems I've found a solution for my issue, although I don't completely understand why I encountered this problem only on MacOS in combination with flakes.

I was tipped off because of a Neovim plugin I have started using more recently that makes language server logs a bit more visible and I noticed that rust_analyzer (RA) was stuck on "roots scanned". Did a little search and came across this: rust-lang/rust-analyzer#8161

The issue mentions RA follows symlinks and potentially tries to traverse large directories, resulting in the language server halting up. Well, nix-direnv creates a .direnv folder in the project directory and it contains symlinks back to the Nix store, which in addition to having many files probably has circular symlinks. This also explains some other things I was seeing with RA: large amounts of CPU usage and RA processes not always terminating after I close my editor.

This is the solution that seems to have worked for me: erikkrieg/envim@3109bd0

What I don't understand is why I only encountered this problem under the following conditions:

1. Using MacOS
2. Using a flake to configure the dev shell

Perhaps the issue lies in Direnv? Maybe the nix-channel I used for nix-shell had a regression not present in the flake input I used for pkgs? In terms of behaviour from Rust Analyzer, I'm surprised it crawls folders that are listed in .gitignore.

So, from what I can tell, this issue isn't associated with rust-overlay.

[shekohex]

That's correct, I faced the same issue way back on macOS, did not know that it will be that issue tho, glad that you got it sorted tho.

https://github.com/shekohex/dotfiles/blob/1d06fb9ff192b5fe241fa5d1e887a84110e5b1cf/modules/programs/neovim/config/plugins/rust-tools.lua#L58-L60