Git sources of child crates are not properly used

[andir]

When building a package from a git source where the git repository contains multiple crates the nix expression (sometimes?) doesn't know about the child directory for dependency in relative child directories.

Example:

Imagine the following section in your Cargo.toml:

[dependencies.rocksdb]
git = "https://github.com/rust-rocksdb/rust-rocksdb"
rev = "bc8520a86e96c38f79caac0d09349a01cbd113e9"
branch = "0.12.2"
default-features = false
features = ["lz4"]

Then looking further into the Cargo.toml of the rocksdb crate:

[dependencies]
libc = "0.2"
librocksdb-sys = { path = "librocksdb-sys", version = "5.18.3" }

(https://github.com/rust-rocksdb/rust-rocksdb/blob/bc8520a86e96c38f79caac0d09349a01cbd113e9/Cargo.toml#L26-L28)

We see that the librocksdb-sys crate is supposed to be at the relative path librocksdb-sys within the rocksdb crate source. When we now look at the definition of librocksdb-sys in the generated Cargo.nix we see the following:

      "librocksdb-sys …"
        = rec {
          crateName = "librocksdb-sys";
          version = "5.18.3";                                                                                                                                                                                                                  
          edition = "2015";                                                                                                                                                                                                                    
          src = builtins.fetchgit {                                                                                                                                                                                                                    
            url = "https://github.com/rust-rocksdb/rust-rocksdb";                                                                                                                                                                        
            rev = "bc8520a86e96c38f79caac0d09349a01cbd113e9";                                                                                                                                                                                  
            sha256 = "…";                                                                                                                                                                   
          };                                                                                                                                                                                                              
          authors = […];                                                                                                                                                                                                                                   
          dependencies = {…};                                                                                                                                                                                                                                   
          buildDependencies = {…};                                                                                                                                                                                                                                   
          features = {                                                                                                                                                                                                                         
            "default" = [ "static" ];                                                                                                                                                                                                          
          };                                                                                                                                                                                                                                   
          resolvedDefaultFeatures = [ "default" "lz4" "static" ];                                                                                                                                                                              
        };                   

What is missing here is the child directory that actually hosts the crate librocksdb-sys. By changing the src attribute to

src = (builtins.fetchgit {                                                                                                                                                                                                                    
            url = "https://github.com/rust-rocksdb/rust-rocksdb";                                                                                                                                                                        
            rev = "bc8520a86e96c38f79caac0d09349a01cbd113e9";                                                                                                                                                                                  
            sha256 = "…";                                                                                                                                                                   
          }) + "/librocksdb-sys";

we can workaround the issue.

I spent some time looking at how cargo handles those internally. It seems like each directory is cloned into your local "db" (in cargo speech?) at ~/.cargo/git/checkouts/<repo>-<somehash>/<git short ref>/. In the above example the package_path of our package then is ~/.cargo/git/checkouts/<repo>-<somehash>/<git short ref>/librocksdb-sys/.

The choices we have here seem to be limited. The trivial approach to fixing this would be to collect path elements from the package_path until the short id has been found. The collected paths can then be used as a suffix of the git cloned sources.

[kolloch]

Thank you for the excellent description of the problem and the analysis! I will look if I have enough info in the cargo metadata output to fix this easily.

[kolloch]

Solving #53 might also solve this.

[kolloch]

I verified my fix against this with an integration tests. I wait until the relevant PR (See #53) hits nixpkgs-unstable. I close this and track this further in #53.