Rustfmt Usage Appveyor

[ThomasdenH]

I have an issue with testing my library on Appveyor using rustfmt. I use the command cargo fmt -- --write-mode=diff on both Travis and Appveyor. For some reason, this command has stopped working on Appveyor with the following error:

error: file not found for module `tests`
  --> \\?\C:\projects\iban-validate\src\lib.rs:54:5
   |
54 | mod tests;
   |     ^^^^^
   |
   = help: name the file either tests.rs or tests/mod.rs inside the directory "\\\\?\\C:\\projects\\iban-validate\\src"

When I remove the line for Appveyor, (keeping it in for Travis), the build succeeds. In the diff you can see that really that line is the only change.

• The failing build log (source)
• The working build (source)

I am sorry if the issue turns out not to be related to rustfmt, but the reason I post it here is the following:

• A nearly identical build script works on Travis
• The directory that is missing really is there in both builds

[nrc]

Can anyone reproduce this locally on Windows?

I'm not sure if this is possible, but is it possible to use the libsyntax branch on Appveyor and see if that works.

[ghost]

Yes, for me the same problem locally:

error: file not found for module `core`
  --> \\?\D:\Projects\emerald-rs\src\lib.rs:34:5
   |
34 | mod core;
   |     ^^^^
   |
   = help: name the file either core.rs or core/mod.rs inside the directory "\\\\?\\D:\\Projects\\emerald-rs\\src"

rustc 1.19.0-nightly (5dfcd85fd 2017-05-19)
binary: rustc
commit-hash: 5dfcd85fd4bae49445383baadf472fbdb414a0e6
commit-date: 2017-05-19
host: x86_64-pc-windows-msvc
release: 1.19.0-nightly
LLVM version: 4.0

> cargo fmt -- --version
0.8.4 ()

[ghost]

After I set the panic:

    /// Returns either a path to a module, or .
    pub fn default_submod_path(id: ast::Ident, dir_path: &Path, codemap: &CodeMap) -> ModulePath
    {
            ...
            (false, false) => {
                panic!(">>>");
                Err(ModulePathError {
                err_msg: format!("file not found for module `{}`", mod_name),
                help_msg: format!("name the file either {} or {} inside the directory {:?}",
                                  default_path_str,
                                  secondary_path_str,
                                  dir_path.display()),
            })}
            ...

thread 'main' panicked at '>>>', C:\Users\dulanov\.cargo\registry\src\github.com-1ecc6299db9ec823\syntex_syntax-0.58.1\src\parse\parser.rs:5317
stack backtrace:
   0: std::sys_common::backtrace::_print
             at C:\projects\rust\src\libstd\sys_common\backtrace.rs:94
   1: std::panicking::default_hook::{{closure}}
             at C:\projects\rust\src\libstd\panicking.rs:354
   2: std::panicking::default_hook
             at C:\projects\rust\src\libstd\panicking.rs:371
   3: std::panicking::rust_panic_with_hook
             at C:\projects\rust\src\libstd\panicking.rs:549
   4: std::panicking::begin_panic<&str>
             at C:\projects\rust\src\libstd\panicking.rs:511
   5: syntex_syntax::parse::parser::Parser::default_submod_path
             at C:\Users\dulanov\.cargo\registry\src\github.com-1ecc6299db9ec823\syntex_syntax-0.58.1\src\parse\parser.rs:5317
   6: syntex_syntax::parse::parser::Parser::submod_path
             at C:\Users\dulanov\.cargo\registry\src\github.com-1ecc6299db9ec823\syntex_syntax-0.58.1\src\parse\parser.rs:5356
   7: syntex_syntax::parse::parser::Parser::parse_item_mod
             at C:\Users\dulanov\.cargo\registry\src\github.com-1ecc6299db9ec823\syntex_syntax-0.58.1\src\parse\parser.rs:5246
   8: syntex_syntax::parse::parser::Parser::parse_item_
             at C:\Users\dulanov\.cargo\registry\src\github.com-1ecc6299db9ec823\syntex_syntax-0.58.1\src\parse\parser.rs:5845
   9: syntex_syntax::parse::parser::Parser::parse_item
             at C:\Users\dulanov\.cargo\registry\src\github.com-1ecc6299db9ec823\syntex_syntax-0.58.1\src\parse\parser.rs:6023
  10: syntex_syntax::parse::parser::Parser::parse_mod_items
             at C:\Users\dulanov\.cargo\registry\src\github.com-1ecc6299db9ec823\syntex_syntax-0.58.1\src\parse\parser.rs:5192
  11: syntex_syntax::parse::parser::Parser::parse_crate_mod
             at C:\Users\dulanov\.cargo\registry\src\github.com-1ecc6299db9ec823\syntex_syntax-0.58.1\src\parse\parser.rs:6111
  12: rustfmt::parse_input
             at D:\Projects\rustfmt\src\lib.rs:550
  13: rustfmt::format_input<std::io::stdio::Stdout>
             at D:\Projects\rustfmt\src\lib.rs:592
  14: rustfmt::run
             at D:\Projects\rustfmt\src\lib.rs:653
  15: rustfmt::execute
             at D:\Projects\rustfmt\src\bin\rustfmt.rs:301
  16: rustfmt::main
             at D:\Projects\rustfmt\src\bin\rustfmt.rs:323
  17: panic_unwind::__rust_maybe_catch_panic
             at C:\projects\rust\src\libpanic_unwind\lib.rs:98
  18: std::rt::lang_start
             at C:\projects\rust\src\libstd\rt.rs:52
  19: main
  20: __scrt_common_main_seh
             at f:\dd\vctools\crt\vcstartup\src\startup\exe_common.inl:253
  21: BaseThreadInitThunk

[ghost]

Probably problem with the syntex_syntax crate Parse impl.:

impl<'a> Parser<'a> {
    pub fn new(sess: &'a ParseSess,
               tokens: Vec<TokenTree>,
               directory: Option<Directory>,
               desugar_doc_comments: bool)
               -> Self {
        ...

        let tok = parser.next_tok();
        parser.token = tok.tok;
        parser.span = tok.sp;
        if let Some(directory) = directory {
            parser.directory = directory;
        } else if parser.span != syntax_pos::DUMMY_SP {
            parser.directory.path = PathBuf::from(sess.codemap().span_to_filename(parser.span));
            parser.directory.path.pop();
            println!(">>> {:?}", parser.directory.path);
        }
        parser
    }

>>> "\\\\?\\D:\\Projects\\emerald-rs\\src"

[ghost]

I have reproduced it, that bug came with - c5f07ed

[ghost]

Probably this issue is related to rust-lang/rust#27706, some other explanations you can find here rust-lang/rust#25505.

[ghost]

Is the main problem with c5f07ed that std::fs::metadata returns false for std::fs::canonicalize on Windows?

/// An abstraction over the fs operations used by the Parser.
pub trait FileLoader {
    /// Query the existence of a file.
    fn file_exists(&self, path: &Path) -> bool;

    /// Return an absolute path to a file, if possible.
    fn abs_path(&self, path: &Path) -> Option<PathBuf>;

    /// Read the contents of an UTF-8 file into memory.
    fn read_file(&self, path: &Path) -> io::Result<String>;
}

/// A FileLoader that uses std::fs to load real files.
pub struct RealFileLoader;

impl FileLoader for RealFileLoader {
    fn file_exists(&self, path: &Path) -> bool {
        fs::metadata(path).is_ok()
    }
...

[nrc]

According to the docs both canonicalize and metadata have Windows-specific implementations, so it seems weird that either would not work work. AFAICT from the Rust issues, it looks like the Windows issues were resovled before stabilisation?

@dulanov could try reproducing with the libsyntax branch please? That does not use Syntex, so it might eliminate some causes of the error.

[ghost]

For libsyntax branch:

cargo build, cargo test, cargo build --release > ok, but

> RUST_BACKTRACE=1 ../rustfmt/target/debug/cargo-fmt.exe
D:/Projects/rustfmt/target/debug/cargo-fmt.exe: error while loading shared libraries: ?: cannot open shared object file: No such file or directory

> RUST_BACKTRACE=1 ../rustfmt/target/release/cargo-fmt.exe
D:/Projects/rustfmt/target/release/cargo-fmt.exe: error while loading shared libraries: ?: cannot open shared object file: No such file or directory

[nrc]

@dulanov I think you need to add the directory with the Rust DLLs to your PATH variable. (Or copy the dlls to the working directory). I'm not 100% sure on this because I'm not very familiar with Windows.

[ghost]

@nrc C:\Windows\SysWOW64 but I firstly try the same directory.

[ghost]

yes, it works with copied dlls into target/release.

[nrc]

@dulanov awesome, thanks for checking this.

I think this means that it will be fixed when we move the main branch to being libsyntax.

[ghost]

Great, thanks a lot, but anyway, a fix workaround for current version who need it now:

1. git checkout tags/0.8.4
2. revert changes from c5f07ed
3. cargo install --force

[ThomasdenH]

Great, thank you both!