Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support getting file metadata on Windows #4067

Open
wants to merge 27 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
92c5f26
Refactor `Handle` slightly
CraftSpider Dec 2, 2024
36c9063
Implement trivial file operations - opening and closing handles. Just…
CraftSpider Dec 2, 2024
1409c81
Fix MAIN_THREAD handle in windows_join_main
CraftSpider Dec 2, 2024
c13a7c8
Try fix for Windows paths on non-Windows machines
CraftSpider Dec 2, 2024
beb13c9
Most review comments - still missing shim tests
CraftSpider Dec 6, 2024
7426d64
Don't leak miri implementation details
CraftSpider Dec 6, 2024
efae1e9
Fix clippy
CraftSpider Dec 6, 2024
0b89004
Test file creation and metadata shims directly
CraftSpider Dec 6, 2024
e8f834d
Move windows-fs to pass-dep and use windows_sys
CraftSpider Dec 8, 2024
fdc23ea
Move FdNum to shims/files.rs, use it in FdTable definitions
CraftSpider Dec 8, 2024
4de4a97
Slightly improve flag handling - parse and validate in one place
CraftSpider Dec 8, 2024
28db729
Fixup imports, compile
CraftSpider Dec 8, 2024
386075b
Make metadata handle store the metadata, instead of just a path. Add …
CraftSpider Dec 8, 2024
9e94fb8
Improve extract_windows_epoch impl
CraftSpider Dec 8, 2024
ecd9e08
Improve extract_windows_epoch impl comments
CraftSpider Dec 8, 2024
d825b1e
Add invalid handle encoding test
CraftSpider Dec 8, 2024
23452ec
Add tests for CREATE_ALWAYS and OPEN_ALWAYS error behavior. Add comme…
CraftSpider Dec 8, 2024
f6d2c76
Extract Windows epoch helpers from GetSystemTimeAsFileTime and use th…
CraftSpider Dec 8, 2024
0225c21
Merge FileHandle implementation between Unix and Windows
CraftSpider Dec 9, 2024
090d7ea
Use u32::MAX constant
CraftSpider Dec 13, 2024
e432714
Some fs improvements
CraftSpider Dec 16, 2024
89ce6d5
Use FdNum more places
CraftSpider Dec 16, 2024
ce74e21
Apply review comments
CraftSpider Jan 30, 2025
7fd7210
Fix build
CraftSpider Jan 30, 2025
75e8284
Fix CreateNew/CreateAlways reversal
CraftSpider Feb 12, 2025
b508a0e
set last error to 0 on function start
CraftSpider Feb 12, 2025
fa3b49a
Update Cargo.lock
CraftSpider Feb 12, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ measureme = "11"
chrono = { version = "0.4.38", default-features = false }
chrono-tz = "0.10"
directories = "5"
bitflags = "2.6"

# Copied from `compiler/rustc/Cargo.toml`.
# But only for some targets, it fails for others. Rustc configures this in its CI, but we can't
Expand Down
115 changes: 105 additions & 10 deletions src/shims/files.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
use std::any::Any;
use std::collections::BTreeMap;
use std::io::{IsTerminal, SeekFrom, Write};
use std::fs::{File, Metadata};
use std::io::{IsTerminal, Seek, SeekFrom, Write};
use std::marker::CoercePointee;
use std::ops::Deref;
use std::rc::{Rc, Weak};
Expand Down Expand Up @@ -192,7 +193,7 @@ pub trait FileDescription: std::fmt::Debug + FileDescriptionExt {
false
}

fn as_unix(&self) -> &dyn UnixFileDescription {
fn as_unix<'tcx>(&self, _ecx: &MiriInterpCx<'tcx>) -> &dyn UnixFileDescription {
panic!("Not a unix file descriptor: {}", self.name());
}
}
Expand Down Expand Up @@ -278,6 +279,97 @@ impl FileDescription for io::Stderr {
}
}

#[derive(Debug)]
pub struct FileHandle {
pub(crate) file: File,
pub(crate) writable: bool,
}

impl FileDescription for FileHandle {
fn name(&self) -> &'static str {
"file"
}

fn read<'tcx>(
self: FileDescriptionRef<Self>,
communicate_allowed: bool,
ptr: Pointer,
len: usize,
ecx: &mut MiriInterpCx<'tcx>,
finish: DynMachineCallback<'tcx, Result<usize, IoError>>,
) -> InterpResult<'tcx> {
assert!(communicate_allowed, "isolation should have prevented even opening a file");

let result = ecx.read_from_host(&self.file, len, ptr)?;
finish.call(ecx, result)
}

fn write<'tcx>(
self: FileDescriptionRef<Self>,
communicate_allowed: bool,
ptr: Pointer,
len: usize,
ecx: &mut MiriInterpCx<'tcx>,
finish: DynMachineCallback<'tcx, Result<usize, IoError>>,
) -> InterpResult<'tcx> {
assert!(communicate_allowed, "isolation should have prevented even opening a file");

let result = ecx.write_to_host(&self.file, len, ptr)?;
finish.call(ecx, result)
}

fn seek<'tcx>(
&self,
communicate_allowed: bool,
offset: SeekFrom,
) -> InterpResult<'tcx, io::Result<u64>> {
assert!(communicate_allowed, "isolation should have prevented even opening a file");
interp_ok((&mut &self.file).seek(offset))
}

fn close<'tcx>(
self,
communicate_allowed: bool,
_ecx: &mut MiriInterpCx<'tcx>,
) -> InterpResult<'tcx, io::Result<()>> {
assert!(communicate_allowed, "isolation should have prevented even opening a file");
// We sync the file if it was opened in a mode different than read-only.
if self.writable {
// `File::sync_all` does the checks that are done when closing a file. We do this to
// to handle possible errors correctly.
let result = self.file.sync_all();
// Now we actually close the file and return the result.
drop(self.file);
interp_ok(result)
} else {
// We drop the file, this closes it but ignores any errors
// produced when closing it. This is done because
// `File::sync_all` cannot be done over files like
// `/dev/urandom` which are read-only. Check
// https://github.com/rust-lang/miri/issues/999#issuecomment-568920439
// for a deeper discussion.
drop(self.file);
interp_ok(Ok(()))
}
}

fn metadata<'tcx>(&self) -> InterpResult<'tcx, io::Result<Metadata>> {
interp_ok(self.file.metadata())
}

fn is_tty(&self, communicate_allowed: bool) -> bool {
communicate_allowed && self.file.is_terminal()
}

fn as_unix<'tcx>(&self, ecx: &MiriInterpCx<'tcx>) -> &dyn UnixFileDescription {
assert!(
ecx.target_os_is_unix(),
"unix file operations are only available for unix targets"
);
self
}
}

/// Like /dev/null
#[derive(Debug)]
pub struct NullOutput;
Expand All @@ -300,10 +392,13 @@ impl FileDescription for NullOutput {
}
}

/// Internal type of a file-descriptor - this is what [`FdTable`] expects
pub type FdNum = i32;
CraftSpider marked this conversation as resolved.
Show resolved Hide resolved

/// The file descriptor table
#[derive(Debug)]
pub struct FdTable {
pub fds: BTreeMap<i32, DynFileDescriptionRef>,
pub fds: BTreeMap<FdNum, DynFileDescriptionRef>,
/// Unique identifier for file description, used to differentiate between various file description.
next_file_description_id: FdId,
}
Expand Down Expand Up @@ -339,21 +434,21 @@ impl FdTable {
}

/// Insert a new file description to the FdTable.
pub fn insert_new(&mut self, fd: impl FileDescription) -> i32 {
pub fn insert_new(&mut self, fd: impl FileDescription) -> FdNum {
let fd_ref = self.new_ref(fd);
self.insert(fd_ref)
}

pub fn insert(&mut self, fd_ref: DynFileDescriptionRef) -> i32 {
pub fn insert(&mut self, fd_ref: DynFileDescriptionRef) -> FdNum {
self.insert_with_min_num(fd_ref, 0)
}

/// Insert a file description, giving it a file descriptor that is at least `min_fd_num`.
pub fn insert_with_min_num(
&mut self,
file_handle: DynFileDescriptionRef,
min_fd_num: i32,
) -> i32 {
min_fd_num: FdNum,
) -> FdNum {
// Find the lowest unused FD, starting from min_fd. If the first such unused FD is in
// between used FDs, the find_map combinator will return it. If the first such unused FD
// is after all other used FDs, the find_map combinator will return None, and we will use
Expand All @@ -379,16 +474,16 @@ impl FdTable {
new_fd_num
}

pub fn get(&self, fd_num: i32) -> Option<DynFileDescriptionRef> {
pub fn get(&self, fd_num: FdNum) -> Option<DynFileDescriptionRef> {
let fd = self.fds.get(&fd_num)?;
Some(fd.clone())
}

pub fn remove(&mut self, fd_num: i32) -> Option<DynFileDescriptionRef> {
pub fn remove(&mut self, fd_num: FdNum) -> Option<DynFileDescriptionRef> {
self.fds.remove(&fd_num)
}

pub fn is_fd_num(&self, fd_num: i32) -> bool {
pub fn is_fd_num(&self, fd_num: FdNum) -> bool {
self.fds.contains_key(&fd_num)
}
}
Expand Down
36 changes: 26 additions & 10 deletions src/shims/time.rs
Original file line number Diff line number Diff line change
Expand Up @@ -219,16 +219,8 @@ pub trait EvalContextExt<'tcx>: crate::MiriInterpCxExt<'tcx> {

let filetime = this.deref_pointer_as(LPFILETIME_op, this.windows_ty_layout("FILETIME"))?;

let NANOS_PER_SEC = this.eval_windows_u64("time", "NANOS_PER_SEC");
let INTERVALS_PER_SEC = this.eval_windows_u64("time", "INTERVALS_PER_SEC");
let INTERVALS_TO_UNIX_EPOCH = this.eval_windows_u64("time", "INTERVALS_TO_UNIX_EPOCH");
let NANOS_PER_INTERVAL = NANOS_PER_SEC / INTERVALS_PER_SEC;
let SECONDS_TO_UNIX_EPOCH = INTERVALS_TO_UNIX_EPOCH / INTERVALS_PER_SEC;

let duration = system_time_to_duration(&SystemTime::now())?
+ Duration::from_secs(SECONDS_TO_UNIX_EPOCH);
let duration_ticks = u64::try_from(duration.as_nanos() / u128::from(NANOS_PER_INTERVAL))
.map_err(|_| err_unsup_format!("programs running more than 2^64 Windows ticks after the Windows epoch are not supported"))?;
let duration = this.system_time_since_windows_epoch(&SystemTime::now())?;
let duration_ticks = this.windows_ticks_for(duration)?;

let dwLowDateTime = u32::try_from(duration_ticks & 0x00000000FFFFFFFF).unwrap();
let dwHighDateTime = u32::try_from((duration_ticks & 0xFFFFFFFF00000000) >> 32).unwrap();
Expand Down Expand Up @@ -280,6 +272,30 @@ pub trait EvalContextExt<'tcx>: crate::MiriInterpCxExt<'tcx> {
interp_ok(Scalar::from_i32(-1)) // Return non-zero on success
}

#[allow(non_snake_case, clippy::arithmetic_side_effects)]
fn system_time_since_windows_epoch(&self, time: &SystemTime) -> InterpResult<'tcx, Duration> {
let this = self.eval_context_ref();

let INTERVALS_PER_SEC = this.eval_windows_u64("time", "INTERVALS_PER_SEC");
let INTERVALS_TO_UNIX_EPOCH = this.eval_windows_u64("time", "INTERVALS_TO_UNIX_EPOCH");
let SECONDS_TO_UNIX_EPOCH = INTERVALS_TO_UNIX_EPOCH / INTERVALS_PER_SEC;

interp_ok(system_time_to_duration(time)? + Duration::from_secs(SECONDS_TO_UNIX_EPOCH))
}

#[allow(non_snake_case, clippy::arithmetic_side_effects)]
fn windows_ticks_for(&self, duration: Duration) -> InterpResult<'tcx, u64> {
let this = self.eval_context_ref();

let NANOS_PER_SEC = this.eval_windows_u64("time", "NANOS_PER_SEC");
let INTERVALS_PER_SEC = this.eval_windows_u64("time", "INTERVALS_PER_SEC");
let NANOS_PER_INTERVAL = NANOS_PER_SEC / INTERVALS_PER_SEC;

let ticks = u64::try_from(duration.as_nanos() / u128::from(NANOS_PER_INTERVAL))
.map_err(|_| err_unsup_format!("programs running more than 2^64 Windows ticks after the Windows epoch are not supported"))?;
interp_ok(ticks)
}

fn mach_absolute_time(&self) -> InterpResult<'tcx, Scalar> {
let this = self.eval_context_ref();

Expand Down
7 changes: 4 additions & 3 deletions src/shims/unix/fd.rs
Original file line number Diff line number Diff line change
Expand Up @@ -121,7 +121,8 @@ pub trait EvalContextExt<'tcx>: crate::MiriInterpCxExt<'tcx> {
throw_unsup_format!("unsupported flags {:#x}", op);
};

let result = fd.as_unix().flock(this.machine.communicate(), parsed_op)?;
let result = fd.as_unix(this).flock(this.machine.communicate(), parsed_op)?;
drop(fd);
// return `0` if flock is successful
let result = result.map(|()| 0i32);
interp_ok(Scalar::from_i32(this.try_unwrap_io_result(result)?))
Expand Down Expand Up @@ -273,7 +274,7 @@ pub trait EvalContextExt<'tcx>: crate::MiriInterpCxExt<'tcx> {
let Ok(offset) = u64::try_from(offset) else {
return this.set_last_error_and_return(LibcError("EINVAL"), dest);
};
fd.as_unix().pread(communicate, offset, buf, count, this, finish)?
fd.as_unix(this).pread(communicate, offset, buf, count, this, finish)?
}
};
interp_ok(())
Expand Down Expand Up @@ -333,7 +334,7 @@ pub trait EvalContextExt<'tcx>: crate::MiriInterpCxExt<'tcx> {
let Ok(offset) = u64::try_from(offset) else {
return this.set_last_error_and_return(LibcError("EINVAL"), dest);
};
fd.as_unix().pwrite(communicate, buf, count, offset, this, finish)?
fd.as_unix(this).pwrite(communicate, buf, count, offset, this, finish)?
}
};
interp_ok(())
Expand Down
94 changes: 3 additions & 91 deletions src/shims/unix/fs.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,9 @@

use std::borrow::Cow;
use std::fs::{
DirBuilder, File, FileType, Metadata, OpenOptions, ReadDir, read_dir, remove_dir, remove_file,
rename,
DirBuilder, File, FileType, OpenOptions, ReadDir, read_dir, remove_dir, remove_file, rename,
};
use std::io::{self, ErrorKind, IsTerminal, Read, Seek, SeekFrom, Write};
use std::io::{self, ErrorKind, Read, Seek, SeekFrom, Write};
use std::path::{Path, PathBuf};
use std::time::SystemTime;

Expand All @@ -14,98 +13,11 @@ use rustc_data_structures::fx::FxHashMap;

use self::shims::time::system_time_to_duration;
use crate::helpers::check_min_vararg_count;
use crate::shims::files::{EvalContextExt as _, FileDescription, FileDescriptionRef};
use crate::shims::files::FileHandle;
use crate::shims::os_str::bytes_to_os_str;
use crate::shims::unix::fd::{FlockOp, UnixFileDescription};
use crate::*;

#[derive(Debug)]
struct FileHandle {
file: File,
writable: bool,
}

impl FileDescription for FileHandle {
fn name(&self) -> &'static str {
"file"
}

fn read<'tcx>(
self: FileDescriptionRef<Self>,
communicate_allowed: bool,
ptr: Pointer,
len: usize,
ecx: &mut MiriInterpCx<'tcx>,
finish: DynMachineCallback<'tcx, Result<usize, IoError>>,
) -> InterpResult<'tcx> {
assert!(communicate_allowed, "isolation should have prevented even opening a file");

let result = ecx.read_from_host(&self.file, len, ptr)?;
finish.call(ecx, result)
}

fn write<'tcx>(
self: FileDescriptionRef<Self>,
communicate_allowed: bool,
ptr: Pointer,
len: usize,
ecx: &mut MiriInterpCx<'tcx>,
finish: DynMachineCallback<'tcx, Result<usize, IoError>>,
) -> InterpResult<'tcx> {
assert!(communicate_allowed, "isolation should have prevented even opening a file");

let result = ecx.write_to_host(&self.file, len, ptr)?;
finish.call(ecx, result)
}

fn seek<'tcx>(
&self,
communicate_allowed: bool,
offset: SeekFrom,
) -> InterpResult<'tcx, io::Result<u64>> {
assert!(communicate_allowed, "isolation should have prevented even opening a file");
interp_ok((&mut &self.file).seek(offset))
}

fn close<'tcx>(
self,
communicate_allowed: bool,
_ecx: &mut MiriInterpCx<'tcx>,
) -> InterpResult<'tcx, io::Result<()>> {
assert!(communicate_allowed, "isolation should have prevented even opening a file");
// We sync the file if it was opened in a mode different than read-only.
if self.writable {
// `File::sync_all` does the checks that are done when closing a file. We do this to
// to handle possible errors correctly.
let result = self.file.sync_all();
// Now we actually close the file and return the result.
drop(self.file);
interp_ok(result)
} else {
// We drop the file, this closes it but ignores any errors
// produced when closing it. This is done because
// `File::sync_all` cannot be done over files like
// `/dev/urandom` which are read-only. Check
// https://github.com/rust-lang/miri/issues/999#issuecomment-568920439
// for a deeper discussion.
drop(self.file);
interp_ok(Ok(()))
}
}

fn metadata<'tcx>(&self) -> InterpResult<'tcx, io::Result<Metadata>> {
interp_ok(self.file.metadata())
}

fn is_tty(&self, communicate_allowed: bool) -> bool {
communicate_allowed && self.file.is_terminal()
}

fn as_unix(&self) -> &dyn UnixFileDescription {
self
}
}

impl UnixFileDescription for FileHandle {
fn pread<'tcx>(
&self,
Expand Down
Loading
Loading