Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ensure lookup_source_file_idx has a sorted list of source files to use #115328

Closed
wants to merge 1 commit into from
Closed

Ensure lookup_source_file_idx has a sorted list of source files to use #115328

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions compiler/rustc_span/src/lib.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@
#![feature(round_char_boundary)]
#![feature(read_buf)]
#![feature(new_uninit)]
#![feature(is_sorted)]
#![deny(rustc::untranslatable_diagnostic)]
#![deny(rustc::diagnostic_outside_of_impl)]
#![allow(internal_features)]
Expand Down
47 changes: 36 additions & 11 deletions compiler/rustc_span/src/source_map.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -190,6 +190,11 @@ pub struct SourceMap {
/// The address space below this value is currently used by the files in the source map.
used_address_space: AtomicU32,

/// A list of source files starting positions along with their index in `self.files` sorted
/// by the starting positions. This is used for fast lookup of a position in
/// `lookup_source_file_idx`.
sorted_files: Lock<Vec<(BytePos, usize)>>,

files: RwLock<SourceMapFiles>,
file_loader: IntoDynSyncSend<Box<dyn FileLoader + Sync + Send>>,
// This is used to apply the file path remapping as specified via
Expand All @@ -216,6 +221,7 @@ impl SourceMap {
) -> SourceMap {
SourceMap {
used_address_space: AtomicU32::new(0),
sorted_files: Default::default(),
files: Default::default(),
file_loader: IntoDynSyncSend(file_loader),
path_mapping,
Expand Down Expand Up @@ -327,17 +333,39 @@ impl SourceMap {
// the ID we generate for the SourceFile we just created.
debug_assert_eq!(StableSourceFileId::new(&source_file), file_id);

let mut files = self.files.borrow_mut();

files.source_files.push(source_file.clone());
files.stable_id_to_source_file.insert(file_id, source_file.clone());
self.insert_source_file(&source_file, file_id);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was wondering if we could keep the same behaviour by locking files between allocate_address_space and the insertion. For instance:

  • allocate_address_space locks files, returns both the last file's end pos and the lock guard;
  • this line finalises the push to files and releases the lock.

This would get rid of used_address_space at the same time.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That would require holding the lock during SourceFile::new, which is expensive and rather complex.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also this moves lookup_source_file_idx from RwLock to Lock which is probably better performance wise. Not sure how hot it is for regular compilation though.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at SourceFile::new, I understand that it indexes all the lines in the file using the result of allocate_address_space.
Would using relative indexing inside the file avoid this dependency:

  • all indices in-file are done wrt. the beginning of the file;
  • the contents of SourceFile get independent of the absolute byte pos;
  • we can assign the start byte pos after SourceFile::new returns.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that would work, but I'm not looking to change a lot of source related function now.

SourceFile / SourceMap is very much due a refactoring to remove the mutable state, but it probably makes sense to wait until we can use the query system there.


source_file
}
};
Ok(lrc_sf)
}

fn insert_source_file(&self, file: &Lrc<SourceFile>, file_id: StableSourceFileId) {
let file_idx = {
let mut files = self.files.borrow_mut();

let file_idx = files.source_files.len();
files.source_files.push(file.clone());
files.stable_id_to_source_file.insert(file_id, file.clone());

file_idx
};

{
let mut sorted_files = self.sorted_files.lock();
let idx = sorted_files.partition_point(|&(start_pos, _)| start_pos < file.start_pos);
sorted_files.insert(idx, (file.start_pos, file_idx));
debug_assert!(sorted_files.iter().map(|&(start_pos, _)| start_pos).is_sorted());
}

// Ensure both start and end belong to the new source file
debug_assert!(
Lrc::as_ptr(&self.lookup_byte_offset(file.start_pos).sf) == Lrc::as_ptr(file)
);
debug_assert!(Lrc::as_ptr(&self.lookup_byte_offset(file.end_pos).sf) == Lrc::as_ptr(file));
}

/// Allocates a new `SourceFile` representing a source file from an external
/// crate. The source code of such an "imported `SourceFile`" is not available,
/// but we still know enough to generate accurate debuginfo location
Expand Down Expand Up @@ -411,12 +439,7 @@ impl SourceMap {
cnum,
});

let mut files = self.files.borrow_mut();

files.source_files.push(source_file.clone());
files
.stable_id_to_source_file
.insert(StableSourceFileId::new(&source_file), source_file.clone());
self.insert_source_file(&source_file, StableSourceFileId::new(&source_file));

source_file
}
Expand Down Expand Up @@ -1082,7 +1105,9 @@ impl SourceMap {
/// This index is guaranteed to be valid for the lifetime of this `SourceMap`,
/// since `source_files` is a `MonotonicVec`
pub fn lookup_source_file_idx(&self, pos: BytePos) -> usize {
self.files.borrow().source_files.partition_point(|x| x.start_pos <= pos) - 1
let sorted_files = self.sorted_files.lock();
let idx = sorted_files.partition_point(|&(start_pos, _)| start_pos <= pos) - 1;
sorted_files[idx].1
}

pub fn count_lines(&self) -> usize {
Expand Down