Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

make date-check more lightweight #1394

Merged
merged 20 commits into from
Aug 2, 2022
Merged
Show file tree
Hide file tree
Changes from 8 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
185 changes: 130 additions & 55 deletions ci/date-check/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,12 @@ use std::{
convert::TryInto as _,
env, fmt, fs,
path::{Path, PathBuf},
str::FromStr,
};

use chrono::{Datelike as _, TimeZone as _, Utc};
use chrono::{Datelike as _, Month, TimeZone as _, Utc};
use glob::glob;
use regex::Regex;
use regex::{Regex, RegexSet};

#[derive(Debug, Copy, Clone, PartialEq, Eq)]
struct Date {
Expand Down Expand Up @@ -35,50 +36,56 @@ impl fmt::Display for Date {
}
}

fn make_date_regex() -> Regex {
Regex::new(
r"(?x) # insignificant whitespace mode
<!--\s*
[dD]ate:\s*
(?P<y>\d{4}) # year
-
(?P<m>\d{2}) # month
\s*-->",
)
.unwrap()
fn make_date_regex() -> Vec<Regex> {
let patterns = [
r"<!--\s+date-check:\s+(\w+)\s+(\d+{4})\s+-->",
r"<!--\s+date-check\s+-->\s+(\w+)\s+(\d+{4})",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think the + after \d is correct (it wasn't there before either):

Suggested change
r"<!--\s+date-check:\s+(\w+)\s+(\d+{4})\s+-->",
r"<!--\s+date-check\s+-->\s+(\w+)\s+(\d+{4})",
r"<!--\s+date-check:\s+(\w+)\s+(\d{4})\s+-->",
r"<!--\s+date-check\s+-->\s+(\w+)\s+(\d{4})",

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, could you combine these regexes into one to avoid all this complexity with making multiple passes? It might be a little more permissive wrt accepted input but I think it'd be better overall.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did this with an earlier iteration, but it moved complexity elsewhere, where I had 4 regex groups (instead of just 2). Is that preferable, or is there a better alternative?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, it's not so bad 2a5b586

];
let set = RegexSet::new(&patterns).unwrap();
set.patterns()
.iter()
.map(|pattern| Regex::new(pattern).unwrap())
.collect()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does this usage of RegexSet do? It seems like has the same result as just returning vec![Regex::new(r1), Regex::new(r2)].

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a confused mess is what it is

}

fn collect_dates_from_file(date_regex: &Regex, text: &str) -> Vec<(usize, Date)> {
let mut line = 1;
let mut end_of_last_cap = 0;
date_regex
.captures_iter(&text)
.map(|cap| {
(
cap.get(0).unwrap().range(),
Date {
year: cap["y"].parse().unwrap(),
month: cap["m"].parse().unwrap(),
},
)
})
.map(|(byte_range, date)| {
line += text[end_of_last_cap..byte_range.end]
.chars()
.filter(|c| *c == '\n')
.count();
end_of_last_cap = byte_range.end;
(line, date)
})
.collect()
fn collect_dates_from_file(date_regexes: &[Regex], text: &str) -> Vec<(usize, Date)> {
let mut output = Vec::new();
for date_regex in date_regexes {
let mut line = 1;
let mut end_of_last_cap = 0;
let results: Vec<_> = date_regex
.captures_iter(text)
.filter_map(|cap| {
if let (Some(year), Some(month)) = (cap.get(2), cap.get(1)) {
let year = year.as_str().parse().expect("year");
let month = Month::from_str(month.as_str())
.expect("month")
.number_from_month();
Some((cap.get(0).expect("all").range(), Date { year, month }))
} else {
None
}
})
.map(|(byte_range, date)| {
line += text[end_of_last_cap..byte_range.end]
.chars()
.filter(|c| *c == '\n')
.count();
end_of_last_cap = byte_range.end;
(line, date)
})
.collect();
output.extend(results);
}
output
camelid marked this conversation as resolved.
Show resolved Hide resolved
}

fn collect_dates(paths: impl Iterator<Item = PathBuf>) -> BTreeMap<PathBuf, Vec<(usize, Date)>> {
let date_regex = make_date_regex();
let date_regexes = make_date_regex();
let mut data = BTreeMap::new();
for path in paths {
let text = fs::read_to_string(&path).unwrap();
let dates = collect_dates_from_file(&date_regex, &text);
let dates = collect_dates_from_file(&date_regexes, &text);
if !dates.is_empty() {
data.insert(path, dates);
}
Expand Down Expand Up @@ -182,61 +189,129 @@ mod tests {

#[test]
fn test_date_regex() {
let regex = make_date_regex();
assert!(regex.is_match("foo <!-- date: 2021-01 --> bar"));
}

#[test]
fn test_date_regex_capitalized() {
let regex = make_date_regex();
assert!(regex.is_match("foo <!-- Date: 2021-08 --> bar"));
let regexes = &make_date_regex();
assert!(regexes[0].is_match("<!-- date-check: jan 2021 -->"));
assert!(regexes[0].is_match("<!-- date-check: january 2021 -->"));
assert!(regexes[0].is_match("<!-- date-check: Jan 2021 -->"));
assert!(regexes[0].is_match("<!-- date-check: January 2021 -->"));
assert!(regexes[1].is_match("<!-- date-check --> jan 2021"));
assert!(regexes[1].is_match("<!-- date-check --> january 2021"));
assert!(regexes[1].is_match("<!-- date-check --> Jan 2021"));
assert!(regexes[1].is_match("<!-- date-check --> January 2021"));
}

#[test]
fn test_collect_dates_from_file() {
let text = "Test1\n<!-- date: 2021-01 -->\nTest2\nFoo<!-- date: 2021-02 \
-->\nTest3\nTest4\nFoo<!-- date: 2021-03 -->Bar\n<!-- date: 2021-04 \
-->\nTest5\nTest6\nTest7\n<!-- date: \n\n2021-05 -->\nTest8
let text = r"
Test1
<!-- date-check: jan 2021 -->
Test2
Foo<!-- date-check: february 2021
-->
Test3
Test4
Foo<!-- date-check: Mar 2021 -->Bar
<!-- date-check: April 2021
-->
Test5
Test6
Test7
<!-- date-check:

may 2021 -->
Test8
Test1
<!-- date-check --> jan 2021
Test2
Foo<!-- date-check
--> february 2021
Test3
Test4
Foo<!-- date-check --> mar 2021 Bar
<!-- date-check
--> apr 2021
Test5
Test6
Test7
<!-- date-check

--> may 2021
Test8 \
";
assert_eq!(
collect_dates_from_file(&make_date_regex(), text),
vec![
(
2,
3,
Date {
year: 2021,
month: 1,
}
),
(
6,
Date {
year: 2021,
month: 2,
}
),
(
9,
Date {
year: 2021,
month: 3,
}
),
(
11,
Date {
year: 2021,
month: 4,
}
),
(
17,
Date {
year: 2021,
month: 5,
}
),
(
20,
Date {
year: 2021,
month: 1,
}
),
(
4,
23,
Date {
year: 2021,
month: 2,
}
),
(
7,
26,
Date {
year: 2021,
month: 3,
}
),
(
8,
28,
Date {
year: 2021,
month: 4,
}
),
(
14,
34,
Date {
year: 2021,
month: 5,
}
),
]
],
);
}
}
2 changes: 1 addition & 1 deletion src/backend/backend-agnostic.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

<!-- toc -->

As of <!-- date: 2021-10 --> October 2021, `rustc_codegen_ssa` provides an
As of <!-- date-check --> October 2021, `rustc_codegen_ssa` provides an
abstract interface for all backends to implement, to allow other codegen
backends (e.g. [Cranelift]).

Expand Down
4 changes: 2 additions & 2 deletions src/backend/updating-llvm.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,8 +66,8 @@ Example PRs look like:

## Feature updates

> Note that this information is as of the time of this writing <!-- date:
2021-10 --> (October 2021). The process for updating LLVM changes with
> Note that this information is as of the time of this writing,
<!-- date-check --> October 2021. The process for updating LLVM changes with
practically all LLVM updates, so this may be out of date!

Unlike bugfixes, updating to pick up a new feature of LLVM typically requires a
Expand Down
2 changes: 1 addition & 1 deletion src/borrow_check/region_inference/member_constraints.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,7 @@ member constraints come in.
## Choices are always lifetime parameters

At present, the "choice" regions from a member constraint are always lifetime
parameters from the current function. As of <!-- date: 2021-10 --> October 2021,
parameters from the current function. As of <!-- date-check --> October 2021,
this falls out from the placement of impl Trait, though in the future it may not
be the case. We take some advantage of this fact, as it simplifies the current
code. In particular, we don't have to consider a case like `'0 member of ['1,
Expand Down
30 changes: 20 additions & 10 deletions src/contributing.md
Original file line number Diff line number Diff line change
Expand Up @@ -437,17 +437,28 @@ Just a few things to keep in mind:
the project.

- The date the comment was added, e.g. instead of writing _"Currently, ..."_
or _"As of now, ..."_, consider writing
_"As of January 2021, ..."_.
Try to format the date as `<MONTH> <YEAR>` to ease search.
tshepang marked this conversation as resolved.
Show resolved Hide resolved
or _"As of now, ..."_,
consider adding the date, in one of the following formats:
- Jan 2021
- January 2021
- jan 2021
- january 2021

- Additionally, include a machine-readable comment of the form `<!-- date:
2022-04 -->` (if the current month is April 2022). We have an automated
tool that uses these (in `ci/date-check`).
There is a CI action (in "~/.github/workflows/date-check.yml")
tshepang marked this conversation as resolved.
Show resolved Hide resolved
that generates a monthly issue with any of these that are over 6 months old.

So, for the month of April 2022, the comment would look like: `As of <!--
date: 2022-04 --> April 2022`. Make sure to put the comment *between* `as of`
and `April 2022`; see [PR #1066][rdg#1066] for the rationale.
For the action to pick the date, add this annotation:

<!-- date-check -->

Example:

As of <!-- date-check --> Jul 2022, the foo did the bar.

For cases where the date should not be part of the visible rendered output,
use the following instead:

<!-- date-check: Jul 2022 -->
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: could you use fenced code blocks and mark it as markdown?


- A link to a relevant WG, tracking issue, `rustc` rustdoc page, or similar, that may provide
further explanation for the change process or a way to verify that the information is not
Expand All @@ -459,7 +470,6 @@ Just a few things to keep in mind:

[rdg]: https://rustc-dev-guide.rust-lang.org/
[rdgrepo]: https://github.com/rust-lang/rustc-dev-guide
[rdg#1066]: https://github.com/rust-lang/rustc-dev-guide/pull/1066

## Issue Triage

Expand Down
4 changes: 2 additions & 2 deletions src/conventions.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,14 +14,14 @@ special config, so this may result in different style from normal [`rustfmt`].
Therefore, formatting this repository using `cargo fmt` is not recommended.

Instead, formatting should be done using `./x.py fmt`. It's a good habit to run
`./x.py fmt` before every commit, as this reduces conflicts later.
`./x.py fmt` before every commit, as this reduces conflicts later.

Formatting is checked by the `tidy` script. It runs automatically when you do
`./x.py test` and can be run in isolation with `./x.py fmt --check`.

If you want to use format-on-save in your editor, the pinned version of
`rustfmt` is built under `build/<target>/stage0/bin/rustfmt`. You'll have to
pass the <!-- date: 2022-04 --> `--edition=2021` argument yourself when calling
pass the <!-- date-check: April 2022 --> `--edition=2021` argument yourself when calling
`rustfmt` directly.

[fmt]: https://github.com/rust-dev-tools/fmt-rfcs
Expand Down
2 changes: 1 addition & 1 deletion src/crates-io.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ reasons:
- The dependency may have transitive dependencies that have one of the above
problems.

As of <!-- date: 2022-02 --> February 2022, there is no official policy for vetting
As of <!-- date-check --> February 2022, there is no official policy for vetting
new dependencies to the compiler. Generally, new dependencies are not added
to the compiler unless there is a good reason to do so.

Expand Down
2 changes: 1 addition & 1 deletion src/diagnostics/diagnostic-items.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ A new diagnostic item can be added with these two steps:
For the naming conventions of diagnostic items, please refer to
[*Naming Conventions*](#naming-conventions).

2. As of <!-- date: 2022-02 --> February 2022, diagnostic items in code are
2. As of <!-- date-check --> February 2022, diagnostic items in code are
accessed via symbols in [`rustc_span::symbol::sym`]. To add your newly
created diagnostic item simply open the module file and add the name (In
this case `Cat`) at the correct point in the list.
Expand Down
2 changes: 1 addition & 1 deletion src/diagnostics/lintstore.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ default lint level and other metadata come from. These are normally defined by
way of the [`declare_lint!`] macro, which boils down to a static with type
`&rustc_session::lint::Lint`.

As of <!-- date: 2022-02 --> February 2022, we lint against direct declarations
As of <!-- date-check --> February 2022, we lint against direct declarations
without the use of the macro today (although this may change in the future, as
the macro is somewhat unwieldy to add new fields to, like all macros).

Expand Down
2 changes: 1 addition & 1 deletion src/diagnostics/translation.md
Original file line number Diff line number Diff line change
Expand Up @@ -217,7 +217,7 @@ returned by `Emitter::fluent_bundle`. This bundle is used preferentially when
translating messages, the fallback bundle is only used if the primary bundle is
missing a message or not provided.

As of <!-- date: 2022-06 --> June 2022, there are no locale bundles
As of <!-- date-check --> June 2022, there are no locale bundles
distributed with the compiler, but mechanisms are implemented for loading
bundles.

Expand Down
2 changes: 1 addition & 1 deletion src/git.md
Original file line number Diff line number Diff line change
Expand Up @@ -157,7 +157,7 @@ no changes added to commit (use "git add" and/or "git commit -a")
These changes are not changes to files: they are changes to submodules (more on
this [later](#git-submodules)). To get rid of those, run `git submodule update`
(or run any `x.py` command, which will automatically update the submodules).
Note that there is (as of <!-- date: 2022-02 --> February 2022) a [bug][#77620] if you use
Note that there is (as of <!-- date-check --> February 2022) a [bug][#77620] if you use
worktrees, submodules, and `x.py` in a commit hook. If you run into an error
like:

Expand Down
Loading