Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

extract library #314

Merged
merged 46 commits into from
Feb 21, 2023
Merged
Show file tree
Hide file tree
Changes from 17 commits
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
ae3313e
wip
marcoieni Jan 21, 2023
d3dc47b
wip
marcoieni Jan 21, 2023
edda25a
wip
marcoieni Jan 21, 2023
62286f2
expand match
marcoieni Jan 21, 2023
3777e0a
clippy
marcoieni Jan 21, 2023
039b163
Merge branch 'main' into lib
marcoieni Jan 21, 2023
1d72ef7
clippy
marcoieni Jan 21, 2023
c1ba81a
move match
marcoieni Jan 21, 2023
fb3f9d2
move rev
marcoieni Jan 21, 2023
4a495fd
easier
marcoieni Jan 21, 2023
2ef3486
Update src/lib.rs
marcoieni Jan 22, 2023
7712fae
Merge remote-tracking branch 'origin' into lib
marcoieni Jan 23, 2023
94bd75f
introduce Rustdoc struct
marcoieni Jan 25, 2023
9e9d3c6
adapt code to rustdoc refactor
marcoieni Jan 25, 2023
afe48e1
Merge branch 'main' into lib
marcoieni Jan 25, 2023
b4e9aa6
clippy
marcoieni Jan 25, 2023
b27b7ab
fix
marcoieni Jan 25, 2023
daa8426
inline function
marcoieni Jan 27, 2023
1190770
avoid mixing including and excluded packages
marcoieni Jan 27, 2023
1324805
Merge branch 'main' into lib
marcoieni Jan 27, 2023
9d35505
clippy
marcoieni Jan 27, 2023
c5a7ae8
Merge branch 'main' into lib
marcoieni Jan 27, 2023
b3d22d6
Merge branch 'main' into lib
obi1kenobi Feb 3, 2023
ad9d81c
add derive debug
marcoieni Feb 5, 2023
8363bd8
Update src/lib.rs
marcoieni Feb 5, 2023
368ee5b
add non_exhaustive
marcoieni Feb 5, 2023
f626d7e
non_exhaustive
marcoieni Feb 5, 2023
19ba58f
Update src/lib.rs
marcoieni Feb 5, 2023
ab2dd0a
Update src/lib.rs
marcoieni Feb 5, 2023
eac165e
non_exhaustive
marcoieni Feb 5, 2023
64509d7
fix
marcoieni Feb 5, 2023
9990b69
debug
marcoieni Feb 5, 2023
5a334ba
doc comment
marcoieni Feb 5, 2023
a496d42
rustdoc
marcoieni Feb 5, 2023
bae60ec
Merge branch 'main' into lib
tonowak Feb 5, 2023
b0f7102
Path handling
tonowak Feb 8, 2023
371ccab
remove useless check
marcoieni Feb 8, 2023
72ee617
Fix baseline_root when no baseline-path given and baseline-rev passed
tonowak Feb 8, 2023
2449acd
Fallback to XDG cache dir
tonowak Feb 8, 2023
8485cc5
Renamed registry/version items
tonowak Feb 9, 2023
4988c0a
Merge branch 'main' into lib
obi1kenobi Feb 12, 2023
a6bd76b
Merge branch 'main' into lib
marcoieni Feb 17, 2023
e5da9d8
fmt
marcoieni Feb 18, 2023
22c0081
use directories crate
marcoieni Feb 18, 2023
7516ab0
Added panic in 'unknown' branch
tonowak Feb 20, 2023
09bd405
Merge branch 'main' into lib
obi1kenobi Feb 21, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion src/config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,14 +3,20 @@ use termcolor::{ColorChoice, StandardStream};
use crate::templating::make_handlebars_registry;

#[allow(dead_code)]
pub(crate) struct GlobalConfig {
pub struct GlobalConfig {
level: Option<log::Level>,
is_stderr_tty: bool,
stdout: StandardStream,
stderr: StandardStream,
handlebars: handlebars::Handlebars<'static>,
}

impl Default for GlobalConfig {
fn default() -> Self {
Self::new()
}
}
marcoieni marked this conversation as resolved.
Show resolved Hide resolved

impl GlobalConfig {
pub fn new() -> Self {
let is_stdout_tty = atty::is(atty::Stream::Stdout);
Expand Down
367 changes: 367 additions & 0 deletions src/lib.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,367 @@
#![forbid(unsafe_code)]

mod baseline;
mod check_release;
mod config;
mod dump;
mod manifest;
mod query;
mod templating;
mod util;

pub use config::*;
pub use query::*;

use check_release::run_check_release;
use trustfall_rustdoc::{load_rustdoc, VersionedCrate};

use dump::RustDocCommand;
use itertools::Itertools;
use semver::Version;
use std::collections::HashSet;
use std::path::{Path, PathBuf};

/// Test a release for semver violations.
pub struct Check {
tonowak marked this conversation as resolved.
Show resolved Hide resolved
/// Which packages to analyze.
scope: Scope,
current: Rustdoc,
baseline: Rustdoc,
log_level: Option<log::Level>,
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this different than GlobalConfig.log_level? I'm confused by the duplication.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I decided not to expose GlobalConfig to the user. I'm only exposing the log level.
That's because if the user wants to use cargo-semver-checks as a library, I imagine cargo-semver-checks won't write to stdout or stderr in the future. So it doesn't make sense to allow the user to customize stdout and stderr.

}

pub struct Rustdoc {
source: RustdocSource,
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried to group conflicting cli arguments together in enums.


impl Rustdoc {
/// Use an existing rustdoc file.
pub fn from_path(rustdoc_path: impl Into<PathBuf>) -> Self {
Self {
source: RustdocSource::Rustdoc(rustdoc_path.into()),
}
}

/// Generate the rustdoc file from the project root directory, i.e. the directory containing the crate source.
marcoieni marked this conversation as resolved.
Show resolved Hide resolved
/// It can be a workspace or a single package.
/// Same as `from_git_revision`, but with the current git revision.
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is Rust doc comment style to include or omit the parens when naming functions? I genuinely am not sure, we should probably mirror what std does here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By looking at this page, it seems that the std uses the () convention.
I changed this in a496d42

pub fn from_root(project_root: impl Into<PathBuf>) -> Self {
Self {
source: RustdocSource::Root(project_root.into()),
}
}

/// Generate the rustdoc file from the project at a given git revision.
pub fn from_git_revision(
project_root: impl Into<PathBuf>,
revision: impl Into<String>,
) -> Self {
Self {
source: RustdocSource::Revision(project_root.into(), revision.into()),
}
}

/// Generate the rustdoc file from the largest-numbered non-yanked non-prerelease version
/// published to the cargo registry. If no such version, uses
/// the largest-numbered version including yanked and prerelease versions.
pub fn from_latest_version() -> Self {
Self {
source: RustdocSource::Version(None),
}
}

pub fn from_version(version: impl Into<String>) -> Self {
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We probably want a doc comment here as well.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done in 5a334ba

Self {
source: RustdocSource::Version(Some(version.into())),
}
}

fn registry(&self, config: &mut GlobalConfig) -> anyhow::Result<baseline::RegistryBaseline> {
let manifest_dir = match &self.source {
RustdocSource::Root(manifest_dir) | RustdocSource::Revision(manifest_dir, _) => {
manifest_dir
}
RustdocSource::Version(_) | RustdocSource::Rustdoc(_) => {
anyhow::bail!("not supported yet")
obi1kenobi marked this conversation as resolved.
Show resolved Hide resolved
}
};
let metadata = manifest_metadata_no_deps(manifest_dir)?;
let target = metadata.target_directory.as_std_path().join(util::SCOPE);
let registry = baseline::RegistryBaseline::new(&target, config)?;
Ok(registry)
}
}

enum RustdocSource {
tonowak marked this conversation as resolved.
Show resolved Hide resolved
/// Path to the Rustdoc json file. Use this option when you have already generated the rustdoc file.
marcoieni marked this conversation as resolved.
Show resolved Hide resolved
Rustdoc(PathBuf),
/// Project root directory, i.e. the directory containing the crate source.
/// It can be a workspace or a single package.
Root(PathBuf),
/// Project root directory and Git Revision.
Revision(PathBuf, String),
/// Version from cargo registry to lookup. E.g. "1.0.0".
/// If `None`, uses the largest-numbered non-yanked non-prerelease version
/// published to the cargo registry. If no such version, uses
/// the largest-numbered version including yanked and prerelease versions.
Version(Option<String>),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe it would be better to call it Registry instead of Version? It would make it more clear that it requires internet access to use it and from where this specific version is being taken from.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about RegistryVersion? 😄
The user when seeing Registry with a string might think about Registry("crates.io") for example

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that might be better 👍 .

... but going with this logic, the user might think that in RegistryVersion one has to pass the version of the registry to use, not the version of the crate to compare with.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Naming is hard 😁
VersionFromRegistry is more verbose, but more clear maybe. 🤔

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm fine with either RegistryVersion or VersionFromRegistry. I'm not that worried about people thinking that they need to pass a version of a registry, but I'm willing to accept I may be overly optimistic if you think the other option is better.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you both think about 8485cc5?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's fine :)
I would not try to make the library perfect in this PR. Otherwise we will have to deal with too many merge conflicts.

I guess (and hope) that the library will be extracted in a separate crate, so we can still change the library API in future PRs.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR is on my to-do list to review today. Sorry for the delay! Trustfall ended up in the spotlight (it was GitHub's #​1 most-starred project a couple of days ago) and I've been super busy over there for the last couple of days.

I know I probably just merged another PR that causes another merge conflict, and I'm happy to fix the mess of my own making 😅

}

#[derive(Default)]
struct Scope {
selection: ScopeSelection,
excluded_packages: Vec<String>,
}

/// Which packages to analyze.
#[derive(Default, PartialEq, Eq)]
enum ScopeSelection {
/// Package to process (see `cargo help pkgid`)
Packages(Vec<String>),
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we define explicitly what happens if ScopeSelection::Packages contains a package that's also in excluded_packages? I'm not sure what should happen, or what cargo does assuming the same thing can happen in cargo. But our lib docs should state the expected behavior in any case.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In 1190770 I have made it impossible for this to happen

/// All packages in the workspace. Equivalent to `--workspace`.
Workspace,
/// Default members of the workspace.
#[default]
DefaultMembers,
}

impl Scope {
fn selected_packages<'m>(
obi1kenobi marked this conversation as resolved.
Show resolved Hide resolved
&self,
meta: &'m cargo_metadata::Metadata,
) -> Vec<&'m cargo_metadata::Package> {
let workspace_members: HashSet<_> = meta.workspace_members.iter().collect();
let base_ids: HashSet<_> = match &self.selection {
ScopeSelection::DefaultMembers => {
// Deviating from cargo because Metadata doesn't have default members
let resolve = meta.resolve.as_ref().expect("no-deps is unsupported");
match &resolve.root {
Some(root) => {
let mut base_ids = HashSet::new();
base_ids.insert(root);
base_ids
}
None => workspace_members,
}
}
ScopeSelection::Workspace => workspace_members,
ScopeSelection::Packages(patterns) => {
meta.packages
.iter()
// Deviating from cargo by not supporting patterns
// Deviating from cargo by only checking workspace members
.filter(|p| workspace_members.contains(&p.id) && patterns.contains(&p.name))
.map(|p| &p.id)
.collect()
}
};

meta.packages
.iter()
.filter(|p| base_ids.contains(&p.id) && !self.excluded_packages.contains(&p.name))
.collect()
}
}

impl Check {
pub fn new(current: Rustdoc) -> Self {
Self {
scope: Scope::default(),
current,
baseline: Rustdoc::from_latest_version(),
log_level: Default::default(),
}
}
Copy link
Contributor Author

@marcoieni marcoieni Jan 21, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I used a builder pattern that matches the cli args.
style of builder pattern: https://rust-lang.github.io/api-guidelines/type-safety.html#non-consuming-builders-preferred


pub fn with_workspace(&mut self) -> &mut Self {
self.scope.selection = ScopeSelection::Workspace;
self
}

pub fn with_packages(&mut self, packages: Vec<String>) -> &mut Self {
self.scope.selection = ScopeSelection::Packages(packages);
self
}

pub fn with_excluded_packages(&mut self, excluded_packages: Vec<String>) -> &mut Self {
self.scope.excluded_packages = excluded_packages;
self
}

pub fn with_baseline(&mut self, baseline: Rustdoc) -> &mut Self {
self.baseline = baseline;
self
}

pub fn with_log_level(&mut self, log_level: log::Level) -> &mut Self {
self.log_level = Some(log_level);
self
}

pub fn check_release(&self) -> anyhow::Result<Report> {
let mut config = GlobalConfig::new().set_level(self.log_level);

let loader: Box<dyn baseline::BaselineLoader> = match &self.baseline.source {
RustdocSource::Rustdoc(path) => {
Box::new(baseline::RustdocBaseline::new(path.to_owned()))
}
RustdocSource::Root(root) => Box::new(baseline::PathBaseline::new(root)?),
RustdocSource::Revision(root, rev) => {
let metadata = manifest_metadata_no_deps(root)?;
let source = metadata.workspace_root.as_std_path();
let slug = util::slugify(rev);
let target = metadata
.target_directory
.as_std_path()
.join(util::SCOPE)
.join(format!("git-{slug}"));
Box::new(baseline::GitBaseline::with_rev(
source,
&target,
rev,
&mut config,
)?)
}
RustdocSource::Version(version) => {
let mut registry = self.current.registry(&mut config)?;
if let Some(ver) = version {
let semver = semver::Version::parse(ver)?;
registry.set_version(semver);
}
Box::new(registry)
}
};
let rustdoc_cmd = dump::RustDocCommand::new()
.deps(false)
.silence(!config.is_verbose());

let all_outcomes: Vec<anyhow::Result<bool>> = match &self.current.source {
RustdocSource::Rustdoc(current_rustdoc_path) => {
let name = "<unknown>";
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While developing #207 I've found a bug: when both crates are generated from registry, this code tries to find a package named <unknown> in registry. From how the interface looked, I was convinced that it is enough to pass .with_packages(vec![name]) to Check, but now when I look at the lib.rs, I don't think it's what it's supposed to accept. So, in the scenario "from registry & from registry", there is no way to pass the name of the crate to check.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In #207 I've made a temporary fix 7ecf587 so that I can progress further. In the case where this fix is good enough, I can move it here, otherwise we should discuss better alternatives.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In 7ecf587 it seems like there's still an unknown branch. That looks like it could work if current is given by gitrev, but won't work with a registry version nor with a premade rustdoc file.

Can we add an explicit panic!() in the cases we know for sure won't work, instead of setting a bogus value and then breaking in a confusing way?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 7516ab0.

let version = None;
let (current_crate, baseline_crate) = generate_versioned_crates(
&mut config,
CurrentCratePath::CurrentRustdocPath(current_rustdoc_path),
&*loader,
&rustdoc_cmd,
name,
version,
)?;

let success = run_check_release(&mut config, name, current_crate, baseline_crate)?;
vec![Ok(success)]
}
RustdocSource::Root(project_root) => {
let metadata = manifest_metadata(project_root)?;
let selected = self.scope.selected_packages(&metadata);
selected
.iter()
.map(|selected| {
let manifest_path = selected.manifest_path.as_std_path();
let crate_name = &selected.name;
let version = &selected.version;

let is_implied = self.scope.selection == ScopeSelection::Workspace;
if is_implied && selected.publish == Some(vec![]) {
config.verbose(|config| {
config.shell_status(
"Skipping",
format_args!("{crate_name} v{version} (current)"),
)
})?;
Ok(true)
} else {
config.shell_status(
"Parsing",
format_args!("{crate_name} v{version} (current)"),
)?;

let (current_crate, baseline_crate) = generate_versioned_crates(
&mut config,
CurrentCratePath::ManifestPath(manifest_path),
&*loader,
&rustdoc_cmd,
crate_name,
Some(version),
)?;

Ok(run_check_release(
&mut config,
crate_name,
current_crate,
baseline_crate,
)?)
}
})
.collect()
}
RustdocSource::Revision(_, _) | RustdocSource::Version(_) => todo!(),
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe these are currently unimplemented in the CLI, so I'm fine with leaving a panic in this branch.

But let's make it unimplemented!() with a suitable message instead of a bare todo!(), which makes it seem like we forgot to add it into the PR rather than making an explicit choice to not include it for now.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After the changes from tonowak the todo is not there anymore 👍

};
let success = all_outcomes
.into_iter()
.fold_ok(true, std::ops::BitAnd::bitand)?;

Ok(Report { success })
}
}

pub struct Report {
marcoieni marked this conversation as resolved.
Show resolved Hide resolved
success: bool,
}
marcoieni marked this conversation as resolved.
Show resolved Hide resolved

impl Report {
pub fn success(&self) -> bool {
self.success
}
}

// Argument to the generate_versioned_crates function.
enum CurrentCratePath<'a> {
CurrentRustdocPath(&'a Path), // If rustdoc is passed, it is just loaded into the memory.
ManifestPath(&'a Path), // Otherwise, the function generates the rustdoc.
}

fn generate_versioned_crates(
config: &mut GlobalConfig,
current_crate_path: CurrentCratePath,
loader: &dyn baseline::BaselineLoader,
rustdoc_cmd: &RustDocCommand,
crate_name: &str,
version: Option<&Version>,
) -> anyhow::Result<(VersionedCrate, VersionedCrate)> {
let current_crate = match current_crate_path {
CurrentCratePath::CurrentRustdocPath(rustdoc_path) => load_rustdoc(rustdoc_path)?,
CurrentCratePath::ManifestPath(manifest_path) => {
let rustdoc_path = rustdoc_cmd.dump(manifest_path, None, true)?;
load_rustdoc(&rustdoc_path)?
}
};

// The process of generating baseline rustdoc can overwrite
// the already-generated rustdoc of the current crate.
// For example, this happens when target-dir is specified in `.cargo/config.toml`.
// That's the reason why we're immediately loading the rustdocs into memory.
// See: https://github.com/obi1kenobi/cargo-semver-checks/issues/269
let baseline_path = loader.load_rustdoc(config, rustdoc_cmd, crate_name, version)?;
let baseline_crate = load_rustdoc(&baseline_path)?;

Ok((current_crate, baseline_crate))
}

fn manifest_from_dir(manifest_dir: &Path) -> PathBuf {
manifest_dir.join("Cargo.toml")
}

fn manifest_metadata(manifest_dir: &Path) -> anyhow::Result<cargo_metadata::Metadata> {
let manifest_path = manifest_from_dir(manifest_dir);
let mut command = cargo_metadata::MetadataCommand::new();
let metadata = command.manifest_path(manifest_path).exec()?;
Ok(metadata)
}

fn manifest_metadata_no_deps(manifest_dir: &Path) -> anyhow::Result<cargo_metadata::Metadata> {
let manifest_path = manifest_from_dir(manifest_dir);
let mut command = cargo_metadata::MetadataCommand::new();
let metadata = command.manifest_path(manifest_path).no_deps().exec()?;
Ok(metadata)
}
Loading