Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Metadata-Version 2.2 #564

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion Changelog.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,6 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
* Users should migrate away from the old `[package.metadata.maturin]` table of `Cargo.toml` to this new `[project]` table of `pyproject.toml`
* Add PEP 656 musllinux support in [#543](https://github.com/PyO3/maturin/pull/543)
* `--manylinux` is now called `--compatibility` and supports musllinux
* The pure rust install layout changed from just the shared library to a python module that reexports the shared library. This should have now observable consequences for users of the created wheel expect that `my_project.my_project` is now also importable (and equal to just `my_project`)

## 0.10.6 - 2021-05-21

Expand Down
2 changes: 0 additions & 2 deletions Readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -426,5 +426,3 @@ The `sysconfig` folder contains the output of `python -m sysconfig` for differen
You need to install `cffi` and `virtualenv` (`pip install cffi virtualenv`) to run the tests.

There are two optional hacks that can speed up the tests (over 80s to 17s on my machine). By running `cargo build --release --manifest-path test-crates/cargo-mock/Cargo.toml` you can activate a cargo cache avoiding to rebuild the pyo3 test crates with every python version. Delete `target/test-cache` to clear the cache (e.g. after changing a test crate) or remove `test-crates/cargo-mock/target/release/cargo` to deactivate it. By running the tests with the `faster-tests` feature, binaries are stripped and wheels are only stored and not compressed.

You might want to have look into my by now slightly outdated [blog post](https://blog.schuetze.link/2018/07/21/a-dive-into-packaging-native-python-extensions.html) which explains the intricacies of building native python packages.
20 changes: 10 additions & 10 deletions src/build_context.rs
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ use crate::module_writer::write_python_part;
use crate::module_writer::WheelWriter;
use crate::module_writer::{write_bin, write_bindings_module, write_cffi_module};
use crate::source_distribution::source_distribution;
use crate::Metadata21;
use crate::Metadata22;
use crate::PyProjectToml;
use crate::PythonInterpreter;
use crate::Target;
Expand Down Expand Up @@ -125,12 +125,12 @@ pub struct BuildContext {
/// Whether this project is pure rust or rust mixed with python
pub project_layout: ProjectLayout,
/// Python Package Metadata 2.1
pub metadata21: Metadata21,
pub metadata22: Metadata22,
/// The name of the crate
pub crate_name: String,
/// The name of the module can be distinct from the package name, mostly
/// because package names normally contain minuses while module names
/// have underscores. The package name is part of metadata21
/// have underscores. The package name is part of metadata22
pub module_name: String,
/// The path to the Cargo.toml. Required for the cargo invocations
pub manifest_path: PathBuf,
Expand Down Expand Up @@ -191,7 +191,7 @@ impl BuildContext {
Ok(pyproject) => {
let sdist_path = source_distribution(
&self.out,
&self.metadata21,
&self.metadata22,
&self.manifest_path,
&self.cargo_metadata,
pyproject.sdist_include(),
Expand Down Expand Up @@ -237,7 +237,7 @@ impl BuildContext {
let platform = self.target.get_platform_tag(platform_tag, self.universal2);
let tag = format!("cp{}{}-abi3-{}", major, min_minor, platform);

let mut writer = WheelWriter::new(&tag, &self.out, &self.metadata21, &[tag.clone()])?;
let mut writer = WheelWriter::new(&tag, &self.out, &self.metadata22, &[tag.clone()])?;

write_bindings_module(
&mut writer,
Expand Down Expand Up @@ -292,7 +292,7 @@ impl BuildContext {
) -> Result<BuiltWheelMetadata> {
let tag = python_interpreter.get_tag(platform_tag, self.universal2);

let mut writer = WheelWriter::new(&tag, &self.out, &self.metadata21, &[tag.clone()])?;
let mut writer = WheelWriter::new(&tag, &self.out, &self.metadata22, &[tag.clone()])?;

write_bindings_module(
&mut writer,
Expand Down Expand Up @@ -382,7 +382,7 @@ impl BuildContext {
.target
.get_universal_tags(platform_tag, self.universal2);

let mut builder = WheelWriter::new(&tag, &self.out, &self.metadata21, &tags)?;
let mut builder = WheelWriter::new(&tag, &self.out, &self.metadata22, &tags)?;

write_cffi_module(
&mut builder,
Expand Down Expand Up @@ -420,11 +420,11 @@ impl BuildContext {
.target
.get_universal_tags(platform_tag, self.universal2);

if !self.metadata21.scripts.is_empty() {
if !self.metadata22.scripts.is_empty() {
bail!("Defining entrypoints and working with a binary doesn't mix well");
}

let mut builder = WheelWriter::new(&tag, &self.out, &self.metadata21, &tags)?;
let mut builder = WheelWriter::new(&tag, &self.out, &self.metadata22, &tags)?;

match self.project_layout {
ProjectLayout::Mixed {
Expand All @@ -443,7 +443,7 @@ impl BuildContext {
let bin_name = artifact
.file_name()
.expect("Couldn't get the filename from the binary produced by cargo");
write_bin(&mut builder, &artifact, &self.metadata21, bin_name)?;
write_bin(&mut builder, &artifact, &self.metadata22, bin_name)?;

let wheel_path = builder.finish()?;
Ok((wheel_path, "py3".to_string()))
Expand Down
18 changes: 9 additions & 9 deletions src/build_options.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ use crate::cross_compile::{find_sysconfigdata, is_cross_compiling, parse_sysconf
use crate::python_interpreter::InterpreterKind;
use crate::BuildContext;
use crate::CargoToml;
use crate::Metadata21;
use crate::Metadata22;
use crate::PythonInterpreter;
use crate::Target;
use anyhow::{bail, format_err, Context, Result};
Expand Down Expand Up @@ -130,7 +130,7 @@ impl BuildOptions {

let cargo_toml = CargoToml::from_path(&manifest_file)?;
let manifest_dir = manifest_file.parent().unwrap();
let metadata21 = Metadata21::from_cargo_toml(&cargo_toml, &manifest_dir)
let metadata22 = Metadata22::from_cargo_toml(&cargo_toml, &manifest_dir)
.context("Failed to parse Cargo.toml into python metadata")?;
let extra_metadata = cargo_toml.remaining_core_metadata();

Expand Down Expand Up @@ -201,7 +201,7 @@ impl BuildOptions {
// User given list of interpreters
Some(interpreter) => find_interpreter(&bridge, &interpreter, &target, None)?,
// Auto-detect interpreters
None => find_interpreter(&bridge, &[], &target, get_min_python_minor(&metadata21))?,
None => find_interpreter(&bridge, &[], &target, get_min_python_minor(&metadata22))?,
};

let rustc_extra_args = split_extra_args(&self.rustc_extra_args)?;
Expand Down Expand Up @@ -229,7 +229,7 @@ impl BuildOptions {
target,
bridge,
project_layout,
metadata21,
metadata22,
crate_name: crate_name.to_string(),
module_name,
manifest_path: self.manifest_path,
Expand All @@ -249,8 +249,8 @@ impl BuildOptions {

/// Uses very simple PEP 440 subset parsing to determine the
/// minimum supported python minor version for interpreter search
fn get_min_python_minor(metadata21: &Metadata21) -> Option<usize> {
if let Some(requires_python) = &metadata21.requires_python {
fn get_min_python_minor(metadata22: &Metadata22) -> Option<usize> {
if let Some(requires_python) = &metadata22.requires_python {
let regex = Regex::new(r#">=3\.(\d+)(?:\.\d)?"#).unwrap();
if let Some(captures) = regex.captures(&requires_python) {
let min_python_minor = captures[1]
Expand Down Expand Up @@ -760,8 +760,8 @@ mod test {
fn test_get_min_python_minor() {
// Nothing specified
let cargo_toml = CargoToml::from_path("test-crates/pyo3-pure/Cargo.toml").unwrap();
let metadata21 =
Metadata21::from_cargo_toml(&cargo_toml, &"test-crates/pyo3-pure").unwrap();
assert_eq!(get_min_python_minor(&metadata21), None);
let metadata22 =
Metadata22::from_cargo_toml(&cargo_toml, &"test-crates/pyo3-pure").unwrap();
assert_eq!(get_min_python_minor(&metadata22), None);
}
}
12 changes: 6 additions & 6 deletions src/develop.rs
Original file line number Diff line number Diff line change
Expand Up @@ -48,11 +48,11 @@ pub fn develop(
})?;

// Install dependencies
if !build_context.metadata21.requires_dist.is_empty() {
if !build_context.metadata22.requires_dist.is_empty() {
let mut args = vec!["-m", "pip", "install"];
args.extend(
build_context
.metadata21
.metadata22
.requires_dist
.iter()
.map(|x| x.as_str()),
Expand All @@ -74,7 +74,7 @@ pub fn develop(
//
// Uninstalling the actual code is done individually for each bridge model
let base_path = target.get_venv_site_package(venv_dir, &interpreter);
let dist_info_dir = base_path.join(build_context.metadata21.get_dist_info_dir());
let dist_info_dir = base_path.join(build_context.metadata22.get_dist_info_dir());
if dist_info_dir.is_dir() {
fs::remove_dir_all(&dist_info_dir).context(format!(
"Failed to uninstall existing installation by removing {}",
Expand Down Expand Up @@ -178,18 +178,18 @@ pub fn develop(
}
};

write_dist_info(&mut writer, &build_context.metadata21, &tags)?;
write_dist_info(&mut writer, &build_context.metadata22, &tags)?;

// https://packaging.python.org/specifications/recording-installed-packages/#the-installer-file
writer.add_bytes(
build_context
.metadata21
.metadata22
.get_dist_info_dir()
.join("INSTALLER"),
env!("CARGO_PKG_NAME").as_bytes(),
)?;

writer.write_record(&build_context.metadata21)?;
writer.write_record(&build_context.metadata22)?;

Ok(())
}
2 changes: 1 addition & 1 deletion src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ pub use crate::build_options::BuildOptions;
pub use crate::cargo_toml::CargoToml;
pub use crate::compile::compile;
pub use crate::develop::develop;
pub use crate::metadata::{Metadata21, WheelMetadata};
pub use crate::metadata::{Metadata22, WheelMetadata};
pub use crate::module_writer::{
write_dist_info, ModuleWriter, PathWriter, SDistWriter, WheelWriter,
};
Expand Down
18 changes: 9 additions & 9 deletions src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ use human_panic::setup_panic;
use keyring::{Keyring, KeyringError};
use maturin::{
develop, source_distribution, write_dist_info, BridgeModel, BuildOptions, CargoToml,
Metadata21, PathWriter, PlatformTag, PyProjectToml, PythonInterpreter, Target,
Metadata22, PathWriter, PlatformTag, PyProjectToml, PythonInterpreter, Target,
};
use std::env;
use std::path::PathBuf;
Expand Down Expand Up @@ -410,8 +410,8 @@ fn pep517(subcommand: Pep517Command) -> Result<()> {
};

let mut writer = PathWriter::from_path(metadata_directory);
write_dist_info(&mut writer, &context.metadata21, &tags)?;
println!("{}", context.metadata21.get_dist_info_dir().display());
write_dist_info(&mut writer, &context.metadata22, &tags)?;
println!("{}", context.metadata22.get_dist_info_dir().display());
}
Pep517Command::BuildWheel {
build_options,
Expand All @@ -428,7 +428,7 @@ fn pep517(subcommand: Pep517Command) -> Result<()> {
} => {
let cargo_toml = CargoToml::from_path(&manifest_path)?;
let manifest_dir = manifest_path.parent().unwrap();
let metadata21 = Metadata21::from_cargo_toml(&cargo_toml, &manifest_dir)
let metadata22 = Metadata22::from_cargo_toml(&cargo_toml, &manifest_dir)
.context("Failed to parse Cargo.toml into python metadata")?;
let cargo_metadata = MetadataCommand::new()
.manifest_path(&manifest_path)
Expand All @@ -437,7 +437,7 @@ fn pep517(subcommand: Pep517Command) -> Result<()> {

let path = source_distribution(
sdist_directory,
&metadata21,
&metadata22,
&manifest_path,
&cargo_metadata,
None,
Expand Down Expand Up @@ -559,7 +559,7 @@ fn run() -> Result<()> {
eprintln!("⚠ Warning: You're publishing debug wheels");
}

let metadata21 = build_context.metadata21.to_vec();
let metadata22 = build_context.metadata22.to_vec();
let mut wheels = build_context.build_wheels()?;
if !no_sdist {
if let Some(sd) = build_context.build_source_distribution()? {
Expand All @@ -569,7 +569,7 @@ fn run() -> Result<()> {

let items = wheels
.into_iter()
.map(|wheel| UploadItem::from_built_wheel(wheel, metadata21.clone()))
.map(|wheel| UploadItem::from_built_wheel(wheel, metadata22.clone()))
.collect::<Vec<_>>();

upload_ui(&items, &publish)?
Expand Down Expand Up @@ -625,7 +625,7 @@ fn run() -> Result<()> {
.context("A pyproject.toml with a PEP 517 compliant `[build-system]` table is required to build a source distribution")?;

let cargo_toml = CargoToml::from_path(&manifest_path)?;
let metadata21 = Metadata21::from_cargo_toml(&cargo_toml, &manifest_dir)
let metadata22 = Metadata22::from_cargo_toml(&cargo_toml, &manifest_dir)
.context("Failed to parse Cargo.toml into python metadata")?;

let cargo_metadata = MetadataCommand::new()
Expand All @@ -643,7 +643,7 @@ fn run() -> Result<()> {

source_distribution(
&wheel_dir,
&metadata21,
&metadata22,
&manifest_path,
&cargo_metadata,
pyproject.sdist_include(),
Expand Down
Loading