Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[pyupgrade] Add rules to use PEP 695 generics in classes and functions (UP046, UP047) #15565

Merged
merged 77 commits into from
Jan 22, 2025
Merged
Show file tree
Hide file tree
Changes from 61 commits
Commits
Show all changes
77 commits
Select commit Hold shift + click to select a range
86146c7
rename old up040 tests
ntBre Jan 16, 2025
72f966d
handle simplest generic class case
ntBre Jan 16, 2025
7d0c1f9
run new test and add snap
ntBre Jan 16, 2025
12874e8
extend rule to TypeVarTuple
ntBre Jan 17, 2025
b27b281
extend the rule to ParamSpec
ntBre Jan 17, 2025
4050486
factor out From<TypeVar> for TypeParam
ntBre Jan 17, 2025
a58cd26
handle generic function with TypeParams
ntBre Jan 17, 2025
3240173
preserve comments in functions
ntBre Jan 17, 2025
3cadd12
use Edit::insertion for classes too
ntBre Jan 17, 2025
d2dabc0
factor out fmt_type_vars, sort by kind
ntBre Jan 17, 2025
278286b
test that comments are preserved
ntBre Jan 17, 2025
3af9428
test that methods are not currently changed, but add todo
ntBre Jan 17, 2025
8176629
clippy
ntBre Jan 17, 2025
ca92868
separate up040 and (new) up046 with shared super module
ntBre Jan 17, 2025
9ad9642
finish separating UP040 and new UP046
ntBre Jan 17, 2025
ea0e276
Revert "rename old up040 tests"
ntBre Jan 17, 2025
7d616f8
document fmt_type_vars
ntBre Jan 17, 2025
c6c434c
tidy Violation code after split
ntBre Jan 18, 2025
821df96
bail out on duplicate type variables
ntBre Jan 20, 2025
3891fec
delete runtime documentation from type aliases
ntBre Jan 20, 2025
57c65d8
preserve order
ntBre Jan 20, 2025
9315d27
import TypeVar and fix mkdocs formatting
ntBre Jan 20, 2025
080ab06
rename TypeVarKind -> TypeParamKind and variants
ntBre Jan 20, 2025
f27da8d
document and test that multiple base classes are not handled
ntBre Jan 20, 2025
5997743
limit class diagnostic to Generic parameter
ntBre Jan 20, 2025
224311b
comment on python version checks
ntBre Jan 20, 2025
1e4c20d
avoid AST names in documentation
ntBre Jan 20, 2025
188f26d
TypeParam -> TypeVar in docs
ntBre Jan 20, 2025
55be0d8
impl Display for DisplayTypeVar and DisplayTypeVars wrappers
ntBre Jan 20, 2025
01c7c12
Remove Ord and PartialOrd
ntBre Jan 20, 2025
f62a68d
Fix type alias wording
ntBre Jan 20, 2025
cfe9c4a
Apply docs suggestions
ntBre Jan 20, 2025
f12036e
factor out `check_type_vars` and check vars.is_empty first
ntBre Jan 20, 2025
30a0f04
add `See also` section pointing to `unused-private-type-var`
ntBre Jan 20, 2025
933002e
update type alias docs around isinstance
ntBre Jan 20, 2025
274cf71
add constrained class test
ntBre Jan 20, 2025
28d9b4a
add constrained function test
ntBre Jan 20, 2025
bfb0b29
bail out on `default` for now
ntBre Jan 20, 2025
e7749ed
skip nested classes and functions
ntBre Jan 20, 2025
108e5be
document nesting and `default` limitations
ntBre Jan 20, 2025
550fc44
move shared function and class code to parent module
ntBre Jan 20, 2025
8f96a26
rename to mod.rs
ntBre Jan 20, 2025
d2c84eb
separate tests
ntBre Jan 20, 2025
7326ee3
update rule codes
ntBre Jan 20, 2025
811e949
rename rules and separate
ntBre Jan 20, 2025
450dfd4
update ruff schema
ntBre Jan 20, 2025
5930151
Revert "update ruff schema"
ntBre Jan 20, 2025
4a6dbf1
only add UP047 to schema (don't remove PLW0101)
ntBre Jan 20, 2025
b9e03bc
Simplify `check_type_vars`
ntBre Jan 21, 2025
1531bf1
Fix many documentation issues
ntBre Jan 21, 2025
96c36dd
fix typo
ntBre Jan 21, 2025
07d3ca3
update see also sections
ntBre Jan 21, 2025
39d0a5b
track default value on TypeVar but don't use it yet
ntBre Jan 21, 2025
e0f1251
only emit diagnostics if there's an unknown type in class def
ntBre Jan 21, 2025
d034f62
mark fixes as unsafe, document them, and test
ntBre Jan 21, 2025
d55e2ae
add a special case for typing.AnyStr
ntBre Jan 21, 2025
c3f3858
clippy
ntBre Jan 21, 2025
cee09a4
re-unify visitors
ntBre Jan 21, 2025
550cf6d
use unsafe_edit
ntBre Jan 21, 2025
06993aa
fix typo
ntBre Jan 21, 2025
9fadf46
add a test with multiple generics
ntBre Jan 21, 2025
6c696f1
slightly more realistic class code
ntBre Jan 22, 2025
a15703e
slightly more realistic function code
ntBre Jan 22, 2025
9948059
update snapshots
ntBre Jan 22, 2025
4500160
fix imports, make Multiple -> MultipleBaseClasses
ntBre Jan 22, 2025
c6abe12
fix AnyStr handling
ntBre Jan 22, 2025
8bc0d9e
add UP040 tests with `default` and bail early
ntBre Jan 22, 2025
bfdb491
check that AnyStr constraints are bound to builtins
ntBre Jan 22, 2025
c975385
add note on type checkers
ntBre Jan 22, 2025
f3e6693
update module comment
ntBre Jan 22, 2025
e2c92ff
move str shadow test to separate fixture
ntBre Jan 22, 2025
fa1007a
clarify comment
ntBre Jan 22, 2025
76d03b2
fix tuple
ntBre Jan 22, 2025
e47f478
update snapshots
ntBre Jan 22, 2025
196b905
update module names to match rule names
ntBre Jan 22, 2025
35a1c7e
simplify with ?
ntBre Jan 22, 2025
971cb09
fix module names in docs
ntBre Jan 22, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
80 changes: 80 additions & 0 deletions crates/ruff_linter/resources/test/fixtures/pyupgrade/UP046.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
from typing import Any, AnyStr, Generic, ParamSpec, TypeVar, TypeVarTuple

S = TypeVar("S", str, bytes) # constrained type variable
T = TypeVar("T", bound=float)
Ts = TypeVarTuple("Ts")
P = ParamSpec("P")
ntBre marked this conversation as resolved.
Show resolved Hide resolved


class A(Generic[T]):
ntBre marked this conversation as resolved.
Show resolved Hide resolved
# Comments in a class body are preserved
pass


class B(Generic[*Ts]):
pass


class C(Generic[P]):
pass


class Constrained(Generic[S]):
pass


# This case gets a diagnostic but not a fix because we can't look up the bounds
# or constraints on the generic type from another module
class ExternalType(Generic[T, SupportsRichComparisonT]):
ntBre marked this conversation as resolved.
Show resolved Hide resolved
pass


# typing.AnyStr is a common external type variable, so treat it specially as a
# known TypeVar
class MyStr(Generic[AnyStr]):
pass


class MultipleGenerics(Generic[S, T, Ts, P]):
pass


# These cases are not handled
class D(Generic[T, T]): # duplicate generic variable, runtime error
pass


# TODO(brent) we should also apply the fix to methods, but it will need a
# little more work. these should be left alone for now but be fixed eventually.
class NotGeneric:
# -> generic_method[T: float](t: T)
def generic_method(t: T):
pass
ntBre marked this conversation as resolved.
Show resolved Hide resolved


# This one is strange in particular because of the mix of old- and new-style
# generics, but according to the PEP, this is okay "if the class, function, or
# type alias does not use the new syntax." `more_generic` doesn't use the new
# syntax, so it can use T from the module and U from the class scope.
class MixedGenerics[U]:
def more_generic(u: U, t: T):
pass


# TODO(brent) we should also handle multiple base classes
class Multiple(NotGeneric, Generic[T]):
pass


# TODO(brent) default requires 3.13
V = TypeVar("V", default=Any, bound=str)


class DefaultTypeVar(Generic[V]): # -> [V: str = Any]
pass


# nested classes and functions are skipped
class Outer:
class Inner(Generic[T]):
pass
59 changes: 59 additions & 0 deletions crates/ruff_linter/resources/test/fixtures/pyupgrade/UP047.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
from collections.abc import Callable
from typing import Any, AnyStr, ParamSpec, TypeVar, TypeVarTuple

from somewhere import Something

S = TypeVar("S", str, bytes) # constrained type variable
T = TypeVar("T", bound=float)
Ts = TypeVarTuple("Ts")
P = ParamSpec("P")


def f(t: T):
pass


def g(ts: tuple[*Ts]):
pass


def h(
p: Callable[P, T],
# Comment in the middle of a parameter list should be preserved
another_param,
and_another,
):
pass


def i(s: S):
pass


# NOTE this case is the reason the fix is marked unsafe. If we can't confirm
# that one of the type parameters (`Something` in this case) is a TypeVar,
# which we can't do across module boundaries, we will not convert it to a
# generic type parameter. This leads to code that mixes old-style standalone
# TypeVars with the new-style generic syntax and will be rejected by type
# checkers
def broken_fix(okay: T, bad: Something):
pass


def any_str_param(s: AnyStr):
pass


# these cases are not handled

# TODO(brent) default requires 3.13
V = TypeVar("V", default=Any, bound=str)


def default_var(v: V):
pass


def outer():
def inner(t: T):
pass
6 changes: 6 additions & 0 deletions crates/ruff_linter/src/checkers/ast/analyze/statement.rs
Original file line number Diff line number Diff line change
Expand Up @@ -376,6 +376,9 @@ pub(crate) fn statement(stmt: &Stmt, checker: &mut Checker) {
if checker.enabled(Rule::PytestParameterWithDefaultArgument) {
flake8_pytest_style::rules::parameter_with_default_argument(checker, function_def);
}
if checker.enabled(Rule::NonPEP695GenericFunction) {
pyupgrade::rules::non_pep695_generic_function(checker, function_def);
}
}
Stmt::Return(_) => {
if checker.enabled(Rule::ReturnOutsideFunction) {
Expand Down Expand Up @@ -554,6 +557,9 @@ pub(crate) fn statement(stmt: &Stmt, checker: &mut Checker) {
if checker.enabled(Rule::DataclassEnum) {
ruff::rules::dataclass_enum(checker, class_def);
}
if checker.enabled(Rule::NonPEP695GenericClass) {
pyupgrade::rules::non_pep695_generic_class(checker, class_def);
}
}
Stmt::Import(ast::StmtImport { names, range: _ }) => {
if checker.enabled(Rule::MultipleImportsOnOneLine) {
Expand Down
2 changes: 2 additions & 0 deletions crates/ruff_linter/src/codes.rs
Original file line number Diff line number Diff line change
Expand Up @@ -540,6 +540,8 @@ pub fn code_to_rule(linter: Linter, code: &str) -> Option<(RuleGroup, Rule)> {
(Pyupgrade, "043") => (RuleGroup::Stable, rules::pyupgrade::rules::UnnecessaryDefaultTypeArgs),
(Pyupgrade, "044") => (RuleGroup::Preview, rules::pyupgrade::rules::NonPEP646Unpack),
(Pyupgrade, "045") => (RuleGroup::Preview, rules::pyupgrade::rules::NonPEP604AnnotationOptional),
(Pyupgrade, "046") => (RuleGroup::Preview, rules::pyupgrade::rules::NonPEP695GenericClass),
(Pyupgrade, "047") => (RuleGroup::Preview, rules::pyupgrade::rules::NonPEP695GenericFunction),

// pydocstyle
(Pydocstyle, "100") => (RuleGroup::Stable, rules::pydocstyle::rules::UndocumentedPublicModule),
Expand Down
2 changes: 2 additions & 0 deletions crates/ruff_linter/src/rules/pyupgrade/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,8 @@ mod tests {
#[test_case(Rule::YieldInForLoop, Path::new("UP028_1.py"))]
#[test_case(Rule::NonPEP695TypeAlias, Path::new("UP040.py"))]
#[test_case(Rule::NonPEP695TypeAlias, Path::new("UP040.pyi"))]
#[test_case(Rule::NonPEP695GenericClass, Path::new("UP046.py"))]
#[test_case(Rule::NonPEP695GenericFunction, Path::new("UP047.py"))]
fn rules(rule_code: Rule, path: &Path) -> Result<()> {
let snapshot = path.to_string_lossy().to_string();
let diagnostics = test_path(
Expand Down
4 changes: 2 additions & 2 deletions crates/ruff_linter/src/rules/pyupgrade/rules/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ pub(crate) use native_literals::*;
pub(crate) use open_alias::*;
pub(crate) use os_error_alias::*;
pub(crate) use outdated_version_block::*;
pub(crate) use pep695::*;
pub(crate) use printf_string_formatting::*;
pub(crate) use quoted_annotation::*;
pub(crate) use redundant_open_modes::*;
Expand All @@ -36,7 +37,6 @@ pub(crate) use use_pep585_annotation::*;
pub(crate) use use_pep604_annotation::*;
pub(crate) use use_pep604_isinstance::*;
pub(crate) use use_pep646_unpack::*;
pub(crate) use use_pep695_type_alias::*;
pub(crate) use useless_metaclass_type::*;
pub(crate) use useless_object_inheritance::*;
pub(crate) use yield_in_for_loop::*;
Expand All @@ -57,6 +57,7 @@ mod native_literals;
mod open_alias;
mod os_error_alias;
mod outdated_version_block;
mod pep695;
mod printf_string_formatting;
mod quoted_annotation;
mod redundant_open_modes;
Expand All @@ -79,7 +80,6 @@ mod use_pep585_annotation;
mod use_pep604_annotation;
mod use_pep604_isinstance;
mod use_pep646_unpack;
mod use_pep695_type_alias;
mod useless_metaclass_type;
mod useless_object_inheritance;
mod yield_in_for_loop;
Loading
Loading