Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Additional assert methods for unittest #71339

Open
serhiy-storchaka opened this issue May 29, 2016 · 20 comments
Open

Additional assert methods for unittest #71339

serhiy-storchaka opened this issue May 29, 2016 · 20 comments
Labels
stdlib Python modules in the Lib dir tests Tests in the Lib/test dir type-feature A feature request or enhancement

Comments

@serhiy-storchaka
Copy link
Member

serhiy-storchaka commented May 29, 2016

BPO 27152
Nosy @rhettinger, @rbtcollins, @ezio-melotti, @bitdancer, @voidspace, @serhiy-storchaka, @PythonCHB
Files
  • extra_assertions.patch
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = None
    created_at = <Date 2016-05-29.15:25:19.795>
    labels = ['type-feature', 'tests']
    title = 'Additional assert methods for unittest'
    updated_at = <Date 2016-06-22.14:03:21.198>
    user = 'https://github.com/serhiy-storchaka'

    bugs.python.org fields:

    activity = <Date 2016-06-22.14:03:21.198>
    actor = 'r.david.murray'
    assignee = 'none'
    closed = False
    closed_date = None
    closer = None
    components = ['Tests']
    creation = <Date 2016-05-29.15:25:19.795>
    creator = 'serhiy.storchaka'
    dependencies = []
    files = ['43047']
    hgrepos = []
    issue_num = 27152
    keywords = ['patch']
    message_count = 10.0
    messages = ['266600', '266601', '266620', '266627', '266675', '266682', '267175', '267185', '268877', '269074']
    nosy_count = 8.0
    nosy_names = ['rhettinger', 'rbcollins', 'ezio.melotti', 'r.david.murray', 'michael.foord', 'Pam.McANulty', 'serhiy.storchaka', 'ChrisBarker']
    pr_nums = []
    priority = 'normal'
    resolution = None
    stage = 'patch review'
    status = 'open'
    superseder = None
    type = 'enhancement'
    url = 'https://bugs.python.org/issue27152'
    versions = ['Python 3.6']

    Linked PRs

    @serhiy-storchaka
    Copy link
    Member Author

    Proposed patch adds the ExtraAssertions mix-in that provides additional assert methods. These methods provide better failure report than assertTrue/assertFalse with a result of corresponding function. For example assertStartsWith outputs shorten reprs of the start of the string and prefix.

    These checks are quite popular and can be used tens or hundreds times in tests (the patch makes tests using new assert methods):

    assertHasAttr 121 times
    assertNotHasAttr 99 times
    assertIsSubclass 243 times
    assertNotIsSubclass 95 times
    assertStartsWith 131 times
    assertEndsWith 73 times

    @serhiy-storchaka serhiy-storchaka added tests Tests in the Lib/test dir type-feature A feature request or enhancement labels May 29, 2016
    @serhiy-storchaka
    Copy link
    Member Author

    Additional statistics that proves that specified checks are not specific for narrow group of tests:

    assertHasAttr 121 times in 57 files
    assertNotHasAttr 99 times in 31 files
    assertIsSubclass 243 times in 30 files
    assertNotIsSubclass 95 times in 5 files
    assertStartsWith 131 times in 59 files
    assertEndsWith 73 times in 36 files

    @bitdancer
    Copy link
    Member

    Why a mixin?

    (As an aside, there is a proposal to move all assert methods to a place where they can be accessed outside test cases.)

    @serhiy-storchaka
    Copy link
    Member Author

    I have this proposal (bpo-19645) in mind. We can add these method just in TestCase.

    In any case I'm going to add the ExtraAssertions mixin in test.support in maintained versions to help keeping branches in sync.

    @rhettinger
    Copy link
    Contributor

    I don't really like the assertEndsWith method which triggers a mental hiccup when taking a familiar method call, making it into function call, and giving an awkward looking camelcase name.

    Also, Guido is usually opposed to broad, sweeping search/replace patches. Instead, he has advocated "holistic refactoring" where updates are done by someone actively working on the module rather than a disinterested party churning the code without thinking deeply about the topic at hand.

    @serhiy-storchaka
    Copy link
    Member Author

    Changes in tests are provided as a demonstration that these methods are useful. I propose to commit only the unittest part, and use new methods on a case-by-case basis. For example some tests already define methods like assertHasAttr or assertIsSubclass, They can now use standard methods.

    assertEndsWith() can be used 73 times in 36 files. It produces more useful error report than assertTrue(a.endswith(b)).

    @PythonCHB
    Copy link
    Mannequin

    PythonCHB mannequin commented Jun 3, 2016

    Why a mixin rather than adding to TestCase? If they are useful they should be easy to find.

    Also, see bpo-27198 for another possible new assert.

    @rbtcollins
    Copy link
    Member

    I'm fine with these as a mixin - they are all very generic and unambiguously named. I'd marginally prefer the opt-in mixin over adding them to the base class.

    Ideally they'd be matchers, but since I haven't ported that upstream yet, thats asking for more work, and we can always migrate later.

    (https://rbtcollins.wordpress.com/2010/05/10/maintainable-pyunit-test-suites/ and http://testtools.readthedocs.io/en/latest/for-test-authors.html#matchers - sorry about the formatting in the blog post, wordpress changed theme details some time after I wrote the post and it now renders horribly :( )

    @rhettinger
    Copy link
    Contributor

    Ask five people to spell "assertEndsWith" and see how many of them capitalize the "W".

    @bitdancer
    Copy link
    Member

    I would expect it to be assertEndswith, etc. You will note that all the other cases of multiple capitals are either englishification of symbolic operators or concatenations of separate type words and/or syntactically distinct operators.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    @gpshead
    Copy link
    Member

    gpshead commented Jun 13, 2023

    absl.testing.absltest includes some of these methods.

    In particular assertStartsWith and assertEndsWith (and Not variants) are its simplest addition. I "only" see those used 2-3000 test files in our huge internal codebase which puts it in a <10% use them category. Their goal is to provide a nice error message, it does that. Those 2-4 would be trivial additions with no maintenance burden. (I'd go with 2, the negative Not variants see almost no use in practice).

    Outside of the stdlib I assume pytest may provide nice output on startswith and endwith comparison failures?

    As for the method name spelling, anyone typing it wrong gets a failure immediately so that isn't a big deal, rapid feedback when wrong. They're fully consistent with the junit javaCaseStyle of unittest assertion methods with words delineated by a case change.

    @voidspace
    Copy link
    Contributor

    +1 to assertStartsWith and assertEndsWith, they sound useful

    @AlexWaygood
    Copy link
    Member

    assertIsSubclass and assertNotIsSubclass are used quite a bit in test_typing and the tests for the third-party typing_extensions backport. Their implementations are simple, and they seem like pretty natural extensions, given that assertIsInstance and assertNotIsInstance already exist.

    @xsdg
    Copy link

    xsdg commented Dec 15, 2023

    Hi, y'all. Just to keep things narrowly-scoped and specific, what would it take to get assertStartsWith and assertEndsWith into unittest.TestCase?

    @gpshead
    Copy link
    Member

    gpshead commented Dec 15, 2023

    A PR adding those two and relevant unittests.

    serhiy-storchaka added a commit to serhiy-storchaka/cpython that referenced this issue Jan 10, 2025
    Add the following methods:
    
    * assertHasAttr() and assertNotHasAttr()
    * assertIsSubclass() and assertNotIsSubclass()
    * assertStartswith() and assertNotStartswith()
    * assertEndswith() and assertNotEndswith()
    
    Also improve error messages for assertIsInstance() and
    assertNotIsInstance().
    @serhiy-storchaka
    Copy link
    Member Author

    #128707 is based on my old patch. The differences:

    • New methods are added to TestCase instead of the ExtraAssertions mixing.
    • assertStartsWith() and assertEndsWith() has been renamed to assertStartswith() and assertEndswith().
    • assertStartswith() and assertEndswith() support tuples of prefixes/suffixes.
    • Added negative assertions assertNotStartswith() and assertNotEndswith().
    • Improved error messages.
    • Tests have been extended.
    • Changes in other tests were not included, except removing similar assertion methods defined in tests.

    @serhiy-storchaka
    Copy link
    Member Author

    Since there are different opinions about names, I created a poll: https://discuss.python.org/t/assertstartwith-vs-assertstartwith/76701. @rhettinger and @bitdancer, please participate.

    @serhiy-storchaka
    Copy link
    Member Author

    Support for assertStartsWith against assertStartswith is unanimous, so I have reverted to this spelling. I'll merge the PR tomorrow.

    @picnixz picnixz added the stdlib Python modules in the Lib dir label Jan 13, 2025
    @hugovk
    Copy link
    Member

    hugovk commented Jan 13, 2025

    If you merge it early tomorrow, we will included it in 3.14.0a4 :)

    serhiy-storchaka added a commit that referenced this issue Jan 14, 2025
    Add the following methods:
    
    * assertHasAttr() and assertNotHasAttr()
    * assertIsSubclass() and assertNotIsSubclass()
    * assertStartsWith() and assertNotStartsWith()
    * assertEndsWith() and assertNotEndsWith()
    
    Also improve error messages for assertIsInstance() and
    assertNotIsInstance().
    serhiy-storchaka added a commit to serhiy-storchaka/cpython that referenced this issue Jan 14, 2025
    …honGH-128707)
    
    Add a mix-in class ExtraAssertions containing the following methods:
    
    * assertHasAttr() and assertNotHasAttr()
    * assertIsSubclass() and assertNotIsSubclass()
    * assertStartsWith() and assertNotStartsWith()
    * assertEndsWith() and assertNotEndsWith()
    
    (cherry picked from commit 06cad77)
    @serhiy-storchaka
    Copy link
    Member Author

    #128815 backports new assertion methods as a mix-in class in test.support. This will help to backport future tests that use these methods.

    serhiy-storchaka added a commit that referenced this issue Jan 20, 2025
    …-128707) (GH-128815)
    
    Add a mix-in class ExtraAssertions containing the following methods:
    
    * assertHasAttr() and assertNotHasAttr()
    * assertIsSubclass() and assertNotIsSubclass()
    * assertStartsWith() and assertNotStartsWith()
    * assertEndsWith() and assertNotEndsWith()
    
    (cherry picked from commit 06cad77)
    serhiy-storchaka added a commit to serhiy-storchaka/cpython that referenced this issue Jan 20, 2025
    serhiy-storchaka added a commit to serhiy-storchaka/cpython that referenced this issue Jan 20, 2025
    serhiy-storchaka added a commit to serhiy-storchaka/cpython that referenced this issue Jan 20, 2025
    serhiy-storchaka added a commit to serhiy-storchaka/cpython that referenced this issue Jan 20, 2025
    serhiy-storchaka added a commit to serhiy-storchaka/cpython that referenced this issue Jan 20, 2025
    serhiy-storchaka added a commit to serhiy-storchaka/cpython that referenced this issue Jan 20, 2025
    serhiy-storchaka added a commit to serhiy-storchaka/cpython that referenced this issue Jan 20, 2025
    serhiy-storchaka added a commit to serhiy-storchaka/cpython that referenced this issue Jan 20, 2025
    serhiy-storchaka added a commit to serhiy-storchaka/cpython that referenced this issue Jan 20, 2025
    …t.support (pythonGH-128707) (pythonGH-128815)
    
    Add a mix-in class ExtraAssertions containing the following methods:
    
    * assertHasAttr() and assertNotHasAttr()
    * assertIsSubclass() and assertNotIsSubclass()
    * assertStartsWith() and assertNotStartsWith()
    * assertEndsWith() and assertNotEndsWith()
    
    (cherry picked from commit c6a566e)
    
    Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
    (cherry picked from commit 06cad77)
    serhiy-storchaka added a commit to serhiy-storchaka/cpython that referenced this issue Jan 20, 2025
    Fix test_cmd_line.
    
    Remove redundant msgs.
    
    Move some changes out to other PRs.
    
    Fix test_cmd_line again.
    serhiy-storchaka added a commit that referenced this issue Jan 20, 2025
    …-128707) (GH-128815) (GH-129059)
    
    Add a mix-in class ExtraAssertions containing the following methods:
    
    * assertHasAttr() and assertNotHasAttr()
    * assertIsSubclass() and assertNotIsSubclass()
    * assertStartsWith() and assertNotStartsWith()
    * assertEndsWith() and assertNotEndsWith()
    
    (cherry picked from commit c6a566e)
    
    Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
    (cherry picked from commit 06cad77)
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    stdlib Python modules in the Lib dir tests Tests in the Lib/test dir type-feature A feature request or enhancement
    Projects
    Status: No status
    Development

    No branches or pull requests

    10 participants