various amendments #21

dornech · 2024-12-22T01:40:59Z

Hi @FlorianWilhelm,
I spent some time on your Hatchlor.
I did some amendments you left over: I included two options for local comitting and versioning - bump-my-version/generate-changelog or semantic-release. Both are - if set up - easy to use. I included a basic config for both tools in the pyproject.toml template so everbody is free to use the one or the other.
Furthermore I added gitlint which I find useful to enforce conventional commit compliant commit messages - pretty useful if you automate the changelog generation with generate_changelog or semantic-release. gitlint is cool stuff and for example works also within PyCharm which is my favorite IDE (which is probably not surprising for the more professional Python developer than I would claim for myself).
Finally I did some rework on the GitHub workflows.
These are the main changes I think. Please find approriate comments in the README.md as well.

FlorianWilhelm

Thanks @dornech! I added a few comments.

Would you be interested in becoming a co-maintainer for The Hatchlor? 🌹

FlorianWilhelm · 2024-12-23T13:13:18Z

{{cookiecutter.project_slug}}/src/{{cookiecutter.pkg_name}}/__init__.py

+    # own developed alternative variant to hatch-vcs-footgun overcoming problem of ignored setuptools_scm settings
+    # from hatch-based pyproject.toml libraries
+    from hatch.cli import hatch
+    from click.testing import CliRunner


What is the advantage of this compared to the original version? It expects way more packages to be installed during execution time like click and hatch itself, which normally is not required to be present when you just want to install the package you have developed.

This is required for retrieving the actual build version. It is an adjusted version of the footgun as commented.
I agree this might be overhead for non-developer installation but if hatch is installed anyway, there is no additional overhead.

Not sure I really get it. My original code accomplished the same thing. It read out the version but only using modules that are shipped directly with Python. So the big advantage of my original code is that you don't need to have hatch installed to use the package. I know that developers of a package need to have hatch installed but you should always think about the users of what you are building. They should not be required to install hatch.

If you do an "pip install -e" then you always see the last install version -- not the last commit version.
It is explained here:
https://github.com/maresb/hatch-vcs-footgun-example

But I agree: this is probably a small advantage compared to the setuptool_scm dependency. And it is only neceesary if one has very strict version dependencies in the code.
Probably refering to metadata or the local _version.py after a hatch build is a good compomise?

Argh, I did miss that I eliminated the setuptools_scm dependency already because of using Hatch compared to the original footgun-version. the setuptools_scm did not work correctly with the version bumping.
Explained in more detail here - maresb/hatch-vcs-footgun-example#3: @maresb made also an suggestion how to call hatchling directly (without the click-overhead of Hatch as the front-end to hatchling).

FlorianWilhelm · 2024-12-23T13:15:27Z

{{cookiecutter.project_slug}}/pyproject.toml

@@ -71,10 +77,104 @@ version-file = "src/{{ cookiecutter.pkg_name }}/_version.py"
 packages = ["src/{{ cookiecutter.pkg_name }}"]

 [tool.hatch.build.targets.sdist]
+artifacts = ["_version.py"]


Smart, so this is necessary for source distributions so that it is included?

The _verison.py file is generated during build and is to be included for the installable installation I think.

This worked out of the box for wheel files but not sure about sdists. Good point.

To be honest I am not experienced with the difference between wheel and sdist-distributions.

However, what I see from my testpackage the _version.py is included in sdist and wheel distribution.

FlorianWilhelm · 2024-12-23T13:18:46Z

{{cookiecutter.project_slug}}/pyproject.toml

    "Programming Language :: Python :: Implementation :: CPython",
 ]
 # direct dependencies of this package, installed when users `pip install {{ cookiecutter.project_slug }}` later.
 dependencies = [ # ToDo: Modify according to your needs!
-    "typer",
+    "setuptools_scm",


setuptools_scm should not be needed for a normal, i.e. non-development, installation. It's not necessary to have it as a runtime requirement.

I agree but it is the issue as above: it is the hatch-vcs-footgun in init.py ...

Argh, I did miss that I eliminated the setuptools_scm dependency already because of using hatch compared to the original footgun-version. the setuptools_scm did not work correctly with the version bumping.

BTW - the issue with setuptools_scm was discussed here
maresb/hatch-vcs-footgun-example#3

FlorianWilhelm · 2024-12-23T13:20:42Z

{{cookiecutter.project_slug}}/.pre-commit-config.yaml

  hooks:
    - id: mypy
      args: ["--install-types", "--non-interactive"]
      additional_dependencies: [types-tabulate, types-cachetools]
+
+-   repo: https://github.com/jorisroovers/gitlint


I really like the idea but would it be better to have this optional via cookiecutter? Gitlint is still having less than 1k Github stars and is it really used that often? I like it but it's not a commonly found best practice, is it?

Well, to be honest: gitlint was the only commit-message linter I found. For automated generation of a changelog it is necessary to have proper commit messages. I struggled somewhat on that ...
Probably more professional developers than I am are much more familiar with this and do it while asleep but for me it was helpful.
I do not knoy how cookiecutter would be of any help here but probably I missed something there.

Ah, I guess now I know what you mean with "optional via cookiecutter". You mean similar to the lock-file support i. e. controlling via y/n prompt during cookiecutter execution? I am afraid it is me lacking some cookiecutter / Python packaging experience that I did not get it directly. Sounds like a valid option.

Probably also an option fot the complete footgun-stuff ...

Exactly, to make it gitlint an additional option. Bear in mind though that with every additional option, running unittests becomes more complicated since you need to test every possible path, not only the one that includes all features.

Question: to do it perfect, I would have to delete the .gitlint config file after generation via the post_gen_project.py hook ?

I tried to make the gitlint optional. Struggling with the post-generation hook to delete the .gitlint file, other error fixed.

FlorianWilhelm · 2024-12-23T13:21:22Z

{{cookiecutter.project_slug}}/.github/workflows/tests.yml

@@ -38,4 +45,4 @@ jobs:
          hatch run lint:all
      - name: Run Tests
        run: |
-          hatch run test:pytest
+          hatch test


Nice, the new way of doing this! Thanks.

My idea was to leave the testing for different versions and environments mainly with GitHub. This is achieved by setting the matrix for os / python version in the GitHub action and executing a hatch test in the standard hatch-test environment.
I was only struggling a little bit with the package dependencies, there might be some room for optimization.

dornech · 2024-12-23T21:03:42Z

General remark: I am afraid this pull request still has some escaping errors. Fix see subsequent commits from today.

dornech · 2024-12-23T21:06:41Z

Regarding co-maintainer: I feel honoured you are asking. I am just not sure if I am experienced enough. I mean I think I am not a rookie in IT and in development but at least a newbie in the Python Packaging universe ...

dornech · 2024-12-24T12:18:50Z

I would love to see PyScaffold become Hatch-enabled to allow to work with Hatch-based scaffolds and probably migrating Hatchlor to a PyScaffold template. So far I am struggling with Cruft which is less well maintained and does not work in my environment.

FlorianWilhelm · 2024-12-25T21:44:02Z

PyScaffold follows a completely different approach and uses traditional tools like setuptools only. To make it hatch-based would mean to rewrite almost everything... Not sure that really makes sense and that this should be done. What would you gain from it? The idea was to have a lightweight Hatchlor for people using hatch (and there aren't that many) and PyScaffold for developers that want to stick with a traditional toolchain.

dornech · 2024-12-25T23:05:52Z

Hm. My understanding was that PyScaffold helps to follow and adopt changes in an underlying template. I wonder why this should not be of interest for Hatch based delopment as for "traditional" toolchain. Only other tool I found is cruft but it is less well maintained and I am not yet too confident in this tool.

…tch-vcs/discussions/12

FlorianWilhelm · 2024-12-27T08:43:59Z

{{cookiecutter.project_slug}}/pyproject.toml

 exclude = [
    "/.github",
 ]

+[tool.bumpversion]


@dornech, how does bumpversion play together with setuptools_scm? The latter takes the info from git, bumpversion uses a different concept, right?

I am not sure if this is relevant. I do not really use setuptools_scm any more and also removed it as dependency for the "footgun" as dicsussed earlier. Setuptools_scm does not work with hatch / pyproject.toml because it does not get the raw-options.
I use bump-my-version (with calling generate-changelog as hook) or semantic-release to bump the version and generating a changelog automatically (what setuptools_scm does not do as far as I understood). What I wanted to achieve ist that the changelog is generated locally and together with creating a version before pushing to github and that works with these two toolchain amendments. The advantage here is also that with conventional commits (enforced by gitlint) this really works automatically.
What one could do: optional cookiecutter to use one or the other toolchain amendment - i. e. bump-my-version/generate-changelog or semantic-release. I have no real picture which one is better, both are pretty long "on the market" and properly maintained. handling is also pretty straightforward.

maresb · 2025-01-04T17:45:57Z

Hey @dornech and @FlorianWilhelm! It's great to see all the activity in this PR. PyScaffold was actually my intro some years back to Python packaging best practices. 🤩

Based on @dornech's very helpful feedback I recently revamped hatch-vcs-footgun-example.

One important technical point regarding the version determination logic: it shouldn't be done within __init__.py but rather in some submodule like version.py. Otherwise it risks introducing circular imports if __version__ is imported from elsewhere. (I just added a brief explanation of the problem here).

Another thing is that I think you give me too much credit for hatch-vcs-footgun-example. It's a hack, and I think it should only be used when there's a specific need. When creating something like a project template for everyone to use, usually less is more. So in my subjective opinion, I think the sweet spot may be to create a simple version.py using just the importlib.metadata.version method, but then add a comment like:

from importlib.metadata import version


# To update the version number in the project metadata it's necessary to reinstall
# the package. In case you are working with an editable install and need up-to-date
# version numbers without reinstallation, check out the alternative methods suggested
# in <https://github.com/maresb/hatch-vcs-footgun-example>.
__version__ = version(__project__)

Finally, from the perspective of a maintainer, in my experience it can be really challenging to approve larger PRs like this one. It requires the maintainer to carve out a big block of time to read the whole thing on each review cycle. If you could instead make several small PRs, then it becomes much easier / faster / safer to check correctness and merge each one individually. (That's the reason I stopped early in pypa/hatch#1870 before achieving anything substantial; it's already probably way too big.)

FlorianWilhelm · 2025-01-04T21:25:36Z

Hi @maresb and thanks for pointing that out and you feedback. I totally agree.

Would you also mind @dornech to split that PR a bit up? You have some fantastic improvements in there, which I would like to just merge but others really need to be discussed. It's much simpler if every isolated feature or improvement has its own PR instead of mixing them in one huge PR. Thank you.

dornech · 2025-01-12T14:41:34Z

Hi, I admit i am not yet too experienced with Git / GitHub and "reasonable" assigning of changes and commits to pull requests. So I need some time and probaly assistance to sort that out.

maresb · 2025-01-12T15:09:54Z

PRs are effectively just branches with a tiny bit of GitHub (not git) infrastructure on top to support conversations. So just focus on branches for now. Opening a PR for a branch is easy. Understanding Git is the eternal struggle.

It looks like you're developing from your main branch. Since your main is different from Florian's main, that's basically blocking you from submitting other PRs, so you should probably clear that up first...

From your main, run git checkout -b various-amendments to create a duplicate of your main. Then run git push and follow the instructions to set the upstream. Then go on GitHub and open a draft PR from various-amendments. You should add a reference to this PR. As far as I know, you can't change the source branch of a PR, so that means you need to close this one, but you'll have all the conversation history indirectly through the link on the new draft PR.

Now you need to reset your main branch via a hard reset. WARNING: this will clobber any tracked changes, meaning existing files that have been modified; new files you created will be unaffected. If you have any such files, run git stash first. (Run git stash pop afterwards to restore the tracked changes.) Run git reset --hard upstream/main. Now your main branch will be in order.

Finally, we can look at splitting this PR. The easiest way is if you can split the PR from the beginning. Like if you want to submit the first two commits as a PR, then check out the hash of the 2nd commit git checkout 709b655. You'll get a "detached head" warning, simply meaning you're not on a branch. Run git checkout -b second-commit-branch to create one and avoid decapitation. Now you can push this branch and create a PR for just the first two commits.

The next easiest way is via cherry-picking. Run git checkout main to get a clean starting point. Then if you want to bring in individual commits, run git cherry-pick commit-hash. Unfortunately you might have to solve merge conflicts when cherry-picking stuff out-of-order.

The hardest is rebasing, which involves rewriting the history of a branch. I'm not going to go there right now. 😂

Let me know if you have any questions.

dornech added 11 commits December 22, 2024 01:41

Feat: rework GitHub workflows

cbee59c

Feat: add config files for linting of commit messages

709b655

Feat: add config file for generate-changelog

a84383e

Feat: adjust pre-commit.yaml for linting commit-msg and version update

81c989d

Fix: correct build.yaml

a2b6e2e

Fix: amend .gitignore for ruff cache

7e26c6e

Feat: include hatch-vcs-footgun-example in __init__.py

834b155

Feat: various changes to pyproject.toml to include additional features

fd0418b

Update README.md

e24f745

feat: code blocks in mkdocs.yaml

3cae85e

feat: add. settings pymdownx.highlight

f2f65b6

FlorianWilhelm reviewed Dec 23, 2024

View reviewed changes

dornech added 7 commits December 23, 2024 18:34

fix: escaping of variable in .changelog-config.yml

7c5e882

fix: escaping of variable in .changelog-config.yml /2.

f45280a

fix: escaping of semantic-release config in pyproject.toml

d5ff3a1

fix: correct build.yml (incorrect escaping)

10ae8ff

fix: correct build.yml (incorrect escaping) /2.

60a35d3

fix: correct build.yml (incorrect escaping) /3.

4b816fb

fix: correct pyproject.toml (incomplete escaping)

2807155

fix: editing ...

3f45e9f

feat: add basic Cruft config (template update helper)

4698acf

FlorianWilhelm added the enhancement New feature or request label Dec 25, 2024

dornech added 2 commits December 25, 2024 23:49

fix: clean __init__.py to avoid Hatch dependency

42c03b1

feat: make gitlint optional via cookiecutter variable

67a7df1

feat: make gitlint optional via cookiecutter variable /2.

1826263

dornech added 3 commits December 26, 2024 00:16

feat: make gitlint optional via cookiecutter variable /2.

0f0908e

fix: postgen_project.py hook for optional .gitlint

2ddb160

fix: postgen_project.py hook for optional .gitlint /2.

f3e85af

This was referenced Dec 26, 2024

Support for changelog generation #12

Open

errors in test.yaml #19

Open

error in build.yaml / wrong call to build coverage #20

Open

fix: hatch.version.raw-options according tohttps://github.com/ofek/ha…

628dee2

…tch-vcs/discussions/12

FlorianWilhelm reviewed Dec 27, 2024

View reviewed changes

various amendments #21

Are you sure you want to change the base?

various amendments #21

Conversation

dornech commented Dec 22, 2024 • edited Loading

FlorianWilhelm left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dornech Dec 23, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dornech Dec 25, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dornech Dec 25, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dornech commented Dec 23, 2024 • edited Loading

dornech commented Dec 23, 2024

dornech commented Dec 24, 2024

FlorianWilhelm commented Dec 25, 2024

dornech commented Dec 25, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

maresb commented Jan 4, 2025

FlorianWilhelm commented Jan 4, 2025 • edited Loading

dornech commented Jan 12, 2025

maresb commented Jan 12, 2025

dornech commented Dec 22, 2024 •

edited

Loading

dornech Dec 23, 2024 •

edited

Loading

dornech Dec 25, 2024 •

edited

Loading

dornech Dec 25, 2024 •

edited

Loading

dornech commented Dec 23, 2024 •

edited

Loading

FlorianWilhelm commented Jan 4, 2025 •

edited

Loading