Use vcs_version for Pants' version

3rdparty/python/pants_dev_requirements.txt

@@ -0,0 +1,4 @@
+# These requirements exist solely for the venv Pants runs in in this repo.
+debugpy==1.6.0
+pydevd-pycharm==203.5419.8
+setuptools_scm==8.0.4

3rdparty/python/requirements.txt

@@ -34,12 +34,6 @@ typing-extensions==4.3.0
 mypy-typing-asserts==0.1.1
 node-semver==0.9.0
 
-
-# These dependencies are only for debugging Pants itself (in VSCode/PyCharm respectively),
-# and should never be imported.
-debugpy==1.6.0
-pydevd-pycharm==203.5419.8
-
 # These dependencies must only be used from the explorer backend, and no code outside that backend
 # may import anything from it, so these libraries are not ending up as requirements of Pants itself.
 fastapi==0.78.0

build-support/pants_venv

@@ -9,6 +9,7 @@ source "${REPO_ROOT}/build-support/common.sh"
 
 REQUIREMENTS=(
   "${REPO_ROOT}/3rdparty/python/requirements.txt"
+  "${REPO_ROOT}/3rdparty/python/pants_dev_requirements.txt"
 )
 
 platform=$(uname -mps)

cargo

@@ -7,6 +7,9 @@ REPO_ROOT=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd -P)
 # shellcheck source=build-support/common.sh
 source "${REPO_ROOT}/build-support/common.sh"
 
+GIT_TAG=$(git describe --tags)
+export GIT_TAG
+
 PY="$(determine_python)"
 export PY
 export PYO3_PYTHON="${PY}"

docs/docs/contributions/releases/release-process.mdx

@@ -53,59 +53,39 @@ If there is deprecated code that must be removed, you can either:
 2. Remove it yourself, in a precursor PR.
 3. Bump the deprecation removal target back by one dev release.
 
-### `rc` - Check for cherry-picks
-
-If this is a release candidate, ensure that pending cherry-picks have been applied in the release branch. Cherry-picks are usually applied automatically, but this may not always succeed, so [check for any pending cherry-picks](https://github.com/pantsbuild/pants/pulls?q=is%3Apr+label%3Aneeds-cherrypick+is%3Aclosed), and find the relevant ones by looking at the milestone: for instance, if doing a release for 2.16, the relevant cherry-picks are those for milestone `2.16.x` or earlier.
-
-The process may fail in one of two ways:
-
-- The cherry-picking process failed, and tagged the PR with `auto-cherry-picking-failed`: follow the instructions in the comment on the pull request. (This likely means there are merge conflicts that require manual resolution.)
-- the cherry-pick hasn't (yet) run: trigger the automation manually by going to [the GitHub Action](https://github.com/pantsbuild/pants/actions/workflows/auto-cherry-picker.yaml), clicking on the "Run workflow" button, and providing the PR number.
-
-## Step 1: Create the release commit
-
-The release commit is the commit that bumps the VERSION string. For `dev`/`a0` releases this happens in the `main` branch, in the same commit that updates the release notes and the `CONTRIBUTORS.md` file. For `rc` and stable releases, this happens in the relevant stable branch.
-
 ### `dev0` - set up the new release series
 
 If this is the first dev release in a new series:
 
 1. Create a new file in ` src/python/pants/notes`, e.g. create `src/python/pants/notes/2.9.x.md`.
    1. Copy the title and template over from the prior release, e.g. `2.8.x.md`.
 
-### Bump the VERSION
-
-From the `main` branch, run `pants run src/python/pants_release/start_release.py -- --new 2.9.0.dev1 --release-manager your_github_username --publish` with the relevant version and your own GitHub username.
-
-This will create a pull request that:
+### `rc` - Check for cherry-picks
 
-1. updates `CONTRIBUTORS.md`
-2. bumps the `VERSION` on the branch
+If this is a release candidate, ensure that pending cherry-picks have been applied in the release branch. Cherry-picks are usually applied automatically, but this may not always succeed, so [check for any pending cherry-picks](https://github.com/pantsbuild/pants/pulls?q=is%3Apr+label%3Aneeds-cherrypick+is%3Aclosed), and find the relevant ones by looking at the milestone: for instance, if doing a release for 2.16, the relevant cherry-picks are those for milestone `2.16.x` or earlier.
 
-### Merge the pull request
+The process may fail in one of two ways:
 
-Post the PR to the `#development` channel in Slack. Merge once approved and green.
+- The cherry-picking process failed, and tagged the PR with `auto-cherry-picking-failed`: follow the instructions in the comment on the pull request. (This likely means there are merge conflicts that require manual resolution.)
+- the cherry-pick hasn't (yet) run: trigger the automation manually by going to [the GitHub Action](https://github.com/pantsbuild/pants/actions/workflows/auto-cherry-picker.yaml), clicking on the "Run workflow" button, and providing the PR number.
 
 ### `a0` - create a new Git branch
 
 If you're releasing an `a0` release, you must create the stable branch for that version.
 
-For example, if you're releasing `2.9.0a0`, create the branch `2.9.x` by running the command below. Make sure you are on your release commit before doing this.
+For example, if you're releasing `2.9.0a0`, create the branch `2.9.x` by running the command below.
+Make sure you are on the latest commit of `main` before doing this.
 
 ```bash
 $ git checkout -b 2.9.x
-$ git push upstream 2.9.x
+$ git push git@github.com:pantsbuild/pants.git 2.9.x
 ```
 
-## Step 2: Tag the release to trigger publishing
-
-Once you have merged the `VERSION` bump — which will be on `main` for `dev` and `a0` releases, and on the release branch for release candidates — tag the release commit to trigger wheel building and publishing.
+## Step 1: Tag the release to trigger publishing
 
-First, ensure that you are on your release branch at your version bump commit.
+Once you are ready to release a new version, tag the a commit to trigger wheel building and publishing.
 
-:::note Tip: if new commits have landed after your release commit
-You can reset to your release commit by running `git reset --hard <sha>`.
-:::
+First, ensure that you are on your release branch on the latest commit.
 
 Then, run:
 
@@ -115,7 +95,7 @@ Then, run:
 
 This will tag the release with your PGP key, and push the tag to origin, which will kick off a [`Release` job](https://github.com/pantsbuild/pants/actions/workflows/release.yaml) to build the wheels and publish them to PyPI.
 
-## Step 3: Test the release
+## Step 2: Test the release
 
 Run this script as a basic smoke test:
 
@@ -125,7 +105,7 @@ Run this script as a basic smoke test:
 
 You should also check [GitHub Releases](https://github.com/pantsbuild/pants/releases) to ensure everything looks good. Find the version you released, then click it and confirm that the "Assets" list includes PEXes for macOS and Linux.
 
-## Step 4: Run release testing on public repositories
+## Step 3: Run release testing on public repositories
 
 Manually trigger a run of the [public repositories testing workflow](https://github.com/pantsbuild/pants/actions/workflows/public_repos.yaml), specifying the version just published as the "Pants version".
 

pants

@@ -53,6 +53,8 @@ function exec_pants_bare() {
     echo "Will launch debugpy server at '127.0.0.1:5678' waiting for client connection."
   fi
 
+  export RUNNING_PANTS_FROM_SOURCES=1
+
   if [ -z "${PANTS_NO_NATIVE_CLIENT}" ]; then
     set +e
     "${NATIVE_CLIENT_BINARY}" "$@"
@@ -72,7 +74,7 @@ function exec_pants_bare() {
   fi
 
   # shellcheck disable=SC2086
-  PYTHONPATH="${PANTS_SRCPATH}:${PYTHONPATH}" RUNNING_PANTS_FROM_SOURCES=1 NO_SCIE_WARNING=1 \
+  PYTHONPATH="${PANTS_SRCPATH}:${PYTHONPATH}" NO_SCIE_WARNING=1 \
     exec ${PANTS_PREPEND_ARGS:-} "$(venv_dir)/bin/python" ${DEBUG_ARGS} "${PANTS_PY_EXE}" "$@"
 }
 

pants.toml

@@ -189,6 +189,8 @@ extra_env_vars = [
   "PATH",
   # We'd always like complete backtraces in tests.
   "RUST_BACKTRACE=1",
+  # NB: Set this to some unique sentinel value so its always consistent in tests
+  "_PANTS_VERSION_OVERRIDE=3.0.0",
 ]
 timeout_default = 60
 

src/python/pants/BUILD

@@ -6,7 +6,6 @@ python_sources(
     overrides={
         # Enable `python -m pants ...` style execution ala `json.tool` or `venv`.
         "__main__.py": {"dependencies": ["src/python/pants/bin:pants_loader"]},
-        "version.py": {"dependencies": ["src/python/pants/_version:resources"]},
     },
 )
 
@@ -20,6 +19,8 @@ python_distribution(
         ":resources",
         # Include the native client binary in the distribution.
         "src/python/pants/bin:native_client",
+        # Embed the VERSION file so it can be read at runtime
+        "src/python/pants/_version:VERSION",
     ],
     # Because we have native code, this will cause the wheel to use whatever the ABI is for the
     # interpreter used to run setup.py, e.g. `cp39m-macosx_10_15_x86_64`.

src/python/pants/_version/BUILD

@@ -3,4 +3,19 @@
 
 python_sources()
 
-resources(name="resources", sources=["VERSION"])
+vcs_version(
+    name="generated-version",
+    generate_to="pants/_version/VERSION",
+    # @TODO: This won't be as meaningful on `main`, as the dev number gets replaced with the "distance".
+    #   However, of all of the schemes the `-semver` ones are the only ones that allow for `.devN`
+    #   tags, and produce a valid version. "release-branch-semver" increments minor version which is
+    #   more misleading than "python-simplified-semver" which increments patch.
+    #   Use https://github.com/pypa/setuptools_scm/pull/1005 when we can.
+    version_scheme=None if env("CI", "0") == "1" else "python-simplified-semver",
+    # NB: Don't dirty the version in CI
+    local_scheme="no-local-version" if env("CI", "0") == "1" else None,
+    tag_regex=r"^release_(?P<version>[vV]?\d+(?:\.\d+){0,2}[^\+]*)$",
+    template="{version}",
+)
+
+experimental_wrap_as_resources(name="VERSION", inputs=[":generated-version"])

src/python/pants/backend/python/target_types.py

@@ -1674,11 +1674,33 @@ class VersionTemplateField(StringField):
     )
 
 
+class VersionVersionSchemeField(StringField):
+    alias = "version_scheme"
+    help = help_text(
+        """
+        The version scheme to configure `setuptools_scm` to use.
+        See https://setuptools-scm.readthedocs.io/en/latest/extending/#available-implementations
+        """
+    )
+
+
+class VersionLocalSchemeField(StringField):
+    alias = "local_scheme"
+    help = help_text(
+        """
+        The local scheme to configure `setuptools_scm` to use.
+        See https://setuptools-scm.readthedocs.io/en/latest/extending/#available-implementations_1
+        """
+    )
+
+
 class VCSVersion(Target):
     alias = "vcs_version"
     core_fields = (
         *COMMON_TARGET_FIELDS,
         VersionTagRegexField,
+        VersionVersionSchemeField,
+        VersionLocalSchemeField,
         VCSVersionDummySourceField,
         VersionGenerateToField,
         VersionTemplateField,

src/python/pants/backend/python/util_rules/vcs_versioning.py

@@ -22,9 +22,11 @@
     PythonSourceField,
     VCSVersion,
     VCSVersionDummySourceField,
+    VersionLocalSchemeField,
     VersionGenerateToField,
     VersionTagRegexField,
     VersionTemplateField,
+    VersionVersionSchemeField,
 )
 from pants.backend.python.util_rules.pex import PexRequest, VenvPex, VenvPexProcess
 from pants.core.util_rules.stripped_source_files import StrippedFileName, StrippedFileNameRequest
@@ -78,9 +80,12 @@ async def generate_python_from_setuptools_scm(
     # directory" and "where should I write output to".
     config: dict[str, dict[str, dict[str, str]]] = {}
     tool_config = config.setdefault("tool", {}).setdefault("setuptools_scm", {})
-    tag_regex = request.protocol_target[VersionTagRegexField].value
-    if tag_regex:
+    if tag_regex := request.protocol_target[VersionTagRegexField].value:
         tool_config["tag_regex"] = tag_regex
+    if version_scheme := request.protocol_target[VersionVersionSchemeField].value:
+        tool_config["version_scheme"] = version_scheme
+    if local_scheme := request.protocol_target[VersionLocalSchemeField].value:
+        tool_config["local_scheme"] = local_scheme
     config_path = "pyproject.synthetic.toml"
 
     input_digest_get = Get(

src/python/pants/bin/pants_runner.py

@@ -59,11 +59,11 @@ def _should_run_with_pantsd(self, global_bootstrap_options: OptionValueContainer
     def scrub_pythonpath() -> None:
         # Do not propagate any PYTHONPATH that happens to have been set in our environment
         # to our subprocesses.
-        # Note that don't warn (but still scrub) if RUNNING_PANTS_FROM_SOURCES is set. This allows
-        # scripts that run pants directly from sources, and therefore must set PYTHONPATH, to mute
+        # Note that don't warn if RUNNING_PANTS_FROM_SOURCES is set. This allows scripts that
+        # run pants directly from sources, and therefore must set PYTHONPATH, to mute
         # this warning.
         pythonpath = os.environ.pop("PYTHONPATH", None)
-        if pythonpath and not os.environ.pop("RUNNING_PANTS_FROM_SOURCES", None):
+        if pythonpath and not os.environ.get("RUNNING_PANTS_FROM_SOURCES", "0") == "1":
             logger.debug(f"Scrubbed PYTHONPATH={pythonpath} from the environment.")
 
     def run(self, start_time: float) -> ExitCode:

src/python/pants/version.py

@@ -56,11 +56,32 @@ def __ge__(self, other: Any):
 _PANTS_VERSION_OVERRIDE = "_PANTS_VERSION_OVERRIDE"
 
 
+# @TODO: Use https://github.com/pypa/setuptools_scm/pull/1005 when we can
+def _determine_version_from_pants_source():
+    from setuptools_scm import get_version  # pants: no-infer-dep
+
+    def version_scheme(version):
+        return version.format_with("{tag}")
+
+    val = get_version(
+        version_scheme=version_scheme,
+        tag_regex=r"^release_(?P<version>[vV]?\d+(?:\.\d+){0,2}[^\+]*)$",
+    )
+    return val
+
+
 VERSION: str = (
     # Do not remove/change this env var without coordinating with `pantsbuild/scie-pants` as it is
     # being used when bootstrapping Pants with a released version.
     os.environ.get(_PANTS_VERSION_OVERRIDE)
     or
+    # NB: This is only relevant for the Pants repo itself
+    (
+        _determine_version_from_pants_source()
+        if os.environ.get("RUNNING_PANTS_FROM_SOURCES", "0") == "1"
+        else None
+    )
+    or
     # NB: We expect VERSION to always have an entry and want a runtime failure if this is false.
     # NB: Since "pants" is the namespace for multiple packages, we need to put VERSION underneath
     # the tree that only the `pantsbuild.pants` package owns. Hence `pants._version`.