Release Django 5.0 compatible package to PyPI

[nijel]

Summary:

Include a brief description of the problem here, and fill out the version info below.

• Celery Version: 5.3.5
• Celery-Beat Version: 2.5.0

Exact steps to reproduce the issue:

1. Try to install Django 5.0a1 with django-celery-beat
2. django-celery-beat 2.5.0 has requirement Django<5.0,>=2.2, but you have django 5.0a1.

Detailed information

See https://docs.djangoproject.com/en/dev/releases/5.0/ for release notes.

[auvipy]

• Update runtime.txt to include Django 5.0 #681

[yanivtoledano]

@auvipy - is there a timeline for the next release (with 5.0 support)?

[auvipy]

Hopefully early next week.

[kozlek]

Hi 👋

Based on what I understood, Django 5.0 support was simply a matter of relaxing the dependency constraints.
I'm not challenging the fact we have a dependency constraint with an upper bound.

The patch was made on the Sep 19, which is the release date of the first Django 5.0 alpha. This is an achievement: we managed to add support for the new Django release very early in the development process 🥳

However, 2 months later and 4 days after the final Django release, users are still not able to use it (or even test it easily) because we failed to release the working code we produced 2 months ago 😕

Far from judging (I've done manual releases myself for some time so I know how time consuming it is 😉), what can we do to help you @auvipy to ease the release process of this version and the following ones?

I'll be happy to look into a CI pipeline to provide an automated release if you are okay for it, but if you prefer to do it manually tell us if you need new maintainers for reviewing PRs or doing issues triage 🙂

[cclauss]

% python3.12 -m venv .venv
% source .venv/bin/activate
% pip install --upgrade pip
% pip install django==5.0 django-celery-beat==2.5.0 --> Failure
% pip install django==5.0 django-celery-beat --> Success but with

• django-celery-beat downgraded from 2.5.0 --> v2.1.0 and
• django-timezone-field downgraded from v6.1.0--> v4.2.3

Please release a new version to PyPI.

[kozlek]

I can't afford to further delay my Django 5.0 upgrade as I have planned features that depend on the new version capabilities.
As a workaround, I released a custom build of django-celery-beat on PyPi under another name.

As I don't expected anyone to trust my build, here are the steps I used to build and upload it if someone needs to do it for himself.
You need a PyPi account to do so, and my instructions require pipx, pyenv and a GNU version of sed.

# install twine to upload package to PyPi
pipx install twine

# install latest version of python (not mandatory) and enable it
pyenv install 3.12.1
pyenv virtualenv 3.12.1 dev
pyenv shell dev
pip install setuptools wheel # python 3.12.0+ does not install them by default

# get source code
git clone git@github.com:celery/django-celery-beat.git
cd django-celery-beat

# rename the package (replace custom by what you want)
sed -i 's/django-celery-beat/django-celery-beat-custom/g' setup.py 

# build the package
python setup.py sdist bdist_wheel

# upload to pypi
twine upload dist/*

[cclauss]

Were you able to get the tests to pass on Py3.12 and Dj5.0?

[kozlek]

Launching tox with the latest commit of the master branch using tox -e py312 installs Django 5.0 and reports 1 failure.

I choose to ignore it because it's related to the removal of django.utils.timezone.utc alias and it only affects systems running with USE_TZ=False which is not my case.

[kozlek]

It's quite easy to fix, as we simply need to use datetime.timezone.utc here instead, as we already do here.

However, we might also want to fix in the same time warnings regarding the deprecation of datetime.utcnow(), as I did for djangorestframework-simplejwt: jazzband/djangorestframework-simplejwt#765

IMHO, we should simply get rid of old python / django versions, remove support for USE_TZ=False and native datetime.
It would simplify library code a lot and it seems to be the direction taken by both Django and Python.

[cclauss]

• Django v5.0: django.utils.timezone.utc alias --> datetime.timezone.utc #703

.utcnow() is a Python 3.13 thing but upstream groundwork is in progress... celery/celery#8726

[chirag-jn]

Is there an ETA for django 5 support? I see that there's work in progress but I am eagerly waiting for the release.

[chirag-jn]

I believe that the celery project has removed references to datetime.utcnow so we can proceed with the release of django-celery-beat for django 5.0

[cclauss]

@nijel Please consider changing the title of this issue to Django 5.0 release to PyPI since I believe that is the only step that is still required.

Others, If you want to provide maintainers more confidence to make this release then please consider installing the main branch of this repo with pip install git+https://github.com/celery/django-celery-beat.git and then run your tests and exercise the Celery pieces of your Django app(s) on both Django 4.2 and 5.0. If you see problems, please comment here. I did this and saw no problems on localhost and our staging server.

[heckad]

Can you please release at least a release candidate? Otherwise, testing without hashes is somehow not very good in modern world.

[heckad]

We have a command in ci to install from pip with the --require-hashes flag. If you just add a link to git there will be an error. Have you really never used the --require-hashes flag in your life and did not understand what hashes we are talking about?

[amweiss]

Our requirements.txt files has been using:

django-celery-beat @ git+https://github.com/celery/django-celery-beat@f66796b9c5c9e47d118cb5b41d970f07cce72bbe

For the past 2 weeks against Django 5.0.1 and Python 3.11 without any issues.

[heckad]

Doesn't work

# pip install -r requirements.txt --require-hashes                                                                                                                                                                                                  
Collecting django-celery-beat@ git+https://github.com/celery/django-celery-beat@f66796b9c5c9e47d118cb5b41d970f07cce72bbe (from -r requirements.txt (line 1))
ERROR: Can't verify hashes for these requirements because we don't have a way to hash version control repositories:
    django-celery-beat@ git+https://github.com/celery/django-celery-beat@f66796b9c5c9e47d118cb5b41d970f07cce72bbe from git+https://github.com/celery/django-celery-beat@f66796b9c5c9e47d118cb5b41d970f07cce72bbe (from -r requirements.txt (line 1))

[cclauss]

@heckad

• Validate VCS urls in hash-checking mode using their commit hashes pypa/pip#6469

[DrJfrost]

any news about the release to the pipy package? I think there is way too much focus on django 5.1 without proper implementation for django 5 support

[csulit]

Any news regarding the upgrade? :)

[Couapy]

Hello 👋
Any updates ? Still blocked on the subject

[cclauss]

@csulit @Couapy

• Release Django 5.0 compatible package to PyPI #680 (comment)
• django 5.0 #698 (comment) --> Please stop posting "any news ?" messages...
• https://pypi.org/project/django-celery-beat

[amweiss]

What else needs to happen here other than tag the version and push it to pypi?

[cclauss]

@amweiss #680 (comment)

[amweiss]

@cclauss yes, I guess I'm wondering what else needs to happen before a maintainer tags and releases as we have all been using it from the git repo for a while now without issues. If there's some further evidence needed I'm happy to supply it. We have been running against the git version for about 4 weeks now without issue.

[deronnax]

Can I suggest to give up upper bound constraints?

It's considered a pretty negative practice, see this article on the subject and Poetry's FAQ (EDIT: link updated).
It hit us in our migration to Django 5, preventing to upgrading django without cheating the package manager, whereas as for 99% of upper version constrained packages, it works actually just fine with the unduly forbidden upper version.

Upper-bound constraints should be solely reserved for the very rare case where you tried with the new version and you know it doesn't work.

[ddahan]

@deronnax thanks!
PS: the second link should be this one I guess

[cclauss]

The euphemism upper-bound constraints should be replaced with the traditional name — a software release process.

Why deprive the maintainers of this project the alpha, beta, production release process that Linux, macOS, Windows, Python, Django, and even Poetry all have? This software operates in complex combinations with other systems and is used to process mission-critical data.

If maintainers are prohibited from testing/repairing before production use then they will need to deal with software quality rumors/concerns and respond to complex issues opened by unhappy users.

Just ask yourself why Poetry releases with alpha, beta, and production releases but encourages others not to follow their own example.

Occasional "spring cleaning" from multiple contributors to prepare for new versions of Python and Django helps to keep the code modernized and functioning properly. Pull requests focused on getting this repo ready for Django v5:

• Update runtime.txt to include Django 5.0 #681
• Upgrade GitHub Actions and PyPy 3.10 and Django 5.0 #699
• Django v5.0: django.utils.timezone.utc alias --> datetime.timezone.utc #703
• Testing Django v5.0 on pypy-3.10-v7.3.14 passes #705
• Prepare for release v2.6.0 to support Py3.12 and Dj5.0 #712
• GitHub Actions: Do not hardcode an out-of-date version of PyPy #715
• Remove requirements/test-djangoXY.txt #728
• Remove code for unsupported django.VERSION < (3, 2) #729

[FluxZA]

#680 (comment) with regard to this issue, we have upgraded as suggested and are experiencing no issues on Django 5.

[deronnax]

@cclauss I think you are confusing two different subjects. I don't see why you oppose relax-bound constraints for libraries and a software release process. A ton (actually most) of python libraries do have a release process and have no upper-bound constraints ; Django being the first of them (see its setup.cfg dependencies list). As for Poetry, they are not a library, they are an application. An application is the very final chain link of its dependency chain, so an app can do whatever it wants on the expression of its dependencies. 99% of the time, they are all pinned to the exact version. That's what Poetry does, you can see it because they have a poetry.lock.

[nlsfnr]

Building on @deronnax's points, I want to emphasize the option for downstream applications to lock their dependency versions (e.g., django==4.2) at their discretion.

This decision should, in my opinion, not be made by upstream library authors, especially if there is no concrete reason / evidence to believe that newer versions of the library's dependencies will break the library. This is especially true for frameworks or libraries, such as Django, that have a disproportionate impact on downstream code bases where unnecessary version locking can cause headaches.

As a side note, we are also not experiencing issues running Django 5.0.

[gav-fyi]

#680 (comment) with regard to this issue, we have upgraded as suggested and are experiencing no issues on Django 5.

I'd like to also confirm that I have been running a fairly large project (with its fair share of celery-related quirks) with the recent changes (Pinned at acd10a390c85414a59e9b37be6226e4a9ea4fb81) with no issues.

[auvipy]

I would like to revisit this issue very soon.

[cclauss]

there is no concrete reason / evidence to believe that newer versions of the library's dependencies will break the library.

Please review the eight pull requests above. For instance... utils.make_aware() will crash on Dj5 without #703 (comment)

[csulit]

The only issue with my setup now is that the task is not registered on the Django admin. However, it works if I register the task via a custom property.

requirements.txt

[sandeepbol]

When can we expect a release? Any ETAs?

[Nusnus]

When can we expect a release? Any ETAs?

Are we ready for a release? @cclauss

[cclauss]

Yes. Please look at #737 and then let's release!

[osuthorpe]

Was this released?

[cclauss]

Was this released?

YES... https://pypi.org/project/django-celery-beat is v2.6.0