build: Python requirements upgrade and lxml build without binary

[edx-requirements-bot]

Description

Python requirements update. Please review the changelogs for the upgraded packages.

• lxml>5.0 needs to be build with --no-binary flag so updated the requirements and test workflows to resolve this issue.
• Copied solution from https://stackoverflow.com/a/25974164.

[edx-requirements-bot]

List of packages in the PR without any issue.

• acid-xblock changes from 0.3.0 to 0.3.1
• aiohttp changes from 3.9.3 to 3.9.5
• asgiref changes from 3.7.2 to 3.8.1
• boto3 changes from 1.34.26 to 1.34.93
• boto3 changes from 1.34.45 to 1.34.93
• botocore changes from 1.34.45 to 1.34.93
• botocore changes from 1.34.26 to 1.34.93
• build changes from 1.0.3 to 1.2.1
• cachetools changes from 5.3.2 to 5.3.3
• camel-converter[pydantic] changes from 3.1.1 to 3.1.2
• celery changes from 5.3.6 to 5.4.0
• click-didyoumean changes from 0.3.0 to 0.3.1
• coverage changes from 7.4.1 to 7.5.0
• coverage[toml] changes from 7.4.1 to 7.5.0
• cssutils changes from 2.9.0 to 2.10.2
• ddt changes from 1.7.1 to 1.7.2
• django changes from 4.2.10 to 4.2.11
• django changes from 4.2.9 to 4.2.11
• django-mysql changes from 4.12.0 to 4.13.0
• django-statici18n changes from 2.4.0 to 2.5.0
• drf-spectacular changes from 0.27.1 to 0.27.2
• edx-braze-client changes from 0.2.2 to 0.2.3
• edx-i18n-tools changes from 1.5.0 to 1.6.0
• exceptiongroup changes from 1.2.0 to 1.2.1
• execnet changes from 2.0.2 to 2.1.1
• faker changes from 24.14.0 to 24.14.1
• fastapi changes from 0.109.2 to 0.110.2
• filelock changes from 3.13.1 to 3.13.4
• freezegun changes from 1.4.0 to 1.5.0
• gitpython changes from 3.1.42 to 3.1.43
• google-api-core changes from 2.15.0 to 2.18.0
• google-api-python-client changes from 2.115.0 to 2.127.0
• google-auth changes from 2.26.2 to 2.29.0
• googleapis-common-protos changes from 1.62.0 to 1.63.0
• icalendar changes from 5.0.11 to 5.0.12
• idna changes from 3.6 to 3.7
• importlib-resources changes from 6.1.1 to 6.4.0
• joblib changes from 1.3.2 to 1.4.0
• jwcrypto changes from 1.5.4 to 1.5.6
• kombu changes from 5.3.5 to 5.3.7
• lti-consumer-xblock changes from 9.10.0 to 9.11.0
• mako changes from 1.3.2 to 1.3.3
• markupsafe changes from 2.1.4 to 2.1.5
• maxminddb changes from 2.5.2 to 2.6.1
• meilisearch changes from 0.30.0 to 0.31.0
• mongoengine changes from 0.27.0 to 0.28.2
• moto changes from 4.2.13 to 4.2.14
• mypy changes from 1.8.0 to 1.10.0
• newrelic changes from 9.6.0 to 9.9.0
• newrelic changes from 9.5.0 to 9.9.0
• pact-python changes from 2.1.1 to 2.2.0
• peewee changes from 3.17.1 to 3.17.3
• pillow changes from 10.2.0 to 10.3.0
• pip-tools changes from 7.4.0 to 7.4.1
• platformdirs changes from 4.1.0 to 4.2.1
• pluggy changes from 1.3.0 to 1.5.0
• pluggy changes from 1.4.0 to 1.5.0
• protobuf changes from 4.25.2 to 4.25.3
• pyasn1 changes from 0.5.1 to 0.6.0
• pyasn1-modules changes from 0.3.0 to 0.4.0
• pycparser changes from 2.21 to 2.22
• pydantic changes from 2.6.3 to 2.7.1
• pydantic-core changes from 2.16.3 to 2.18.2
• pyparsing changes from 3.1.1 to 3.1.2
• pyproject-hooks changes from 1.0.0 to 1.1.0
• pytest changes from 8.1.2 to 8.2.0
• pytest-xdist[psutil] changes from 3.5.0 to 3.6.1
• python-dateutil changes from 2.8.2 to 2.9.0.post0
• python-swiftclient changes from 4.4.0 to 4.5.0
• redis changes from 5.0.1 to 5.0.4
• referencing changes from 0.33.0 to 0.35.0
• regex changes from 2024.4.16 to 2024.4.28
• requests-mock changes from 1.11.0 to 1.12.1
• responses changes from 0.24.1 to 0.25.0
• rich changes from 13.7.0 to 13.7.1
• s3transfer changes from 0.10.0 to 0.10.1
• setuptools changes from 69.1.0 to 69.5.1
• shapely changes from 2.0.3 to 2.0.4
• simple-salesforce changes from 1.12.5 to 1.12.6
• sniffio changes from 1.3.0 to 1.3.1
• snowflake-connector-python changes from 3.7.0 to 3.9.1
• sphinx-mdinclude changes from 0.5.3 to 0.6.0
• sqlparse changes from 0.4.4 to 0.5.0
• starlette changes from 0.36.3 to 0.37.2
• stevedore changes from 5.1.0 to 5.2.0
• testfixtures changes from 8.0.0 to 8.1.0
• tomlkit changes from 0.12.3 to 0.12.4
• tox changes from 4.11.4 to 4.15.0
• types-pytz changes from 2024.1.0.20240203 to 2024.1.0.20240417
• types-pyyaml changes from 6.0.12.12 to 6.0.12.20240311
• typing-extensions changes from 4.9.0 to 4.11.0
• uvicorn changes from 0.27.1 to 0.29.0
• virtualenv changes from 20.25.0 to 20.26.0
• wcmatch changes from 8.5 to 8.5.1
• werkzeug changes from 3.0.1 to 3.0.2
• wheel changes from 0.42.0 to 0.43.0
• xblock-google-drive changes from 0.6.1 to 0.7.0
• xblock[django] changes from 4.0.0 to 4.0.1
• zipp changes from 3.17.0 to 3.18.1

[edx-requirements-bot]

These Packages need manual review..

• [MAJOR] certifi changes from 2023.11.17 to 2024.2.2
• [MAJOR] django-ipware changes from 6.0.4 to 7.0.1
• [REMOVED] importlib-resources (6.1.1) removed from the requirements
• [MAJOR] lxml changes from 4.9.3 to 5.2.1
• [REMOVED] lxml (4.9.4) removed from the requirements
• [NEW] lxml-html-clean (0.1.1) added to the requirements
• [NEW] lxml[html-clean] (5.2.1) added to the requirements
• [MAJOR] mistune changes from 2.0.5 to 3.0.2
• [MAJOR] packaging changes from 23.2 to 24.0
• [REMOVED] pendulum (3.0.0) removed from the requirements
• [MAJOR] platformdirs changes from 3.11.0 to 4.2.1
• [NEW] proto-plus (1.23.0) added to the requirements
• [REMOVED] pyjwt (2.8.0) removed from the requirements
• [NEW] pyjwt[crypto] (2.8.0) added to the requirements
• [MAJOR] pytest changes from 7.4.4 to 8.2.0
• [MAJOR] python-ipware changes from 2.0.1 to 3.0.0
• [MAJOR] pytz changes from 2023.3.post1 to 2024.1
• [MAJOR] pywatchman changes from 1.4.1 to 2.0.0
• [MAJOR] requests-file changes from 1.5.1 to 2.0.0
• [REMOVED] time-machine (2.13.0) removed from the requirements
• [REMOVED] tzdata (2023.4) removed from the requirements
• [REMOVED] zipp (3.17.0) removed from the requirements

[UsamaSadiq]

All the upgrades which were causing test failures have been pinned in separate PRs. The remaining packages are all inter-dependent and the upgrade job fails to upgrade the package individually hence all of these needs to be merged in the same PR.

[timmc-edx]

Need to revert this to unblock the build pipeline, more details in PR description: #34661

[edx-pipeline-bot]

2U Release Notice: This PR has been deployed to the edX staging environment in preparation for a release to production.

[edx-pipeline-bot]

2U Release Notice: This PR has been deployed to the edX production environment.

[edx-pipeline-bot]

2U Release Notice: This PR has been deployed to the edX production environment.