Update celery to 5.4.0

[pyup-bot]

This PR updates Celery from 5.2.7 to 5.4.0.

Changelog

5.4

environments. The release candidate version is available for testing.
The official release is planned for March-April 2024.

- New Config: worker_enable_prefetch_count_reduction (8581)
- Added "Serverless" section to Redis doc (redis.rst) (8640)
- Upstash's Celery example repo link fix (8665)
- Update mypy version (8679)
- Update cryptography dependency to 41.0.7 (8690)
- Add type annotations to celery/utils/nodenames.py (8667)
- Issue 3426. Adding myself to the contributors. (8696)
- Bump actions/setup-python from 4 to 5 (8701)
- Fixed bug where chord.link_error() throws an exception on a dict type errback object (8702)
- Bump github/codeql-action from 2 to 3 (8725)
- Fixed multiprocessing integration tests not running on Mac (8727)
- Added make docker-docs (8729)
- Fix DeprecationWarning: datetime.datetime.utcnow() (8726)
- Remove `new` adjective in docs (8743)
- add type annotation to celery/utils/sysinfo.py (8747)
- add type annotation to celery/utils/iso8601.py (8750)
- Change type annotation to celery/utils/iso8601.py (8752)
- Update test deps (8754)
- Mark flaky: test_asyncresult_get_cancels_subscription() (8757)
- change _read_as_base64 (b64encode returns bytes) on celery/utils/term.py (8759)
- Replace string concatenation with fstring on celery/utils/term.py (8760)
- Add type annotation to celery/utils/term.py (8755)
- Skipping test_tasks::test_task_accepted (8761)
- Updated concurrency docs page. (8753)
- Changed pyup -> dependabot for updating dependencies (8764)
- Bump isort from 5.12.0 to 5.13.2 (8772)
- Update elasticsearch requirement from <=8.11.0 to <=8.11.1 (8775)
- Bump sphinx-click from 4.4.0 to 5.1.0 (8774)
- Bump python-memcached from 1.59 to 1.61 (8776)
- Update elastic-transport requirement from <=8.10.0 to <=8.11.0 (8780)
- python-memcached==1.61 -> python-memcached>=1.61 (8787)
- Remove usage of utcnow (8791)
- Smoke Tests (8793)
- Moved smoke tests to their own workflow (8797)
- Bugfix: Worker not consuming tasks after Redis broker restart (8796)
- Bugfix: Missing id on chain (8798)

.. _version-5.3.6:

5.4.0

=====

:release-date: 2024-04-17
:release-by: Tomer Nosrati

Celery v5.4.0 and v5.3.x have consistently focused on enhancing the overall QA, both internally and externally.
This effort led to the new pytest-celery v1.0.0 release, developed concurrently with v5.3.0 & v5.4.0.

This release introduces two significant QA enhancements:

- **Smoke Tests**: A new layer of automatic tests has been added to Celery's standard CI. These tests are designed to handle production scenarios and complex conditions efficiently. While new contributions will not be halted due to the lack of smoke tests, we will request smoke tests for advanced changes where appropriate.
- `Standalone Bug Report Script <https://docs.celeryq.dev/projects/pytest-celery/en/latest/userguide/celery-bug-report.html>`_: The new pytest-celery plugin now allows for encapsulating a complete Celery dockerized setup within a single pytest script. Incorporating these into new bug reports will enable us to reproduce reported bugs deterministically, potentially speeding up the resolution process.

Contrary to the positive developments above, there have been numerous reports about issues with the Redis broker malfunctioning
upon restarts and disconnections. Our initial attempts to resolve this were not successful (8796).
With our enhanced QA capabilities, we are now prepared to address the core issue with Redis (as a broker) again.

The rest of the changes for this release are grouped below, with the changes from the latest release candidate listed at the end.

Changes
-------
- Add a Task class specialised for Django (8491)
- Add Google Cloud Storage (GCS) backend (8868)
- Added documentation to the smoke tests infra (8970)
- Added a checklist item for using pytest-celery in a bug report (8971)
- Bugfix: Missing id on chain (8798)
- Bugfix: Worker not consuming tasks after Redis broker restart (8796)
- Catch UnicodeDecodeError when opening corrupt beat-schedule.db (8806)
- chore(ci): Enhance CI with `workflow_dispatch` for targeted debugging and testing (8826)
- Doc: Enhance "Testing with Celery" section (8955)
- Docfix: pip install celery[sqs] -> pip install "celery[sqs]" (8829)
- Enable efficient `chord` when using dynamicdb as backend store (8783)
- feat(daemon): allows daemonization options to be fetched from app settings (8553)
- Fix DeprecationWarning: datetime.datetime.utcnow() (8726)
- Fix recursive result parents on group in middle of chain (8903)
- Fix typos and grammar (8915)
- Fixed version documentation tag from 8553 in configuration.rst (8802)
- Hotfix: Smoke tests didn't allow customizing the worker's command arguments, now it does (8937)
- Make custom remote control commands available in CLI (8489)
- Print safe_say() to stdout for non-error flows (8919)
- Support moto 5.0 (8838)
- Update contributing guide to use ssh upstream url (8881)
- Update optimizing.rst (8945)
- Updated concurrency docs page. (8753)

Dependencies Updates
--------------------
- Bump actions/setup-python from 4 to 5 (8701)
- Bump codecov/codecov-action from 3 to 4 (8831)
- Bump isort from 5.12.0 to 5.13.2 (8772)
- Bump msgpack from 1.0.7 to 1.0.8 (8885)
- Bump mypy from 1.8.0 to 1.9.0 (8898)
- Bump pre-commit to 3.6.1 (8839)
- Bump pre-commit/action from 3.0.0 to 3.0.1 (8835)
- Bump pytest from 8.0.2 to 8.1.1 (8901)
- Bump pytest-celery to v1.0.0 (8962)
- Bump pytest-cov to 5.0.0 (8924)
- Bump pytest-order from 1.2.0 to 1.2.1 (8941)
- Bump pytest-subtests from 0.11.0 to 0.12.1 (8896)
- Bump pytest-timeout from 2.2.0 to 2.3.1 (8894)
- Bump python-memcached from 1.59 to 1.61 (8776)
- Bump sphinx-click from 4.4.0 to 5.1.0 (8774)
- Update cryptography to 42.0.5 (8869)
- Update elastic-transport requirement from <=8.12.0 to <=8.13.0 (8933)
- Update elasticsearch requirement from <=8.12.1 to <=8.13.0 (8934)
- Upgraded Sphinx from v5.3.0 to v7.x.x (8803)

5.4.0rc2

========

:release-date: 2024-03-27
:release-by: Tomer Nosrati

- feat(daemon): allows daemonization options to be fetched from app settings (8553)
- Fixed version documentation tag from 8553 in configuration.rst (8802)
- Upgraded Sphinx from v5.3.0 to v7.x.x (8803)
- Update elasticsearch requirement from <=8.11.1 to <=8.12.0 (8810)
- Update elastic-transport requirement from <=8.11.0 to <=8.12.0 (8811)
- Update cryptography to 42.0.0 (8814)
- Catch UnicodeDecodeError when opening corrupt beat-schedule.db (8806)
- Update cryptography to 42.0.1 (8817)
- Limit moto to <5.0.0 until the breaking issues are fixed (8820)
- Enable efficient `chord` when using dynamicdb as backend store (8783)
- Add a Task class specialised for Django (8491)
- Sync kombu versions in requirements and setup.cfg (8825)
- chore(ci): Enhance CI with `workflow_dispatch` for targeted debugging and testing (8826)
- Update cryptography to 42.0.2 (8827)
- Docfix: pip install celery[sqs] -> pip install "celery[sqs]" (8829)
- Bump pre-commit/action from 3.0.0 to 3.0.1 (8835)
- Support moto 5.0 (8838)
- Another fix for `link_error` signatures being `dict`s instead of `Signature` s (8841)
- Bump codecov/codecov-action from 3 to 4 (8831)
- Upgrade from pytest-celery v1.0.0b1 -> v1.0.0b2 (8843)
- Bump pytest from 7.4.4 to 8.0.0 (8823)
- Update pre-commit to 3.6.1 (8839)
- Update cryptography to 42.0.3 (8854)
- Bump pytest from 8.0.0 to 8.0.1 (8855)
- Update cryptography to 42.0.4 (8864)
- Update pytest to 8.0.2 (8870)
- Update cryptography to 42.0.5 (8869)
- Update elasticsearch requirement from <=8.12.0 to <=8.12.1 (8867)
- Eliminate consecutive chords generated by group | task upgrade (8663)
- Make custom remote control commands available in CLI (8489)
- Add Google Cloud Storage (GCS) backend (8868)
- Bump msgpack from 1.0.7 to 1.0.8 (8885)
- Update pytest to 8.1.0 (8886)
- Bump pytest-timeout from 2.2.0 to 2.3.1 (8894)
- Bump pytest-subtests from 0.11.0 to 0.12.1 (8896)
- Bump mypy from 1.8.0 to 1.9.0 (8898)
- Update pytest to 8.1.1 (8901)
- Update contributing guide to use ssh upstream url (8881)
- Fix recursive result parents on group in middle of chain (8903)
- Bump pytest-celery to 1.0.0b4 (8899)
- Adjusted smoke tests CI time limit (8907)
- Update pytest-rerunfailures to 14.0 (8910)
- Use the "all" extra for pytest-celery (8911)
- Fix typos and grammar (8915)
- Bump pytest-celery to 1.0.0rc1 (8918)
- Print safe_say() to stdout for non-error flows (8919)
- Update pytest-cov to 5.0.0 (8924)
- Bump pytest-celery to 1.0.0rc2 (8928)

.. _version-5.4.0rc1:

5.4.0rc1

========

:release-date: 2024-01-17 7:00 P.M GMT+2
:release-by: Tomer Nosrati

5.3.7

=====
:release-date: 11 April, 2024
:release-by: Tomer Nosrati

The release of v5.3.6 was missing the bumbversion commit so v5.3.7 is only released to sync it back.

.. _version-5.3.6:

5.3.6

=====
:release-date: 27 Mar, 2024
:release-by: Tomer Nosrati

- boto3>=1.26.143 (1890)
- Always convert azureservicebus namespace to fully qualified (1892)
- Pin pytest-sugar to latest version 1.0.0 (1912)
- Upgrade to pytest v8 that removed nose compatibility (1914)
- fix warning for usage of utcfromtimestamp (1926)
- Update pytest to 8.0.2 (1942)
- Hotfix: Fix CI failures (limit redis to <5.0.2 instead of <6.0.0) (1961)
- Expose cancel callback from py-amqp channel.basic_consume (1953)
- Update mypy to 1.9.0 (1963)
- Update pytest to 8.1.1 (1965)
- Pin hypothesis to hypothesis<7 (1966)
- redis>=4.5.2,<5.0.2,!=4.5.5 -> redis>=4.5.2,!=5.0.2,!=4.5.5 (1969)
- add escape hatch for custom JSON serialization (1955)
- Pin pytest-cov to latest version 5.0.0 (1972)

.. _version-5.3.5:

5.3.5

=====
:release-date: 12 Jan, 2024
:release-by: Tomer Nosrati

- Fix ReadTheDocs CI (1827).
- fix(docs): add Redis to the list of transports where SSL is supported (1826).
- Fixed Improper Method Call: Replaced `mktemp` (1828).
- Bump actions/setup-python from 4 to 5 (1829).
- Bump github/codeql-action from 2 to 3 (1832).
- fix: freeze set during ticks iter in async hub (1830).
- azure service bus: fix TypeError when using Managed Identities (1825).
- Fix unacknowledge typo in restore_visible() (1839).
- Changed pyup -> dependabot for updating dependencies (1842).
- Bump pytest from 7.4.3 to 7.4.4 (1843).
- Bump flake8 from 6.0.0 to 7.0.0 (1845).
- Bump mypy from 1.3.0 to 1.8.0 (1844).
- Fix crash when using global_keyprefix with a sentinel connection (1838)
- Fixed version_dev in docs/conf.py (1875).

.. _version-5.3.4:

5.3.4

=====
:release-date: 16 Nov, 2023
:release-by: Asif Saif Uddin

- Use the correct protocol for SQS requests (1807).


.. _version-5.3.3:

5.3.3

=====
:release-date: 6 Nov, 2023
:release-by: Asif Saif Uddin

- Raise access denied error when ack.
- test redis 5.0.0.
- fix azure servicebus using managed identity support (1801).
- Added as_uri method to MongoDB transport - Fixes 1795 (1796).
- Revert "[fix 1726] Use boto3 for SQS async requests (1759)" (1799).
- Create a lock on cached_property if not present (1811).
- Bump kafka deps versions & fix integration test failures (1818).
- Added Python 3.12 support.
- Fix: redis requeue concurrency bug 1800 (1805).


.. _version-5.3.2:

5.3.2

=====
:release-date: 31 Aug, 2023
:release-by: Tomer Nosrati

- Reverted unwanted constraint introduced in 1629 with max_retries (1755)
- Doc fix (hotfix for 1755) (1758)
- Python3.12: fix imports in kombu/utils/objects.py (1756)
- [fix 1726] Use boto3 for SQS async requests (1759)
- docs: Remove SimpleQueue import (1764)
- Fixed pre-commit issues (1773)
- azure service bus: add managed identity support (1641)
- fix: Prevent redis task loss when closing connection while in poll (1733)
- Kombu & celery with SQS 222 (1779)
- syntax correction (1780)

.. _version-5.3.1:

5.3.1

=====
:release-date: 15 Jun, 2023
:release-by: Asif Saif Uddin

- Update pycurl version.
- Declare python 3.11 support (1425).
- Fix: allow deserializing any version of UUID.
- Update PyCurl version in SQS (1747).


.. _version-5.3.0:

5.3.0

=====
:release-date: 03 Jun, 2023
:release-by: Asif Saif Uddin

- Support for Kafka as transport.
- Add fanout to filesystem (1499).
- Added possibility to serialize and deserialize binary messages in json (1516).
- Support pymongo 4.x (1536).
- Support redis-py 4.5.x.
- Upgrade Azure Storage Queues transport to version 12 (1539).
- Add support to SQS DelaySeconds (1567).
- Add WATCH to prefixed complex commands.
- Avoid losing type of UUID when serializing/deserializing (1575).
- Added HLEN to the list of prefixed redis commands (1540).
- Add managed identity support to azure storage queue (1631).
- Add Support of sqlalchemy v2.0.
- Deprecate pytz and use zoneinfo (1680)


.. _version-5.3.0rc2:

5.3.0rc2

========
:release-date: 31 May, 2023
:release-by: Asif Saif Uddin

- add missing zoneinfo dependency (1732).
- Support redis >= 4.5.2
- Loosen urlib3 version range for botocore compat


.. _version-5.3.0rc1:

5.3.0rc1

========
:release-date: 24 May, 2023
:release-by: Asif Saif Uddin

- Moved to pytest-freezer (1683).
- Deprecate pytz and use zoneinfo (1680).
- handle keyerror in azureservicebus transport when message is not
found in qos and perform basic_ack (1691).
- fix mongodb transport obsolete calls (1694).
- SQS: avoid excessive GetQueueURL calls by using cached queue url (1621).
- Update confluentkafka.txt version (1727).
- Revert back to pyro4 for now.


.. _version-5.3.0b3:

5.3.0b3

=======
:release-date: 20 Mar, 2023
:release-by: Asif Saif Uddin

- Use SPDX license expression in project metadata.
- Allowing Connection.ensure() to retry on specific exceptions given by policy (1629).
- Redis==4.3.4 temporarilly in an attempt to avoid BC (1634).
- Add managed identity support to azure storage queue (1631).
- Support sqla v2.0 (1651).
- Switch to Pyro5 (1655).
- Remove unused _setupfuns from serialization.py.
- Refactor: Refactor utils/json (1659).
- Adapt the mock to correctly mock the behaviors as implemented on Python 3.10. (Ref 1663).


.. _version-5.3.0b2:

5.3.0b2

=======
:release-date: 19 Oct, 2022
:release-by: Asif Saif Uddin

- fix: save QueueProperties to _queue_name_cache instead of QueueClient.
- hub: tick delay fix (1587).
- Fix incompatibility with redis in disconnect() (1589).
- Solve Kombu filesystem transport not thread safe.
- importlib_metadata remove deprecated entry point interfaces (1601).
- Allow azurestoragequeues transport to be used with Azurite emulator in docker-compose (1611).


.. _version-5.3.0b1:

5.3.0b1

=======
:release-date: 1 Aug, 2022
:release-by: Asif Saif Uddin

- Add ext.py files to setup.cfg.
- Add support to SQS DelaySeconds (1567).
- Add WATCH to prefixed complex commands.
- Avoid losing type of UUID when serializing/deserializing (1575).
- chore: add confluentkafka to extras.

.. _version-5.3.0a1:

5.3.0a1

=======
:release-date: 29 Jun, 2022
:release-by: Asif Saif Uddin

- Add fanout to filesystem (1499).
- Protect set of ready tasks by lock to avoid concurrent updates. (1489).
- Correct documentation stating kombu uses pickle protocol version 2.
- Use new entry_points interface.
- Add mypy to the pipeline (1512).
- Added possibility to serialize and deserialize binary messages in json (1516).
- Bump pyupgrade version and add __future__.annotations import.
- json.py cleaning from outdated libs (1533).
- bump new py-amqp to 5.1.1 (1534).
- add GitHub URL for PyPi.
- Upgrade pytest to ~=7.1.1.
- Support pymongo 4.x (1536).
- Initial Kafka support (1506).
- Upgrade Azure Storage Queues transport to version 12 (1539).
- move to consul2 (1544).
- Datetime serialization and deserialization fixed (1515).
- Bump redis>=4.2.2 (1546).
- Update sqs dependencies (1547).
- Added HLEN to the list of prefixed redis commands (1540).
- Added some type annotations.


.. _version-5.2.4:

Links

• PyPI: https://pypi.org/project/celery
• Changelog: https://data.safetycli.com/changelogs/celery/
• Homepage: https://docs.celeryq.dev/
• Docs: https://pythonhosted.org/celery/