Support Redis + TLS URLs

[dstufft]

redis-py supports redis instances which are secured by TLS by using rediss://URLs instead of redis:// URLs. Celery should support this too.

[ask]

Does it not have an option for that? Or is this a special option that can only be set in the new url format?

[dstufft]

Celery doesn't know that rediss:// (note the two s) is a redis backend. See https://github.com/pypa/warehouse/blob/master/warehouse/celery.py

[ask]

Yeah, but we don't pass this URL to Redis, we need an option to set

[dstufft]

https://github.com/pypa/warehouse/blob/master/warehouse/celery.py#L38

[ask]

Closing this, as we don't have the resources to complete this task.

[lern]

Any plan to support the rediss:// schema?

[smcoll]

How does this issue correspond to #3831?

[chriskuehl]

It should work fine, we're using SSL on the broker and the backend for Redis. Broker support has been around for a while (not sure when it was added), backend support for SSL was added in #3831. I think this ticket can be closed.

Here are the docs:
http://docs.celeryproject.org/en/latest/userguide/configuration.html

The relevant options are redis_backend_use_ssl and broker_use_ssl (which has a section on redis).

[dstufft]

Those options existed when I originally opened this issue, but this issue isn't about supporting TLS at all, it was specifically about supporting rediss:// URLs to turn on the TLS support.

[chriskuehl]

The backend option definitely didn't exist when you opened the ticket (I added it in #3831 only a few months ago). But that's a good point, maybe the ticket should be re-opened.

[remeika]

I think this ticket should actually be broken out into two closely-related issues:

• Add Redis + TLS connections to broker_url setting (Add Redis + TLS connections to broker_url setting #4694)
• Add Redis + TLS connections to result_backend setting (Add Redis + TLS connections to result_backend setting #4695)

Celery processes the value of result_backend in its own code, and directly instantiates a redis connection from redis-py. But the value of broker_url is passed to Kombu, which also must be patched.

[gonzalodiaz]

+1 We need the fix from Kombu as well.