[Bug]: Cannot use redis cache without TTL

[eliorc]

What happened?

The current implementation of the Redis cache, prevents users from configuring cache as never-expiring.

This occurs mainly because of thes lines:

litellm/litellm/caching/redis_cache.py

Lines 142 to 149 in 24dd655

	ttl = self.get_ttl(**kwargs)
	print_verbose(
	f"Set Redis Cache: key: {key}\nValue {value}\nttl={ttl}, redis_version={self.redis_version}"
	)
	key = self.check_and_fix_namespace(key=key)
	try:
	start_time = time.time()
	self.redis_client.set(name=key, value=str(value), ex=ttl)

jointly with these:

litellm/litellm/caching/redis_cache.py

Lines 120 to 123 in 24dd655

	if litellm.default_redis_ttl is not None:
	super().__init__(default_ttl=int(litellm.default_redis_ttl))
	else:
	super().__init__() # defaults to 60s

litellm/litellm/caching/base_cache.py

Lines 22 to 34 in 24dd655

	class BaseCache(ABC):
	def __init__(self, default_ttl: int = 60):
	self.default_ttl = default_ttl
	def get_ttl(self, **kwargs) -> Optional[int]:
	kwargs_ttl: Optional[int] = kwargs.get("ttl")
	if kwargs_ttl is not None:
	try:
	return int(kwargs_ttl)
	except ValueError:
	return self.default_ttl
	return self.default_ttl

Essentially the line .set(..., ex=ttl) combined with the other settings makes sure we can't get an infinite ttl. I've tried setting -1 in the settings (using the proxy) but that didn't work. Going into redis documentation I'm quite sure the only way to set an infinite ttl is to not pass it at all - e.g. have it set to the default None. But when we set it to None, litellm's internals convert it to the default of 60 seconds.

I thought about opening my own PR on this - but I'm not sure what is the correct way to go. Should the default be None making default TTLs infinite? Should defaults be configured on a framework level? Should we offer a constant value to enable litellm users to set an inifinite TTL (like an enum TTL.INFINITE or something).

Relevant log output

No response

Are you a ML Ops Team?

No

What LiteLLM version are you on ?

v1.58.1

Twitter / LinkedIn details

No response