Cache tag purger

[mhf-ir]

I need a solution to my front end proxcy/cdn server to store cache in tags:

Example:
https://support.cloudflare.com/hc/en-us/articles/206596608-How-to-Purge-Cache-Using-Cache-Tags

Example this url:

http://example.tld/category34/article32.html?method=1

Orgin server response the header:

Pragma: public
Cache-Control: max-age=86400
Expires: Mon, 12 Mar 2018 20:58:27
Content-Type: text/html
X-Cache-Tags: ["category34", "article32"]

So the trafficserver knows this reposne must be cache but with two tag category34 and article32.

Now think my backend app need to have update whole cache system with tag category34 send purger request to trafficserver and trafick server remove whole cache of any items that contain category34 tag:

example:

curl -X DELETE -H "Content-Type: application/json" -d '["category34"]' http://trafficservercdn.tld/_cache/_purger

it's can be authentication via special header or special ips

[SolidWallOfCode]

What would such tags do? Distinguish the object from other objects with different tags? You might be able to use the cachekey plugin to do that.

[mhf-ir]

Thank you for response 😄

Caching the response front of cdn/proxy/webserver is fastest way to send data to client. Like cloudflare if the backend server response the special header to tell cache server thats my tags and beside the cache id/cache key the tags also stored. now if in any process we know how to tell cache server to purge all cached contents by cachetags not cachekey.

Another explanation, think about 2 response from backend server(s):

Response 1:
  http://example.tld/category34/article32.html?method=1
  X-Cache-Tags: ["category34", "article32"]
    
    CDN/Proxy/Cache server now store the response with cachekey bd9561eca6c3bf92f8af92a75e5f8dc8 and cachetags: ["category34", "article32"]

Response 2:
  http://example.tld/category33/article32.html?method=1
  X-Cache-Tags: ["category33", "article32"]
  
      CDN/Proxy/Cache server now store the response with cachekey d660279e6fe81bde4fcbc3ae2d1744f7 and cachetags: ["category33", "article32"]

Response 3:
  http://example.tld/category99/article99.html?method=1
  X-Cache-Tags: ["category99", "article99"]
  
      CDN/Proxy/Cache server now store the response with cachekey 953cb2849369cbb4026ef1423137b4d7 and cachetags: ["category34", "article32"]

No we need to purge any response that has by backend application article32 so we dont know anything about cachekey. but backend knows the tags of edited objects.
So we send purge method to cdn/proxy/cache server to purge all caches with any id/keys with our tags

curl -X DELETE -H "Content-Type: application/json" -d '["article32"]' http://trafficservercdn.tld/_cache/_purger

Then our responses 1 and 2 will be purge from caches cause have article32 tag and response 3 will be untouched.

Seems be CacheKey plugin just can manipulate cachekey. but we need another attribute to purge and manipulate caches, cache key is the way. I cannot find out any method to use cache tag mechanism

This is way in many layers use this mechanism, but in front of cdn/proxy/cache server will be awesome. See some example of php cache in application layer:
https://framework.zend.com/manual/1.10/en/zend.cache.theory.html#zend.cache.tags
https://www.drupal.org/docs/8/api/cache-api/cache-tags
https://laravel.com/docs/5.6/cache#cache-tags

and as i mentioned before cloudflare also use it:
https://support.cloudflare.com/hc/en-us/articles/206596608-How-to-Purge-Cache-Using-Cache-Tags

[jbergstroem]

@mhf-ir: You can just create your own cache-keys containing these tags. With the now-released lua plugin it is probably even easier to modify the state of requests and how you want them to live in cache.

[mhf-ir]

@jbergstroem Cache tags is different concept if you see the samples you will see. cache id is key value but cache tags is relational data type and need different implementation for retrieve or save or delete by them.

[jbergstroem]

@mhf-ir I am merely stating that you can reimplement it yourself without too much headache in lua.

[mhf-ir]

@jbergstroem of course we can reimplement, lua work on nginx well but from trafficker we expect more than simple cache server. and according cloudflare sample it's a very useful feature .

[shukitchan]

you can take a look at this POC, implementing the cache tag mechanism using lua - https://github.com/shukitchan/ats_lua_scripts/blob/master/cache_tags.lua

[zwoop]

The POC is pretty interesting, but storing "state" in librecords is really not that great. API metrics are limited in numbers, it would not be able to deal with many thousands. Also, you have to send the BAN to every ATS instance. Using some sort of distributed database would be better. Or, if a distributed DB is not feasible, then a local DB like sqlite or BDB or something would be better as well.

Also, look at the regex_revalidate plugin, it doesn't revalidate on tags, but could likely be modified to work on something like these tags in a similar way that this Lua plugin deals with it. I think the concept in the Lua and regex_revalidate is very, very similar.

[mhf-ir]

Seems be next generation of developer initialize it :
https://github.com/valeriansaliou/bloom

[bryancall]

I think this would be better handled by an external db and a plugin. If someone would like to work on this please assign yourself the issue or ask to be assigned to it.