[RFC 7234] Cache secondary keys with Vary

[krizhanovsky]

Tempesta must process Vary header as special header. TfwCacheEntry must be extended by vary member which contains normalized and compiled request headers mentioned in Vary header.

Note that RFC 7234 4.1 requires matching header fields in different order and with varying spaces. Actually browsers aren't nice enough to send header fields in the same order and w/o unnecessary spaces.

TfwCacheEntry->vary must have following form:

      vary = *hdr
      hdr = special-hdr | common-hdr
      special-hdr = type_len hdr-id hdr-val
      common-hdr = type_len field-name ":" hdr-val
      type_len = 4 byte unsigned int (31-bit length of hdr token, type of the header in most significant bit)
      hdr-id = 2 byte unsigned integer (special header index)
      field-name = field-name (as in RFC 7230 3.2)
      hdr-val = normalize(field-value)

#733 (TDB secondary index) implements true secondary keys at TDB level, so there is no need to sort the headers in vary in any way, that's OK for now just to scan all the header values. normalize() algorithms does (see an example here):

1. merge duplicate headers;
2. remove all q attributes since they don't guarantee anything;
3. remove all spaces;
4. sort all comma separated tokens.

Algorithm to find appropriate TfwCacheEntry should be as follwoing:

1. Match cache entry key with the request key, get next entry if not match;

2. Check value of Vary header of the entry, return the entry of TfwCacheEntry->vary is empty;

3. Compute a digest over request headers mentioned in Vary header:

   3.1. for each hdr from TfwCacheEntry->vary calculate appropriate normalized header value for the request;

   3.2. match the values and get next TfwCacheEntry if doesn't match;

4. Match the computed value with TfwCacheEntry->vary, get next entry if not match;

5. Return the entry as cache hit.

If a response has Vary: *, then it shound't be cached at all since * never match.

The use case for the feature.

Consider the Nginx case from the presentation, slide 19: if Vary contains Accept-Encoding: gzip,deflate (the first curl request), then if a client sends no Accept-Encoding at all (the second request), then there is no Vary headers matching and we should not return stored response.

Vary header must not be set for HTTP/2 push responses in server mode since all browsers just ignore it. See Understanding The Vary Header for discussion of the Vary header.

Upon the release deploy content compression for tempesta-tech.com site.