feat(proxy-cache): support memory-based strategy

[agile6v]

What this PR does / why we need it:

Fix #1304

Pre-submission checklist:

• Did you explain what problem does this PR solve? Or what new features have been added?
• Have you added corresponding test cases?
• Have you modified the corresponding document?
• Is this PR backward compatible? If it is not backward compatible, please discuss on the mailing list first

[spacewander]

Could you provide a benchmark between the disk & the memory cache?

[agile6v]

Hi @spacewander

I think these two strategies are suitable for different scenarios. It may be difficult to get a reasonable test result.

For example:

These are suitable for disk strategy,

1. Large files (> 1MB)
2. Lots of large files
3. Small files (< 1MB) with hotspots
4. Persistence

For memory strategy,

1. Small files (< 1MB)
2. The amount of the cached data doesn't exceed the host memory

As a result, these two scenarios can compare is small files with hotspots. But Nginx's proxy-cache send cache file requires 2 disk io (One for header part, another for body part), here ignores pagecache because it's uncertain. Maybe the result is already known.

In addition, sorry for the limited environment, I cannot finish the benchmark. If someone can help to accomplish this test, I can provide the test steps.

Configure proxied server

1. generate test files in /home/nginx/data/

dd if=/dev/sda of=4k bs=4 count=1024
dd if=/dev/sda of=64k bs=64 count=1024
dd if=/dev/sda of=512k bs=512 count=1024
dd if=/dev/sda of=1024k bs=1024 count=1024

2. configure location in proxied server

   server {
       listen 1996;
       server_name localhost;

   	location / {
   	    if ($arg_size) {
   	        rewrite .* /${arg_size} break;
   	    }
   	
   	    root /home/nginx/data/;
   	}

   }

Configure the APISIX

need to download master branch: https://github.com/agile6v/apisix

When testing memory strategy, we need to use this configuration.

curl http://127.0.0.1:9080/apisix/admin/routes/1  -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
{
    "plugins": {
        "proxy-cache": {
            "cache_strategy": "memory",
            "cache_key":  ["$uri", "-cache-id"],
            "cache_bypass": ["$arg_bypass"],
            "cache_zone": "memory_cache",
            "cache_method": ["GET"],
            "cache_http_status": [200],
            "hide_cache_headers": true,
            "no_cache": ["$arg_nocache"]
        }
    },
    "upstream": {
        "nodes": {
            "127.0.0.1:1996": 1
        },
        "type": "roundrobin"
    },
    "uri": "/*"
}'

This for disk strategy.

curl http://127.0.0.1:9080/apisix/admin/routes/1  -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
{
    "plugins": {
        "proxy-cache": {
            "cache_strategy": "disk",
            "cache_key":  ["$uri", "-cache-id"],
            "cache_bypass": ["$arg_bypass"],
            "cache_zone": "disk_cache_one",
            "cache_method": ["GET"],
            "cache_http_status": [200],
            "hide_cache_headers": true,
            "no_cache": ["$arg_nocache"]
        }
    },
    "upstream": {
        "nodes": {
            "127.0.0.1:1996": 1
        },
        "type": "roundrobin"
    },
    "uri": "/*"
}'

Verify the configuration

curl -v -o /dev/null http://127.0.0.1:9080/hello?size=4k

Benckmark tool use wrk

./wrk http://127.0.0.1:9080 -s scripts/random_url.lua

random_url.lua contains the following code.

math.randomseed(os.time())

function Random(n)
    local t = {
        "a","b","c","d","e","f","g","h","i","j","k","l","m","n","o","p","q","r","s","t","u","v","w","x","y","z",
    }
    local s = ""
    for i =1, n do
        s = s .. t[math.random(#t)]
    end;
    return s
end

local arg_size = "?size=4k"

request = function()
   return wrk.format("GET", "/" .. Random(3) .. arg_size)
end{}

Note:
Random(3) will generate 17000+ unique random url
Random(4) will generate 456976+ unique random url

[spacewander]

Please merge master to make Chaos test pass

[spacewander]

There is still CI failure: https://github.com/apache/apisix/runs/3719358180

[spacewander]

@shuaijinchao
Please take a look.

[shuaijinchao]

good feature. I benchmarked this PR. In most scenarios, disk cache is better than shared memory cache.
@agile6v can look my test results: Apache APISIX Proxy Cache Plugin Benchmark

[agile6v]

good feature. I benchmarked this PR. In most scenarios, disk cache is better than shared memory cache. @agile6v can look my test results: Apache APISIX Proxy Cache Plugin Benchmark

Good work. Is WRK and APISIX on the same machine? If the machine memory is too small, SWAP will be used and pls take a look at the MISS ratio. Could you use the larger memory machine to test? Thanks.

[spacewander]

Would you provide your own benchmark? It would satisfy your idea better.

[agile6v]

Would you provide your own benchmark? It would satisfy your idea better.

Hi @spacewander Sorry for the limited environment reason, there is no resources to do this test now.

[spacewander]

Would you provide your own benchmark? It would satisfy your idea better.

Hi @spacewander Sorry for the limited environment reason, there is no resources to do this test now.

OK, let me look at the implementation first.

[spacewander]

t/plugin/proxy-cache-memory.t                 (Wstat: 2048 Tests: 123 Failed: 8)
  Failed tests:  12, 31, 60, 75, 83, 90, 94, 98
  Non-zero exit status: 8
Files=262, Tests=10124, 2074 wallclock secs ( 2.54 usr  0.46 sys + 322.91 cusr 110.99 csys = 436.90 CPU)
Result: FAIL

Look like some tests are not passed?