kong decode json empty array to empty object?

[wuyachao]

Summary

I want to change the body proxy kong. So I use the plugin response-transformer.But kong decode json empty array to empty object.

Steps To Reproduce

1. add a api and plugin to kong. the config of the plugin is this:

{
    "data": [
        {
            "api_id": "d48f92ee-09dd-4970-8e7a-a853aeac30bd",
            "config": {
                "add": {
                    "headers": {},
                    "json": [
                        "age:111",
                        "age2:222"
                    ]
                },
                "append": {
                    "headers": {},
                    "json": {}
                },
                "remove": {
                    "headers": {},
                    "json": {}
                },
                "replace": {
                    "headers": {},
                    "json": {}
                }
            },
            "created_at": 1515747762000,
            "enabled": true,
            "id": "ea5ea2d2-3a3f-4a0a-a719-a41eb124b483",
            "name": "response-transformer"
        }
    ],
    "total": 1
}

2. request this api:

curl -X POST \
  http://localhost:8000/notifi \
  -H 'content-type: application/json' \
  -d '{"name":"wyc","info":{},"list":[]}'

3. the reponse is ：

{
    "list": {},
    "info": {},
    "age2": "222",
    "age": "111",
    "name": "wyc"
}

4. the list is empty array. it should be [],but the result is {}.

5. The dkjson can do it. The master branch of openresty/cjson can do it too.It has the method decode_array_with_array_mt. But kong bundle openresty/1.11.2.4. So I only install the master cjson to replace the current cjson.

6. Kong can resolve it the next version.May be the latest version is 1.13.6.1 also hasn't the method.
   attempt to call field 'decode_array_with_array_mt' (a nil value)? openresty/lua-cjson#33

Additional Details & Logs

• Kong version :0.11.0 or 0.10.4

[wuyachao]

The request body have empty object and empty list is common. encode_empty_table_as_object can't resolve it. ecode_array_with_array_mt is in master branch, I must compile the master branch cjson. Maybe the latest version 1.13.6.1 also hasn't included the method.

[kikito]

Hi, I have investigated this and it is as you say: in order to fix this problem we need the latest cjson, which is not even included in openresty yet, as pointed in openresty/lua-cjson#33

I will leave this issue open but for now there's little we can do.

[wuyachao]

@kikito thank you. Now I have compiled it to replace the old cjson.so.

[erran]

This seems related to #2309 which I was bit by today. I ended up updating my client code as described in #2309 (comment). Unfortunately doesn't look like this issue has the same type of work around.

also see #1208

[wuyachao]

@erran From #2309 , We can know it will be empty array so encode it with cjson.empty_table.It's also harmless. But this response-transformer,we get data from front-bacnend and add some field.I do not know its format, It can cause backend error.

[erran]

Unfortunately doesn't look like this issue has the same type of work around.

That's right @wuyachao. I was just commenting to reference related issues when I found #2309.

[maksimlikharev]

I see OpenResty incorporated the fix:
https://openresty.org/en/changelog-1013006.html

I don't think this is harmless, as the client expects [] and gets {} instead.

Any ETA on this issue?

Thanks

[bungle]

We will base our 0.14.0 on OpenResty that contains the fix.

[hbagdi]

I can reproduce this issue with 0.14.0.

[thibaultcha]

Now that our underlying OpenResty version has the fix, we need to actually use it, which means updating the plugin. Our new Admin API already makes use of it along with a few plugins that received contributions to make use of the fix.

[hbagdi]

Now that our underlying OpenResty version has the fix, we need to actually use it, which means updating the plugin

@thibaultcha I suspected this but didn't actually look into it. Thanks for the quick response here!

[p0pr0ck5]

Is this solved in the new DAO?

[hbagdi]

So, the new DAO returns an empty list for or an explicit null, based on the default:

{
    "config": {
        "blacklist": null,
        "hide_groups_header": false,
        "whitelist": [
            "foo"
        ]
    },
    "consumer": null,
    "created_at": 1547255075,
    "enabled": true,
    "id": "a43ddcb0-6559-4ed1-bab9-cda909c4584b",
    "name": "acl",
    "route": null,
    "run_on": "first",
    "service": null
}

        {
            "config": {
                "add": {
                    "body": [],
                    "headers": [
                        "fp:bar"
                    ],
                    "querystring": []
                },
                "append": {
                    "body": [],
                    "headers": [],
                    "querystring": []
                },
                "http_method": null,
                "remove": {
                    "body": [],
                    "headers": [],
                    "querystring": []
                },
                "rename": {
                    "body": [],
                    "headers": [],
                    "querystring": []
                },
                "replace": {
                    "body": [],
                    "headers": [],
                    "querystring": []
                }
            }