Nested JSON causes quoting issue

[passcod]

Consider this call to normalize_needed_jobs_status taken from a real run in an internal repo:

  python -m normalize_needed_jobs_status \
  "$(cat << EOM
    []
  EOM
  )" \
  "$(cat << EOM
    []
  EOM
  )" \
  "$(cat << EOM
    {
    "config": {
      "result": "success",
      "outputs": {
        "build-images": "true",
        "down": "[]",
        "go-down": "false",
        "go-up": "true",
        "ref": "refs/heads/main",
        "sha": "3c9153355eb2d6a0a8985b5974c77a38094d135d",
        "up": "[{\"name\":\"main\",\"options\":\"{\\\"facilities\\\":2,\\\"config\\\":\\\"pr\\\",\\\"timezone\\\":\\\"Pacific/Auckland\\\",\\\"ip\\\":null,\\\"dbstorage\\\":5,\\\"arch\\\":\\\"arm64\\\",\\\"opsref\\\":\\\"main\\\",\\\"opsstack\\\":\\\"tamanu/on-k8s\\\",\\\"k8score\\\":\\\"tamanu-internal-main\\\",\\\"pause\\\":false,\\\"apis\\\":2,\\\"centralapis\\\":2,\\\"facilityapis\\\":1,\\\"tasks\\\":1,\\\"centraltasks\\\":1,\\\"facilitytasks\\\":0,\\\"webs\\\":2,\\\"centralwebs\\\":2,\\\"facilitywebs\\\":2,\\\"dbs\\\":2,\\\"centraldbs\\\":2,\\\"facilitydbs\\\":2}\"}]"
      }
    },
    "check-images-exist": {
      "result": "success",
      "outputs": {
        "exists": "no"
      }
    },
    "images": {
      "result": "success",
      "outputs": {}
    },
    "multi-arch": {
      "result": "success",
      "outputs": {}
    }
  }
  EOM
  )"

This fails with:

Traceback (most recent call last):
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/runner/work/_actions/re-actors/alls-green/release/v1/src/normalize_needed_jobs_status.py", line 195, in <module>
    sys.exit(main(sys.argv))
  File "/home/runner/work/_actions/re-actors/alls-green/release/v1/src/normalize_needed_jobs_status.py", line 133, in main
    inputs = parse_inputs(
  File "/home/runner/work/_actions/re-actors/alls-green/release/v1/src/normalize_needed_jobs_status.py", line 66, in parse_inputs
    'jobs': json.loads(raw_jobs),
  File "/usr/lib/python3.10/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3.10/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python3.10/json/decoder.py", line 353, in raw_decode
    obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Expecting ',' delimiter: line 11 column 52 (char 299)

Specifically, here's a reduced case:

$ python
>>> import json
>>> json.loads('{"up":"[{\"options\":\"{\\\"facilities\\\":2}\"}]"}')
...
json.decoder.JSONDecodeError: Expecting ',' delimiter: line 1 column 11 (char 10)

Initially I thought Python's JSON decoder had a bug, but I think what's happening is instead a quoting issue, such that the string is interpreted something like:

{"up":"[{" options...

(space before options added for clarity)

Indeed, using a file works:

$ cat > valid.json
{"up":"[{\"options\":\"{\\\"facilities\\\":2}\"}]"}
^D

$ python
>>> import json
>>> with open('valid.json') as f:
...   json.load(f)
...
{'up': '[{"options":"{\\"facilities\\":2}"}]'}
>>>

Upvote & Fund

• We're using Polar.sh so you can upvote and help fund this issue.
• We receive the funding once the issue is completed & confirmed by you.
• Thank you in advance for helping prioritize & fund our backlog.

[webknjaz]

Oh, so the job output is what's nested. Interesting.. I'll need to think how to best handle this case.

[passcod]

Current workaround is to do this:

- id: workaround
  uses: actions/github-script@v7
  with:
    # Workaround for https://github.com/re-actors/alls-green/issues/29
    # we strip outputs from the needs object to avoid nested JSON
    result-encoding: string
    script: |
      return JSON.stringify(Object.fromEntries(Object.entries(${{ toJSON(needs) }})
        .map(([name, { result }]) => [name, { result, outputs: {} }])));
- uses: re-actors/alls-green@release/v1
  with:
    jobs: ${{ steps.workaround.outputs.result }}

[artpelling]

Facing the same issue, would be great to have this fixed.