RDS Policy post-finding is not working as expected

[BinduHK]

Describe the bug

We are facing below issue with RDS policy where we are trying to detect RDS instances are not set to be accessed publicly.

custodian.actions: ERROR import_response={'ResponseMetadata': {'RequestId': '1cddc74e-f580-4d3c-bc34-c136f8cd114a', 'HTTPStatusCode': 200, 'HTTPHeaders': {'date': 'Wed, 18 Jan 2023 07:05:16 GMT', 'content-type': 'application/json', 'content-length': '387', 'connection': 'keep-alive', 'x-amzn-requestid': '1cddc74e-f580-4d3c-bc34-c136f8cd114a', 'access-control-allow-origin': '*', 'access-control-allow-headers': 'Authorization,Date,X-Amz-Date,X-Amz-Security-Token,X-Amz-Target,content-type,x-amz-content-sha256,x-amz-user-agent,x-amzn-platform-id,x-amzn-trace-id', 'x-amz-apigw-id': 'e7UG6G9PiYcF14A=', 'access-control-allow-methods': 'GET,POST,OPTIONS,PUT,PATCH,DELETE', 'access-control-expose-headers': 'x-amzn-errortype,x-amzn-requestid,x-amzn-errormessage,x-amzn-trace-id,x-amz-apigw-id,date', 'x-amzn-trace-id': 'Root=1-63c79a2c-7ddf2fdc3accee816d2fab47', 'access-control-max-age': '86400'}, 'RetryAttempts': 0}, 'FailedCount': 1, 'SuccessCount': 0, 'FailedFindings': [{'Id': 'us-east-2/123456789224/da29b6687193395d6e49704fa708e49b/716bdb342539547bfaa6d86ea5731b88', 'ErrorCode': 'InvalidInput', 'ErrorMessage': 'Finding does not adhere to Amazon Finding Format. data.Resources[0].Details.Other should NOT have more than 50 properties, data.Resources[0].Details should pass "$merge" keyword validation.'}]}

What did you expect to happen?

It should have posted the finding on security hub

Cloud Provider

Amazon Web Services (AWS)

Cloud Custodian version and dependency information

custodian version --debug

Please copy/paste the following info along with any bug reports:

Custodian:   0.9.21
Python:      3.7.10 (default, Jun  3 2021, 00:02:01) 
             [GCC 7.3.1 20180712 (Red Hat 7.3.1-13)]
Platform:    posix.uname_result(sysname='Linux', nodename='ip-10-106-177-187.aws.lilly.com', release='4.14.219-161.340.amzn2.x86_64', version='#1 SMP Thu Feb 4 05:54:19 UTC 2021', machine='x86_64')
Using venv:  False
Docker: False
Installed: 

argcomplete==2.0.0
attrs==22.1.0
boto3==1.26.30
botocore==1.29.51
docutils==0.16
importlib-metadata==4.13.0
importlib-resources==5.10.1
jmespath==1.0.1
jsonschema==4.17.3
pkgutil-resolve-name==1.3.10
pyrsistent==0.19.2
python-dateutil==2.8.2
pyyaml==5.4.1
s3transfer==0.6.0
six==1.16.0
tabulate==0.8.10
typing-extensions==4.4.0
urllib3==1.26.13
zipp==3.11.0

Policy

policies:

  - name: aws-rds-PubliclyAccessible-On
    resource: rds
    description: Verifies that RDS instances are not set to be accessed publicly. 
    filters:
      - type: value
        key: "DBInstanceArn"
        op: not-in
        value_from:
          url: "s3://test/exceptions/exceptions.csv"
          format: csv
      - type: value
        key: PubliclyAccessible
        op: eq
        value: true
    actions:
      - type: post-finding
        severity_normalized: 90
        severity_label: CRITICAL
        types:
          - "test"
        recommendation: "Ensure that RDS database is not publicly accessible."
        recommendation_url: "https://testurl/RDS"
        confidence: 100
        compliance_status: FAILED

Relevant log/traceback output

custodian.actions: ERROR import_response={'ResponseMetadata': {'RequestId': '1cddc74e-f580-4d3c-bc34-c136f8cd114a', 'HTTPStatusCode': 200, 'HTTPHeaders': {'date': 'Wed, 18 Jan 2023 07:05:16 GMT', 'content-type': 'application/json', 'content-length': '387', 'connection': 'keep-alive', 'x-amzn-requestid': '1cddc74e-f580-4d3c-bc34-c136f8cd114a', 'access-control-allow-origin': '*', 'access-control-allow-headers': 'Authorization,Date,X-Amz-Date,X-Amz-Security-Token,X-Amz-Target,content-type,x-amz-content-sha256,x-amz-user-agent,x-amzn-platform-id,x-amzn-trace-id', 'x-amz-apigw-id': 'e7UG6G9PiYcF14A=', 'access-control-allow-methods': 'GET,POST,OPTIONS,PUT,PATCH,DELETE', 'access-control-expose-headers': 'x-amzn-errortype,x-amzn-requestid,x-amzn-errormessage,x-amzn-trace-id,x-amz-apigw-id,date', 'x-amzn-trace-id': 'Root=1-63c79a2c-7ddf2fdc3accee816d2fab47', 'access-control-max-age': '86400'}, 'RetryAttempts': 0}, 'FailedCount': 1, 'SuccessCount': 0, 'FailedFindings': [{'Id': 'us-east-2/123456789224/da29b6687193395d6e49704fa708e49b/716bdb342539547bfaa6d86ea5731b88', 'ErrorCode': 'InvalidInput', 'ErrorMessage': 'Finding does not adhere to Amazon Finding Format. data.Resources[0].Details.Other should NOT have more than 50 properties, data.Resources[0].Details should pass "$merge" keyword validation.'}]}

Extra information or context

No response

[thisisshi]

interesting, looking at the finding that it tries to post it looks like there are 55 keys as we're stuffing the entire resource definition into the details, I'm not really sure what we would want to cull from the resource details

[BinduHK]

Yes, they are expecting 50 properties so that the validation gets passed.