Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(clients): trim values in parsed xml only if result is empty #2194

Merged
merged 6 commits into from
Mar 31, 2021

Conversation

trivikr
Copy link
Member

@trivikr trivikr commented Mar 30, 2021

Issue

Fixes: #1893

Description

Sets trimValues=false in fast-xml-parser configuration, so that values are not trimmed while parsing.

Testing

CI


By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@trivikr trivikr force-pushed the xml-parser-trimValues-false branch from a9b620d to 38e0f59 Compare March 30, 2021 22:37
@trivikr
Copy link
Member Author

trivikr commented Mar 30, 2021

The unit tests fail, as the output returned by fast-xml-parser now contains "#text"

Code
import { parse } from "fast-xml-parser";

const xmlData = `<Response>
<Errors>
    <Error>
        <Code>InvalidGreeting</Code>
        <Message>Hi</Message>
    </Error>
</Errors>
<RequestId>foo-id</RequestId>
</Response>`;

console.log(
  `\noutput without trimValues set: ${JSON.stringify(parse(xmlData), null, 2)}`
);
console.log(
  `\noutput with trimValues set to false: ${JSON.stringify(
    parse(xmlData, {
      trimValues: false,
    }),
    null,
    2
  )}`
);
Output
output without trimValues set: {
  "Response": {
    "Errors": {
      "Error": {
        "Code": "InvalidGreeting",
        "Message": "Hi"
      }
    },
    "RequestId": "foo-id"
  }
}

output with trimValues set to false: {
  "Response": {
    "#text": "\n\n\n",
    "Errors": {
      "#text": "\n    \n",
      "Error": {
        "#text": "\n        \n        \n    ",
        "Code": "InvalidGreeting",
        "Message": "Hi"
      }
    },
    "RequestId": "foo-id"
  }
}

The trimming of "#text" needs to be done on value returned by parser.

@trivikr
Copy link
Member Author

trivikr commented Mar 30, 2021

Wrote a wrapper function to delete empty text nodes to work on parsed object.

Code
import { parse } from "fast-xml-parser";

const xmlData = `<Response>
<Errors>
    <Error>
        <Code>InvalidGreeting</Code>
        <Message>Hi</Message>
    </Error>
</Errors>
<RequestId>foo-id</RequestId>
</Response>`;

console.log(
  `\noutput without trimValues set: ${JSON.stringify(parse(xmlData), null, 2)}`
);

const textNodeName = "#text";
const deleteEmptyTextNodes = (data, textNodeName) => {
  if (Array.isArray(data)) {
    return data.map((item) => deleteEmptyTextNodes(item, textNodeName));
  }
  if (data.constructor.name !== "Object") {
    return data;
  }
  if (data[textNodeName] && data[textNodeName].trim() === "") {
    delete data[textNodeName];
  }
  return Object.entries(data).reduce(
    (acc, [key, value]) => ({
      ...acc,
      [key]: deleteEmptyTextNodes(value, textNodeName),
    }),
    {}
  );
};

console.log(
  `\noutput with trimValues set to false with wrapper: ${JSON.stringify(
    deleteEmptyTextNodes(
      parse(xmlData, {
        trimValues: false,
      }),
      textNodeName
    ),
    null,
    2
  )}`
);
Output
output without trimValues set: {
  "Response": {
    "Errors": {
      "Error": {
        "Code": "InvalidGreeting",
        "Message": "Hi"
      }
    },
    "RequestId": "foo-id"
  }
}

output with trimValues set to false with wrapper: {
  "Response": {
    "Errors": {
      "Error": {
        "Code": "InvalidGreeting",
        "Message": "Hi"
      }
    },
    "RequestId": "foo-id"
  }
}

@trivikr
Copy link
Member Author

trivikr commented Mar 30, 2021

An easier solution would be to add the following code in tagValueProcessor:

if (val.trim() === "") {
  return "";
}

This way, value will be trimmed only if it's empty, and no additional processing of parsed object would be required.

@aws-sdk-js-automation
Copy link

AWS CodeBuild CI Report

  • CodeBuild project: sdk-staging-test
  • Commit ID: e9d1489
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@trivikr trivikr changed the title fix(clients): set trimValues=false in fast-xml-parser configuration fix(clients): trim values in parsed xml only if result is empty Mar 31, 2021
@codecov-io
Copy link

Codecov Report

❗ No coverage uploaded for pull request base (main@325d6b4). Click here to learn what that means.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##             main    #2194   +/-   ##
=======================================
  Coverage        ?   61.83%           
=======================================
  Files           ?      453           
  Lines           ?    23287           
  Branches        ?     5520           
=======================================
  Hits            ?    14399           
  Misses          ?     8888           
  Partials        ?        0           

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 325d6b4...e9d1489. Read the comment docs.

@trivikr
Copy link
Member Author

trivikr commented Mar 31, 2021

Verified that integration tests are successful:

$ yarn test:integration-legacy
yarn run v1.22.5
$ cucumber-js --fail-fast
...............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................

150 scenarios (150 passed)
523 steps (523 passed)
1m30.715s
Done in 95.50s.

$ yarn test:integration       
yarn run v1.22.5
$ jest --config jest.config.integ.js --passWithNoTests
 PASS  clients/client-transcribe-streaming/test/index.integ.spec.ts (31.71 s)
  TranscribeStream client
    ✓ should stream the transcript (28978 ms)

Test Suites: 1 passed, 1 total
Tests:       1 passed, 1 total
Snapshots:   0 total
Time:        32.169 s
Ran all test suites.
Done in 32.83s.

Copy link
Contributor

@alexforsyth alexforsyth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@trivikr trivikr merged commit a990db7 into aws:main Mar 31, 2021
@trivikr trivikr deleted the xml-parser-trimValues-false branch March 31, 2021 22:46
@github-actions
Copy link

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs and link to relevant comments in this thread.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Apr 15, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

SQS receiveMessage not handling new lines correctly
4 participants