Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make ensure_ascii Dynamic with Default Set to True in JSON Serialization #168

Merged
merged 3 commits into from
Aug 13, 2024

Conversation

sgandhi1311
Copy link
Member

@sgandhi1311 sgandhi1311 commented Aug 8, 2024

Description of change

This PR introduces a dynamic setting for the ensure_ascii parameter in the json.dumps function, with its default value set to True. This change allows for greater flexibility in JSON serialization, enabling non-ASCII characters to be preserved when needed by explicitly passing ensure_ascii=False.

Manual QA steps

  • Unit Tests: Added and updated unit tests to verify the correct behaviour of JSON serialization with both ensure_ascii=True (default) and ensure_ascii=False.
  • Alpha: Performed PR-alpha on the customer connection to test JSON serialization for the special characters

Risks

  • If not ensure_ascii is set differently per the expectations within the tap code, then customers can experience the data discrepancy.

Rollback steps

  • revert this branch

@sgandhi1311 sgandhi1311 changed the title Ensure nonascii characters Preserve Special Characters in JSON Output by Setting ensure_ascii=False Aug 8, 2024
@sgandhi1311 sgandhi1311 changed the title Preserve Special Characters in JSON Output by Setting ensure_ascii=False Make ensure_ascii Dynamic with Default Set to True in JSON Serialization Aug 8, 2024
@sgandhi1311 sgandhi1311 merged commit 0cb2288 into master Aug 13, 2024
1 check passed
jesuejunior added a commit to sixcodes/singer-python that referenced this pull request Oct 5, 2024
* parse dev mode argument (singer-io#158)

* parse dev mode argument

* update the short flag for dev mode

Replace `-dev` to `-D`

* Updated the short flag for dev mode

* - remove dev mode short flag

Co-authored-by: RushiT0122 <rtodkar@stitchdata-talend.com>

* Bump version 5.13.0 (singer-io#160)

* Bump version 5.13.0

* update changelog for dev mode argument parsing

* change versioning to minor version bump

Co-authored-by: RushiT0122 <rtodkar@stitchdata-talend.com>

* Bump backoff to be compatible with newer python versions (singer-io#165)

* bump backoff for pyton 3.11 compatibility

* update pip version

* try new circleci yml

* remove 'make'

* make pylint happy

* make pylint happy

* make pylint happy again

* backoff version is a breaking change for old python versions

* Changelog update

* Relax dependency version requirements (singer-io#167)

* Relax dependency constraints

* Bump version to `6.0.1`

* pin backoff and simplejson to major version

* Don't allow older versions

* Update changelog

* Pin minumum and major versions

* Make `ensure_ascii` Dynamic with Default Set to `True` in JSON Serialization (singer-io#168)

* add parameter - ensure_ascii to load non ascii characters when set to false

* add unit test for ensuring ascii characters while loading

* update setup and changelog

* Enable copilot usage in PR template according to Qlik policy

* removing simplejson dep and improve dep version

* Adding

* bumping version

* Removing use_decimal param for json.dumps

* Adding a consig to be able to send message to pubsub

* Extracting to a F

* pubsub doesnt expect None as attr

* Parsing state to be used as attr in pubsub

* Set state as stream name when type is state

* feat: supporting a simple entry from lakehouse lib to send messages to a queue

* chore(deps) make jsonschema version flexible

---------

Co-authored-by: Rushikesh Todkar <98420315+RushiT0122@users.noreply.github.com>
Co-authored-by: RushiT0122 <rtodkar@stitchdata-talend.com>
Co-authored-by: Leslie VanDeMark <38043390+leslievandemark@users.noreply.github.com>
Co-authored-by: Bryant Gray <bryant.gray@qlik.com>
Co-authored-by: Sourabh Gandhi <105213416+sgandhi1311@users.noreply.github.com>
Co-authored-by: Eivin Giske Skaaren <eivin.skaaren@qlik.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants