Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add JSON pre-processor #17125

Merged
merged 2 commits into from
Mar 11, 2025
Merged

Add JSON pre-processor #17125

merged 2 commits into from
Mar 11, 2025

Conversation

RobinMalfait
Copy link
Member

@RobinMalfait RobinMalfait commented Mar 11, 2025

This PR adds a small JSON pre processor to improve parsing JSON files. This is because the extractor creates "sub machines" whenever it encounters a [ or a { in the input. We do this because of things like %w[…] strings in Ruby or className={clsx({flex: true})} in JSX.

Due to the sheer amount of potential [ and ] brackets, it could be that parsing JSON files are way slower than they need to be.

To tackle this, after this PR, when given an input like this:

[1,[2,[3,4,["flex flex-1 content-['hello_world']"]]], {"flex": true}]

We'll preprocess all the important brackets and braces by replacing them with spaces so the extractor doesn't need special casing:

1, 2, 3,4, "flex flex-1 content-['hello_world']" , "flex": true

We saw this while debugging this issue: #17092

Test plan

  1. Added test to verify the pre processing works
  2. Existing tests still pass

@RobinMalfait RobinMalfait requested a review from a team as a code owner March 11, 2025 12:06
@RobinMalfait RobinMalfait force-pushed the feat/add-json-pre-processor branch from c0ed038 to 08b0203 Compare March 11, 2025 12:06
@RobinMalfait RobinMalfait force-pushed the feat/add-json-pre-processor branch from 3fe4791 to a45a6c4 Compare March 11, 2025 12:12
@RobinMalfait RobinMalfait enabled auto-merge (squash) March 11, 2025 12:15
@RobinMalfait RobinMalfait merged commit 9ddeb09 into main Mar 11, 2025
5 checks passed
@RobinMalfait RobinMalfait deleted the feat/add-json-pre-processor branch March 11, 2025 12:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants