Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: JSON NestedMap + add tests #1482

Merged
merged 3 commits into from
Jan 28, 2025
Merged

fix: JSON NestedMap + add tests #1482

merged 3 commits into from
Jan 28, 2025

Conversation

SpencerTorres
Copy link
Member

Summary

The NestedMap function converts a flat map[string]any into a structure usable by json.Marshal. The behavior has been updated to match that of ClickHouse server. A good example of this feature in use is the Grafana integration.

The main change here is the exclusion of nil values from encoding.

Comparison

Here is the outputs before/after for comparison (the sorting is inconsistent since there's no sorting key)

ClickHouse Server (expected output):

SELECT *
FROM test2

Query id: e7dc18ee-ddd1-47ee-9263-3750f6a298c4

   ┌─json─┐
1. │ {}   │
   └──────┘
   ┌─json────────────────────────────────────────┐
2. │ {"a":{"b":"42"},"c":["1","2","3"]}          │
3. │ {"f":"Hello, World!"}                       │
4. │ {"a":{"b":"43","e":"10"},"c":["4","5","6"]} │
   └─────────────────────────────────────────────┘
   ┌─json────────────────────────────────────────┐
5. │ {"a":{"b":"42"},"c":["1","2","3"]}          │
6. │ {"f":"Hello, World!"}                       │
7. │ {"a":{"b":"43","e":"10"},"c":["4","5","6"]} │
   └─────────────────────────────────────────────┘
    ┌─json────────────────────────────────────────┐
 8. │ {"a":{"b":"42"},"c":["1","2","3"]}          │
 9. │ {"f":"Hello, World!"}                       │
10. │ {"a":{"b":"43","e":"10"},"c":["4","5","6"]} │
    └─────────────────────────────────────────────┘

10 rows in set. Elapsed: 0.001 sec. 

Grafana output (incorrect, nulls are included):
json output in Grafana not matching ClickHouse server

Grafana output (corrected):
json output in Grafana matching ClickHouse server

Checklist

  • Unit and integration tests covering the common scenarios were added

@SpencerTorres SpencerTorres merged commit 2d72850 into main Jan 28, 2025
12 checks passed
@SpencerTorres SpencerTorres deleted the json_nested_map_fix branch January 28, 2025 17:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant