Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add support for maps #258

Merged
merged 3 commits into from
Jun 3, 2024
Merged

feat: add support for maps #258

merged 3 commits into from
Jun 3, 2024

Conversation

anthony-swirldslabs
Copy link
Contributor

@anthony-swirldslabs anthony-swirldslabs commented May 28, 2024

Description:
Introducing support for map fields in PBJ.

I apologize for the size of the PR but it would be close to impossible to break it up into reasonable pieces because all the code is deeply interconnected and needs to be updated all at once when adding a new field type.

Short summary:

  1. The compiler is updated to generate Protobuf and JSON models/codecs for messages with map fields.
  2. In models, the maps are present as instances of a new PbjMap class that implements an immutable map and provides access to a list of sorted keys to support deterministic iteration over the map entries.
  3. Per the spec and the implementation of maps in protoc, they are... strange. In general, on the wire they're presented as repeated entry messages with key/value fields. However, unlike normal protobuf messages, the entries always serialize key and value values, even when the values are "default" (e.g. "" for a string, or 0 for an int, etc.) This peculiarity required quite a bit of refactoring in the codec generator. All those skipDefault-related changes are here to support just this.

Related issue(s):

Fixes #254

Notes for reviewer:

  1. All the tests, including the integration and fuzz tests pass.
  2. A new test model map.proto is added to test various maps.

Checklist

  • Documented (Code comments, README, etc.)
  • Tested (unit, integration, etc.)

Signed-off-by: Anthony Petrov <anthony@swirldslabs.com>
@anthony-swirldslabs anthony-swirldslabs self-assigned this May 28, 2024
Signed-off-by: Anthony Petrov <anthony@swirldslabs.com>
Copy link

github-actions bot commented May 29, 2024

JUnit Test Report

     48 files  ±0       48 suites  ±0   4m 20s ⏱️ + 1m 40s
1 138 tests ±0  1 137 ✔️ ±0    1 💤 ±0  0 ±0 
6 993 runs  ±0  6 976 ✔️ ±0  17 💤 ±0  0 ±0 

Results for commit a0fc753. ± Comparison against base commit 6da76df.

♻️ This comment has been updated with latest results.

Copy link
Contributor

codacy-production bot commented May 29, 2024

Coverage summary from Codacy

See diff coverage on Codacy

Coverage variation Diff coverage
-43.15% (target: -1.00%) 0.00%
Coverage variation details
Coverable lines Covered lines Coverage
Common ancestor commit (6da76df) 4361 1977 45.33%
Head commit (a0fc753) 2151 (-2210) 47 (-1930) 2.19% (-43.15%)

Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch: <coverage of head commit> - <coverage of common ancestor commit>

Diff coverage details
Coverable lines Covered lines Diff coverage
Pull request (#258) 284 0 0.00%

Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified: <covered lines added or modified>/<coverable lines added or modified> * 100%

See your quality gate settings    Change summary preferences

Codacy will stop sending the deprecated coverage status from June 5th, 2024. Learn more

Copy link

github-actions bot commented May 29, 2024

Integration Test Report

     269 files   -        5       269 suites   - 5   8m 3s ⏱️ + 1m 57s
90 327 tests  - 6 327  90 327 ✔️  - 6 327  0 💤 ±0  0 ±0 
90 555 runs   - 6 327  90 555 ✔️  - 6 327  0 💤 ±0  0 ±0 

Results for commit a0fc753. ± Comparison against base commit 6da76df.

♻️ This comment has been updated with latest results.

imalygin
imalygin previously approved these changes May 29, 2024
Copy link
Member

@jasperpotts jasperpotts left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Impressive work, that was a lot of work :-)

Signed-off-by: Anthony Petrov <anthony@swirldslabs.com>
@anthony-swirldslabs anthony-swirldslabs merged commit 79ac1de into main Jun 3, 2024
7 of 10 checks passed
@anthony-swirldslabs anthony-swirldslabs deleted the issue-254-1 branch June 3, 2024 20:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add support of map field type to PBJ
3 participants