Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PBJ promotes sub-type references to global scope. #263

Open
jsync-swirlds opened this issue Jun 5, 2024 · 2 comments
Open

PBJ promotes sub-type references to global scope. #263

jsync-swirlds opened this issue Jun 5, 2024 · 2 comments
Assignees

Comments

@jsync-swirlds
Copy link
Member

jsync-swirlds commented Jun 5, 2024

PBJ compiler fails when it encounters a reference to a message contained in another message. This makes it impossible to use intuitive message naming and well designed message scope.

Example

File distro_response.proto

syntax = "proto3";

package com.zyxyl.distribution.partition;

message PartitionResponse {
    oneof response {
        FullResponse.Acknowledgement acknowledgement = 1;
        FullResponse.EndOfStream status = 2;
    }
}
message FullResponse {
    // several hundred lines elided here for clarity
    message Acknowledgement {
        oneof acknowledgements {
            FullResponse.ChildAcknowledgement child_ack = 271;
            FullResponse.ParentAcknowledgement parent_ack = 272;
        }
    }
    // several hundred lines elided here for clarity
    message ParentAcknowledgement {
        uint32 child_count = 872;
    }
    // several hundred lines elided here for clarity
    message ChildAcknowledgement {
        uint32 partition_count = 1438;
    }
    // several hundred lines elided here for clarity
    enum EndOfStream {
        UNKNOWN = 0;
        CANCELED = 1;
       // several hundred lines elided here for clarity
    }
}

The canonical protoc compiler compiles these and handles the references as expected (note, the file may contain several other messages with child messages named Acknowledgement, EndOfStream, etc...)
The PBJ compiler fails with an error similar to the following:

Not found pbj package for message or enum [FullResponse.Acknowledgement] in file [protobufs/distribution/partition/distro_response.proto]

Worse, if there are two EndOfStream subtypes and both are used only within their enclosing type, PBJ will silently promote one to global, and use the one type for all cases. This causes the PBJ code to incorrectly create or interpret the messages (due to using the wrong subtype) and only fails at runtime (fortunately our extensive automated testing does detect this).
This is a subtle data corruption bug, and it needs to be fixed.

@jsync-swirlds
Copy link
Member Author

PBJ also promotes subtypes to global scope. This is incompatible with protoc, and is no longer something we can just ignore. PBJ cannot compile the block node block service (nor many common protocol buffer/gRPC APIs!) without a fix for this bug.

@jsync-swirlds
Copy link
Member Author

Too be updated: Note here how this blocks implementing the "publish" protocol for Block Stream.

@jsync-swirlds jsync-swirlds self-assigned this Jan 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants