Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: metadata protocol #1732

Merged
merged 19 commits into from
Dec 5, 2023
Merged

feat: metadata protocol #1732

merged 19 commits into from
Dec 5, 2023

Conversation

danisharora099
Copy link
Collaborator

@danisharora099 danisharora099 commented Nov 27, 2023

This PR introduces the Metadata Protocol as a service

Notes:

@danisharora099 danisharora099 force-pushed the feat/metadata-protocol branch 2 times, most recently from c7fee6e to 47b95f1 Compare November 29, 2023 07:18
Copy link

github-actions bot commented Nov 29, 2023

size-limit report 📦

Path Size Loading time (3g) Running time (snapdragon) Total time
Waku core 78.48 KB (+0.14% 🔺) 1.6 s (+0.14% 🔺) 152 ms (+43.69% 🔺) 1.8 s
Waku Simple Light Node 241.3 KB (+0.19% 🔺) 4.9 s (+0.19% 🔺) 319 ms (+16.66% 🔺) 5.2 s
ECIES encryption 72.96 KB (+0.16% 🔺) 1.5 s (+0.16% 🔺) 182 ms (+27.25% 🔺) 1.7 s
Symmetric encryption 72.95 KB (+0.16% 🔺) 1.5 s (+0.16% 🔺) 155 ms (+33.51% 🔺) 1.7 s
DNS discovery 120.91 KB (0%) 2.5 s (0%) 219 ms (-5.31% 🔽) 2.7 s
Privacy preserving protocols 126.01 KB (+0.08% 🔺) 2.6 s (+0.08% 🔺) 201 ms (-2.93% 🔽) 2.8 s
Light protocols 76.06 KB (+0.14% 🔺) 1.6 s (+0.14% 🔺) 185 ms (+100.62% 🔺) 1.8 s
History retrieval protocols 74.97 KB (+0.14% 🔺) 1.5 s (+0.14% 🔺) 147 ms (+26.36% 🔺) 1.7 s
Deterministic Message Hashing 5.65 KB (0%) 113 ms (0%) 23 ms (-29.66% 🔽) 136 ms

@danisharora099 danisharora099 force-pushed the feat/metadata-protocol branch 2 times, most recently from f6976e5 to 4efcad5 Compare November 29, 2023 11:13
@danisharora099 danisharora099 force-pushed the feat/metadata-protocol branch 2 times, most recently from fa922b4 to 6524032 Compare November 29, 2023 12:09
@danisharora099 danisharora099 marked this pull request as ready for review November 29, 2023 12:46
@danisharora099 danisharora099 requested a review from a team as a code owner November 29, 2023 12:46
packages/core/src/lib/metadata/index.ts Outdated Show resolved Hide resolved
packages/core/src/lib/metadata/index.ts Outdated Show resolved Hide resolved
packages/core/src/lib/metadata/index.ts Outdated Show resolved Hide resolved
});
const response = proto_metadata.WakuMetadataResponse.decode(bytes);
if (!response) {
throw new Error("Error decoding metadata response");
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's move away from exceptions and use enum error across all req-res interfaces

Copy link
Collaborator Author

@danisharora099 danisharora099 Nov 30, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes. updated the issue for metadata & tracked part of #1694

packages/tests/tests/metadata.spec.ts Outdated Show resolved Hide resolved
await tearDownNodes([nwaku1], waku);
});

it("interop", async function () {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

More tests are needed:

  1. same cluster, same shard: nodes connect
  2. same cluster, different shardL nodes connects
  3. different cluster, same shard, node disconnect
  4. different cluster, different shard, node disconnects

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

right, thanks!
ca17b30

Copy link
Collaborator

@fryorcraken fryorcraken left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some comments.

I think it would have been easier to get the API right (enum error) from the get go.

this.shardInfo
);

await pipe([encodedResponse], lp.encode, streamData.stream);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The metadata request contains the shardinfo from the remote node. We should save it locally.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, that part was intended be tracked part of #1684 but addressed it here and actually makes more sense! thanks:) d0c9683

await waku.libp2p.dialProtocol(nwaku1Ma, MetadataCodec);

await expect(
waku.libp2p.services.metadata?.query(nwaku1PeerId)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is that?

It seems better to check whether you have an active connection with the node, as the expectation is no active connection.

Copy link
Collaborator Author

@danisharora099 danisharora099 Dec 5, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the metadata query fails (with a throw) if the clusters are different between the service node it's attempting to dial therefore the expecting a catch

you're right in pointing out that there will no active connection, you're right in pointing out that the connection will be closed so good to add an expect for that as well: d53eddb

--
wanted to be explicit with making the query, but perhaps it's simpler to remove the query() attempt and directly expect the connection to be dropped:d53eddb

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@danisharora099 danisharora099 merged commit 9ac2a3f into master Dec 5, 2023
10 of 11 checks passed
@danisharora099 danisharora099 deleted the feat/metadata-protocol branch December 5, 2023 13:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

feat: add new metadata protocol
2 participants