refactor(chain): expunge rocksdb from near-jsonrpc-primitives dependency tree

[miraclx]

Under the initiative to build near-api-rs, we need to ensure its dependencies from within nearcore are as lightweight as possible. On initial inspection, near-jsonrpc-primitives, which should be lightweight enough to be used across the ecosystem, actually isn't, as it's dependency tree currently includes rocksdb. To remedy this, we extract useful structures from near-chunks and near-network into their own -primitives counterpart crates and depend on those instead.

New Crates

• near-chunks-primitives from near-chunks
• near-network-primitives from near-network

Analysis

Dump of cargo tree -p near-jsonrpc-primitives | wc -l:

   624 master (before this refactor)
   610 after introducing `near-chunks-primitives` (`rocksdb` still present)
   494 after introducing `near-network-primitives` (bye, bye, rocksdb)
   459 after refining `near-network{,-primitives}`
   457 after moving metrics back to `near-network` (thanks, @bowenwang1996)
   417 after removing `metric_recorder` and moving it's dependent back to `near-network` 

Test Plan

• Ensure unit tests pass

_{An unfortunate consequence of this kind of code movearound is that git looses track of blame, but for reference, modules in either of the new crates that git thinks of as a "new file", was extracted from their original counterparts.}

[frol]

This PR improved the compilation time and other metrics of near-cli-rs significantly! (see the before / after figures)

debug clean build time: 7m 44s / 2m 7s
debug incremental build time: 34s / 17s
debug binary size: 55.9 MB / 38.2 MB
dependencies: 442 / 390

@miraclx Great job!

[bowenwang1996]

What are the criteria for deciding what goes into network-primitives?

[bowenwang1996]

What's the reason for moving this file to network-primitives? It feels to me that those counters should be defined in the network crate itself

[miraclx]

You're right. Nice catch!

---

What are the criteria for deciding what goes into network-primitives?

In this case, we just initialize the crate and move whatever's necessary to liberate near-client-primitives from the excess dependencies.
Going forward, we can incrementally move whatever's appropriate into their equivalent -primitives crates.

[bowenwang1996]

Where does the dependency on Edge come from? I don't think it is used outside of the network crate

[miraclx]

So, while near-client-primitives only required;

near_network::{recorder::MetricRecorder, types::{AccountOrPeerIdOrHash, KnownProducer}, PeerInfo}

In the case of near_network::types, I believed it was safe to extract as much as was possible since that module name typically encapsulates reusable primitives. And that worked fine with the only exception being Consolidate which is retained in near_network::types.

Following that logic, routing::Edge has a number of dependents in near_network_primitives::types. So, naturally, it's extracted alongside it.

---

That aside, recorder::MetricRecorder depends on types::PeerMessage which depends on types::Edge

[bowenwang1996]

I believed it was safe to extract as much as was possible since that module name typically encapsulates reusable primitives

Well that is exactly my concern. Edge should not be used outside of the network crate. Why do we need MetricRecorder in near-client-primitives?

[bowenwang1996]

I see. @miraclx please remove metric_recorder from NetworkInfoResponse

[frol]

@mfornet @bowenwang1996 should the whole feature be removed altogether or just drop the field from the NetworkInfoResponse? Is the feature still useful without the exposed data?

[bowenwang1996]

@mfornet @bowenwang1996 should the whole feature be removed altogether or just drop the field from the NetworkInfoResponse? Is the feature still useful without the exposed data?

I think the feature should be removed altogether, but my point is that we can now revert the change that extracts network primitives since I don't think they should be in a separate crate to begin with @miraclx.

[miraclx]

MetricRecorder was removed, which freed up the dependency on PeerMessage which, in-turn, freed up the dependency on Edge. This means ::routing and a number of structures in ::types we're returned to near-network.
But we (near_client_primitives) still need near_network_primitives::types::{AccountOrPeerIdOrHash, KnownProducer, PeerInfo}

[frol]

@bowenwang1996 @mfornet Could you review the PR after the update?

[frol]

This seems to also need to be cleaned up

[frol]

Or maybe we want to keep the feature in the Cargo.toml file since there are other places where metric_recorder feature is used