[RelayMiner] Add an ability to test RelayMiner configuration without application

[okdas]

Objective

The current Morse mainnet client has a feature to send relays without application signature. We should add this functionality to RelayMiner: an ability to send relay through a supplier without providing meta information (session, signature, etc.) so operators can verify their RelayMiner configuration.

Origin Document

Requests from the community:

• Discord;

First mentioned in internal communication.

In the current Morse client, the feature is enabled by adding --simulateRelay argument. In our implementation, we can consider adding this to the relayminer config file.

Goals

• Operators can configure their RelayMiners to send requests to the backend service to make sure applications will get expected service.

General deliverables

• Comments: Add/update TODOs and comments alongside the source code so it is easier to follow.
• Testing: Add new tests (unit and/or E2E) to the test suite.
• Makefile: Add new targets to the Makefile to make the new functionality easier to use.
• Documentation: Update architectural or development READMEs; use mermaid diagrams where appropriate.

---

Creator: @okdas

[Olshansk]

@okdas to provide a link to pocket-core and an example of how it works on Morse.

[okdas]

In pocket-core guidelines, we recommend node runners to start the process with --simulateRelay prior to staking, so they can test before going live. That way, they would avoid being ignored by the gateway (or getting jailed/slashed - though I don't think this is functionality is turned on).

This argument then adds a new http route where the relay simulation request can be sent to. For example, this is how I used to do that:

Fuse

curl -X POST \
--data '{"relay_network_id":"0005","payload":{"data":"{\"jsonrpc\":\"2.0\",\"method\":\"eth_blockNumber\",\"params\":[],\"id\":0}","method":"POST","path":"","headers":{}}}' \
http://localhost:8081/v1/client/sim

xdai

curl -X POST \
--data '{"relay_network_id":"0027","payload":{"data":"{\"jsonrpc\":\"2.0\",\"method\":\"eth_blockNumber\",\"params\":[],\"id\":0}","method":"POST","path":"","headers":{}}}' \
http://localhost:8081/v1/client/sim

I think we have an opportunity to make this functionality more user-friendly and still secure at the same time. The simplest way to implement I think would be to add some sort of "password" or a token that can be passed via HTTP header.

[eddyzags]

Hey folks!

I've been considering integrating the capability to test the connection between the Relay Server and the suppliers' endpoints to increase confidence in the Relay Miner configuration for operators. Based on what is already implemented in Morse, the discussion around the topic, and this issue, I've come up with two use cases for operators:

1. As an operator, I want to ensure the suppliers' backend URL is routable.
2. As an operator, I want to forward a request to the supplier backend URL and then assert the response.

Use case 1

We can add a safeguard at the startup process to ensure every supplier's backend URLs are reachable (before starting the relay servers). This safeguard logic will be responsible for dialing every supplier's backend URL to test the connectivity. Then, we could expose this capability as an endpoint (/ping) for synchronous verifications (e.g., Livesness probe in Kubernetes).

• RelayServer (see here): Add RelayServer.Ping(...) method capability to test the connection between a Relay Server and one or more suppliers.
• synchronousRPCServer (see here): Implement this Ping(...) method for HTTP servers as suppliers.
• RelayProxy (see here): RelayServer.Ping(...) every supplier backend URLs at startup (RelayProxy.Start(...) logic)
• RelayMiner (see here): Add the ability to listen and serve ping requests for synchronous verification, then Ping every relay servers.

   ping:
     enabled: <string>
     addr: <string>

Use case 2

We can set up an authenticated endpoint (/services/{service_id}/forward) whose responsibility would be to forward raw requests from the Relay Server to the service ID (more precisely, service node or data node) without providing meta information (like Session, Signature). As we plan to have service nodes that support different transport layers (TCP, UPD, QUICK ect...), we could delegate the responsibility of defining the actual logic to forward a request to the Relay Server.

• RelayServer (see here): Add RelayServer.Forward(...) method to send request to the suppliers.
• synchronousRPCServer (see here): Implement synchronousRPCServer.Forward(...) method for HTTP requests.
• RelayMiner (see here): Expose HTTP server to forward incoming request to a service id.

We could also make this feature configurable through the Relay Miner's configuration:

   forward:
     enabled: true
     auth-token: <token>

Questions

• Is the Relayer Server and the Service ID a one-to-one relationship?
  IIUIC, the RelayMinerServerConfig implies that one Relay Server can forward requests to one or more service IDs. But Server Config Hydrator doesn't allow that (see code).
  Example: If I execute a relayminer using this configuration, the ollama:mistral:7b service id will be ignored because it has the same listen_url as ethereum.

I am still grasping the code, but I would like your input. Thanks 🙏🏾

(I've just linked a draft PR so you can get a better idea of what I am proposing)

[red-0ne]

@eddyzags Thank you for your contribution. You've correctly identified the use cases, and the proposed solution aligns with our goals.

Let me address your questions:

There isn't a one-to-one relationship between RelayServer and ServiceId. Here's why:

1. The ServiceId is already included in the RelayRequest, which is sufficient for routing requests to the appropriate BackendService.
2. RelayServers don't need to handle service-specific logic. Their main functions are:
3. Receiving a RelayRequest
4. Unpacking and forwarding the RelayRequest.Payload to the BackendService
5. Packaging the response into a RelayResponse
6. Multiple RelayServers are intended to improve communication between Gateway/Application and RelayMiner while maintaining backward compatibility (e.g., supporting QUIC while still accommodating gateways using HTTP/1.1).

Regarding your comment about the Server Config Hydrator: It actually does allow for multiple servers, but we should clarify this in the comments. We're ensuring unique URLs/listen addresses for starting RelayServers, while allowing services to specify which server type to use. There's potential for further refactoring in this area.

Thank you for submitting the PR. I'll examine it and respond with comments before EOW.

[Olshansk]

@eddyzags Is this something you still plan to tackle?

[eddyzags]

@Olshansk Yes. Sorry for the delay; I am re-prioritizing this.
I should come up with an implementation by the end of the week.

[eddyzags]

Hey @okdas & @Olshansk

Here is the first PR to add the capability to test the connectivity between the Relay Miner and the Backend URLs. I am working on another PR for adding the forward request capability.

Thanks!

[Olshansk]

Thanks @eddyzags, we'll take a look by end of week!

[Olshansk]

@eddyzags Please see this comment: #744 (comment)