sdk-ts : Support multiple stacks per network

[xmonader]

threefoldtech/home#1550

[0oM4R]

Issue update :
started to implement the logic but can't use the monitor package

[0oM4R]

also current mentor services logs the aliveness of the services but do not return any values,i think it need some refactor adding new methods

[0oM4R]

WIP: creating the logic to pick the first available stack per service add this init class to the alivenessCheck in monitoring

export class serviceStackPicker {
  private result: ServicesUrls = {};
  constructor(public options: StackPickerOptions) {}

  private async pingService(service: ILivenessChecker) {
    const status = await service.isAlive();
    if ("disconnect" in service) {
      await (service as IDisconnectHandler).disconnect();
    }
    return status;
  }
  async GetAvailableServices(): Promise<ServicesUrls> {
    if ("tfChain" in this.options)
      this.result.tfChain = await this.getAvailableServiceStack(
        this.options.tfChain.slice(1),
        new TFChainMonitor(this.options.tfChain[0]),
      );
    if ("gridProxy" in this.options)
      this.result.GirdProxy = await this.getAvailableServiceStack(
        this.options.GirdProxy.slice(1),
        new GridProxyMonitor(this.options.GirdProxy[0]),
      );


    return this.result;
  }
  async getAvailableServiceStack(urls: string[], service: ILivenessChecker) {
    let index = 0;
    do {
      const status = await this.pingService(service);
      if (status.alive) return service.serviceUrl();
      console.log(`${service.serviceName()}: failed to ping ${service.serviceUrl()}, due to ${status.error}`);
      service.setServiceURl(urls[index++]);
    } while (index < urls.length);
  }
}

and should be called anywhere with

 const test = new serviceStackPicker({
    tfChain: ["faf", "wss://tfchain.dev.grid.tf/ws", "wsss://tfchain.dev.grid.tf/ws"],
    GirdProxy: ["hahah", "https://gridproxy.dev.grid.tf"],
  });
  console.log(await test.GetAvailableServices());

also we had to add setServicesUrl method to IServiceBase and add some changes to update the effected props

this snips is the initial phase of coding, feel free to suggest any refactors

[0oM4R]

Issue Update:
monitoring part is almost ready, will support accessing the pick function directly and make the #3105 ready for review.

[0oM4R]

Issue update:
was investigating how to read and pars array form user, but i can't reach anything;

WIP: creating a script to read from user the stacks for each service and then will convert it to string and export it
current behavior

[AhmedHanafy725]

it shouldn't be interactive. you can export them with comma-separate then split them in the script

[0oM4R]

it shouldn't be interactive. you can export them with comma-separate then split them in the script

done export target stacks separated with comma will work fine

[0oM4R]

Issue update:
#3105 got some changes:

• add stats and activation services
• fix build for browser
• replace fetch with axios
• refactor sendRequest to adapt graphQl requests
• add options to allow validating tfchain while validating rmb, or take the provided tfchain url as it is

[0oM4R]

Issue update:
#3105 had some refactors
WIP: finding a way to integrate this logic with playground

[0oM4R]

Issue update: applied pr comments and introduce new features

[0oM4R]

New monitoring pr is ready : #3134

[0oM4R]

All review comments applied on #3134

[0oM4R]

Issue update:
fixed some issues in monitroring while integrating it in UI,

Blocker:

tfgrid-sdk-ts/packages/playground/src/clients/index.ts

Lines 1 to 10 in 0a5568e

	import TFGridGqlClient from "@threefold/graphql_client";
	import GridProxyClient from "@threefold/gridproxy_client";
	import { QueryClient } from "@threefold/tfchain_client";
	const gqlClient = new TFGridGqlClient(window.env.GRAPHQL_URL);
	const gridProxyClient = new GridProxyClient(window.env.GRIDPROXY_URL);
	const queryClient = new QueryClient(window.env.SUBSTRATE_URL);
	export { gqlClient, gridProxyClient, queryClient };
	export * from "./indexedDB";

those clients got loaded before initializing the envs and got undifined urls, suggest moving those clients to grid store

[0oM4R]

Issue Update:
Monitoring integrated in playground
Blocker: Not sure yet should we provide fake mnemonic for RMB monitor or what

[0oM4R]

all requested changes applied, and ui is ready as well.

[0oM4R]

As we discussed with @sameh-farouk, we can use chain /health endpoint to verify node rpc status,
if it responds with 200 OK, then it is alive and we can rely on it

[0oM4R]

Issue update

• now all services are monitored via fetch requests
• applied some review comments

Blocker
while pinging all urls in parallel, by passing url to the alive method, all urls give an error even if one of them is reachable
work on testing branch

[0oM4R]

Issue update:
I added the required fallback mechanism, but I'm facing a very wired issue, when we have more than one invalid url in the stack, the whole requested urls for all services got effected and gives Timeout error

but if we remove one of the invalid/unreachable urls from the array it works fine.

tfgrid-sdk-ts/packages/monitoring/example/serviceURLManager.ts

Line 10 in 889190d

const gridproxy = ["http://www.nonexistentgridProxy.com", "http://www.gooogleTF.com", "https://gridproxy.dev.grid.tf/"];

details of the new applied mechanism:

• isAlive method now takes an optional url param, and the default value is service.url.
• store the promise result of isAliveWithretires for all stacks in promise array
  

  tfgrid-sdk-ts/packages/monitoring/src/serviceMonitor/serviceURLManager.ts

  Line 128 in 889190d

  requestPromises.push(this.isAliveWithRetries(urls[i], service));

• resovle using Promise.allSattled and return the first fullfiled url in the promise array "first based the order of promises array"
  

  tfgrid-sdk-ts/packages/monitoring/src/serviceMonitor/serviceURLManager.ts

  Lines 130 to 131 in 889190d

  	const result = await Promise.allSettled(requestPromises);
  	for (const url of result) if (url.status == "fulfilled") return url.value;

Concerns:
this approach takes a lot of time, as Promise.allSattled waits for all stacks to resolve
so if i have 2 urls first one has a response on the first try, and the second is unreachable
i have to wait for the second one to exhausts all its retries, for our case, with base timeout 2 it will take about 12 seconds to response with the first url that already replied within the first 2 seconds

[0oM4R]

Issue update :
resolved the issue of fetch by providing valid or existing urls
work completed:
support url monitor in gridclient, code needs some cleanup and test then the pr will be ready

[0oM4R]

I faced an issue while testing playground integration: getDefaultUrls had braking changes as it used in playground and tests so will create new function that contain the new added logic

[0oM4R]

work completed:

• monitoring : Monitoring: add service URL manager #3134
• dashboard: Support multiple stacks per network in Dashboard #3162
• grid client: Support multiple stacks per network in gird client #3200

[A-Harby]

Verified, Devnet.

The stack is working fine, checking all the URLs at once with 3 retries and picking the working one in the same order as they were written.

For one failing but the other working.

For all of them to fail.

For Grid client.

Created an issue for the Grid Client to work as config in playground: #3399.

New test Case: TC2825 - Run locally with multiple stack

[khaledyoussef24]

verified
used Harby's test case and it is working fine

changed the stacks in the file of tfgrid-sdk-ts/packages/playground/public/config.js

and also tried to implement wrong urls in the file to check the behaviour

if many stalk are there it would switch automatically to another one
as here : (left the same wrong urls but added also the valid ones)
and the dashboard worked (but left some errors for the wrong urls