Goal remote node config

[barnjamin]

Problem

When getting started, goal is often the easiest method to start sending transactions. This ease is complicated by the need to have a node installed to get config/make requests.

Solution

Allow a config option for an Algod client that makes requests to a remote API.

@nullun pointed out that the algod.net file has the ip/port for a REST api. Pointing it to algoexplorerapi.io:443 worked to get node status on the cli after tweaking:

ben@LAPTOP-4EV1BSD6:~/go/src/github.com/algorand/go-algorand$ git diff
diff --git a/nodecontrol/algodControl.go b/nodecontrol/algodControl.go
index d6aed9f4..b180dfd4 100644
--- a/nodecontrol/algodControl.go
+++ b/nodecontrol/algodControl.go
@@ -86,7 +86,7 @@ func (nc NodeController) ServerURL() (url.URL, error) {
        if err != nil {
                return url.URL{}, err
        }
-       return url.URL{Scheme: "http", Host: addr}, nil
+       return url.URL{Scheme: "https", Host: addr}, nil
 }

 // GetHostAddress retrieves the REST address for the node from its algod.net file.

ben@LAPTOP-4EV1BSD6:~$ cat .algorand/algod.net
algoexplorerapi.io:443
ben@LAPTOP-4EV1BSD6:~$ goal node status
Last committed block: 16279285
Time since last block: 1.2s
Sync Time: 0.0s
Last consensus protocol: https://github.com/algorandfoundation/specs/tree/abc54f79f9ad679d2d22f0fb9909fb005c16f8a1
Next consensus protocol: https://github.com/algorandfoundation/specs/tree/abc54f79f9ad679d2d22f0fb9909fb005c16f8a1
Round for next consensus protocol: 16279286
Next consensus protocol supported: true
Last Catchpoint: 16270000#QK7RJHDC6FDDZ3QR4M5IDOSWXPD4XEQQP7ARI3EHKHVWRWOXMQQQ
Genesis ID: mainnet-v1.0
Genesis hash: wGHE2Pwdvd7S12BL5FaOP20EGYesN73ktiC1qzkkit8=
ben@LAPTOP-4EV1BSD6:~$ goal asset info  --assetid 438831
Error processing command: json: cannot unmarshal array into Go struct field Account.assets of type map[uint64]v1.AssetHolding

It looks like the rest client has a number of the endpoints hardcoded to v1 endpoints/data structures

go-algorand/daemon/algod/api/client/restClient.go

Lines 361 to 401 in 4b67562

	func (client RestClient) TransactionsByAddr(addr string, first, last, max uint64) (response v1.TransactionList, err error) {
	err = client.get(&response, fmt.Sprintf("/v1/account/%s/transactions", addr), transactionsByAddrParams{first, last, max})
	return
	}
	// PendingTransactionsByAddr returns all the pending transactions for a PK [addr].
	func (client RestClient) PendingTransactionsByAddr(addr string, max uint64) (response v1.PendingTransactions, err error) {
	err = client.get(&response, fmt.Sprintf("/v1/account/%s/transactions/pending", addr), pendingTransactionsByAddrParams{max})
	return
	}
	// AssetInformation gets the AssetInformationResponse associated with the passed asset index
	func (client RestClient) AssetInformation(index uint64) (response v1.AssetParams, err error) {
	err = client.get(&response, fmt.Sprintf("/v1/asset/%d", index), nil)
	return
	}
	// Assets gets up to max assets with maximum asset index assetIdx
	func (client RestClient) Assets(assetIdx, max uint64) (response v1.AssetList, err error) {
	err = client.get(&response, "/v1/assets", assetsParams{assetIdx, max})
	return
	}
	// AssetInformationV2 gets the AssetInformationResponse associated with the passed asset index
	func (client RestClient) AssetInformationV2(index uint64) (response generatedV2.Asset, err error) {
	err = client.get(&response, fmt.Sprintf("/v2/assets/%d", index), nil)
	return
	}
	// ApplicationInformation gets the ApplicationInformationResponse associated
	// with the passed application index
	func (client RestClient) ApplicationInformation(index uint64) (response generatedV2.Application, err error) {
	err = client.get(&response, fmt.Sprintf("/v2/applications/%d", index), nil)
	return
	}
	// AccountInformation also gets the AccountInformationResponse associated with the passed address
	func (client RestClient) AccountInformation(address string) (response v1.Account, err error) {
	err = client.get(&response, fmt.Sprintf("/v1/account/%s", address), nil)
	return
	}

I haven't tested all the endpoints yet.

Dependencies

goal, libgoal

[jannotti]

It is possible to "trick" goal by creating a directory with algod.net and algos.token in it. That should certainly be made easier by using environment variables or command line switches to supply address and token.

To really ease getting started, goal probably also needs a way to accept private keys for signing operations, rather than always expecting a kmd to talk to.

[barnjamin]

It is possible to "trick" goal by creating a directory with algod.net and algos.token in it. That should certainly be made easier by using environment variables or command line switches to supply address and token.

yep, got this sort of working by just pointing to algoexplorerapi.io but there are a number of functions expecting v1 algod payloads, which algoexplorer doesn't seem to support anymore. I can start to run thru them if you think this is worthwhile.

To really ease getting started, goal probably also needs a way to accept private keys for signing operations, rather than always expecting a kmd to talk to.

I figured a local KMD assumption would be ok, easier to setup than a node. But happy to see anything that would make it even easier to setup. Especially something that'd let you install the goal bin as a standalone package

[jannotti]

I'm reluctant to change one hardcoded protocol (http) for another (https). An option: If ALGOD_ADDRESS and ALGOD_TOKEN environment variables are set, then prefer them to the files in the datadir, and, in fact, don't require a data directory at all. (Obviously, some of the commands goal supports need a datadir, like node start, but many do not.)

Switching to v2 is somewhat orthogonal, but I guess the issue is that some public algod providers have v1 turned off?

Some other ways that goal could be improved to help new users.

• Allow specifying keys (either base64 encoded, or by mnemonic) for signing operations, rather than require kmd.
• Break goal out of repo so that it is go get installable
• Have a command line switch for many commands that cause them to emit their info in json, so it's scriptable.

[barnjamin]

These are great ideas to improve goal especially for new devs.

I expect that to rip out goal would need some design considerations to be discussed, maybe internally?

How should I proceed on this? File some more issues? Just start hacking? Just chill tf out and be patient?

[jannotti]

My personal opinion is that removing the need to specify -d for some operations, and instead get the info from the environment (or other switches, like --algod-token, --algod-address, --kmd-token, --kmd-address) would make a nice self-contained PR that could be accepted on its own.

I think that's also true of v1 -> v2 (as a separate PR though).

The other ideas (allow specifying keys, make goal go gettable) would probably need considerably more buy-in, so I wouldn't jump into working on them. I think you could make issues, with as much detail as possible, and that might be the first step in getting that buy-in.

[brianolson]

Env var $ALGORAND_DATA=foo currently exists as an alternative to goal -d foo; extending that along with extending other ways of specifying net&token makes sense to me.

[jannotti]

But that var requires you set up files to "trick" algod. You have to create a directory, and create files to hold the token and address. It's pretty weird if your real intent is just to make `goal` talk to an internet accessible `algod` instead of a local one. Also, algod.net only has the host and port, it's presumed to be `http`. An ALGOD_ADDRESS var would probably contain the protocol. (I suppose that would be the truly smallest useful PR - make it so algod.net can specify the protocol if it wants, so at least the "trick" works) I have noticed as I browse goal source that it would be super invasive to make the earlier ALGOD_ADDRESS change I recommended, by the way. We use the datadir all over the place as a way to pass around a lot of context - how to contact kmd, algod, etc.

…

On Tue, Sep 28, 2021 at 11:23 AM Brian Olson ***@***.***> wrote: Env var $ALGORAND_DATA=foo currently exists as an alternative to goal -d foo; extending that along with extending other ways of specifying net&token makes sense to me. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#2904 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AADL7TZJMKERI6INGS2HCHTUEHMYHANCNFSM5EGR2T5Q> .

[aetherplex]

+1. Developing locally against a node using shell scripts and goal is integral to our workflow. We have a private node running on a hosting provider that we can only currently interact with via the SDKs, which is not ideal for the prototyping stage of our contracts, so we have resorted to spinning up local networks and populating them with the requisite accounts and assets in order to test out specific contracts.

[Blackglade]

+1 to this as well. It would be a great quality of life improvement.

[algochoi]

PR to migrate the goal client from v1->v2 might allow goal to use remote v2 providers: #4684

[win-t]

Hi, I created this PR #4922
I hope it can help this issue, although it is not an elegant solution