Fix machine discovery on compute nodes

[xylar]

This merge uses an E3SM-Unified environment variable, E3SMU_MACHINE, to determine the machine if this environment variable has been set (e.g. by sourcing an E3SM-Unified load script). This makes it possible
to detect the machine on compute nodes if you are using E3SM-Unified.

This merge also cleans up a few issues with the docs that I noticed.

Partially addresses #406

[xylar]

@forsyth2, let me know if you'd like me to make a separate PR for the docs changes.

[forsyth2]

From reading the code changes, this looks good to me.

Partially addresses #406

Why only partially? This seems like a full resolution.

[xylar]

Why only partially? This seems like a full resolution.

This only fixes the problem if someone is using E3SMU, not if you use a development env.

[chengzhuzhang]

@xylar e3sm_diags has this same challenge that when use a development env, and run on compute node, it won't auto detect the machine and use machine specific info. I'm wondering if it is possible to update mache for a full solution?

[xylar]

@chengzhuzhang, I'm sorry but I don't know of any solution to this. The compute nodes have super generic names and there is not a consistent way to detect what machine you are on using things like environment variables or files. If you have a suggestion, I'm up for it. But otherwise we simply need users to identify with a config option or command-line flag what machine they are on when they launch on compute nodes.

[chengzhuzhang]

I was thinking for the machines that we support, if we can find some pattern of compute nodes names and filter them in mache. But if there is not a pattern, i guess asking users to provide a machine name is not too bad.

[xylar]

@chengzhuzhang, on Chrysalis, it might work since the login nodes are named something like chr-0472. I can add that.

But on Anvil, they are named: b677, which seems way too generic to use, but maybe folks feel like it's fine for our limit number of to assume b and 3 numbers is Anvil?

Compy is the same story: n0432. Again is this safe to assume that anything like n and 4 numbers is a Compy compute node? I have been hesitant.

On Perlmutter, even the login nodes are too generic so we have to fall back to an environment variable, and that probably works on compute nodes, though I can't say I've checked.

Frontier was similar with the added problem that there was no definitive environment variable to use so I had to fall back on an environment variable related to modules. Again, that likely works on compute nodes.

Other supported machines like Chicoma, Andes and ACME1, I don't know what the deal is there since I don't use the that much.

[xylar]

@chengzhuzhang can you open this issue on https://github.com/E3SM-Project/mache/issues? This is not the right place to discuss this further.