Implement webgw in zos instead of the current infrastructure

[LeeSmet]

Intro

We experienced frequent issues with yggdrasil, such that it is not really usable as backbone for the current gateway infrastructure on the zos nodes. To this end, webgw was created to improve the connection between the backend service and the gateway node. We now need to change the implementation of gateway nodes to use the new webgw isntead.

This means that changes are needed in zos (cc @muhamadazmy ). Next to this, we might also want to include the client in images we deploy with the current gateway infrastructure, otherwise people will have to do this themselves. Also, as part of this change, we want to bill gateway traffic.

Required work

• General
  • Add the "hashed secret" needed to configure the webgw in the workload of a name/subomain
  • Images will need to be updated to include the client
• zos
  • Remove existing traeffik in favor of webgw
  • Add webgw
  • Implement configuration of webgw, when a new (sub) domain is reserved, a secret will have to be configured
  • Add billing for webgw traffic. There are counters in place already (as prometheus metrics) for data going to individual backends (based on domain name) in webgw.
• Terraform/weblets
  • Update reservations to include the hashed secret

Caveats

Known problems in the deployment:

• Subdomain TLS termination needs to be done in the client node now
• There is no backward compatibility with the current gateway infrastructure
  • Existing deployments using gateways will break
• There is a manual step involved now when using a gateway from the user side (needs to create a config for the client with the gateway address + secret used)
• Existing images won't have the client, until they are updated users need to download the client manually, which might lead to problems for inexperienced users
• Community won't like network billing, an attempt to bill private network usage was reverted in 3.8

[LeeSmet]

Let's start by adding estimates on the work required and potential problems if any are forseen, to get a good overview of the scope of the work required.

[muhamadazmy]

Closed in favor of threefoldtech/zos#1674