balancerd: per-tenant metrics for http

[maddyblue]

Extract the tenant from the CNAME.

Motivation

• This PR adds a feature that has not yet been specified.

Checklist

• This PR has adequate test coverage / QA involvement has been duly considered.
• This PR has an associated up-to-date design doc, is a design doc (template), or is sufficiently small to not require a design.
• If this PR evolves an existing $T ⇔ Proto$T mapping (possibly in a backwards-incompatible way), then it is tagged with a T-proto label.
• If this PR will require changes to cloud orchestration or tests, there is a companion cloud PR to account for those changes that is tagged with the release-blocker label (example).
• This PR includes the following user-facing behavior changes:
  • n/a

[pH14]

Will this capture webhook calls as well?

[alex-hunt-materialize]

This assumes the SNI servername and frontegg tenants are the same.

Unfortunately, the SNI servername is not the tenant. The CNAME response to looking up that SNI servername does include the tenant ID, though.

[maddyblue]

Will this capture webhook calls as well?

Yes!

[maddyblue]

Lint is producing errors like

error[duplicate]: found 2 duplicate entries for crate 'enum-as-inner'
    ┌─ /home/mjibson/materialize/Cargo.lock:173:1
    │  
173 │ ╭ enum-as-inner 0.5.1 registry+https://github.com/rust-lang/crates.io-index
174 │ │ enum-as-inner 0.6.0 registry+https://github.com/rust-lang/crates.io-index
    │ ╰─────────────────────────────────────────────────────────────────────────^ lock entries
    │  
    = enum-as-inner v0.5.1
      └── sysctl v0.5.4
          └── mz-environmentd v0.92.0-dev
              ├── (dev) mz-balancerd v0.92.0-dev
              ├── (dev) mz-environmentd v0.92.0-dev (*)
              └── mz-sqllogictest v0.0.1
    = enum-as-inner v0.6.0
      └── hickory-proto v0.24.0
          └── hickory-resolver v0.24.0
              └── mz-balancerd v0.92.0-dev

and uh I have no idea what to do about that.

[alex-hunt-materialize]

This looks great!

It seems that sysctl is the thing pulling in the older enum-as-inner, and there is a newer v0.5.5 that uses the newer version. Perhaps upgrading to that would resolve the lint issues?

[alex-hunt-materialize]

Oof, looks like it has other duplicates, which aren't so simple.

[alex-hunt-materialize]

Hickory DNS has idna 0.5 on their main branch. I'm not sure what to do about the quick-error thing, though, as resolv-conf 0.7.0 is the latest, and seems abandoned for the last four years, despite 24 million downloads.

[alex-hunt-materialize]

I've opened hickory-dns/resolv-conf#35 to hopefully remove the dependency on quick-error in resolv-conf, but I doubt it will get merged any time soon.

[alex-hunt-materialize]

It seems like a pain, but a possible solution:

1. Update to sysctl v0.5.5.
2. Point at a newer commit of hickory-resolver (main), which uses idna v0.5. I think our git dependency rules require us to fork for this.
3. Add a patch directive for resolv-conf v0.7.0 to point at a fork that contains Remove dependency on quick-error hickory-dns/resolv-conf#35

[maddyblue]

There are other ideas too like use the domain crate which seems to have somewhat fewer deps, but still quite a lot.

[maddyblue]

Reworked to use the domain crate, so only 2 new crates needed to be audited. That also removed the Cargo.lock duplicates.

[maddyblue]

@alex-hunt-materialize the mzcompose tests do not test the CNAME -> tenant stuff because I don't know if docker compose can set up arbitrary CNAMES and didn't investigate that. After deploying this to staging we will need to do a few tests to make sure the http tenants are being correctly extracted.