CDS with strict/logical_dns

[krm1312]

I would like to have CDS (currently file based) contain clusters of type strict_dns. It appears the solution in #5186 no longer works of inlining in the CDS cluster definition.

If this is no longer supported, is it still a valid feature request? There is at least one issue related to this in the go-control-plane project, but, I suspect it should be here (envoyproxy/go-control-plane#87 (comment)).

Specifically:

dynamic_resources:
  cds_config:
    path: "conf/cds.yaml"

cds.yaml

resources:
- "@type": type.googleapis.com/envoy.api.v2.Cluster
  name: cl_acme-service-any-compute-cluster
  connect_timeout: 0.5s
  http2_protocol_options: {}
  lb_policy: least_request
  type: strict_dns
    dns_lookup_family: v4_only
    dns_resolvers:
      socket_address:
        address: 172.22.0.100
        port_value: 8600
    load_assignment:
      cluster_name: cl_acme-service-any-compute-cluster
      endpoints:
      - lb_endpoints:
        - endpoint:
            address:
              socket_address:
                address: acme-service.service.consul
                port_value: 4435

- "@type": type.googleapis.com/envoy.api.v2.Cluster
  name: cl_acme-service-c001
  connect_timeout: 0.5s
  http2_protocol_options: {}
  # Sticky
  lb_policy: ring_hash
  type: strict_dns
    dns_lookup_family: v4_only
    dns_resolvers:
      socket_address:
        address: 172.22.0.100
        port_value: 8600
    load_assignment:
      cluster_name: cl_acme-service-c001
      endpoints:
      - lb_endpoints:
        - endpoint:
            address:
              socket_address:
                address: c001.acme-service.service.consul
                port_value: 4435
                
# c002...etc.

Basically, in our app the first request would goes to the 'any' compute cluster. After that, the rest of the "interaction" is handled by a specific cluster (say c001 above). The client sends this in a cluster_header.

These auto-register with consul - so, the idea was that all that was needed in envoy to add a new compute cluster was to drop an additional cluster in cds.yaml with its (consul) DNS.

All the clusters are just ignored, so, it is possible I'm missing something obvious. Trace logs show file reloading but, it appears everything is silently ignored.

[2019-09-09 18:03:12.876][1][debug][file] [external/envoy/source/common/filesystem/inotify/watcher_impl.cc:75] notification: fd: 1 mask: 80 file: cds.yaml~
[2019-09-09 18:03:12.877][1][debug][file] [external/envoy/source/common/filesystem/inotify/watcher_impl.cc:75] notification: fd: 1 mask: 2 file: cds.yaml
[2019-09-09 18:03:12.880][1][debug][file] [external/envoy/source/common/filesystem/inotify/watcher_impl.cc:75] notification: fd: 1 mask: 2 file: .cds.yaml.swp

Nothing relevant before or after those entries.

[root@d73e06c31888 conf]# envoy --version

envoy  version: e349fb6139e4b7a59a9a359be0ea45dd61e589c5/1.11.1/clean-getenvoy-930d4a5/RELEASE/BoringSSL

[zuercher]

I'll mark this as an enhancement request, but I don't think this has ever been possible.

[zuercher]

For your case, though, I think you can just use the hosts field on the Cluster object instead of load_assignment. That won't help if you want to make use of more advanced features available only via load_assignment (localities, lb weights, metadata, etc), though.

[krm1312]

I didn't fully process the note in the docs about inotify on ConfigSource yesterday. Just editing the file in-place won't work. When I move a file into the watched path with changes, it does work with 'hosts'. Thanks.

[krm1312]

And, it seems that it works with load_assignment after all (w/ the config above w/o the indentation typo). So, you can probably close this out.

Since this is CDS, it just logs invalid config at startup, but does not fail startup (just rejects it).