Clarify Cluster ID, UUID, Name in API

[dgoodwin]

Right now we have the cluster deployment name, a ClusterID, and a ClusterUUID.

The installer has a Name, and a ClusterID. Name is used in DNS in combination with the base domain.

Our ClusterUUID -> their ClusterID
Our ClusterID -> their cluster Name.
Our cluster deployment name is not used.

SD has raised a use case where name is not unique even within one account, as it could be combined with different base domains. As such we can't really map our cluster name to their cluster name. (We're going to need something validating uniqueness)

We should probably rename our ClusterID to ClusterName to eliminate the confusion around ClusterID in each API. Should we make it "DNSName" instead?

SHould we then rename our ClusterUUID to just ClusterID to match installer? Or is the UUID more precise and clear.

The result would be:

ClusterDeployment:
Name: foo-xldkj
DNSName: foo
BaseDomain: example.com
ClusterUUID: UUIDHERE

[wking]

We should probably rename our ClusterID to ClusterName to eliminate the confusion around ClusterID in each API. Should we make it "DNSName" instead?

Just to clarify, the installer uses its ClusterName to generate subdomains (e.g. ${ClusterName}-api.${BaseDomain}). But I don't know if there is anything that is just ${ClusterName}.${BaseDomain} (I don't know what ingress uses, though). I dunno if there's an easy way to hint at that in the variable name, but maybe SubdomainPrefix or some such would make it more obvious than DNSNamee?

[dgoodwin]

@csrwng put forth a slight change to my plan in separating the DNS from the naming, which would leave us with:

Hive Cluster Deployment Name = Installer Cluster Name. It's used for tagging resources in AWS, both their Name tag as well as the k8s.io cluster tag. Generate name can be used if the particular deployment wants that additional randomness.

Add SubdomainPrefix to both APIs, used for DNS when combined with -api.basedomain etc.

Hive ClusterUUID = Installer ClusterID. (propose a rename to UUID in both)

Does this seem clearer, and ok with your team @wking ?

[wking]

I don't want to speak for the rest of the installer folks. @abhinavdahiya, did you want to chime in here too?

Hive Cluster Deployment Name = Installer Cluster Name. It's used for tagging resources in AWS, both their Name tag as well as the k8s.io cluster tag. Generate name can be used if the particular deployment wants that additional randomness.

I personally don't have much need for a name with an explicit goal of both human-recognition and uniqueness. If you want that, I'd rather you just stuffed it into ClusterID (and took responsibility for ensuring sufficient uniqueness). More discussion along these lines in openshift/installer#458.

Hive ClusterUUID = Installer ClusterID. (propose a rename to UUID in both)

While we want this variable to hold a reasonably-unique ID, it only needs to be unique within an AWS account (or libvirt host, etc.). It doesn't have to be an RFC 4122 UUID (that's just what we happen to default to). So I'm ok with the current ClusterID name here, but I'd be ok switching if other folks had a stronger preference for ClusterUUID.

[dgoodwin]

Ok another stab:

• Cluster Deployment Name is a Kube only construct and has no bearing on anything. Presumably will be used with GenerateName. i.e. foo-xlskj
• SubdomainPrefix = Desired DNS prefix. i.e. foo.
• ClusterID = a unique enough identifier for the cluster. Typically generated, but if you were to provide one we would respect it. Used for tagging AWS resources k8s.io/cluster, and tectonicClusterID. Full UUID not required. i.e. foo-xlskj-pqosi

What would we use for the "name" tag on AWS resources? If installer generates a full UUID for ClusterID, this is not great. However if Hive passes it's semi-friendly ClusterID (foo-xlskj-pqosi) then this isn't so bad.

Should the k8s cluster deployment name -> install config cluster name -> AWS resource name prefix?

[wking]

• Cluster Deployment Name is a Kube only construct and has no bearing on anything.

Would this be passed to the installer? I don't see an analog there now, but I'm obviously fine with you folks doing whatever you want internally ;).

• SubdomainPrefix = Desired DNS prefix. i.e. foo.

This is fine with me. On the "is this just a prefix?" front, openshift/installer/pull/717 is landing an ingress config to use apps.${clusterName}.${baseDomain}, so in at least some cases it won't be a prefix. I dunno if that's enough reason to weaken to the more generic DNSName?

ClusterID = a unique enough identifier for the cluster. Typically generated, but if you were to provide one we would respect it. Used for tagging AWS resources k8s.io/cluster, and tectonicClusterID.

Works for me, and this addresses my concerns about kubernetes.io/cluster/{clusterName} by offloading the uniqueness responsibility to the caller who overrides clusterID.

However if Hive passes it's semi-friendly ClusterID (foo-xlskj-pqosi) then this isn't so bad.

Yeah, that addresses @joelddiaz's concerns about using opaque IDs in the tags by including a human-readable portion.

Should the k8s cluster deployment name -> install config cluster name -> AWS resource name prefix?

I'm not entirely clear on what you mean here. It sounds like you want to map your SubdomainPrefix (or whatever) to the installer's clusterName (metadata.name in the install-config YAML). And you want to add some random characters to that to create the generated name (I don't know what this is for). And you want to add some more random characters to create the clusterID. That sounds fine to me; as long as you're comfortable asserting the uniqueness of your clusterID, I don't mind how many steps you take to generate it ;).

[dgoodwin]

Would this be passed to the installer? I don't see an analog there now, but I'm obviously fine with you folks doing whatever you want internally ;).

I think cluster deployment name -> install config metadata.name, and it should get used for AWS name tags.

We can't use the ClusterID for those if the installer still uses a full UUID because standalone installer users should not login to aws and see UUIDs in names in the console. (regardless if Hive is working around it)

Seem reasonable?

[wking]

Would this be passed to the installer? I don't see an analog there now, but I'm obviously fine with you folks doing whatever you want internally ;).

I think cluster deployment name -> install config metadata.name, and it should get used for AWS name tags.

But metadata.name also shows up in DNS. I don't think folks want foo-xlskj-api.example.com.

We can't use the ClusterID for those if the installer still uses a full UUID because standalone installer users should not login to aws and see UUIDs in names in the console. (regardless if Hive is working around it)

I have no problem with users logging into AWS and seeing kubernetes.io/cluster/439543aa-e9bf-45b1-9993-ba67672074ec: owned. I'm also fine with them setting their own clusterID so they can see kubernetes.io/cluster/foo-xlskj-pqosi: owned, or with them adding a user-tag so they can see for-questions-about-this-cluster-call: Alice (or whatever they want to see). I'd rather have the installer stay out of the business of re-inventing UUIDs (but shorter while still being reliably unique enough?) to use as suffixes for a human-readable name, but I'm fine if Hive wants to take that on.

[dgoodwin]

But metadata.name also shows up in DNS. I don't think folks want foo-xlskj-api.example.com.

I thought we were both going to be adding SubdomainPrefix/DNSDomain, our config mirrors yours as much as possible and ideally we should probably stick pretty close together. This would leave your metadata.name largely unused though, so its probably fine if our SubdomainPrefix becomes your metadata.name.

After discussion with @smarterclayton and @markturansky yesterday it sounds like we need to drop the functionality where callers can specify their UUID, always let the installer do it, and probably implicitly stick with UUIDs permanently.

I'm ok with kubernetes.io/cluster/439543aa-e9bf-45b1-9993-ba67672074ec: owned, however we still need to sort out the name tag for the AWS resources, but for that we can probably stick with SubdomainPrefix/metadata.name.

Being able to specify additional tags for AWS resources would be nice but we can look at in future. It's probably going to be more complexity than it's worth when it comes to cluster provisioned post-install resources.

So in summary:

• Hive ClusterDeployment.Name is not used for anything and callers can use GenerateName if desired.
• Hive will add ClusterDeployment.SubdomainPrefix, pass to installer as install config metadata.name.
• Installer will use metadata.name for the subdomain prefix, and the friendly "name" tags in AWS.
• Hive and Installer will drop ClusterID/UUID from APIs, Installer will always generate a RFC 4112 UUID for each cluster install invocation. Hive will read it from metadata.json and upload to ClusterDeployment.Status.ClusterUUID as soon as possible in the install process.
• Installer will tag AWS resources with kubernetes.io/cluster/439543aa-e9bf-45b1-9993-ba67672074ec: owned. Hopefully this will eventually be the only tag we scrub for to deprovision.

[wking]

Installer will use metadata.name for the subdomain prefix, and the friendly "name" tags in AWS.

What are the friendly "name" tags? Things like this?

[dgoodwin]

No the literal "name" tags in AWS, i.e. the first thing you see when browsing in the AWS console on the leftmost column.

[dgoodwin]

Although I guess that is technically what you linked to becomes. :)

[cben]

Being able to specify additional tags for AWS resources would be nice but we can look at in future. It's probably going to be more complexity than it's worth when it comes to cluster provisioned post-install resources.

A possible use case for custom tags: it's possible in aws to get cost allocation report broken down by tags:
https://aws.amazon.com/blogs/aws/aws-cost-allocation/
https://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/cost-alloc-tags.html
Sounds like it is doable to add kubernetes.io/cluster/439543aa-e9bf-45b1-9993-ba67672074ec tags as cost allocation tags, but this is different key per cluster so gets annoying if customer wants multiple clusters on same aws account...
Or if they have non-openshift stuff on same account already tagged with say departments / cost centers, it would be nice to fit clusters into their existing tagging scheme, so they'll be categorized correctly in aws cost report.
(I don't know enough about aws, may well be other ways to achieve this?)

[dgoodwin]

https://jira.coreos.com/browse/CO-127

[dgoodwin]

Looping back I think I'd like to move Hive's Spec.Config.ClusterID to Spec.Config.ClusterName, rather than SubdomainPrefix. It gets used in other ways and probably will continue to, that makes more sense to me with a more generic name. Any objections?