Add support for IBM Cloud

[ashcrow]

In order to properly support this, we should:

• reserve and document a platform ID for this environment
• check if we have some metadata endpoint (or hypervisor back-channel) available for Ignition to use
• add support in coreos-assembler to buildextend images for this platform
• add support for ore to upload images
• add support for afterburn if needed

/cc @imcleod

[ashcrow]

Keying off #276 (comment)

there doesn't seem to be a provider for "IBM Cloud" in Terraform. How about we use ibmcloud?

[ashcrow]

cc @arithx @ajeddeloh @darkmuggle @vrutkovs @lucab

[ashcrow]

It looks like ibm is used for the provider via a third party repo.

[ashcrow]

In terms of metadata endpoint I've found docs pointing to https://cloudinit.readthedocs.io/en/latest/topics/datasources/configdrive.html (which isn't really an endpoint ..). Still digging.

[arithx]

It looks like ibm is used for the provider via a third party repo.

In that same link they also have a variable called ibmcloud_api_key so I don't think even that is sure what to call it.

I'd personally lean towards ibmcloud

[ashcrow]

I'd personally lean towards ibmcloud

I do as well

[lucab]

cloud-init calls it IBMCloud. It seems to be a variant of openstack-configdrive.
https://github.com/cloud-init/cloud-init/blob/master/cloudinit/sources/DataSourceIBMCloud.py

[ashcrow]

cloud-init calls it IBMCloud. It seems to be a variant of openstack-configdrive.
https://github.com/cloud-init/cloud-init/blob/master/cloudinit/sources/DataSourceIBMCloud.py

Side note: That points to config drive as well instead of a metadata endpoint. So far I've been unable to find a metadata endpoint in the docs 🙁.

[ashcrow]

The other day @darkmuggle found https://cloud.ibm.com/apidocs/vpc. However, I can't seem to fund a location for user-data. I'll continue to look a bit more, but I'm not optimistic that a traditional user-data enabled endpoint is provided.

@tuan-hoang1 do you happen to know if there is a user-data endpoint within IBM Cloud? (a location we can put Ignition data that is retrievable by the instance)

[ashcrow]

OK, for the platform id we will go with ibmcloud.

[ashcrow]

Updating docs: coreos/fedora-coreos-docs#15

[lucab]

The docs/URLs referenced in #277 (comment) are for the Internet-public API, not for link-local metadata.

According to current @ajeddeloh research, we think that there is no metadata endpoint on this cloud provider (which is consistent with the cloud-init implementation I linked above), but we are waiting for confirmation.

[ajeddeloh]

I got confirmation on userdata, though I didn't explicitly ask about other metadata endpoints.

[ashcrow]

I got confirmation on userdata, though I didn't explicitly ask about other metadata endpoints.

To clarify, the confirmation is there is no endpoint for userdata and ConfigDrive is the expected solution, correct?

[ajeddeloh]

Correct

[lucab]

Misc notes from my exploration of the environment so far.

In my IBM cloud console I see there are three types of infrastructure/compute-node available.
Each one has a specific and very different approach to networking (all cases with a floating-ip attached):

• classic: no DHCP, multiple interfaces, no NAT, legacy interface naming (ethX)
• VPC gen1: no DHCP, single interface, NAT, legacy interface naming (eth0)
• VPC gen2: DHCP, single interface, NAT, predictable interface naming (ens3)

None of those of seems to to have reachable link-local metadata endpoint. They all have a local config-drive, however location and content is NOT uniform:

• classic and gen1: traditional openstack-like VFAT disk with JSON files
• gen2: iso9660 disk (labeled cidata) with user-data and meta-data files. The latter consists of multiple lines in key-name: value format

Things that I didn't check but would be good followups for the non-DHCP ones:

• NTP servers
• DNS servers
• hostname and domain search-list

[ashcrow]

classic and VPC Gen 2 should be supported. It may be worth having different platform id's for these so we can tell them apart easily. Thoughts?

[ajeddeloh]

Having both under one name isn't a problem from the Ignition side. We can do it like the openstack provider and just try both at the same time and see which one succeeds.

[ajeddeloh]

Thinking about this more though, we'd need to more for classic to bring up networking in the initramfs, so having it differentiated might be useful there. cc @lucab @bgilbert

[ashcrow]

Thinking about this more though, we'd need to more for classic to bring up networking in the initramfs, so having it differentiated might be useful there. cc @lucab @bgilbert

cc'ing @darkmuggle as well

[darkmuggle]

Breaking the enablement into two separate "clouds" makes sense from a planning and implementation perspective.

[lucab]

After some more chatting and work planning, we decided to start with a reduced scope in order to break a few bootstrapping tasks in an easier way.
For the moment, we are scoping the ibmcloud platform ID to mean "VPC Gen2" only. Reference docs update at coreos/fedora-coreos-docs#16.

Once we get to a reasonable state with "VPC Gen2", we will re-evaluate "classic" infrastructure next and decide whether it is feasible and better (from a technical point of view) to use the same platform ID for both. If not, we'll need to note down the new constraints and start work under a new platform ID.

[lucab]

Reached a further decision point on the chimera-cloud situation, input on coreos/coreos-assembler#1119 appreciated.

[dustymabe]

For this I'm going to shoot to an MVP of just getting the images on the website. After that we can worry about auto uploading via ore and CI testing via kola. Here's the list of things to do. I'll update this list as we go:

• coreos-assembler produces raw image with platform ibmcloud
  • qemuvariants: add ibmcloud coreos-assembler#1118
  • schema: add ibmcloud to schema coreos-assembler#1740
• afterburn
  • ibmcloud: add support for SSH keys afterburn#475
• build images via pipeline
  • pipeline: start creating ibmcloud artifacts fedora-coreos-pipeline#280
• coreos-meta-translator
  • Add support for IBM Cloud #277
• fedora-coreos-stream-generator
  • Add IBM Cloud disk artifacts to stream metadata fedora-coreos-stream-generator#13
• website
  • https://pagure.io/fedora-web/websites/pull-request/149
• booting on IBM cloud docs
  • provisioning: add instructions for IBM Cloud fedora-coreos-docs#185
• fedora CoreOS Release Browser
  • Add IBM Cloud fedora-coreos-browser#17

This list is inspired by the list we had for vultr.

[dustymabe]

I broke out the ore/kola work into #614

[dustymabe]

A few new PRs

• schema: add ibmcloud to schema coreos-assembler#1740
• pipeline: start creating ibmcloud artifacts fedora-coreos-pipeline#280

Updated #277 (comment) to reflect current status.

[dustymabe]

A few more new PRs

• coreos-meta-translator: add IBM Cloud image to trans.py fedora-coreos-releng-automation#124
• Add IBM Cloud disk artifacts to stream metadata fedora-coreos-stream-generator#13
• https://pagure.io/fedora-web/websites/pull-request/149

[dustymabe]

and a few final ones:

• Add IBM Cloud fedora-coreos-browser#17
• provisioning: add instructions for IBM Cloud fedora-coreos-docs#185

[dustymabe]

This is pretty much all done. See updated checklist in #277 (comment)

The only thing left is me pushing the merge button on the docs PR, which we'll do when the round of releases goes out this week.

[dustymabe]

The IBM Cloud images first landed in:

• stable stream release 32.20200923.3.0
• testing stream release 32.20201004.2.0