RedHat provisioning

[ehazlett]

This enables Machine to provision RedHat (RHEL). Tested on RHEL 7.0.

Refs #905

[ehazlett]

I wasn't sure about this. On AWS the extras have to be configured with a separate command. Perhaps there is a better way.

[nathanleclaire]

Oh really? Very interesting. We should ping Nirmal and see what he thinks.

Also, what is this endpoint? We should probably at least put a comment saying what it is.

[ehazlett]

If you are using RHEL with a subscription you can use the Docker docs to install. However, on EC2, that command doesn't work -- you have to use the above. I'm assuming it's subscription related.

[ehazlett]

@nathanleclaire @sthulb @hairyhenderson PTAL

[nathanleclaire]

Why not check whether we're on AWS by attempting a typecast of the driver to amazonec2.Driver like so:

if _, err := provisioner.Driver.(amazonec2.Driver); err != nil {
    // do AWS-specific logic
} else {
    // do logic for everyone else
}

Admittedly this won't work for generic, so maybe we could fall back on the other method in that case.

[ehazlett]

Yeah I think the typecast is better. Thx!

[sthulb]

How do we deal with RHEL7 like distros?

[sthulb]

we should set the terminal to os.Getenv("TERM") and default to xterm

[ibuildthecloud]

I just put in #1096. It's for RancherOS support, but as a part of that work I refactored the ubuntu provisioner out into a GenericProvisioner and UbuntuProvision. Seem like this code could also embed the GenericProvisioner I created.

[nathanleclaire]

For alphabetical order, or...?

[ehazlett]

Yes it was listed out of order from one of the rebases.

[RusDavies]

Hi,

I few days ago, I started coding a FedoraProvisioner for docker-machine. Then I realized this pull request for a (very similar) Redhat provisioner was already in progress. However, there are some differences.

So, I adapted by using the code for this pull request (from the redhat-provisioning branch of ehazlett/machine), and modifying it to allow integration of my code for Fedora. To do so, I split the RedhatProvisioner into two parts, one containing code that is generic to all redhat-family distributions, and another that seems specific to RHEL only. With that done, it was a trivial matter to add a new Fedora provisioner, which others may find useful.

Code is here https://github.com/RusDavies/machine/tree/provisioner-fedora. Important parts are the files redhat_family.go, redhat_family_RHEL.go, and redhat_family_fedora.go, all under libmachine/provision.

I've been using the resulting FedoraProvisioner on Amazon E2, using the standard Fedora 21 cloud images. It seems to works fine. Clearly, however, more thorough testing is required.

At this point I'd like to engage in discussion about the acceptability of this adaptation/extension of the RedhatProvisioner code for inclusion in docker-machine. However, given the RedhatProvisioner code itself hasn't yet been merged, then I'm not sure of the best way to proceed. Which of the following is preferable:

1. Make a new pull request against the ehazlett/machine repository
2. Make a new pull request against the docker/machine repository
3. Wait for the RedhatProvisioner to be merged, and then make a pull request against docker/machine.
4. Something else?

Thanks :o)

[sthulb]

@ehazlett at this point, can this provisioner be a generic for both redhat + centos + fedora? or are they very different?

[RusDavies]

@sthulb , there are some items in this RedhatProvisioner that seem very specific to RHEL only, such as the configureRepos(). Likewise for isAWS(), and installOfficialDocker(). Usage of these RHEL specific methods are hard coded into the Provision() procedure.

However, the differences between RHEL and Fedora seem minor. If a suitable interface is provided, that can be assigned in an embedding structure, thereby allowing virtual dispatch overrides, then those differences can be lifted out into overriding methods in the embedding structure that are called at specific points in the provisioning process. Makes it nice and DRY. That's what I've done in my code (see comment before yours).

[ehazlett]

@RusDavies i have actually been working on Fedora and CentOS provisioners based off of RHEL.

@sthulb yeah there are some minor differences.

@RusDavies you can see the branch here: ehazlett/machine@redhat-provisioning...ehazlett:fedora-provisioner

It's pretty lightweight.

[RusDavies]

@ehazlett , I thought I may be too late to the game. Good to know it's coming, anyway. :o)

Some questions:

1. How will you prevent the FedoraProvisioner from attempting to run your RHEL subscription stuff. Will you gate it only for OsReleaseId="rhel"? For example, this will fail on Fedora: "subscription-manager repos --enable=rhel-7-server-extras-rpms". Similarly, what's the effect on Fedora of, "yum-config-manager --enable rhui-REGION-rhel-server-extras"?

2. What other configuration nuances will other redhat-like distributions have? Are you going to include every nuance in the general code, and gate each case? Is it not better to take such distribution specifics out of the generalized code, and put them in a higher layer?

Thanks

UPDATE: Just tried the commands in configureRepos() on a Fedora machine:

• "subscription-manager repos --enable=rhel-7-server-extras-rpms" fails with an error: command not found.
• "yum-config-manager --enable rhui-REGION-rhel-server-extras" does nothing, and returns without error.

Thus, FedoraProvisioner will work if isAWS() returns true, but will fail otherwise. So, at least, maybe gate the call to configureRepos()?

[ehazlett]

@RusDavies thanks for the feedback. It's not done (I've experienced the same issues in Fedora as well) but I pushed to show the work. It most likely won't make it in for 0.3.0 as we have a bunch of testing for RHEL yet.

The problem with the RedHat distros is each one configures things slightly different then with various repos. This multiplies when you add more flavors.

[RusDavies]

@ehazlett , no problem. I'm a happy to help.

The slight variance between Redhat friends is precisely my concern. As support for other friends are added, it may tend towards a spaghetti of gates and exceptions.

I've updated my own code to more closely match your branch with the FedoraProvisioner. Take another look: using an interface, as I have, to place hooks in the process really doesn't add much complexity, but does help to create separation between distribution specifics.

Oh, and one other thing... given my use-cases tend to involve compliance concerns, seeing a "--nogpgcheck" on a yum command makes me frown just a little ;o)

[ehazlett]

@RusDavies actually after looking we don't need configureRepo at all since we use our own RPMs. I will look into yours but I'm not sure we need all of the hooks -- the Fedora provisioner is now 39 loc. I would lean towards adding the hooks only if needed as it's more overhead and potential code paths.

[RusDavies]

@ehazlett , ok, fair enough.

[RusDavies]

@ehazlett, more feedback ...

RedhatProvisioner.installOfficialDocker() fails if docker is already installed from a different source.

This can happen when using "--driver generic" to connect to an existing machine, or otherwise using a custom image, in either case where docker may already be installed from e.g. distribution repos.

Worth noting, at least in my case, and likely for others, machines are deployed against a BOM which includes software versions. If I have an AMI that includes a particular version of docker, then I don't necessarily want to change the version or source. In my case, this important not just for consistency & stability, but also for compliance reasons.

If docker is already present on the target machine, can we just use it? Trivial gate would be to do the install as:

provisioner.SSHCommand(fmt.Sprintf("sudo sh -c  'type /bin/docker || yum install -y --nogpgcheck  %s'", provisioner.DockerRPMPath))

[denverdino]

@ehazlett Evan
After the 26432b7 the Native SSH client is broken. I got the following error when executing the following lines of "ssh/client.go"

    if err := session.RequestPty("xterm-256color", termHeight, termWidth, modes); err != nil {
        return string(output), err
    }

The error output for provisioning my ubuntu image is

SSH cmd err, output: EOF: 

And I think the logic of the Output method also has some problem for the command will be executed twice. Does it work as expected?

func (client NativeClient) Output(command string) (string, error) 
    ...
    output, err := session.CombinedOutput(command)
    ...
    return string(output), session.Run(command)
}

Thanks

[ehazlett]

@sthulb do you know what is causing the Native to have issues?

@nathanleclaire is the Output func correct or did I munge a merge?

[nathanleclaire]

Mmm, yeah that Output function definitely looks wrong. I'll take a look at it.

[ehazlett]

@nathanleclaire thx

[sampowers]

Is there a way to make this step be conditional/take a shorter amount of time? I already have a mirror on the local network, but this is taking up most of the setup time.

[ehazlett]

Unfortunately this depends on the provider and mirror. If you have a local mirror then that's about all you can do. We must update as some images do not contain the proper device mapper libs that are needed.

[marcellodesales]

Hi @ehazlett, I'm trying to use the generic driver on RHEL 7.1... I have the following setup:

SSK keys already authorized on the host

~ ⌚ 1:04:08
$ ssh 10.132.52.146

Last login: Tue Jun 23 07:59:07 2015 from 172.17.193.49
[mdesales@pppdc9prd8ok ~]$ cat /etc/redhat-release 
Red Hat Enterprise Linux Server release 7.1 (Maipo)

Command to create host fails while importing keys

After a few seconds, the command to create and use the generic driver fails..

~ ⌚ 0:58:57
$ docker-machine create --driver "generic" --generic-ip-address "10.132.52.146" --generic-ssh-user "mdesales" dotci
Creating CA: /home/mdesales/.docker/machine/certs/ca.pem
Creating client certificate: /home/mdesales/.docker/machine/certs/cert.pem
Importing SSH key...
Error creating machine: exit status 1
You will want to check the provider to make sure the machine and associated resources were properly removed.

What should I do?