[Provisioner] Vagrant setup script #268

bertvv · 2018-02-07T14:03:51Z

This is some code to automate the setup of a Vagrant environment. After vagrant up, log in on the control node with vagrant ssh. Ansible should be installed, as well as the lightbulb repository. The VMs node-[123] can immediately be controlled from the control node.

dagwieers · 2018-02-07T14:33:42Z

Tested on Windows 10 and this is working very nice.

For Windows, the following instructions are important:

Windows Subsystem for Linux (WSL) and Cygwin are not recommended, resp. won't work and slow
Using CMD.EXE or Powershell are both fine.

Procedure for Windows (but works equally well for Linux)

Install VirtualBox and Vagrant (on bare machine)
- NOTE: Do not try to run vagrant within a VM or container or WSL as it cannot manage VirtualBox
- A reboot is not required (although VirtualBox and/or Vagrant may indicate it is required)
Run:

mkdir lightbulb
cd lightbulb
wget -o Vagrantfile https://github.com/ansible/lightbulb/blob/master/tools/vagrant_lab_setup/Vagrantfile

Then you can run:

vagrant up

After it has finished, you can do:

vagrant ssh

And everything should be up and running ! You can now run the below and all hosts should be addressable out-of-the-box:

yum install -y ansible
ansible -m ping all

dagwieers · 2018-02-07T14:35:33Z

We now have two options here to make it work identically on Linux and Windows (and macOS)

Ask to install Git on Windows
Download the Vagrantfile in directory lightbulb (instead of using git clone)

Both would work fine.

…to vagrant-setup-script

tima · 2018-02-07T18:20:46Z

Vagrantfile

+readonly workshop_user=vagrant
+
+readonly workshop_key="/home/${workshop_user}/.ssh/id_rsa"
+readonly workshop_inventory="/home/${workshop_user}/hosts.ini"


I think this will conflict with #265 that @dagwieers submitted.

Agreed, but merge this one first and I'll resolve the conflicts...

tima · 2018-02-07T18:21:34Z

Vagrantfile

+
+[defaults]
+
+inventory = ./hosts.ini


Likewise #264 conflict?

Agreed, but merge this one first and I'll resolve the conflicts...

tima · 2018-02-07T18:30:25Z

Vagrantfile

+install_ansible_centos() {
+  info "CentOS: installing Ansible from the EPEL repository"
+  yum -y install epel-release
+  yum -y install git ansible


The first Ansible Essentials workshop assignment is to install ansible itself -- by putting this function in we are making that pointless. I want users of this content to experience install ansible and how easy it is.

That's fine, I agree.

bertvv · 2018-02-08T10:47:22Z

Ok, the Ansible installation was removed from the script.

dagwieers · 2018-02-08T16:31:46Z

LGTM, can we merge this ?

tima · 2018-02-13T21:56:25Z

@bertvv and @dagwieers:

I was starting to merge this, but this doesn't seem to work properly.

First time I run vagrant up I get this error:

==> ansible: >>> Ansible version
==> ansible: /tmp/vagrant-shell: line 51: ansible: command not found
The SSH command responded with a non-zero exit status. Vagrant
assumes that this means the command failed. The output for this command
should be in the log above. Please read the output to determine what
went wrong.

I think the last commit didn't go far enough removing ansible from being installed. I find some other issues I was going to fix after merging.

When I rerun vagrant up it finishes running, but I don't have an inventory file of ansible.cfg file.

Given this hiccup, I'm going to do another code review to address those to see if we can address those before merging.

dagwieers · 2018-02-13T22:11:19Z

@tima The removal of ansible from the installation process broke some other stuff. We tested this thoroughly before that last commit. Shouldn't be too hard to get the other ansible references removed.

tima · 2018-02-13T21:59:42Z

Vagrantfile

+
+}
+
+# Ansible installation


This seems like cruft with the automatic installation of ansible removed now.

tima · 2018-02-13T21:59:53Z

Vagrantfile

+  ansible --version
+}
+
+is_ansible_installed() {


This seems like cruft with the automatic installation of ansible removed now.

tima · 2018-02-13T22:00:22Z

Vagrantfile

+  fi
+}
+
+# Install Ansible on a Fedora system.


We aren't supporting Fedora at this time. Please remove.

tima · 2018-02-13T22:00:58Z

Vagrantfile

+  yum -y install git
+}
+
+# Install Ansible on a recent Ubuntu distribution, from the PPA


We aren't supporting Ubuntu at this time. Please remove.

tima · 2018-02-13T22:04:21Z

Vagrantfile

+[defaults]
+
+inventory = ./hosts.ini
+forks = 50


This is too high for this lab environment (I know that in the source material from another use case) -- it should be 10 or removed entirely.

tima · 2018-02-13T22:07:07Z

Vagrantfile

+          "--memory", "#$NODEMEM",
+          "--cpus", 1
+        ]
+        vb.customize ['modifyvm', :id, '--groups', '/lightbulb_workshop']


Question why "ligthbulb_workshop"? I'd prefer just sticking with "lightbulb".

tima · 2018-02-13T22:09:13Z

Vagrantfile

+
+# Install Ansible on a CentOS system from EPEL
+install_ansible_centos() {
+  info "CentOS: installing Ansible from the EPEL repository"


We're not installing Ansible.

tima · 2018-02-13T22:10:01Z

Vagrantfile

+main() {
+  info "Bootstrapping Ansible on host ${HOSTNAME}."
+
+  ensure_ansible_installed


Function seems misnamed now.

tima · 2018-02-13T22:11:20Z

Vagrantfile

+#
+
+main() {
+  info "Bootstrapping Ansible on host ${HOSTNAME}."


This should read something like "Bootstrapping Lightbulb lab environment on host..." since we aren't installing Ansible.

tima · 2018-02-13T22:11:56Z

Vagrantfile

+  elif [ -f '/etc/lsb-release' ]; then
+
+    # Debian-based distributions
+    grep DISTRIB_ID '/etc/lsb-release' \


We are only supporting centos7 and RHEL7 at this time.

tima · 2018-02-13T22:14:04Z

@dagwieers: Yes I figure as much, but it was enough that it stopped me from merging at the moment to work with you to get it straight.

webknjaz · 2018-02-14T13:29:25Z

Vagrantfile

+ansible ansible_host=10.42.0.2 ansible_connection=local
+
+[web]
+node-1 ansible_host=10.42.0.6


I'd add integration with vagrant-hostmanager plugin instead of hardcoding IPs

Thanks for the suggestion. These fixed static IP addresses are used throughout the content that we want to keep it simple and static like this to demonstrate the static inventory features of Ansible.

Yes, but I had to track all occurrences of them once, when I had some issues with conflicting networking. I'd not want to do this again as patching vagrantfile makes it harder to sync with upstream etc. IMHO this should be easily configurable, but now it's painful sometimes, as opposed to "just works" approach.

webknjaz · 2018-02-14T13:43:25Z

Vagrantfile

@@ -6,6 +6,196 @@ $NODEMEM=256
 # Overwrite host locale in ssh session


I'd move changable options to a separate config like in https://github.com/openstack-dev/devstack-vagrant/blob/master/Vagrantfile#L7-L12
It's very useful to not have to edit local copy of vagrantfile and/or track down all the places where the value is hardcoded

Something to consider though we are trying to keep this experience as straight forward and simple as possible. We purposefully do not want to add too many knobs and switches to avoid issues getting to the task at hand learning how to automate with Ansible. By curl'ing a single Vagrantfile followed by a vagrant up then vagrant ssh it can't get much easier than that.

Ideally the whole bootstrap process would be done by Ansible and not the bash script and embedded files process that @bertvv and @dagwieers are introducing here, but in streamlining this process it makes the most sense.

Hope that reasoning makes sense.

Sounds good, but it won't make it more complicated for user if there would be sane defaults and it will work without config, but support changing things in the independent file seamlessly.
You're talking about public switches, but if they're not explicitly advertised to users it shouldn't be harmful and will enable ones who needs to really do changes make them in a comfortable way.

webknjaz · 2018-02-14T14:13:32Z

Vagrantfile

@@ -47,7 +237,9 @@ Vagrant.configure("2") do |cluster|
        "--memory", "2048",


--name fails for me when I cloned the fork to other folder and tried to have a separate set of VMs.
I would suggest adding support for some (configurable) prefixes.

I'll let @bertvv and @dagwieers and others weigh in on this idea. Perhaps an ENV variable with a default value? Again, we want to err on the side of making this as simple and "foolproof" as possible to get up and running. Our goal is to teach Ansible automation skill, not Vagrant ones. ;)

Right, env var or config with defaults is what I'd expect. Consider supporting dotenv in case of env vars.

webknjaz · 2018-02-14T15:08:26Z

Vagrantfile

+  fi
+
+  info "Ansible version"
+  ansible --version


This fails, because ansible is not being installed during the flow.

Right -- that's the line I hacked out to test the rest of this. I trust @dagwieers and @bertvv will pull that out when they get around to addressing the issues that have been raised.

webknjaz

There's mention of using vi or vim in the second guide. only vi is available. it's better to install vim in vagrantfile as well.

bbaassssiiee · 2018-02-14T21:31:56Z

This looks cluttered to me. Why not create a lightbulb box with Packer and work with that?

download Vagrantfile any way you like...

vagrant init ansible/lightbulb
vagrant up

tima · 2018-02-14T22:42:13Z

@webknjaz Adding vim would be a good thing -- I actually thought centos7 had that sort of knowledge isn't my thing. There are a few other tools we install in the AWS provisioned boxes I'd like to see added to the list (wget, nano, tree, python-pip) in addition to some other VM definitions that go dropped during renovations (ansible/tower, a ubuntu web node for testing multi-os roles, and a haproxy VM for doing a rolling update exercise/demo like #158). Feel free to file an enhancement issue that we can come back to.

@bbaassssiiee Good point. Definitely something to consider since that would streamline this process at the expense of some addition maintenance and bandwidth. Feel free to file an enhancement issue for us to revisit after we get thru some of these other things here.

webknjaz · 2018-02-15T09:23:49Z

@tima why is it labeled windows? this looks misleading to me

tima · 2018-02-15T13:10:36Z

@webknjaz Windows was in reference to this: #268 (comment)

webknjaz · 2018-02-17T22:14:52Z

@bertvv this now needs conflict resolution

tima · 2018-02-19T17:08:32Z

pinging @bertvv and @dagwieers

webknjaz · 2018-02-26T13:45:49Z

@bertvv @dagwieers ping ;)

dagwieers · 2018-02-26T18:00:21Z

I cannot make modifications to this branch.

dagwieers · 2018-02-26T18:04:14Z

IMO We should have done this in Ghent when everyone was together and we were testing this. Now we lost momentum, it conflicts with other changes and things go stale. Short feedback loop wins.

tima · 2018-02-26T22:26:48Z

Noted. I agree with you @dagwieers and ideally we would have so this feedback loop would have been kept short. Unfortunately getting it done in Ghent in-person wasn't an option under the circumstances so we have to go from here.

Do you have a better way of reaching @bertvv than here? Would it be better to start this PR over, inserting the init script features in the current Vagrantfile rather than attempt to resolve the conflicts?

walidshaari · 2018-08-22T10:47:54Z

Hi, what is the status of this pull request.

webknjaz · 2018-08-22T12:37:49Z

cc @dfederlein

dfederlein · 2018-08-22T16:30:29Z

Lightbulb is currently not making any changes in regards to provisioning the labs/examples while we regroup and redesign. We will soon publish new materials on this and update the contributor guide accordingly.

Basically, we'll be removing the provisioning, including vagrant, by separating provisioning out into a role that lives on galaxy so everyone can point to their own provisioner and use the content. We'll then only focus on the content and have spec sheets for the labs so people can design their provisioning accordingly.

nqb · 2018-12-05T15:34:43Z

any updates ?

bertvv added 4 commits February 7, 2018 14:23

Automate bootstrapping the Vagrant environment

b44ebcb

Add README with setup instructions

0d94765

Install Lightbulb Git repo on the control node

88bd024

Removed trailing spaces

64ce3ed

bertvv added 5 commits February 7, 2018 15:36

Run the bootstrap script inline

82df4d5

Update installation instructions

43bf453

Merge branch 'vagrant-setup-script' of github.com:bertvv/lightbulb in…

1d9682a

…to vagrant-setup-script

Move Vagrantfile and instructions to project root

9a80cfa

Fix mdl issues

97a5d48

tima suggested changes Feb 7, 2018

View reviewed changes

tima requested a review from dfederlein February 7, 2018 18:36

tima added provisioner windows labels Feb 7, 2018

Don't install Ansible, left as an exercise

60af23e

tima suggested changes Feb 13, 2018

View reviewed changes

webknjaz reviewed Feb 14, 2018

View reviewed changes

This was referenced Feb 16, 2018

Implement a haproxy box in the Vagrantfile for exercising rolling update scenarios #279

Open

Implement an Ubuntu box in the Vagrantfile for exercising multi-OS scenarios #280

Open

Move Vagrantfile to tools/vagrant-linux folder. #283

Open

nqb mentioned this pull request Nov 24, 2018

[Provisioner] fix issues with Virtualbox provider in Vagrantfile #346

Open

gundalow changed the title ~~Vagrant setup script~~ [Provisioner] Vagrant setup script Dec 3, 2018

		@@ -6,6 +6,196 @@ $NODEMEM=256
		# Overwrite host locale in ssh session

		@@ -47,7 +237,9 @@ Vagrant.configure("2") do \|cluster\|
		"--memory", "2048",

[Provisioner] Vagrant setup script #268

Are you sure you want to change the base?

[Provisioner] Vagrant setup script #268

Conversation

bertvv commented Feb 7, 2018

dagwieers commented Feb 7, 2018 • edited Loading

dagwieers commented Feb 7, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bertvv commented Feb 8, 2018

dagwieers commented Feb 8, 2018

tima commented Feb 13, 2018

dagwieers commented Feb 13, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tima commented Feb 13, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

webknjaz Feb 14, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

webknjaz left a comment

Choose a reason for hiding this comment

bbaassssiiee commented Feb 14, 2018

tima commented Feb 14, 2018

webknjaz commented Feb 15, 2018

tima commented Feb 15, 2018

webknjaz commented Feb 17, 2018

tima commented Feb 19, 2018

webknjaz commented Feb 26, 2018

dagwieers commented Feb 26, 2018

dagwieers commented Feb 26, 2018

tima commented Feb 26, 2018

walidshaari commented Aug 22, 2018

webknjaz commented Aug 22, 2018

dfederlein commented Aug 22, 2018

nqb commented Dec 5, 2018

dagwieers commented Feb 7, 2018 •

edited

Loading

dagwieers commented Feb 7, 2018 •

edited

Loading

webknjaz Feb 14, 2018 •

edited

Loading