Vagrant-hashistack

Hashistack in one click for development & testing

This vagrant box aims to make it dead simple to start a hashistack and emulate how services will be deployed to production.

---

🚧 - current vagrant box runs consul, nomad and vault in dev (development) mode.

• consul development mode
• nomad development mode
• vault development mode

---

Content

1. Description - what & why
   1. Submodules
   2. Services
   3. Why does this exist?
   4. Installed stack
   5. Versions
2. Install prerequisites
   1. General requirements
      1. Proxy
   2. Linux requirements
   3. MacOS requirements
   4. Windows requirements
      1. GNU Make and Git Bash
      2. Installation
3. Build
4. Configuration
   1. Default Configuration
   2. Override default configuration
      1. ENV variables
      2. Config files
5. Usage
   1. Starting a plain default box
      1. Port collisions
         1. Shut down the running machine
         2. Use the auto_correct feature to dynamically allocate ports
   2. Starting a new project based on the template
6. Test
   1. Local run
   2. CI pipeline run
      1. CI test configuration
7. Diagram
8. Contribute

Description - what & why

This repository will build a base-box for different projects to extend on. The base-box contains components, and a setup that makes it ideal for working with the hashistack.

Hashistack, in current repository context, is a set of software products by HashiCorp.

Submodules

Current repository contains submodules

• vagrant-hashistack-template

Check Using submodules in Git - Tutorial

Current repository points to concrete commit in submodule(s)

To point to the latest commit in submodule master

make update-submodule

Services

The default box will start Nomad, Vault, Consul and MinIO bound to loopback and advertising on the IP 10.0.3.10, which should be available on your local machine. Port-forwarding for nomad on port 4646 should bind to 127.0.0.1 and should allow you to use the nomad binary to post jobs directly. Consul and Vault have also been port-forwarded and are available on 127.0.0.1 on ports 8500 and 8200 respectively. Minio is started on port 9000 and shares the /vagrant (your repo) from within the vagrant box.

Service	URL	Token(s)
Nomad	http://10.0.3.10:4646
Consul	http://10.0.3.10:8500	master
Vault	http://10.0.3.10:8200	master
Minio	http://10.0.3.10:9000	minioadmin : minioadmin

Why does this exist?

We needed a Vagrant box with the complete hashistack to use for demo, development and testing. In order to build cloud native, security minded and dependable services, there exists a killer combination;

• Containers - (Docker)
• Simple&Powerful Orchestrator - (Nomad)
• Service-mesh mTLS - (Consul connect)
• Secrets management - (Vault)

Installed stack

• Consul
• Nomad
• Vault
• Terraform
• Docker CE

with a side-play of

• Ansible
• Packer
• Consul-template
• Minio

Versions

vagrant-hashistack	terraform	consul	nomad	vault	packer	consul-template	remarks
0.4.3	0.13.2	1.8.4	0.12.4	1.5.3	1.6.1	0.25.1
0.4.2	0.13.2	1.8.4	0.12.4	1.5.3	1.6.1	0.25.1
0.4.1	0.13.2	1.8.4	0.12.3	1.5.3	1.6.1	0.25.1
0.4.0	0.13.1	1.8.3	0.12.3	1.5.2	1.6.0	0.25.1	vault 1.5.2.1+ent

Install prerequisites

make install

The command, will install:

• VirtualBox
• Packer
• Vagrant with additional plugins
• Additional software dependent on the OS (Linux, MacOS)

General requirements

You will need to have the following binaries pre-installed:

• Make
• Git CLI

---

NB Post installation you might need to reboot your system in order to start the virtual-provider (VirtualBox)

The rest of the Requirements are operative system dependent

---

Proxy

If you for any reason find yourself behind a transparent proxy you need to set the environment variables SSL_CERT_FILE and CURL_CA_BUNDLE. You have three options:

• Prefix vagrant up; SSL_CERT_FILE=<path/to/ca-certificates-file> CURL_CA_BUNDLE=<path/to/ca-certificates-file> vagrant up
• Set the environment variables in your current session by running export SSL_CERT_FILE=<path/to/ca-certificates-file> and export CURL_CA_BUNDLE=<path/to/ca-certificates-file> in the terminal. Eg:export SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt CURL_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt
• Set the environment variables permanently by adding the above export commands to your ~/.bashrc or equivalent.

Linux requirements

• Virtualization must be enabled. Error if it is not.
• Packages gpg and apt must be installed.

MacOS requirements

• Virtualization must be enabled. This is enabled by default on MacOS.
• Homebrew must be installed.

Windows requirements

Most developers are likely to be using Linux or Mac as their development box, however the install procedure should also work on Windows.

GNU Make and Git Bash

• Git. Use the download link and follow the steps.

⚠️ Make sure to install Git Bash when installing Git

• GNU Make. Run the commands below in your Git Bash, or any other terminal.

curl -O https://managedway.dl.sourceforge.net/project/ezwinports/make-4.3-without-guile-w32-bin.zip
unzip -d "/c/Program Files/Git/mingw64" make-4.3-without-guile-w32-bin.zip

Next, make sure to start git bash as admin shell (start > Git Bash > rightclick > Run as administrator). Now you are all set to run

make install

Build

make build

Command above will build a vagrant box based on fredrikhgrelland/bionic64-ansible-docker. The packaged box will be locally available at ´packer/output-hashistack/package.box´

Note: You can refer to the configuration section in order to get a comprehensive overview of the default configurations with which the system is set up. The section also provides information about steps for overriding the default system configuration.

Configuration

In most cases, users need to customize vagrant-box's services configuration.

Scenarios for customization:

1. consul(enterprise) with consul_acl = enabled and default acl policy = deny
2. nomad(oss) with nomad_acl = false
3. vault(enterprise), unsealed and integrated with nomad and consul, the way that it manages their secrets/tokens

In order to simplify making such changes in the configuration, we provide switches. These are sort of switches which are controlled by env variables and provide the user with the opportunity to quickly switch between the configuration setup.

Supported switches are listed under # Control box features section in the following file

---

Default Configuration

• .env_default.j2
• hashistack component versions
• nomad policies
• consul policies

Consul:

• Open source version
• ACL enabled=true
• default_policy=allow

Nomad:

• Open source version
• ACL enabled=false
• Integrated with Consul, using token

Vault

• Open source version
• Unsealed
• PKI enabled at /pki

Override default configuration

Option 1 - env variables

Use env to switch prebuild configuration on/off

When the vagrant box is provisioned, it reads the data from the following environment file /home/vagrant/.env_default in order to set up the system. If you wish to override any of the default values then you can do so by adding that variable name and value in .env file. The property values in the .env file override the property values present in the .env_default file and thus makes it simple to provision systems that suffice the relevant development needs.

For example, in order to override the consul acl default policy from allow to deny, the following needs to be added to the .env file:

consul_acl_default_policy=deny

Option 2 - config files

Overriding config files*

It is possible to add and/or override the hashistack components' configuration files. See documentation here.

NB! Overriding config files will take effect at last. In other words, config files(Option 2) will override any configuration which were setup by the env variables(Option 1)

Usage

Vagrant-hashistack provides these features:

• Deploy & test terraform modules
• Deploy & test nomad jobs
• Upload files to Minio
• Test automation

Option-1 Starting a plain default box

To get a running VM using the latest release of this box run

vagrant init fredrikhgrelland/hashistack
ANSIBLE_ARGS='--extra-vars "local_test=true"' vagrant up --provision

The first command will add a file called Vagrantfile to your directory, and vagrant up will start a box based on the specifications of that file.

NB If you are behind a transparent proxy, follow proxy documentation

Port collisions

If you get the error message

Vagrant cannot forward the specified ports on this VM, since they
would collide with some other application that is already listening
on these ports. The forwarded port to 8500 is already in use
on the host machine.

you do most likely have another version of the vagrant-box already running and using the ports. You can solve this in one of two ways:

Option 1 Shut down the running machine

Run

vagrant status

to see all running boxes. Then run

vagrant destroy <box-name>

to take it down. Doc on what vagrant destroy does.

Option 2 Use the auto_correct feature to dynamically allocate ports

Vagrant has a configuration option called auto_correct which will use another port if the port specified is already taken. To enable it you can add the lines below to the bottom of your Vagrantfile.

Vagrant.configure("2") do |config|
    # Hashicorp consul ui
    config.vm.network "forwarded_port", guest: 8500, host: 8500, host_ip: "127.0.0.1", auto_correct: true
    # Hashicorp nomad ui
    config.vm.network "forwarded_port", guest: 4646, host: 4646, host_ip: "127.0.0.1", auto_correct: true
    # Hashicorp vault ui
    config.vm.network "forwarded_port", guest: 8200, host: 8200, host_ip: "127.0.0.1", auto_correct: true
end

This will enable the autocorrect-feature on the ports used by consul, nomad, and vault.

💡 You can find out more about Vagrantfiles here

Option-2 Starting a new project based on the template

To see a full example of how to start a new project based on this box go to template-repo.

NB If you are behind a transparent proxy, follow proxy documentation

Test

There are two options how to run tests:

• run all tests on local machine
• run all tests in CI (env variable CI)

Options are controlled by environment variable CI.

Local run

CI env variable is not set.

make test

The above command runs the tests by starting the countdash consul-connect example. If ´packer/output-hashistack/package.box´ does not exist, it will run ´make build´.

Pay attention that we pass extra-vars --tags=local_test=true to the ansible provisioner. Full example

CI pipeline run

CI env variable set to any non-null value.

make test

The tests are run using Github Actions feature which makes it possible to automate, customize, and execute the software development workflows right in the repository.

We utilize the matrix testing strategy to cover all the possible and logical combinations of the different properties and values that the components support. The .env_override file is used by the tests to override the values that are available in the .env_default file, as well as the user configurable .env file.

CI test configuration

As of today, the following tests are executed whenever a Pull request is created :

Test name	Consul Acl	Consul Acl Policy	Nomad Acl	Hashicorp binary
test (consul_acl_enabled, consul_acl_deny, nomad_acl_enabled, hashicorp_oss)	true	deny	true	Open source
test (consul_acl_enabled, consul_acl_deny, nomad_acl_enabled, hashicorp_enterprise)	true	deny	true	enterprise
test (consul_acl_enabled, consul_acl_deny, nomad_acl_disabled, hashicorp_oss)	true	deny	false	Open source
test (consul_acl_enabled, consul_acl_deny, nomad_acl_disabled, hashicorp_enterprise)	true	deny	false	enterprise
test (consul_acl_disabled, consul_acl_deny, nomad_acl_enabled, hashicorp_oss)	false	deny	true	Open source
test (consul_acl_disabled, consul_acl_deny, nomad_acl_enabled, hashicorp_enterprise)	false	deny	true	enterprise
test (consul_acl_disabled, consul_acl_deny, nomad_acl_disabled, hashicorp_oss)	false	deny	false	Open source
test (consul_acl_disabled, consul_acl_deny, nomad_acl_disabled, hashicorp_enterprise)	false	deny	false	enterprise

The latest test results can be looked up under the Actions tab Actions

Diagram

Contribute

See here