Skip to content

Start the client agents

Jump to bottom
Chris edited this page Jun 24, 2021 · 17 revisions

Why client agents are important

Having client agents for both Nomad and Consul are essential. The purpose of the client agents is to automatically be able to communicate with the Nomad/Consul servers so that any request to the client agent can be forwarded to the server.

As long as the client agents know the address of one running server on start up, they will automatically know the addresses of the server cluster. This is essential because now applications on different machines that want to talk to the server can just send a request to the local agent instead of having a hard coded server address and run the risk of not being able to contact the cluster if that specific server goes down. We will use this feature when we set up our Consul and Nomad client agents.

Start the Consul client agent

On the second machine download the consul and nomad binaries again. I'm first going to start up the local consul agent and activate some important features:
sudo consul agent -data-dir="/tmp/consul" -bind=<ip> -client=<ip> >> /var/log/consul/output.log &

  • consul agent starts the agent in client mode. Remember that adding -server starts the agent in server mode
  • -data-dir is here for the same reason we need it for the consul server agent
  • -bind=<ip> also will attach the agent to the ip specified. This is required if there are multiple IPs found. Use the IP of the host machine this agent is on.
  • -client=<ip> is the key in making our Manticore system work. It functions the same way as -bind, only now ALL interactions with the agent must now be made on the IP address specified instead of local host (127.0.0.1). Use the same IP as in the -bind flag.
  • /var/log/consul/output.log is a recommended location to print the log stream. Make sure the directory exists first
  • & Run the agent in the background

The -client flag

If you remember, our Consul server didn't set the -client flag and because of that we could run consul without any additional parameters. Now, in order to use the Consul API and the consul command you must append -http-addr=<ip>:8500 to all commands and when using the Consul API you use the address you put in the -client flag. For example, to access the UI when activated, you point your browser to http://<ip in -client flag>:8500. The reason why we do this is because when we schedule docker containers we want the applications to be able to use the Consul API by talking to the local agent. But, the local host inside the docker container is different from the local host in the virtual machine, which is Consul's default address for using the API. Therefore, we specify an address that the container application can reach so that it can use the API.

Join the Consul client agent to the server

If you noticed when you started the agent, one of the logs tells you that there are no known Consul servers. Our agent needs a Consul server address so that it can connect to the entire cluster. To do this, run:
consul join -http-addr=<ip of this machine>:8500 <ip of the consul server agent>
If we didn't specify the -client flag then we could leave out the -http-addr flag.
If you see this message then you are good to go:
Successfully joined cluster by contacting 1 nodes.

Sidenote: Apparently using an address of a consul client agent that is connected to the servers will also allow your agent to find the servers.

Start the Nomad client agent

Similar to how we started the Nomad server agent, we will also use an HCL file to pass in configurations.

# The address the nomad agent will bind to
bind_addr="127.0.0.1" # Replace this with your machine's IP
log_level="DEBUG"

data_dir="/tmp/nomad"

consul {
    # Consul's Client agent Address. The Nomad agent connects to Consul through the running consul client agent
    address = "127.0.0.1:8500" # Keep port 8500, but replace with your machine's IP
}

# Client configuration
client {
    enabled = true
    # A list of Nomad servers to connect to. You only need one running server for this to work
    # Keep port 4647, but replace with the IP of the Nomad server
    servers = ["127.0.0.1:4647"] 
    # Manticore checks the meta data of a client agent if it can place manticore/core/hmi there
    # Setting this agent's meta.manticore to true means Nomad can allocate manticore web apps on this machine
    # Setting this agent's meta.job to true means Nomad can allocate sdl_core and HMI instances on this machine
    meta {
        "manticore" = true
        "job" = true
    }
    # Use this option at your own discretion. Setting docker.cleanup.image to false means Nomad won't remove
    # images that tasks have used when they are stopped. This is good for when your images won't change and 
    # you don't need to pull changes from the docker repo every time.
    options {
        "docker.cleanup.image" = "false"
    }
}

Now run the agent, passing in the file above (assuming it's called client.hcl). Make sure the output directory exists first sudo nomad agent -config client.hcl >> /var/log/nomad/output.log &

Try running a nomad command. Don't forget to pass in the IP address of where Nomad is listening
nomad status --address=http://<ip of nomad client agent>:4646

Jobs

Nomad schedules jobs. It does this by being fed job files. Job files are composed of many groups, where each group can have many tasks, and each task has a service. Tasks in the same group will be scheduled on the same machine. You can also duplicate the same group in order to have many of the same kind of task. A task is where we specify which Docker image we want to run.

These job files can get really large. That's why this project relies on the help of the npm module nomad-helper, which has been created for the purpose of this project.

Since Manticore schedules jobs to start Core and HMI for us, there is not much more you really need to learn about Nomad to get started with using this project. You should at least be familiar with some basic commands, such as stopping jobs, and checking the allocation status of jobs.

Getting the status of a specific job. Returns the allocation ID for that job, too

nomad status --address=http://<ip of nomad client agent>:4646 <job name>

Printing the stdout logs of that job

nomad alloc logs --address=http://<ip of nomad client agent>:4646 <allocation id>

Printing the stderr logs of that job

nomad alloc logs --stderr --address=http://<ip of nomad client agent>:4646 <allocation id>

Stopping a job

nomad stop --address=http://<ip of nomad client agent>:4646 <job name>

Reading all the values in Consul's KV store under the manticore key

consul kv get --http-addr=http://<ip of consul client agent>:8500 -recurse manticore

It's now time to learn how to set up everything else

Clone this wiki locally