Working With Nodes

A cluster is made up of nodes. Hence, node selection for creating a cluster is a common action across all the clusters & is an essential part of any cluster creation. In this section of the Getting Started Guide, a basic part which is common across technologies particularly for node authentication and retrieval is covered. Specific fields related to a particular Technology which are displayed in the node list section post node retrieval are discussed in the individual technology-specific guides.

In order to create a cluster, user needs to provide following information related to nodes. It has to be done in the following predefined sequence:

1. Provide Node Authentication information
2. Retrieve Nodes
3. Inspect Nodes: This activity is optional. Refer Nodes Inspection section for more information. This functionality can be used to confirm whether the below mentioned prerequisites like Firewall, Require TTY, Sudo user, Loopback address, Wget existence etc. are fulfilled or not.

Hardware Recommendations

Processor - Dual Core RAM - 2GB (Minimum)

Supported Environments

The list of environments that are supported for nodes of a cluster are as follows:

Supported Versions OS - Ubuntu Server 64 bit (12.04), CentOS Server 64 bit (6.2, 6.3, 6.4) Java (64 bit) - Java 6 (OpenJava, Oracle Java), Java 7 (OpenJava, Oracle Java)

Note: Although, Ankush supported Java 6 and Java 7 both for the cluster nodes, but the user may need to consider the Java version prerequisites pertaining to individual technology. For e.g. Cassandra 2.x versions necessarily requires Java 7 version on cluster nodes. It does not work with Java 6.

Software Prerequisites

In order to create big data clusters using Ankush, the user must ensure that the following prerequisites are fulfilled on all the nodes.

• All nodes should have a user account that has common credentials for login across all the nodes • /etc/hosts file should contain the proper IP Address & host-name mapping for all the nodes that are part of the cluster. The node loopback IP (e.g. 127.0.0.1) should be disabled on all the nodes • Following software should be installed on all the nodes used for cluster creation: o wget o zip o unzip o tar o ssh server and client o apt-get (for Ubuntu) o yum (for CentOS) o Java (Optional)

• Enable NTP on cluster and Ankush server host The clock must be synchronized between all the nodes in a cluster and machine that runs Ankush server. NTP refers to Network Time Protocol that helps in synchronizing time between all the machines. It is mandatory for Oracle, HBase

• Disable iptables/firewalls Iptables must be disabled on each host of a cluster to make them reachable through Ankush. This can be done by running following command on each host:

CentOS service iptables save service iptables stop chkconfig iptables off Ubuntu ufw disable

• Ensure that the /sbin path is included in path variable

• Configure /etc/hosts /etc/hosts file must be configured for Ankush to function smoothly. Ensure that the user performs the following configuration on all the hosts as well as on Ankush server machine:

1. All loop back entries must be removed/commented

Default view: 127.0.x.x ::1

Commented view: #127.0.x.x #::1

2. Must have proper binding of host with IP

IP hostname localhost e.g. 192.168.XXX.XXX FQDN localhost

• Check RequiresTTY Default “Requirestty” must be disabled for Ankush to function. To disable tty, comment or delete the following line in /etc/sudoers on each host in the cluster:

default requirestty

• Configure /etc/sudoers Make sure that “User” with which cluster deployment is to be done is a pseudo-user or user having administrator privileges. This can be done by having following entry in /etc/sudoers file on each host:

ALL=(ALL) ALL

• Configure NOPASSWD in /etc/sudoers If you are using shared key authentication in case of in premise machines then make sure to configure NOPASSWD setting for the user.

Note: A private key file (e.g. id_rsa) whose corresponding public key file (e.g. id_rsa.pub) is copied to authorized_keys to all the nodes used in the cluster to allow passowrdless ssh.

username ALL=(ALL) NOPASSWD: ALL

Use the following code to configure your system in such a way that you need not to enter your password while executing admin/sudo operations.

Tip: Make your work easy by configuring your system in such a way that need not to enter password every time to engage in admin/sudo activities. Use “username ALL=(ALL) NOPASSWD:ALL” code just below the "%sudo ALL=(ALL:ALL) ALL" line. In case, this code is not placed appropriately as recommended, then "%sudo ALL=(ALL:ALL) ALL" entry will override your configuration.

• Install psmisc rpm Download rpm from the given link: ftp://ftp.muug.mb.ca/mirror/centos/6.5/updates/x86_64/Packages/psmisc-22.6-19.el6_5.x86_64.rpm

Command for downloading: wget ftp://ftp.muug.mb.ca/mirror/centos/6.5/updates/x86_64/Packages/psmisc-22.6-19.el6_5.x86_64.rpm

Command for installing: rpm –Uvh psmisc-22.6-19.el6_5.x86_64.rpm

• Configure repositories (CentOS only)

Ankush installs few software from EPEL repositories which are not a part of default CentOS installation. Therefore, for CentOS systems make sure to have EPEL latest version installed on your hosts. This can be checked by running the following commands:

1. To install epel & atomic repository:

wget http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm

sudo rpm -Uvh epel-release-6*.rpm

2. Confirm repo list for epel

yum repolist

Output of above command should have EPEL and atomic mentioned in the repo list.

repo id repo name status base CentOS-6 - Base 6,381 epel Extra Packages for Enterprise Linux 6 - x86_64 8,626 atomic CentOS / Red Hat Enterprise Linux 6 - atomicrockettur 946 extras CentOS-6 - Extras 12 rpmforge RHEL 6 - RPMforge.net - dag 4,594 updates CentOS-6 - Updates 562 virtualbox Oracle Linux / RHEL / CentOS-6 / x86_64 - VirtualBox 19 repolist: 20,194

Node Authentication

“Node Authentication” section on cluster creation page captures input related to user name & authentication. This is the information related to common user login account that must exist on each node which user wants to apply to the cluster. The three kind of inputs captured are as follows:

![] (https://raw.githubusercontent.com/wiki/impetus-opensource/ankush/images/Image36.png) Figure 36: Node Authentication

• Username: Enter the user name that will be used for login (This must be same across all the nodes as per also mentioned in prerequisites.)

• Authentication Type: A user account can be authenticated either using password or using the second option of shared key. Depending on the type of authentication selected, the next input changes either to “Password” or to “File Upload” to upload the shared key

• Password / File Upload:

o Password: If in above input the selected authentication type is password then the user needs to enter password. (See image given above for password)

• Node Rights: The rights on a node can be sudo or non-sudo. Sudo rights would mean that you would be able to use Ankush seamlessly without any hassles. Contrastingly, if you are a non-sudo user, you may be restricted of few operations like installation of Ganglia component etc.

![] (https://raw.githubusercontent.com/wiki/impetus-opensource/ankush/images/Image37.png) Figure 37: Authentication: Password

o File Upload: If the authentication type selected above is “Shared Key”, then clicking on this input opens up a file upload dialog where user needs to up-load the shared key. (See image given below for shared key)

![] (https://raw.githubusercontent.com/wiki/impetus-opensource/ankush/images/Image38.png) Figure 38: Authentication: Shared Key

Node Retrieval

There are two main sections of inputs for node selection namely:

• Search and Select Nodes: This captures information for node selection criteria

• Node List: This shows the retrieved / detected nodes. The output may vary as per the technology. However, few items or columns remains same regardless of the technology.

Search and Select Nodes

This section takes the following inputs from you:

![] (https://raw.githubusercontent.com/wiki/impetus-opensource/ankush/images/Image39.png) Figure 39: Search & Select Nodes

• Mode: Captures the method of specifying node information which could be Host names or File Upload

o Host Names: Provide comma separated host names in the Host Name List field. For example: m171-74,m171-82,m145-113,m145-115

![] (https://raw.githubusercontent.com/wiki/impetus-opensource/ankush/images/Image40.png) Figure 40: Search Nodes: Host Names List

o File Upload: In this case the next input changes to File Upload. The user needs to upload a file that contains information related to nodes. Each line of the uploaded file can consist of valid IP address or node detection patterns.

![] (https://raw.githubusercontent.com/wiki/impetus-opensource/ankush/images/Image41.png) Figure 41: Search Nodes: File Upload

After clicking “Retrieve” button, its caption changes to “Retrieving” & then node list is populated with nodes matching the given node selection criteria.

Common part from this node list as follows:

In this grid first, second & last column remains same across all technologies. The node filters & search box also remains same.

In this retrieved node list, the nodes which are not available for selection are highlighted with different background color & the checkbox for node selection in those rows are disabled.

![] (https://raw.githubusercontent.com/wiki/impetus-opensource/ankush/images/Image42.png) Figure 42: Node List

Node Inspection

This feature enables the user to inspect nodes prior to cluster deployment. This increases the possibility of cluster being deployed in a good health without any error, alert or warning. It checks for a lot parameters and probable places where an error might occur.

![] (https://raw.githubusercontent.com/wiki/impetus-opensource/ankush/images/Image47.png) Figure 43: Inspect Nodes

The various parameters that it checks for are as follows:

1. Connection Status
2. AnkushAgent service running
3. AnkushAgent directory existence
4. Firewall
5. Port availability
6. Require TTY
7. AnkushAgent Install Dir Disk Space
8. Sudo user
9. Loopback address
10. Wget existence
11. Tar existence
12. Jps existence
13. JPS process list
14. Overall Status

![] (https://raw.githubusercontent.com/wiki/impetus-opensource/ankush/images/Image48.png) Figure 44: Node Inspection