Aquilon doc: add documentation on configuring cluster

.travis-scripts/dictionary.txt

@@ -49,6 +49,7 @@ GPUs
 GRN
 GRNs
 hacky
+Hadoop
 HPC
 HTC
 HEPiX
@@ -144,6 +145,7 @@ vCenter
 versioned
 VirtualEnv
 virtualenv
+virtualisation
 virtualised
 VLAN
 VM

_aquilon/management.md

@@ -365,6 +365,103 @@ remove the service requirement matching the host with the command
 The mapping is actually removed at the next `aq reconfigure` for the host after modifying
 the service requirements.
 
+## Using Aquilon clusters
+
+[Clusters][aquilon_details] are intended to represent in Aquilon a group of hosts that must be configured in
+the same way. It may be hypervisors in a cloud, disk servers in a distributed storage infrastructure like Hadoop
+or Ceph, or hosts behind proxy. 
+
+Clusters, like hosts, have an archetype and personality attached. It allows for cluster-wide configuration
+to be applied after most of the host configuration is done, just before including the host archetype `final.pan`
+(the last thing done in any plenary object template). Thus the cluster personality is added to the host
+personality. Depending on the cluster type, the host personality of a clustered host may be as minimal
+as providing the basic OS configuration (typically the case for hypervisors in a cloud)or also 
+configure some features specific to the host (e.g. the hosts
+behind a proxy may not all run the same services).
+
+Configuring a cluster involves:
+* Creating the cluster archetype and personality
+* Creating the cluster
+* Adding hosts (hypervisors) to the cluster
+* Building the cluster and its hosts
+* Optionally, using the cluster as the "machine location" for virtual machines
+
+### Creating the cluster archetype and personality
+
+A cluster archetype is created the same way as as [host archetype](#adding-archetypes), 
+with the exception that `--cluster_type compute` option must be added. A typical example would be:
+
+```bash
+aq add_archetype --archetype cluster_test --compilable --cluster_type compute
+```
+
+Cluster archetype creation involves the creation of the same templates as for a 
+[host archetype](/aquilon/configuration.html#creating-the-archetypebase-and-archetypefinal-templates), as well
+as a [archetype/declaration.pan](/aquilon/configuration.html#enabling-the-template-library)
+if it is enabled for host archetypes.
+
+Personality is created and managed the same way as for a [host personality](#personalities)). The same personality
+(and archetype) can be used with several clusters.
+
+Like for host archetypes, it is possible to [bind](#binding-features-to-personalities) a feature to a cluster
+archetype.
+
+
+### Creating the cluster
+
+A cluster is created with `aq add_cluster`. For example, to create a cluster `cluster_test` that will
+use the personality `os_hv_test` from archetype `mycloud`, use the following command:
+
+```bash
+aq add cluster --cluster cluster_test --archetype mycloud --personality os_hv_test 
+               --down_hosts 0 --building hq --sandbox your/sandbox
+```
+
+In the previous command, `--down_hosts` is set to 0 as Aquilon doesn't make a direct use of this value
+(an external monitoring tool would be required). For `--building` and `--sandbox`, use values appropriate
+to your site (sandbox name must be in the form `user/sandbox_name`). 
+
+### Adding hosts to the cluster
+
+After creating the cluster, it is necessary to add or more hosts as the cluster members (e.g. hypervisors in a 
+cluster representing a cloud). The host must have been created with the [standard procedure](/aquilon/configuration.html#declaring-hosts)
+before adding it to the cluster. This is done with the command `aq cluster`.
+
+For example, to add a node `hv1.example.com` in the cluster `cluster_test`:
+
+```bash
+aq cluster --cluster cluster_test --hostname hv1.example.com
+```
+
+Note that if the host was previously part of another cluster, it will be removed from the original cluster.
+
+
+### Building the cluster and its hosts
+
+In addition to the configuration added to each cluster host, there is a specific profile built for cluster
+itself. The command `aq make cluster` allows to rebuild both the cluster profile and the host profile for each
+member.
+
+For example, to rebuild the cluster `cluster_test` and its hosts, use:
+
+```bash
+aq make cluster --cluster cluster_test
+```
+
+This command must be used after every change affecting the cluster configuration (in particular the cluster
+personality) to ensure that all the configuration (profile) of each cluster member is updated.
+
+### Binding virtual machines to the cluster
+
+If the cluster is describing a virtualisation infrastructure like a cluster, hosts that are not part of the cluster
+may be associated with it through the (virtual) machine they use.  When creating the machine, use the
+option `--cluster` instead of `--rack`, `--desk` or `--chassis`. One the machine is configured,
+[add the hosts][aquilon_hosts] to Aquilon using the same procedure as for hosts using bare metal machines. 
+
+It is not possible to update a machine that was initially configured to use a bare metal machine to become
+a virtual machine. But it is possible to change the cluster hosting a machine (or change a virtual machine hosted
+by a host to a VM hosted by a cluster) with `aq update_machine` command with the `--cluster` option. 
+
 ## Initial Installation
 
 Initial installation of a node in Quattor is managed by the AII component that generally runs on

_aquilon/technical_details.md

@@ -119,11 +119,18 @@ the AII service used for the initial installation of an Aquilon host.
 ## Clusters
 
 Clusters are collections of hosts that have a similar role/function. They are typically used describe 
-a cluster of hypervisors, like those provided by VMware ESX or clouds. Aquilon can provide some monitoring
+a cluster of hypervisors, like those provided by VMware ESX or clouds, a disk servers in a distributed
+storage technology like Hadoop or Ceph, hosts behind a proxy... 
+
+Aquilon can provide some monitoring
 thresholds associated with the cluster, like the minimum or maximum number of hosts that must be running at
 any time. Clusters can also be used to describe a HA cluster. Note that Aquilon is not a replacement for the
 cluster middleware: it just allows to represent a group of machines managed by such a middleware.
 
+Clusters, like hosts, have an archetype and personality attached. It allows to ensure that all hosts that are
+part of the cluster receive the same configuration. The cluster personality is added to the host, in addition
+to the host personality.
+
 Once a cluster is defined, it can be used as an alternative to a machine object to describe where is running
 a host. In this case, Aquilon doesn't track on which cluster node the host is running: it lets the middleware
 do the scheduling, assuming that all hosts in the cluster are equivalent.

_includes/link_definitions.md

@@ -7,6 +7,7 @@ This file contains link definitions that can be used in reference links.
 [aquilon_configuration]: /aquilon/configuration.html
 [aquilon_details]: /aquilon/technical_details.html
 [aquilon_domains]: /aquilon/technical_details.html#domains
+[aquilon_hosts]: /aquilon/configuration.html#declaring-hosts
 [aquilon_install]: /aquilon/00-install.html
 [aquilon_management]: /aquilon/management.html
 [aquilon_plenary]: /aquilon/technical_details.html#plenary-templates