2.1 Managing cloud providers

NebulOuS supports two cloud providers: OpenStack and AWS. Below you will find the details on how to register accounts for these cloud providers in NebulOuS so it can deploy applications using them.

Before you start

Throughout an application's lifecycle, NebulOuS dynamically manages the application cluster by adding or removing computing nodes to comply with the application SLO. When cloud resources are required, NebulOuS autonomously creates the necessary VMs and decommissions them once they are no longer needed. Currently, NebulOuS supports two cloud providers: OpenStack and Amazon Web Services. Regardless of the specific cloud provider, some requirements must be satisfied:

• The cloud provider API must be reachable by NebulOuS core to provision/decomission nodes.
• The cloud provider must offer clean Images based on Ubuntu 22.04 LTS. These images must:
  • have a user ubuntu that can execute sudo commands without requiring password.
  • allow SSH connections using ssh-rsa algorithm.
  • At least, 20GB of disk so the operating system and necessary software packages for NebulOuS can be installed. Moreover, the disk size must also be aligned with the disk requirements of the application components you intend to deploy.
• The following ports are needed:
  • INBOUND 22 TCP (SSH)
  • INBOUND 51820 UDP (ONM)
  • OUTBOUND ALL TCP & UDP

OpenStack

Before you start

Acquire the OpenRC configuration file. This file is crucial as it contains the necessary settings and configurations

OVH: https://help.ovhcloud.com/csm/en-public-cloud-compute-set-openstack-environment-variables?id=kb_article_view&sysparm_article=KB0050920

OpenStack: https://docs.openstack.org/newton/user-guide/common/cli-set-environment-variables-using-openstack-rc.html

NebulOuS can only instantiate machines on OpenStack when they are based on an Ubuntu 22.04 image.

It is important to give at least, 20GB of disk so the operating system and necessary software packages for NebulOuS can be installed. Moreover, the disk size must also be aligned with the disk requirements of the application components you intend to deploy.

Make sure user ubuntu exists and it can execute sudo commands without requesting the password.

As a result to make sure that your images can be detected and used in the node candidate creation the following should be done:

1. Create an image or copy one from an already existing image based on Ubuntu 22.04. (for example: GOLD Ubuntu 22.04 LTS)

While you can create an image from a running instance, it is often recommended to stop the instance to ensure that the image captures a consistent state.

openstack server stop <instance-id>

To create image, execute:

openstack server image create --name <image-name> <instance-id>

2. Preferably, if you are creating the image from a running machine, we advice you to run:

sudo apt-get update
sudo unattended-upgrade -d

3. In the /etc/ssh/sshd_config add the following at the end of the file:

PubkeyAcceptedAlgorithms=+ssh-rsa,ssh-rsa-cert-v01@openssh.com
HostKeyAlgorithms=+ssh-rsa,ssh-rsa-cert-v01@openssh.com

Registering OpenStack account:

Step 1: On the NebulOuS UI, go to the "Resources" section

Step 2: Click the "ADD RESOURCE" button

Step 3: Fill in the resource registration form

You must fill the following fields:

• Name: A name for your cloud account. NebulOuS supports multiple cloud accounts from the same or different cloud providers. Choose a name that is descriptive enough.
• Platform: OpenStack
• In General section:
  • Auth URL: OS_AUTH_URL Mapped from openrc file.
  • Identity API Version: OS_IDENTITY_API_VERSION Mapped from openrc file.
  • Interface Type: OS_INTERFACE Mapped from openrc file.
  • add Security Group from your OpenStack account (Take into account that security group is used per region). To list all security groups one can execute command:

openstack security group list

To add security group one can execute command (https://docs.openstack.org/python-openstackclient/pike/cli/command-objects/security-group.html):

openstack security group create
    [--description <description>]
    [--project <project> [--project-domain <project-domain>]]
    <name>

This version of NebulOuS requires that the cloud provider has all ports open. Make sure that you have a security group with rules that allow inbound and outbound traffic on all ports. To create such security group, one can execute:

openstack security group create allow-all --description "Security group with rules to allow all inbound and outbound traffic"

Also, one can add such rules to security group: Allow all inbound TCP traffic:

openstack security group rule create --proto tcp --dst-port 1:65535 allow-all

Allow all inbound UDP traffic:

openstack security group rule create --proto udp --dst-port 1:65535 allow-all

Allow all inbound ICMP traffic (e.g., ping):

openstack security group rule create --proto icmp allow-all

• add Default Network from your OpenStack account. To list all networks, one can execute command:

openstack network list

To add network, one can execute command (https://docs.openstack.org/python-openstackclient/pike/cli/command-objects/network.html):

openstack network list
    [--external | --internal]
    [--long]
    [--name <name>]
    [--enable | --disable]
    [--project <project> [--project-domain <project-domain>]]
    [--share | --no-share]
    [--status <status>]
    [--provider-network-type <provider-network-type>]
    [--provider-physical-network <provider-physical-network>]
    [--provider-segment <provider-segment>]
    [--agent <agent-id>]
    [--tags <tag>[,<tag>,...]] [--any-tags <tag>[,<tag>,...]]
    [--not-tags <tag>[,<tag>,...]] [--not-any-tags <tag>[,<tag>,...]]

• In credentials section:

  • Username: OS_USERNAME Mapped from openrc file.

  • Password: OS_PASSWORD Mapped from openrc file.

  • Project Domain Name: OS_PROJECT_DOMAIN_NAME Mapped from openrc file

• In SSH Credentials section

  • Username (e.g.: ubuntu), Key Pair Name and Key Private Key, you can define them in the "Key pair" section of your AWS account

After filling in the form and saving it, you need to click the eye button

To confirm that you OpenStack installation is ready. *Using the credentials OS_USERNAME, OS_PASSWORD and OS_PROJECT_DOMAIN_NAME ** create a VM that uses the image you created:

openstack server create \
--image "<name_of_the_NebulOuS_ready_image" \
--flavor <flavor> \
--key-name <name_of_the_nebulous_ssh_key> \
--network <name_of_the_nebulous_network> \
--security-group <name_of_the_nebulous_sg> \
Test-NebulOuS

If an unauthorized error appears on the UI, please check that you have this step

If you encounter another issue. Please refer to https://github.com/eu-nebulous/nebulous/wiki/6.4-Common-errors#cloud-providers

AWS

Before you start

NebulOuS can only instantiate machines on AWS that fulfill the following conditions:

1. Are owned by the account that is used in the cloud definition.
2. Are tagged with: proactive-list-label:listed-in-proactive and Name:Ubuntu 22.04
3. Are based on an Ubuntu 22.04 Server LTS image

As a result to make sure that your images can be detected and used in the node candidate creation the following should be done: 1- Create an AMI based on Ubuntu 22.04 Server LTS. In this step, you will need to define the disk ammount associated to the AMI. It is important that you select at least, 20GB of disk so the operating system and necessary software packages for NebulOuS can be installed. Moreover, the disk size must also be aligned with the disk requirements of the application components you intend to deploy.

1.1- Run:

sudo apt-get update
sudo unattended-upgrade -d

1.2- Make sure user ubuntu exists and it can execute sudo commands without requesting the password.

1.3- In the /etc/ssh/sshd_config add the following at the end of the file:

PubkeyAcceptedAlgorithms=+ssh-rsa,ssh-rsa-cert-v01@openssh.com
HostKeyAlgorithms=+ssh-rsa,ssh-rsa-cert-v01@openssh.com

1.4- from the running instance save the image following this manual: 
           https://docs.aws.amazon.com/toolkit-for-visual-studio/latest/user-guide/tkv-create-ami-from-instance.html
2- Either at the creation step or after the image is created, add the tag to the AMI: 
Please note that if the image should be present in different regions, then it can be copied from one region to another following this manual: 
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/CopyingAMIs.html#ami-copy-steps Security groups and open ports This version of NebulOuS requires that the cloud provider has all ports open. Make sure that you have a security group with rules that allow inbound and outbound traffic on all ports.

Registering AWS account:

Step 1: On the mNebulOuS UI, go to the "Resources" section

Create a security group for your application:

VMs created by NebulOuS need to be able to be reachable by certain NebulOuS core services. To achieve it, you need to create a security group with specific rules.

To create a security group:

1. Click on the Security group section under the Network & Security section of the left menu.

2. Click on the Create security group button.

3. Give it a name and a description.

4. Configure the following Inbound rules:

5. Configure the following Outbound rules:

6. Keep in mind that these are rules necessary for NebulOuS to operate correctly the VMs created on AWS. If your application needs to expose any other ports, you need to configure them here too.

Create a dedicated user for NebulOuS

You will have to give NebulOuS access right to your AWS account in order to create and destroy VMs. It is best practice to create a dedicated user with limited acces to the account. To do so, you can create a dedicated user.

1. Go to Security credentials section

2. Go to Policies section from the left menu.

3. Click Create policy,

4. Give it a name. e.g: NebulOuSCustomPricingPolicy

5. Copy the following JSON in the JSON Policiy Editor:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "pricing:DescribeServices",
                "pricing:ListPriceLists",
                "pricing:GetAttributeValues",
                "pricing:GetPriceListFileUrl",
                "pricing:GetProducts"
            ],
            "Resource": "*"
        }
    ]
}

6. Save the policy

7. Go to Users section from the left menu.

8. Click Create user

9. Give it a name

10. In the Set permissions step, select Attach policies directly

11. Select the policy created previously and AmazonEC2FullAccess, AWSPriceListServiceFullAccess

12. Save.

13. Go to Security credentials section of the newly created user and click Create access key

14. Select the option Third party service

15. Give the credentials a name.

16. Save Access key and Secret access key to use them later when registering the cloud provider in NebulOuS

Step 2: Click the "ADD RESOURCE" button

Step 3: Fill in the resource registration form

You must fill the following fields:

• Name: A name for your cloud account. NebulOuS supports multiple cloud accounts from the same or different cloud providers. Choose a name that is descriptive enough.
• Platform: AWS
• Regions: Select the region for your AWS account. Only one region is permited. Make sure you have an AMI defined in the region you are configuring.
• In General section:
  • add Security Group from your aws account (in EC2 Dashboard you can find default security groups or you can create your own). Make sure that the security group is in the same region you are configuring.
• In credentials section
  • Provide a value for Username (Access key ID obtained from AWS portal) and Secret (Secret access key obtained form AWS portal)
• In SSH Credentials section
  • Username (must be ubuntu), Key Pair Name and Key Private Key, you can define them in the "Key pair" section of your AWS account.

After filling in the form and saving it, you need to click the eye button

If an unauthorized error appears on the UI, please check that you have this step

If you encounter another issue. Please refer to https://github.com/eu-nebulous/nebulous/wiki/6.4-Common-errors#cloud-providers