Cloud integration - 1st step :OpenNebula

[cduverne]

In order to have a cloud scaling module, let's handle the following points :

1/ VM Image : Requirements ? Base packages ?
2/ API cloud : Create / Read / Update / Delete
3/ Script démarrage -> HCM etc... -> Propose to use ONE contextualization
4/ Callback

[cduverne]

Create image for RoQ user :

• Embed RoQ
• Create init.sh for RoQ
• Add GCMIP to RoQ's script + template + Context
  Create template for RoQ user

[cduverne]

Well, well, I created 2 versions of RoQ image :

They are running Ubuntu 12.04 with base packages. (e.g : SSH)
The credentials to connect to them are : roq:roq (roq is a sudoer)

1/ OpenNebula RoQ image :
This image is especially contextualized to be deployed in an OpenNebula infra.
It's waiting the following parameters in the [CONTEXT] to be normaly running :
HOSTNAME -> Hostname of the VM
FILES -> Files to be sent to the VM which will be launched at boot with context variables, usually at least init.sh
IP_PUBLIC -> IP to be assigned to eth0
DNS -> DNS IP Address
MASK -> Network Mask to be assigned to eth0
GATEWAY -> Network Gateway to be assigned to eth0
TARGET -> Partition which will host the context : usually vdb or hdb
GCM -> IP address of the GCM

The RoQ processes will have to be launched in the init.sh script that will be sent to the VM, this allows us to modify the script without modifying the VM image which is tougher... and time consuming !

2/ A base RoQ image :
This image embed RoQ in /var/lib/RoQ/RoQ without any context, this will be reused for other cloud infrastructure.

I think that we have to store those images outside the git repository since they are 5Gb each and not mandatory for RoQ. But where ?

[cduverne]

1st tests show a provisioning time below 5 min/VM :)

[sskhiri]

what is the most time consuming operation ? below 5 mins is ok but I suppose it is greater than 4, isn't it ? That's still huge ?
Do you think it is the time to transport the image to the host ?

for the image, for sure, git is not appropriate, is there any image sharing hub ? otherwise we can still consider a file server on dev.roq.org if we get enough storage.

[sskhiri]

Roq Startup on VM

Actually to be accurate, we just need to update the HCM.properties in the ROQ_HOME/conf/ for setting the right GCM address.

[cduverne]

Regarding the requirements we implement in the image : Full Ubuntu 12.04, JVM etc... , it's hard to have an image of less than 4Gb.
Even on a DFS you have to clone the image before using it, and that's the most time consuming operation.
Maybe the scheduler etc... could be tweaked but I don't know in which way.

[sskhiri]

Can we try a 2Gb instance ?

[cduverne]

Well... If I cut my leg or my arm, it could fit... :) I'll figure if we could do something on this, but the base system is already a bit heavy, I'll have a look on basic basic basic ubuntu.

[cduverne]

Good news, last test show less than 4 minutes to deploy, with some tweaks, but all has been done on an infra side, not on the image.
Do we have to keep Ubuntu as a base then ? What about tiny debian distro etc... ?

[sskhiri]

well, why not, I do not see any issue, just requires to re-validate a roq installation and the GimmeRoq.sh, otherwise that is a good idea !

[cduverne]

I just shot a bullet in my feet (funny phrase in english indeed :p)
I've a Debian instance running somewhere on the planet, and tried the GimmeRoQ.sh, and get some "package not found etc..." here is an excerpt of the log :

E: Unable to locate package libpgm-dev
E: Unable to locate package nodejs
E: Unable to locate package npm
E: Unable to locate package libzmq-dbg
E: Unable to locate package libzmq-dev
E: Unable to locate package libzmq1

Other errors are consequence of this.
I'll investigate if those package are available in Debian Squeeze.