Skip to content

Home

Jump to bottom Edit New page
tjmcs edited this page Jun 13, 2012 · 36 revisions

Project Overview

The Razor Microkernel is a small, in-memory Linux kernel that is used by the Razor Server for dynamic, real-time discovery and inventory of the nodes that the Razor Server is managing. The Razor Server accomplishes these tasks by using the Razor Microkernel as the default boot image for any nodes in the network (i.e. any new nodes or nodes which the Razor Server doesn't have a matching policy).

There are a number of small, in-memory Linux kernels out there (Damn Small Linux, SliTaz, Porteus, and Puppy LinuX all come to mind), so we applied a number of constraints in order to narrow down our choice of which distribution we should use as the basis for the development of our Microkernel:

  • The image (ISO) for the distribution should be smaller than 256MB in size (to speed up delivery of the image to the node)
  • Only distributions that were being actively developed were considered
  • The distribution should be based on a relatively recent Linux kernel (v3.0.0 or later) so that we could be fairly confident that it would support the newer hardware we knew that we knew we would find in many modern datacenters
  • Since we knew we would be using Facter as part of the node discovery process, the distribution needed to include a pre-built version of Ruby
  • In order to support the development of custom extensions for our base Microkernel down the line the distribution needed to have an easy mechanism for building "custom extensions"
  • In order to support commercial versions of these extensions, the distrubution needed to be licensed under a "commercial friendly" open-source license.

Once we applied all of these constraints, the Tiny Core Linux distribution really stood out above the rest. Tiny Core Linux (or TCL) easily met all of the constraints that we had applied (and even met a few constraints that we hadn't thought of initially):

  1. TCL is very small (the "Core" distribution is an ISO that is only 8MB in size) and is designed to run completely in memory (the default configuration assumes no local storage exists and only takes up about 20MB of system memory when fully booted)
  2. TCL is built using a (very) recent kernel; as of the time of this writing (the latest release of TCL, v4.5.4, uses a v3.0.21 Linux kernel and, at the time that this was written, that release was posted less than a week ago), so we knew that it would provide support for most of the hardware that we were likely to see.
  3. TCL can easily be extended (either during the boot process or dynamically, while the kernel is running) by installing TCL Extensions (which we will call TCEs for short). An extensive set of pre-built TCEs are available for download and installation (including Ruby). The complete set of extensions can be found here.
  4. It is relatively simple to build your own TCE mirror, allowing for download and installation of TCEs from a local server (rather than having to pull down the extensions you need across the network).
  5. Tools exist to build your own TCEs if you can't find a pre-build TCE for a package that you might need.
  6. The licensing terms under which TCL is available (GPLv2) are relatively commercial friendly, allowing for later development of commercial extensions for the Microkernel (as long as those extensions are not bundled directly into the ISO). This would not be the case if a distribution that used a GPLv3 license were used instead.

With the foundation for our Microkernel chosen, the next step was to actually build the components that are needed for the node discovery process (and which, when taken together, make up the Razor Microkernel). In the next section, we'll describe these additional components (and their interaction with the Razor Server) in a bit more detail.

Components that make up the Razor Microkernel

As was mentioned previously, our Razor Microkernel is basically an extension of the "Core" distribution from Tiny Core Linux. We have added a number of standard TCL extensions (and their dependencies) to this "Core" distribution in order to support the node discovery process:

  1. ruby.tcz - an extension that provides everything needed to run Ruby (v1.8.7) within the Microkernel
  2. bash.tcz - an extension containing the "bash" shell; out of the box the TCL distribution only provides users with access to the "ash" shell
  3. dmidecode.tcz - an extension containing the standard dmidecode UNIX command; this command is used by the Microkernel Controller during the discovery process
  4. scsi-3.0.21-tinycore.tcz - an extension that provides the tools, drivers, and kernel modules needed to access SCSI disks; without this extension any SCSI disks attached to the node are not visible to the Microkernel
  5. lshw.tcz - an extension containing the standard lshw UNIX command; this command is used by the Microkernel Controller during the discovery process
  6. firmware-bnx2.tcz - an extension that provides the firmware files necessary to access the network using a Broadcom NetXtreme II networking card during the system boot process; without this extension the network cannot be accessed using this type of NIC (which is fairly common on some newer servers).
  7. openssh.tcz - an extension containing the openssh daemon; this extension is only included in Development Microkernel images, on a Production Microkernel image this package is not included (to prevent unauthorized access to the underlying systems via SSH).

These extensions (which we'll refer to as the "built-in extensions") are bundled into the ISO directly, and they are installed automatically during the boot process. This ensures that these extensions (and the functionality that they provide) are readily available for use during the Microkernel boot, setup and initialization process.

In addition to these built-in extensions, the Razor Microkernel supports the download and installation of "additional extensions" (and their dependencies) from a local TCE mirror once the kernel has completed its boot process (but prior to starting up the Razor Microkernel Controller and any dependencies). Currently, only one of these "additional extensions" is installed during the system initialization process (an Open VM Tools extension that we have built ourselves), and that extension is only installed if the Microkernel is being used to boot a virtual machine that is running in a VMware environment. Because of the way that the Microkernel is configured initially, this Open VM Tools extension is actually "downloaded" and installed from an internal TCE mirror that is built into the Razor Microkernel itself.

This initial configuration can easily be extended to support the inclusion of additional extensions that are installed directly from this internal TCE mirror or that are installed from an external TCE mirror somewhere else in the local network (perhaps this external TCE mirror functionality could be supported from within the Razor Server itself). Changing over from using an internal TCE mirror to using an external TCE mirror is as simple as changing a URL in the Razor Server configuration file. Like all such server-side changes to the Microkernel configuration, this change would be picked up by all of the Microkernel instances in the network during their next (or first) checkin.

In addition to the extensions mentioned previously, we have also pre-installed a number of "Ruby Gems" in our Razor Microkernel. Currently, this list of gems includes the following:

  1. daemons - a gem that provides the capability to wrap existing Ruby classes/scripts as daemon processes (that can be started, stopped, restarted, etc.); this gem is used primarily to wrap the Razor Microkernel Controller as a daemon process.
  2. facter - provides us with access to Facter, which is used to discover many "facts" about the systems that the Microkernel is deployed to (other "facts" are discovered using the standard lshw, lscpu, and dmidecode UNIX commands).
  3. json_pure - provides the functionality needed to parse/construct JSON requests, which is critical when interacting with the Razor Server; the json_pure gem is used because it is purely Ruby based, so we don't have to install any additional packages like we would have to do in order to use the more "performant" (but partly C-based) json gem instead.
  4. stomp - used by the MCollective daemon to communicate with its agents in the network via an ActiveMQ message queue

These Ruby Gems are installed dynamically (as part of the system initialization process), and are used within the Razor Microkernel Controller (and the services that it depends on). Currently these gems are installed based on a local list (and a local set of gems) that are "burned into" the Microkernel ISO, but it would be a fairly simple matter to pull this list (and the gems themselves) from an external server in the local network instead.

It should also be noted that, while the list of gems installed during the boot process is "burned into" the ISO (in the '/opt/gems/gem.list' file), the source for this file is actually the 'opt/gems/gem.list' file from the Razor-Microkernel project. This file is copied over during the Microkernel ISO build process (and is used to determine which gem files to download and include in the Microkernel ISO itself), so any changes to this file in the Razor-Microkernel project will automatically change the gems that are included the resulting Microkernel ISO (and that are installed during the Microkernel boot process). More information on how the ISO file is built from the Razor-Microkernel project is provided elsewherer in this project's Wiki, and a link to thate page is provided (along with links to other pages containing more detailed information about the Microkernel) in the References section at the bottom of this page.

Microkernel Initialization

During the Microkernel initialization process, there are several key services that are started. This set of services includes the following:

  1. The Microkernel Controller - a Ruby-based daemon process that interacts with the Razor Server via HTTP
  2. The Microkernel TCE Mirror - a WEBrick instance that provides a completely internal web-server that can be used to obtain TCL extensions that should be installed once the boot process has completed. As was mentioned previously, the only extension that is currently provided by this mirror is the Open VM Tools extension (and its dependencies).
  3. The Microkernel Web Server - a WEBrick instance that can be used to interact with the Microkernel Controller via HTTP; currently this server is only used by the Microkernel Controller itself to save any configuration changes it might receive from the Razor Server (this action actually triggers a restart of the Microkernel Controller by this web server instance), but this is the most-likely point of interaction between the MCollective and the Microkernel Controller in the future.
  4. The MCollective Daemon - as was mentioned previously, this process is not currently used, but it is available for future use
  5. The OpenSSH Daemon - only installed and running if we are in a "development" Microkernel; in a "production" Microkernel this daemon process is not started (in fact, the package containing this daemon process isn't even installed, as was noted above).

When the system is fully initialized, the components that are running (and the connections between them) look something like this:

Microkernel Components (Post-Boot)

The Razor Microkernel itself includes the four components on the left-hand side of this diagram (the OpenSSH Daemon may also be running but, since it is optional, it is not shown here). There are several interactions that are shown on this diagram that are worth describing in a bit more detail (since those interactions will help new users understand the underlying structure of the Microkernel and how the services that are running in the Microkernel work together to accomplish the task at hand):

  • The Microkernel Controller (a Ruby-daemon process) interacts with the Razor Server and Microkernel Web Server via HTTP/HTTPS (the former via the Microkernel checkin requests, the latter via POSTs that are made to the Microkernel Web Server instance whenever a new configuration is received by the Microkernel Controller from the Razor Server).
  • The Microkernel Web Server may restart the Microkernel Controller instance occasionally (typically this is done in order to force the Microkernel Controller to pick up and use a new configuration that it received from the Razor Server).
  • The Razor Server can communicate with the Microkernel Controller (indirectly) via the MCollective daemon and the HTTP requests that MCollective agents running on the Microkernel submit to the Microkernel Web Server (in order to push out a new configuration to a set of controllers or force a set of controllers to restart, for example). We have set up the framework for this communications channel, and have tested it by changing the Microkernel configuration using this communications channel in the past, but this channel is not actively used in the current release of the Razor Microkernel. Even though it is not used, this channel has been maintained for possible use in future releases of the Microkernel.
  • The Microkernel Controller can download and install TCL Extensions at any time (either from the Local TCE Mirror that is running within the Microkernel itself or from a Remote TCE Mirror). Currently only the local TCE mirror is used (and that mirror is only used to install a few extensions that are needed when the Microkernel first boots or reboots). However, using a remote mirror after the Microkernel Controller has been started is as simple as making change in the Microkernel Controller's configuration, a change that can be made on the Razor Server itself (and that would be picked up on by the Microkernel Controller on its next checkin).
    • It should be noted here that while we are showing the Remote TCE Mirror as being a part of the Razor Server in this diagram, this functionality could actually be provided by any web server in the local network; the Razor Server is just one possible source for this functionality.

References

For more detailed information about the Razor Microkernel (what it is, how it works, and how to build your own Microkernel ISOs), users should check the pages on this project Wiki that discuss these topics in more detail:

  • An-Overview-of-the-Razor-Microkernel - Provides users with an overview of the Razor Microkernel, the Razor Microkernel boot process, and how the Razor Server uses the Razor Microkernel to perform dynamic discovery and registration of nodes in the network.
  • Building-a-new-Razor-Microkernel-ISO - Describes the process of building a new Microkernel ISO using the tools provided by the Razor-Microkernel project.

For more detailed information about Tiny Core Linux (the basis for our Razor Microkernel), users are referred to the following pages:

  • The Tiny Core Linux Wiki - This Wiki contains a lot of detailed information about Tiny Core Linux including the details of how to install and use TCL, remaster TCL, the details about the internals of TCL (including the boot process)
  • The main Tiny Core Linux Project Page - This page contains links to TCL News, FAQs, Downloads, and Forms
  • The Tiny Core Base "Final Releases" Page - This page in the Tiny Core Linux Forum contains news about the current list of "final releases" (including links to Changelog-style information and discussions about each release)
Add a custom footer
Add a custom sidebar
Clone this wiki locally