Skip to content

Commit

Permalink
Initial import v2.2
Browse files Browse the repository at this point in the history
  • Loading branch information
Thomas LEIBOVICI - DSSI/SISR/LIGS committed Aug 3, 2010
0 parents commit 2d1de16
Show file tree
Hide file tree
Showing 192 changed files with 61,253 additions and 0 deletions.
28 changes: 28 additions & 0 deletions KnownBugs
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@

Description:
Process terminates with SEGFAULT when MySQL server restarts.
Cause:
This is due to a bad resilience of MySQL client API to server crash when prepared statements are used.
This is known as bug #33384 in MySQL tracker (see http://bugs.mysql.com/bug.php?id=33384 for current bug status).
Workaround:
Disable prepared statements in Robinhood: to do so, use '--disable-prep-stmts' argument to ./configure
before compiling the program or building the RPM.


Description:
There are a lot of <defunc> process when reading Lustre's Changelogs.
Cause:
In early Lustre 2.0 releases (before alpha8) , liblustreapi forks a process each time the changelog is reopened.
Workaround:
Make Robihood trap SIGCHLD, by specifiying specifying the following option to configure: "--enable-llapi-fork-support".
Side effect:
Trapping events causes some internal calls to be interrupted by signals.

Description:
"with" or "bcond_with" macros are unknown when building the RPM.
Cause:
rpm or rpmbuild are too old.
Workaround:
Replace robinhood.spec.in with robinhood.old_spec.in.
Then re-run "configure" and "make rpm".

62 changes: 62 additions & 0 deletions LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
----------------------------------------------------------------------------
Copyright CEA/DAM/DIF (2004-2010)
Contributeur principal: Thomas LEIBOVICI thomas.leibovici@cea.fr

Ce logiciel est r�gi par la licence CeCILL soumise au droit fran�ais et
respectant les principes de diffusion des logiciels libres. Vous pouvez
utiliser, modifier et/ou redistribuer ce programme sous les conditions
de la licence CeCILL telle que diffus�e par le CEA, le CNRS et l'INRIA
sur le site "http://www.cecill.info".

En contrepartie de l'accessibilit� au code source et des droits de copie,
de modification et de redistribution accord�s par cette licence, il n'est
offert aux utilisateurs qu'une garantie limit�e. Pour les m�mes raisons,
seule une responsabilit� restreinte p�se sur l'auteur du programme, le
titulaire des droits patrimoniaux et les conc�dants successifs.

A cet �gard l'attention de l'utilisateur est attir�e sur les risques
associ�s au chargement, � l'utilisation, � la modification et/ou au
d�veloppement et � la reproduction du logiciel par l'utilisateur �tant
donn� sa sp�cificit� de logiciel libre, qui peut le rendre complexe �
manipuler et qui le r�serve donc � des d�veloppeurs et des professionnels
avertis poss�dant des connaissances informatiques approfondies. Les
utilisateurs sont donc invit�s � charger et tester l'ad�quation du
logiciel � leurs besoins dans des conditions permettant d'assurer la
s�curit� de leurs syst�mes et ou de leurs donn�es et, plus g�n�ralement,
� l'utiliser et l'exploiter dans les m�mes conditions de s�curit�.

Le fait que vous puissiez acc�der � cet en-t�te signifie que vous avez
pris connaissance de la licence CeCILL, et que vous en avez accept� les
termes.

----------------------------------------------------------------------------

Copyright CEA/DAM/DIF (2004-2010)
Main contributor: Thomas LEIBOVICI thomas.leibovici@cea.fr

This software is governed by the CeCILL license under French law and
abiding by the rules of distribution of free software. You can use,
modify and/or redistribute the software under the terms of the CeCILL
license as circulated by CEA, CNRS and INRIA at the following URL
"http://www.cecill.info".

As a counterpart to the access to the source code and rights to copy,
modify and redistribute granted by the license, users are provided only
with a limited warranty and the software's author, the holder of the
economic rights, and the successive licensors have only limited
liability.

In this respect, the user's attention is drawn to the risks associated
with loading, using, modifying and/or developing or reproducing the
software by the user in light of its specific status of free software,
that may mean that it is complicated to manipulate, and that also
therefore means that it is reserved for developers and experienced
professionals having in-depth computer knowledge. Users are therefore
encouraged to load and test the software's suitability as regards their
requirements in conditions enabling the security of their systems and/or
data to be ensured and, more generally, to use and operate it in the
same conditions as regards security.

The fact that you are presently reading this means that you have had
knowledge of the CeCILL license and that you accept its terms.
----------------------------------------------------------------------------
82 changes: 82 additions & 0 deletions Makefile.am
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
ACLOCAL_AMFLAGS = -I autotools/m4

#SUBDIRS=src man scripts
SUBDIRS=src scripts tests

rpm_dir=`pwd`/rpms

if LUSTRE
rpmbuild_opt=--with lustre
lversion=`rpm -qa "lustre[-_]*modules*" --qf "%{Version}\n" 2>/dev/null | tail -1 | cut -d "." -f 1-2`
lpackage=`rpm -qa "lustre[-_]*modules*" --qf "%{Name}\n" 2>/dev/null | tail -1`
rpmbuild_opt+=--define="lversion $(lversion)" --define="lpackage $(lpackage)"
else
rpmbuild_opt=--without lustre
endif

if USE_MYSQL_DB
rpmbuild_opt += --with mysql
dbname=mysql
dbversion=`mysql_config --version | cut -d "." -f 1-2`
else
rpmbuild_opt += --with sqlite
dbname=sqlite
dbversion=`rpm -qa "sqlite-devel*" --qf "%{Version}\n" | tail -1 | cut -d "." -f 1-2`
endif

if TMP_FS_MGR
purpose=tmp_fs_mgr
endif
if LUSTRE_HSM
purpose=lustre_hsm
rpmbuild_opt += --with hsm
endif
if SHERPA
purpose=sherpa
rpmbuild_opt += --with sherpa
endif



new: clean all

rpms: robinhood.spec dist
mkdir -p $(rpm_dir)/BUILD $(rpm_dir)/SPECS $(rpm_dir)/SRPMS $(rpm_dir)/RPMS \
&& rpmbuild $(rpmbuild_opt) --define="_topdir $(rpm_dir)" \
--define="dbversion $(dbversion)" --define="purpose $(purpose)" --define="_prefix $(prefix)" \
-ta $(distdir).tar.gz \
&& echo "RPMs successfully generated in $(rpm_dir)"

rpm: robinhood.spec dist
mkdir -p $(rpm_dir)/BUILD $(rpm_dir)/SPECS $(rpm_dir)/SRPMS $(rpm_dir)/RPMS \
&& rpmbuild $(rpmbuild_opt) --define="_topdir $(rpm_dir)" \
--define="dbversion $(dbversion)" --define="purpose $(purpose)" \
--define="_prefix $(prefix)" -tb $(distdir).tar.gz \
&& echo "RPM successfully generated in $(rpm_dir)/RPMS"

srpm: robinhood.spec dist
mkdir -p $(rpm_dir)/BUILD $(rpm_dir)/SPECS $(rpm_dir)/SRPMS $(rpm_dir)/RPMS \
&& rpmbuild $(rpmbuild_opt) --define="_topdir $(rpm_dir)" --define="_prefix $(prefix)" \
--define="dbversion $(dbversion)" --define="purpose $(purpose)" \
-ts $(distdir).tar.gz \
&& echo "SRPM successfully generated in $(rpm_dir)/SRPMS"

install-exec-local:
install -m 750 scripts/rbh-config $(DESTDIR)/$(sbindir)/rbh-config



EXTRA_DIST= robinhood.spec robinhood.spec.in \
robinhood.old_spec.in \
doc/templates/hsm_policy_basic.conf \
doc/templates/hsm_policy_detailed.conf \
doc/templates/tmp_fs_mgr_tuto.conf \
doc/templates/tmp_fs_mgr_detailed.conf \
LICENSE KnownBugs README.tmp_fs_mgr README.Lustre_HSM \
doc/admin_guides/Lustre-HSM_PolicyEngine_v212_admin_guide.pdf \
doc/admin_guides/robinhood_v212_tmp_fs_mgr-admin_guide.pdf \
doc/admin_guides/robinhood_v212_fs_mgr-tutorial.pdf



# goodies/vim/robinhood.vim cfg_templates/example.conf
151 changes: 151 additions & 0 deletions README.Lustre_HSM
Original file line number Diff line number Diff line change
@@ -0,0 +1,151 @@
I - License
II - Compiling
III - Install
IV - Database configuration
V - Enabling Lustre changelogs
VI - RobinHood config file
VII - First run

I - License
===========

Copyright (C) 2009 CEA/DAM

This program is free software; you can redistribute it and/or modify
it under the terms of the CeCILL-C License.

The fact that you are presently reading this means that you have had
knowledge of the CeCILL-C license (http://www.cecill.info) and that you
accept its terms.

II - Compiling
==============

2.1 - From source tarball
-------------------------

It is advised to build RobinHood on your target system, to ensure the best
compatibility with your Lustre and MySQL versions.

Make sure you have the following products installed on your machine:
* mysql-devel
* liblustreapi (/usr/include/liblustreapi.h and /usr/lib/liblustreapi.a)
It is usually provided by lustre RPM.

Unzip and untar the source distribution:
> tar zxvf robinhood-2.2.0.tar.gz
> cd robinhood-2.2.0

Run the 'configure' script with the following options:
* For Lustre-HSM: --with-purpose=LUSTRE_HSM
* --prefix=<path> to set the prefix of installation path (default is /usr).

E.g.:
> ./configure --with-purpose=LUSTRE_HSM

Then, build the RPM:
> make rpm

A ready-to-install RPM is generated in the 'rpms/RPMS/<arch>' directory.

2.2 - From svn repository
-------------------------

svn checkout https://robinhood.svn.sourceforge.net/svnroot/robinhood/trunk robinhood
cd robinhood
autoreconf --install

# also make sure flex and bison are installed:
yum install flex
yum install bison

Then refer to section 2.1 for next compilation steps.


III - Install
=============

For installing robinhood on your target system, install the previously generated RPM:
> rpm -ivh rpms/RPMS/x86_64/robinhood.lustre_hsm-2.2.0-1.x86_64.rpm

This installs:
* robinhood binaries (for Lustre-HSM: 'rbh-hsm' and 'rbh-hsm-report')
* configuration helper script (rbh-config)
* configuration templates
* The /etc/init.d/robinhood-hsm script (for RobinHood service management)

This also creates the robinhood service (chkconfig --add).

Note: on SLES systems, the default dependency for init scheduling is on "mysql" service.
However, in many cases, it should be too early for starting robinhood daemon,
especially if the filesystem it manages is not yet mounted.
In such case, you have to modify the following lines in "scripts/robinhood.init.sles.in"
before you run "./configure":
# Required-Start: <your filesystem service>


IV - Database Configuration
===========================

MySQL
-----
Robinhood needs a MySQL database for storing information about files.
This database can run on a different node from Robinhood daemon.

* Install MySQL server on the machine (mysql-server and mysql packages).
* Start the DB engine:
service mysqld start
* Run the configuration helper script as root on the database host:
1) check database requirement:
> rbh-config precheck_db
2) create robinhood's database:
> rbh-config create_db

* Write the DB password to a file with read access for root only (600)
e.g. to /etc/robinhood.d/.dbpassword

Note: initially, the database schema is empty. Robinhood will create it the first time it is launched.

SQLite
------
You can use SQLite instead of MySQL for testing, or small filesystems.
For this, you need to compile robinhood with the following option:
--with-db=SQLITE
To setup database access, you just need to give a path where robinhood will
store its sqlite database (in-file), in the ListManager::SQLite block of the
configuration file:

ListManager {
SQLite {
db_file = "/var/robinhood/robinhood_sqlite_db" ;
}
}

V - Enabling Lustre changelogs
==============================
For Lustre-HSM purpose, you must register robinhood as a MDT changelog consummer.

* Run the configuration helper script on Lustre MDT:
> rbh-config enable_chglogs
This registers a changelog consummer and activate required changelog records.

Note: by default, the script checks for a 'cl1' consummer.
If you need to register several changelog consummers on your file system,
refer to lustre documentation.

VI - Configuration file
=======================

You can use the '--template' option to generate a documented configuration file template:
> rbh-hsm --template=<template_file>

Note: by default, "service robinhood-hsm start" looks for configuration files in "/etc/robinhood.d/hsm".

For more details, refer to Robinhood Admin Guide (in the doc/admin_guide directory).

VII - First run
===============

Even if your filesystem is empty, you need to perform an initial scan in order to initialize robinhood database.
This prevents from having entries in filesystem that it wouldn't know about.
rbh-hsm --scan --once
Loading

0 comments on commit 2d1de16

Please sign in to comment.