-
Notifications
You must be signed in to change notification settings - Fork 62
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Thomas LEIBOVICI - DSSI/SISR/LIGS
committed
Aug 3, 2010
0 parents
commit 2d1de16
Showing
192 changed files
with
61,253 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,28 @@ | ||
|
||
Description: | ||
Process terminates with SEGFAULT when MySQL server restarts. | ||
Cause: | ||
This is due to a bad resilience of MySQL client API to server crash when prepared statements are used. | ||
This is known as bug #33384 in MySQL tracker (see http://bugs.mysql.com/bug.php?id=33384 for current bug status). | ||
Workaround: | ||
Disable prepared statements in Robinhood: to do so, use '--disable-prep-stmts' argument to ./configure | ||
before compiling the program or building the RPM. | ||
|
||
|
||
Description: | ||
There are a lot of <defunc> process when reading Lustre's Changelogs. | ||
Cause: | ||
In early Lustre 2.0 releases (before alpha8) , liblustreapi forks a process each time the changelog is reopened. | ||
Workaround: | ||
Make Robihood trap SIGCHLD, by specifiying specifying the following option to configure: "--enable-llapi-fork-support". | ||
Side effect: | ||
Trapping events causes some internal calls to be interrupted by signals. | ||
|
||
Description: | ||
"with" or "bcond_with" macros are unknown when building the RPM. | ||
Cause: | ||
rpm or rpmbuild are too old. | ||
Workaround: | ||
Replace robinhood.spec.in with robinhood.old_spec.in. | ||
Then re-run "configure" and "make rpm". | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,62 @@ | ||
---------------------------------------------------------------------------- | ||
Copyright CEA/DAM/DIF (2004-2010) | ||
Contributeur principal: Thomas LEIBOVICI thomas.leibovici@cea.fr | ||
|
||
Ce logiciel est r�gi par la licence CeCILL soumise au droit fran�ais et | ||
respectant les principes de diffusion des logiciels libres. Vous pouvez | ||
utiliser, modifier et/ou redistribuer ce programme sous les conditions | ||
de la licence CeCILL telle que diffus�e par le CEA, le CNRS et l'INRIA | ||
sur le site "http://www.cecill.info". | ||
|
||
En contrepartie de l'accessibilit� au code source et des droits de copie, | ||
de modification et de redistribution accord�s par cette licence, il n'est | ||
offert aux utilisateurs qu'une garantie limit�e. Pour les m�mes raisons, | ||
seule une responsabilit� restreinte p�se sur l'auteur du programme, le | ||
titulaire des droits patrimoniaux et les conc�dants successifs. | ||
|
||
A cet �gard l'attention de l'utilisateur est attir�e sur les risques | ||
associ�s au chargement, � l'utilisation, � la modification et/ou au | ||
d�veloppement et � la reproduction du logiciel par l'utilisateur �tant | ||
donn� sa sp�cificit� de logiciel libre, qui peut le rendre complexe � | ||
manipuler et qui le r�serve donc � des d�veloppeurs et des professionnels | ||
avertis poss�dant des connaissances informatiques approfondies. Les | ||
utilisateurs sont donc invit�s � charger et tester l'ad�quation du | ||
logiciel � leurs besoins dans des conditions permettant d'assurer la | ||
s�curit� de leurs syst�mes et ou de leurs donn�es et, plus g�n�ralement, | ||
� l'utiliser et l'exploiter dans les m�mes conditions de s�curit�. | ||
|
||
Le fait que vous puissiez acc�der � cet en-t�te signifie que vous avez | ||
pris connaissance de la licence CeCILL, et que vous en avez accept� les | ||
termes. | ||
|
||
---------------------------------------------------------------------------- | ||
|
||
Copyright CEA/DAM/DIF (2004-2010) | ||
Main contributor: Thomas LEIBOVICI thomas.leibovici@cea.fr | ||
|
||
This software is governed by the CeCILL license under French law and | ||
abiding by the rules of distribution of free software. You can use, | ||
modify and/or redistribute the software under the terms of the CeCILL | ||
license as circulated by CEA, CNRS and INRIA at the following URL | ||
"http://www.cecill.info". | ||
|
||
As a counterpart to the access to the source code and rights to copy, | ||
modify and redistribute granted by the license, users are provided only | ||
with a limited warranty and the software's author, the holder of the | ||
economic rights, and the successive licensors have only limited | ||
liability. | ||
|
||
In this respect, the user's attention is drawn to the risks associated | ||
with loading, using, modifying and/or developing or reproducing the | ||
software by the user in light of its specific status of free software, | ||
that may mean that it is complicated to manipulate, and that also | ||
therefore means that it is reserved for developers and experienced | ||
professionals having in-depth computer knowledge. Users are therefore | ||
encouraged to load and test the software's suitability as regards their | ||
requirements in conditions enabling the security of their systems and/or | ||
data to be ensured and, more generally, to use and operate it in the | ||
same conditions as regards security. | ||
|
||
The fact that you are presently reading this means that you have had | ||
knowledge of the CeCILL license and that you accept its terms. | ||
---------------------------------------------------------------------------- |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,82 @@ | ||
ACLOCAL_AMFLAGS = -I autotools/m4 | ||
|
||
#SUBDIRS=src man scripts | ||
SUBDIRS=src scripts tests | ||
|
||
rpm_dir=`pwd`/rpms | ||
|
||
if LUSTRE | ||
rpmbuild_opt=--with lustre | ||
lversion=`rpm -qa "lustre[-_]*modules*" --qf "%{Version}\n" 2>/dev/null | tail -1 | cut -d "." -f 1-2` | ||
lpackage=`rpm -qa "lustre[-_]*modules*" --qf "%{Name}\n" 2>/dev/null | tail -1` | ||
rpmbuild_opt+=--define="lversion $(lversion)" --define="lpackage $(lpackage)" | ||
else | ||
rpmbuild_opt=--without lustre | ||
endif | ||
|
||
if USE_MYSQL_DB | ||
rpmbuild_opt += --with mysql | ||
dbname=mysql | ||
dbversion=`mysql_config --version | cut -d "." -f 1-2` | ||
else | ||
rpmbuild_opt += --with sqlite | ||
dbname=sqlite | ||
dbversion=`rpm -qa "sqlite-devel*" --qf "%{Version}\n" | tail -1 | cut -d "." -f 1-2` | ||
endif | ||
|
||
if TMP_FS_MGR | ||
purpose=tmp_fs_mgr | ||
endif | ||
if LUSTRE_HSM | ||
purpose=lustre_hsm | ||
rpmbuild_opt += --with hsm | ||
endif | ||
if SHERPA | ||
purpose=sherpa | ||
rpmbuild_opt += --with sherpa | ||
endif | ||
|
||
|
||
|
||
new: clean all | ||
|
||
rpms: robinhood.spec dist | ||
mkdir -p $(rpm_dir)/BUILD $(rpm_dir)/SPECS $(rpm_dir)/SRPMS $(rpm_dir)/RPMS \ | ||
&& rpmbuild $(rpmbuild_opt) --define="_topdir $(rpm_dir)" \ | ||
--define="dbversion $(dbversion)" --define="purpose $(purpose)" --define="_prefix $(prefix)" \ | ||
-ta $(distdir).tar.gz \ | ||
&& echo "RPMs successfully generated in $(rpm_dir)" | ||
|
||
rpm: robinhood.spec dist | ||
mkdir -p $(rpm_dir)/BUILD $(rpm_dir)/SPECS $(rpm_dir)/SRPMS $(rpm_dir)/RPMS \ | ||
&& rpmbuild $(rpmbuild_opt) --define="_topdir $(rpm_dir)" \ | ||
--define="dbversion $(dbversion)" --define="purpose $(purpose)" \ | ||
--define="_prefix $(prefix)" -tb $(distdir).tar.gz \ | ||
&& echo "RPM successfully generated in $(rpm_dir)/RPMS" | ||
|
||
srpm: robinhood.spec dist | ||
mkdir -p $(rpm_dir)/BUILD $(rpm_dir)/SPECS $(rpm_dir)/SRPMS $(rpm_dir)/RPMS \ | ||
&& rpmbuild $(rpmbuild_opt) --define="_topdir $(rpm_dir)" --define="_prefix $(prefix)" \ | ||
--define="dbversion $(dbversion)" --define="purpose $(purpose)" \ | ||
-ts $(distdir).tar.gz \ | ||
&& echo "SRPM successfully generated in $(rpm_dir)/SRPMS" | ||
|
||
install-exec-local: | ||
install -m 750 scripts/rbh-config $(DESTDIR)/$(sbindir)/rbh-config | ||
|
||
|
||
|
||
EXTRA_DIST= robinhood.spec robinhood.spec.in \ | ||
robinhood.old_spec.in \ | ||
doc/templates/hsm_policy_basic.conf \ | ||
doc/templates/hsm_policy_detailed.conf \ | ||
doc/templates/tmp_fs_mgr_tuto.conf \ | ||
doc/templates/tmp_fs_mgr_detailed.conf \ | ||
LICENSE KnownBugs README.tmp_fs_mgr README.Lustre_HSM \ | ||
doc/admin_guides/Lustre-HSM_PolicyEngine_v212_admin_guide.pdf \ | ||
doc/admin_guides/robinhood_v212_tmp_fs_mgr-admin_guide.pdf \ | ||
doc/admin_guides/robinhood_v212_fs_mgr-tutorial.pdf | ||
|
||
|
||
|
||
# goodies/vim/robinhood.vim cfg_templates/example.conf |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,151 @@ | ||
I - License | ||
II - Compiling | ||
III - Install | ||
IV - Database configuration | ||
V - Enabling Lustre changelogs | ||
VI - RobinHood config file | ||
VII - First run | ||
|
||
I - License | ||
=========== | ||
|
||
Copyright (C) 2009 CEA/DAM | ||
|
||
This program is free software; you can redistribute it and/or modify | ||
it under the terms of the CeCILL-C License. | ||
|
||
The fact that you are presently reading this means that you have had | ||
knowledge of the CeCILL-C license (http://www.cecill.info) and that you | ||
accept its terms. | ||
|
||
II - Compiling | ||
============== | ||
|
||
2.1 - From source tarball | ||
------------------------- | ||
|
||
It is advised to build RobinHood on your target system, to ensure the best | ||
compatibility with your Lustre and MySQL versions. | ||
|
||
Make sure you have the following products installed on your machine: | ||
* mysql-devel | ||
* liblustreapi (/usr/include/liblustreapi.h and /usr/lib/liblustreapi.a) | ||
It is usually provided by lustre RPM. | ||
|
||
Unzip and untar the source distribution: | ||
> tar zxvf robinhood-2.2.0.tar.gz | ||
> cd robinhood-2.2.0 | ||
|
||
Run the 'configure' script with the following options: | ||
* For Lustre-HSM: --with-purpose=LUSTRE_HSM | ||
* --prefix=<path> to set the prefix of installation path (default is /usr). | ||
|
||
E.g.: | ||
> ./configure --with-purpose=LUSTRE_HSM | ||
|
||
Then, build the RPM: | ||
> make rpm | ||
|
||
A ready-to-install RPM is generated in the 'rpms/RPMS/<arch>' directory. | ||
|
||
2.2 - From svn repository | ||
------------------------- | ||
|
||
svn checkout https://robinhood.svn.sourceforge.net/svnroot/robinhood/trunk robinhood | ||
cd robinhood | ||
autoreconf --install | ||
|
||
# also make sure flex and bison are installed: | ||
yum install flex | ||
yum install bison | ||
|
||
Then refer to section 2.1 for next compilation steps. | ||
|
||
|
||
III - Install | ||
============= | ||
|
||
For installing robinhood on your target system, install the previously generated RPM: | ||
> rpm -ivh rpms/RPMS/x86_64/robinhood.lustre_hsm-2.2.0-1.x86_64.rpm | ||
|
||
This installs: | ||
* robinhood binaries (for Lustre-HSM: 'rbh-hsm' and 'rbh-hsm-report') | ||
* configuration helper script (rbh-config) | ||
* configuration templates | ||
* The /etc/init.d/robinhood-hsm script (for RobinHood service management) | ||
|
||
This also creates the robinhood service (chkconfig --add). | ||
|
||
Note: on SLES systems, the default dependency for init scheduling is on "mysql" service. | ||
However, in many cases, it should be too early for starting robinhood daemon, | ||
especially if the filesystem it manages is not yet mounted. | ||
In such case, you have to modify the following lines in "scripts/robinhood.init.sles.in" | ||
before you run "./configure": | ||
# Required-Start: <your filesystem service> | ||
|
||
|
||
IV - Database Configuration | ||
=========================== | ||
|
||
MySQL | ||
----- | ||
Robinhood needs a MySQL database for storing information about files. | ||
This database can run on a different node from Robinhood daemon. | ||
|
||
* Install MySQL server on the machine (mysql-server and mysql packages). | ||
* Start the DB engine: | ||
service mysqld start | ||
* Run the configuration helper script as root on the database host: | ||
1) check database requirement: | ||
> rbh-config precheck_db | ||
2) create robinhood's database: | ||
> rbh-config create_db | ||
|
||
* Write the DB password to a file with read access for root only (600) | ||
e.g. to /etc/robinhood.d/.dbpassword | ||
|
||
Note: initially, the database schema is empty. Robinhood will create it the first time it is launched. | ||
|
||
SQLite | ||
------ | ||
You can use SQLite instead of MySQL for testing, or small filesystems. | ||
For this, you need to compile robinhood with the following option: | ||
--with-db=SQLITE | ||
To setup database access, you just need to give a path where robinhood will | ||
store its sqlite database (in-file), in the ListManager::SQLite block of the | ||
configuration file: | ||
|
||
ListManager { | ||
SQLite { | ||
db_file = "/var/robinhood/robinhood_sqlite_db" ; | ||
} | ||
} | ||
|
||
V - Enabling Lustre changelogs | ||
============================== | ||
For Lustre-HSM purpose, you must register robinhood as a MDT changelog consummer. | ||
|
||
* Run the configuration helper script on Lustre MDT: | ||
> rbh-config enable_chglogs | ||
This registers a changelog consummer and activate required changelog records. | ||
|
||
Note: by default, the script checks for a 'cl1' consummer. | ||
If you need to register several changelog consummers on your file system, | ||
refer to lustre documentation. | ||
|
||
VI - Configuration file | ||
======================= | ||
|
||
You can use the '--template' option to generate a documented configuration file template: | ||
> rbh-hsm --template=<template_file> | ||
|
||
Note: by default, "service robinhood-hsm start" looks for configuration files in "/etc/robinhood.d/hsm". | ||
|
||
For more details, refer to Robinhood Admin Guide (in the doc/admin_guide directory). | ||
|
||
VII - First run | ||
=============== | ||
|
||
Even if your filesystem is empty, you need to perform an initial scan in order to initialize robinhood database. | ||
This prevents from having entries in filesystem that it wouldn't know about. | ||
rbh-hsm --scan --once |
Oops, something went wrong.