RHive is an R extension facilitating distributed computing via HIVE query. RHive allows easy usage of HQL(Hive SQL) in R, and allows easy usage of R objects and R functions in Hive.
Before installing RHive, you have to have installed Hadoop and Hive
- Single Node
- Cluster Node
- set HADOOP_HOME at local machine on which R runs
- install local machine and remote machine on which NameNode runs or Hive-Server runs.
- Installation Guide
- set HIVE_HOME at local machine on which R runs.
- launch Hive Server with following command on remote machine. it should be as a background process.
$HIVE_HOME/bin/hive --service hiveserver
- install R
- need to install R on all tasktracker nodes
- install rJava
- only install rJava on local machine.
- install Rserve
- need to install Rserve on all tasktracker nodes
- make configuration in path (/etc/Rserv.conf) on all tasktracker nodes. edit this file to add 'remote enable' to allow remote connection.
- launch all Rserve on all tasktracker nodes.
- e.q>
R CMD Rserve
- e.q>
- setting tasktracker nodes
- add R_HOME path at $HADOOP_HOME/conf/hadoop-env.sh
- e.q>
export R_HOME=/usr/lib/R
- e.q>
- add R_HOME path at $HADOOP_HOME/conf/hadoop-env.sh
- install RUnit
- Requirements
- ant (in order to build java files)
- Installing RHive
- Download source code:
git clone https://github.com/nexr/RHive.git
- Change your working directory:
cd RHive
- Set the environment variables HIVE_HOME and HADOOP_HOME:
export HIVE_HOME=/path/to/your/hive/directory
export HADOOP_HOME=/path/to/your/hadoop/directory
- Build java files using ant:
ant build
- Build RHive:
R CMD build RHive
- Install RHive:
R CMD INSTALL RHive_.tar.gz
- Download source code:
- Set the environment variables HIVE_HOME and HADOOP_HOME:
- Set the environment variables:
export HIVE_HOME=/path/to/your/hive/directory
export HADOOP_HOME=/path/to/your/hadoop/directory
export HADOOP_CONF_DIR=/path/to/your/hadoop/conf/directory
- Or, add environment variables into Renviron
HIVE_HOME=/path/to/your/hive/directory
HADOOP_HOME=/path/to/your/hadoop/directory
HADOOP_CONF_DIR=/path/to/your/hadoop/conf/directory
- Set the environment variables:
- launch R
library(RHive)
rhive.connect(host, port, hiveServer2)
- Java 1.6
- R 2.13.0
- Rserve 0.6-0
- rJava 0.9-0
- Hadoop 0.20.x (x >= 1)
- Hive 0.8.x (x >= 0)