Make sure the you have JAVA_HOME set at your .bashrc file
Example:
export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk
export CLASSPATH=.:$JAVA_HOME/jre/lib/rt.jar:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
export PATH=$PATH:$JAVA_HOME/bin
you have set the following envirnoment variables too. Examples:
export HDFS_NAMENODE_USER="root"
export HDFS_DATANODE_USER="root"
export HDFS_SECONDARYNAMENODE_USER="root"
export YARN_RESOURCEMANAGER_USER="root"
export YARN_NODEMANAGER_USER="root"
The example setting of core-site.xml is as follows
<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://localhost:9000</value>
</property>
<property>
<name>io.file.buffer.size</name>
<value>1048576</value>
</property>
</configuration>
The Example hdfs-site.xml as follows:
<configuration>
<property>
<name>dfs.replication</name>
<value>2</value>
</property>
<property>
<name>dfs.http.address</name>
<value>localhost:50070</value>
</property>
<property>
<name>dfs.datanode.data.dir</name>
<value>/data/kmg/hadoop-data/data</value>
<final>true</final>
</property>
<property>
<name>dfs.namenode.name.dir</name>
<value>/data/kmg/hadoop-data/name</value>
<final>true</final>
</property>
<property>
<name>dfs.permissions</name>
<value>false</value>
</property>
<property>
<name>dfs.datanode.use.datanode.hostname</name>
<value>false</value>
</property>
</configuration>
Format the name node Example command:
/data/kmg/hadoop-3.2.0/bin/hdfs namenode -format
To start the DFS
/data/kmg/hadoop-3.2.0/sbin/start-dfs.sh
To stop the DFS
/data/kmg/hadoop-3.2.0/sbin/stop-dfs.sh
SBK Example invocations:
#Write
./build/distributions/sbk/bin/sbk -class hdfs -uri hdfs://localhost:9000 -file kmg-tmp-10.txt -writers 1 -size 1000000 -records 100000
#Read
./build/distributions/sbk/bin/sbk -class hdfs -uri hdfs://localhost:9000 -file kmg-tmp-10.txt -readers 10 -size 1000000 -records 100000