Skip to content

Latest commit

 

History

History
89 lines (64 loc) · 2.63 KB

README.md

File metadata and controls

89 lines (64 loc) · 2.63 KB
Raw

Hadoop cluster - lite

Build a lightweight hadoop cluster with below mentioned hadoop services. These versions match what Hortonworks HDP 2.3.4 distribution has.

Versions

  • Alpine Linux: latest
  • Debian: 8.3
  • Java: Open JDK 7
  • hadoop : 2.7.1
  • pig: 0.15.0
  • Zookeeper: 3.4.6
  • Hbase: 1.1.2
  • Kafka: 0.9.0.1
  • Kafka manager: latest
  • Storm: 0.10.0
  • Phoenix: 4.7.0

Note: Hadoop and storm are built on Debian Linux and all others on Alpine Linux

Dockerhub: https://hub.docker.com/r/anoopnair/

Prerequsities

The Docker project is tested on Ubuntu host. There is no guarantee that this will work on Windows.

Building all images

  • Pull from docker hub OR
  • Build each image individually

This will individually build the above mentioned images. This will take sometime and is required to be done only once.

  • Get zookeeper IP address using

docker inspect --format='{{.NetworkSettings.IPAddress}}' zookeeper

  • Get hbase IP address using

docker inspect --format='{{.NetworkSettings.IPAddress}}' hbase

  • Add zookeeper and hbase entries in the host /etc/hosts file

Verification

  • docker images to see all images created with proper tag version
  • docker logs -f _CONTAINER_NAME_ to follow logs for a specific container

If there is inconsistency in the image creation process, run the affected build script only. The build script is in each component directory.

Starting

./start.sh (background process) OR

docker-compose up (foreground process)

Stopping

docker-compose stop

Web UI

READMEs

Please go through below READMEs to get detailed information about each image

Reference