Skip to content

Latest commit

 

History

History
106 lines (88 loc) · 2.75 KB

README.md

File metadata and controls

106 lines (88 loc) · 2.75 KB
Raw

Fetch Hadoop Cookbook

Apache License 2.0

This cookbook fetches a hadoop distribution and installs it as a local user. Some very basic configuration is done.

Requirements

You should have java installed and the cookbook assumes a zsh on the system for the user setup.

Attributes

Key Type Description Default
['hadoop']['version'] String The hadoop version. 2.4.0
['hadoop']['data-directory'] String Which directory is the root directory for the hadoop data. /hadoop
['hadoop']['user']['name'] String Username for the hadoop user. hadoop
['hadoop']['user']['shell'] String The shell for the hadoop user. Platform dependent /usr/local/bin/zsh or /usr/bin/zsh.
['hadoop']['core-site'] Hash Settings for the core-site.xml. 

      'hadoop.tmp.dir' => "#{data-directory}/tmp",
      'fs.defaultFS' => 'hdfs://localhost:49152',
      'dfs.name.dir' => "#{data-directory}/name",
      'dfs.data.dir' => "#{data-directory}/data",
      'dfs.replication' => 1,
      'mapred.system.dir' => "#{data-directory}/mapreduce/system",
      'mapred.local.dir' => "#{data-directory}/mapreduce/local",
      'mapred.job.tracker' => 'localhost:49153',
      'mapred.tasktracker.tasks.maximum' => 3,
      'mapred.child.java.opts' => '-Xmx1024'

Usage

Just include fetch-hadoop in your node's run_list:

{
  "name":"my_node",
  "run_list": [
    "recipe[fetch-hadoop]"
  ]
}

You can include the recipe fetch-hadoop::format-namenode to format the name node directly after the installation.

If the DFS service is running you can use fetch-hadoop::dfs-create-user-dir to create the following directories:

  • /user
  • /user/#{node['hadoop']['user']['name']}

Contributing

  1. Fork the repository on Github
  2. Create a named feature branch (like add_component_x)
  3. Write your change
  4. Write tests for your change (if applicable)
  5. Run the tests, ensuring they all pass
  6. Submit a Pull Request using Github

License and Authors

Authors: