Thursday, October 04, 2012

Hadoop - Single Node Setup

Wanted to try hadoop for a longtime, got a chance to take a test drive today. Primed a fresh  CentOS6 Linux VM on my windows box (thanks to VMware Player) & setup hadoop with the help of below instruction.

Everthing went well, until this error throwed up on ./

2012-10-04 19:18:13,900 ERROR org.apache.hadoop.hdfs.server.namenode.NameNode: java.lang.IllegalArgumentException: Does not contain a valid host:port authority: file:///
    at org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(
    at org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(
    at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(
    at org.apache.hadoop.hdfs.server.namenode.NameNode.(
    at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(
    at org.apache.hadoop.hdfs.server.namenode.NameNode.main(

Same issue has been reported before -

Command "strace -fe open" tells you exactly where the configuration is getting loaded.


Even thought I configured the core-site.xml, mapred-site.xml & hdfs-site.xml under /usr/local/hadoop/conf/ folder, by default the system is referring to /etc/hadoop/ *.xml. Once I update the configuration files in /etc/hadoop location everything started working.

Very exciting, now that the environment is ready have to try some samples & step into mutli node cluster setup.

1 comment:

Hadoop online training said...

Thanks for nice data provided easy way to learn training with real time on
hadoop online training
through the experienced experts