Apache Hadoop 1.0.0 released

Posted: January 8, 2012 | Author: | Filed under: FOSS, hadoop | Tags: , , | Leave a comment

Apache Hadoop 1.0.0 released. Release Note

Hadoop 1.0.0 released from 0.20.2xx.x development tree.  There is another new development in Hadoop space with version 0.23.0. New 0.23.0 version contains HDFS Federation and NextGen MapReduce (YARN).

Howto start Apache hadoop in debug mode

Posted: June 28, 2011 | Author: | Filed under: hadoop, HDFS | Tags: , | Leave a comment

Inset JPDA parameters to $HADOOP_HOME/bin/hadoop script.

HADOOP_OPTS=”$HADOOP_OPTS -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8000″

start hadoop with command:

$HADOOP_HOME/bin/hadoop namenode

Apache hadoop with start and wait for a debugger to connect via port 8000

How to Read/Write to a Apache hadoop file system (HDFS) file system using JAVA

Posted: May 18, 2011 | Author: | Filed under: hadoop, HDFS | Tags: , | Leave a comment

Deploy a HDFS cluster.

Create HDFS client using JAVA.

Genarate Maven2 project.

mvn archetype:generate -DgroupId=org.wso2.carbon -DartifactId=HDFSClient
hadoop-core is the only dependency you want in maven2 based project.
<dependency>
    <groupId>org.apache.hadoop</groupId>
    <artifactId>hadoop-core</artifactId>
    <version>0.20.2</version>
</dependency>

set cluster config in the client

do file operation via JAVA API

find maven2 based sample in in OT svn.