Debuging Hadoop applications in Eclipse (Linux & Windows)

1. Install Java 1.6.x, ssh and download Haddop. For windows need also install Cygwin.
2. Then configure environments for running haddop in standeloune mode. See Hadoop quick start guide for how to do it.
3. Go to ${hadoop}/bin folder and create hdebug file using existing hadoop file (simple copy and rename).
4. In the  hdebug file find strings:
   
   1 HADOOP_OPTS="$HADOOP_OPTS -Dhadoop.log.dir=$HADOOP_LOG_DIR"
   2 HADOOP_OPTS="$HADOOP_OPTS -Dhadoop.log.file=$HADOOP_LOGFILE"
   3 HADOOP_OPTS="$HADOOP_OPTS -Dhadoop.home.dir=$HADOOP_HOME"
   4 HADOOP_OPTS="$HADOOP_OPTS -Dhadoop.id.str=$HADOOP_IDENT_STRING"
   5 HADOOP_OPTS="$HADOOP_OPTS -Dhadoop.root.logger=${HADOOP_ROOT_LOGGER:-INFO,console}"
   6 if [ "x$JAVA_LIBRARY_PATH" != "x" ]; then
   7 HADOOP_OPTS="$HADOOP_OPTS -Djava.library.path=$JAVA_LIBRARY_PATH"
   8 fi
   and add new:
   
   1 HADOOP_OPTS="$HADOOP_OPTS -Xdebug -Xrunjdwp:transport=dt_socket,address=8001,server=y,suspend=y"
   you can get something like:
   
   1 HADOOP_OPTS="$HADOOP_OPTS -Dhadoop.log.dir=$HADOOP_LOG_DIR"
   2 HADOOP_OPTS="$HADOOP_OPTS -Dhadoop.log.file=$HADOOP_LOGFILE"
   3 HADOOP_OPTS="$HADOOP_OPTS -Dhadoop.home.dir=$HADOOP_HOME"
   4 HADOOP_OPTS="$HADOOP_OPTS -Dhadoop.id.str=$HADOOP_IDENT_STRING"
   5 HADOOP_OPTS="$HADOOP_OPTS -Dhadoop.root.logger=${HADOOP_ROOT_LOGGER:-INFO,console}"
   6 HADOOP_OPTS="$HADOOP_OPTS -Xdebug -Xrunjdwp:transport=dt_socket,address=8001,server=y,suspend=y"
   7 if [ "x$JAVA_LIBRARY_PATH" != "x" ]; then
   8 HADOOP_OPTS="$HADOOP_OPTS -Djava.library.path=$JAVA_LIBRARY_PATH"
   9 fi
5. Run your application using hdebug file instead of hadoop. Example start command:
   
   1 ${hadoop}/bin/hdebug jar your-app.jar
   You can see log:
   
   
6. Open Eclipse. Go to menu Run->Debug configurations... and create new Remoute Java Application configuration.
   On Connect tab: choice your project, select Standart connection type, set host (local), set Port: 8001.
   On Source tab: Add your project.
   
   
7. Starts debug, and enjoy :)