GitHub - eliast/edoop: A module for writing Hadoop Streaming jobs using Erlang

eliast / edoop Public

Notifications You must be signed in to change notification settings
Fork 1
Star 8

A module for writing Hadoop Streaming jobs using Erlang

Notifications

Name		Name	Last commit message	Last commit date
Latest commit History 3 Commits
README		README
brian.txt		brian.txt
edoop.erl		edoop.erl
wordcount.erl		wordcount.erl

Repository files navigation

DESCRIPTION
"""""""""""
 
edoop is just a simple erlang module that makes it easy to write
and run Streaming programs for Hadoop very easy.
 
INSTALLATION
""""""""""""

All you need to do is run: erlc edoop.erl 
 
USAGE
"""""
 
hadoop fs -put brian.txt brian.txt

hadoop jar $STREAMING_PATH/hadoop-*-streaming.jar -input brian.txt -output wordcount \
    -mapper 'escript wordcount.erl map' -reducer 'escript wordcount.erl reduce' \
    -file wordcount.erl -file edoop.beam 

hadoop fs -get wordcount .

cat wordcount/part*
 
MORE INFO
"""""""""
 
http://github.com/eliast/edoop

testing