Mac OS X, Hadoop 0.19.1, and Java 1.6

If you're excited, like I am, about Amazon's recent announcement that they are now offering Elastic Map Reduce you probably want to try a quick Hadoop MapReduce application to test the waters. I found out quickly that if you are on a Mac (as I am) you'll need to perform a few quick configurations before things work correctly. Below is what I needed to do to get Hadoop running on my Mac with Java 1.6.

This post assumes you are running the latest Mac OS X 10.5 with all updates applied.

Enable Java 1.6 Support

To enable Java 1.6, open up the Java Preferences application. This can be found in /Applications/Utilities/Java Preferences.

You will need to drag Java 1.6 up and place at the top of both the applet and application versions.

Open up a terminal and type java -version and you should see something like the following:

java version "1.6.0_07"
Java(TM) SE Runtime Environment (build 1.6.0_07-b06-153)
Java HotSpot(TM) 64-Bit Server VM (build 1.6.0_07-b06-57, mixed mode)

Configure JAVA_HOME

This was the tough part. Once you have Java 1.6 as the default runtime, you need to configure your JAVA_HOME environment variable to reflect this as well. Hadoop wants JAVA_HOME set before it will run correctly, otherwise you'll see:

Exception in thread "main" java.lang.UnsupportedClassVersionError: Bad version number in .class file

This assumes you are using the standard bash shell. Open up ~/.bash_profile (or create it if it doesn't exist) and add this line:

export JAVA_HOME=/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Home

Save the file. Return to the terminal and type . ~/.bash_profile which will set your JAVA_HOME correctly. To verify, you can type: echo $JAVA_HOME.

While you are in .bash_profile you might as well add a HADOOP_HOME variable, which will make running hadoop from the command line much easier. Below is my full .bash_profile:

export JAVA_HOME=/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Home
export HADOOP_HOME=~sethladd/Development/hadoop-0.19.1


Note that I also placed $HADOOP_HOME/bin into my path, so now I can type hadoop from anywhere.

Now that you are all setup, try the simple WordCount Hadoop Tutorial.

Popular posts from this blog

I ported a JavaScript app to Dart. Here's what I learned.

Converting Array to List in Scala

Minification is not enough, you need tree shaking