Montag, 3. Dezember 2012

NPE: org.apache.hadoop.io.serializer.SerializationFactory.getSerializer(SerializationFactory.java:73) (SOLVED)


My NullPointerException looked something like this:
java.lang.NullPointerException
    at org.apache.hadoop.io.serializer.SerializationFactory.getSerializer(SerializationFactory.java:73)
    at org.apache.hadoop.io.SequenceFile$Writer.init(SequenceFile.java:959)
    at org.apache.hadoop.io.SequenceFile$Writer.<init>(SequenceFile.java:892)
    at org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:393)
    at org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:354)
    at org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:476)
    at org.apache.hadoop.mapreduce.lib.output.SequenceFileOutputFormat.getRecordWriter(SequenceFileOutputFormat.java:61)
    at org.apache.hadoop.mapred.ReduceTask$NewTrackingRecordWriter.<init>(ReduceTask.java:569)
    at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:638)
    at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:417)
    at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:260)
After a few google searches I found the solution. Turns out that you need to add the serialization lib's to the Job configuration manually.. So my conf and the setup looks like this:
Configuration conf = new Configuration();
        conf.set("io.serializations","org.apache.hadoop.io.serializer.JavaSerialization,"
        + "org.apache.hadoop.io.serializer.WritableSerialization");

I cannot Understand chinese, but good thing that Programming code is (mostly) universal! :D

Link to the original thread: Link

Keine Kommentare:

Kommentar veröffentlichen