Home/ Questions/Q 7021073
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-05-27T23:24:06+00:00

I am running this code using oozie workflow and getting type mismatch error: public

  • 0

I am running this code using oozie workflow and getting type mismatch error:

 public static class mapClass extends Mapper<Object, Text, LongWritable, LongWritable> {
        public void map(Object, Text, Context..)
..
         context.write(<LongWritable type> , <LongWritable type> )
}

public static class reduceClass extends Reducer<LongWritable, LongWritable,LongWritable, LongWritable> {
        public void reduce(LongWritable, LongWritable, context)
..
    context.write(<LongWritable type>, <LongWritable type>)
{

}

java.io.IOException: Type mismatch in value from map: expected org.apache.hadoop.io.LongWritable, recieved org.apache.hadoop.io.Text

I’m using new-api in my workflow. The same code works fine without using oozie.

Any help would be appreciated. Thanks.

—–code sample —

package org.apache.hadoop;
import java.io.IOException;

import org.apache.hadoop.io.LongWritable;
import org.apache.hadoop.io.Text;

import org.apache.hadoop.mapreduce.Mapper;


public class MapperLong extends Mapper<LongWritable, Text, LongWritable, LongWritable> {
    public final static int COL_ZERO = 0;
    public final static int COL_ONE = 1;
    public final static int COL_TWO = 2;
    public final static int COL_THREE = 3;

@Override
    public void map(LongWritable offset, Text line, Context context)
throws IOException, InterruptedException {
        String[] parts = (line.toString()).split(" ");

        LongWritable one = new LongWritable(Integer.parseInt(parts[COL_ONE]));
        LongWritable two = new LongWritable(Integer.parseInt(parts[COL_TWO]));
        context.write(one, two);

    }
}




package org.apache.hadoop;

import java.io.IOException;
import java.util.HashSet;
import java.util.Set;

import org.apache.hadoop.io.LongWritable;

import org.apache.hadoop.mapreduce.Reducer;


public class ReducerLong extends Reducer<LongWritable, LongWritable, LongWritable, LongWritable> {
@Override
    public void reduce(LongWritable colOneKey, Iterable<LongWritable> values,
        Context context) throws IOException, InterruptedException{
        Set<Integer> colTwo = new HashSet<Integer>();

        for (LongWritable val : values) {
            colTwo.add(Integer.valueOf((int)val.get()));
        }
        context.write(colOneKey, new LongWritable(colTwo.size()));
    }
}
}

java.io.IOException: Type mismatch in value from map: expected org.apache.hadoop.io.LongWritable, recieved org.apache.hadoop.io.Text
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:876)
at org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:574)
at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
at org.apache.hadoop.mapreduce.Mapper.map(Mapper.java:124)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:647)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323)
at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
at org.apache.hadoop.mapred.Child.main(Child.java:264)

Input :
34 342 1 1
45 23 0 1
..
..

Note: I changed Object type to LongWritable which did not make any difference. The above exception is thrown while using the following property in workflow.xml. Without the following property, code executes producing output same as input prefixed with offset!

        <property>
            <name>mapred.output.key.class</name>
            <value>org.apache.hadoop.io.LongWritable</value>
        </property>
        <property>
            <name>mapred.output.value.class</name>
            <value>org.apache.hadoop.io.LongWritable</value>
        </property>
Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    Ok, I figured out. The problem was in the oozie workflow I defined. 

    I had 
<name>mapreduce.mapper.class</name>
and 
<name>mapreduce.reducer.class</name>

instead of
<name>mapreduce.map.class</name>
and 
<name>mapreduce.reduce.class</name>

    [refer https://github.com/yahoo/oozie/wiki/Oozie-WF-use-cases MR Api ] For some reason that could not catch my eyes 🙁 since I was using the modified version of non-api workflow!

    Thank you all for your time.

    • 0