Question

I am unable to fix these typechecking errors in Scala:

package junk

import org.apache.hadoop.conf.Configuration
import org.apache.hadoop.io.IntWritable
import org.apache.hadoop.io.Text
import org.apache.hadoop.mapreduce.lib.input.TextInputFormat
import org.apache.hadoop.mapreduce.lib.output.TextOutputFormat
import org.apache.hadoop.mapreduce.Job

object Sample {
  val conf: Configuration = new Configuration()
  val job: Job = new Job(conf, "sample Hadoop MapReduce program");

  job.setInputFormatClass(classOf[TextInputFormat])
  job.setOutputFormatClass(classOf[TextOutputFormat])
  job.setOutputFormatClass(classOf[TextOutputFormat[Text, IntWritable]])
}    

The error messages are

Sample.scala:14: error: type mismatch;
found   : java.lang.Class[org.apache.hadoop.mapreduce.lib.input.TextInputFormat](classOf[org.apache.hadoop.mapreduce.lib.input.TextInputFormat])
required: java.lang.Class[_ <: org.apache.hadoop.mapreduce.InputFormat]
 job.setInputFormatClass(classOf[TextInputFormat])
                                ^
Sample.scala:15: error: class TextOutputFormat takes type parameters
 job.setOutputFormatClass(classOf[TextOutputFormat])
                                  ^
Sample.scala:16: error: type mismatch;
found   : java.lang.Class[org.apache.hadoop.mapreduce.lib.output.TextOutputFormat[org.apache.hadoop.io.Text,org.apache.hadoop.io.IntWritable]](classOf[org.apache.hadoop.mapreduce.lib.output.TextOutputFormat])
required: java.lang.Class[_ <: org.apache.hadoop.mapreduce.OutputFormat]
 job.setOutputFormatClass(classOf[TextOutputFormat[Text, IntWritable]])

Note that I have tried to setOutputFormatClass in two different ways and both fail.

I am trying to write Hadoop WordCount: http://wiki.apache.org/hadoop/WordCount in Scala. The equivalent lines in Java are

job.setInputFormatClass(TextInputFormat.class);
job.setOutputFormatClass(TextOutputFormat.class);

API docs for Job: http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/mapreduce/Job.html

Answer 1

As of Scala 2.9.1 (not Scala 2.9.0 — see https://issues.scala-lang.org/browse/SI-4603), this also works:

job.setInputFormatClass(classOf[TextInputFormat])