Question

I’m doing an application to process some data stored in mongoDB using Hadoop. I’m writing the program in java.

The thing is that I have a subdocument that contains an array, and I want to take the value of one attribute of the array. I’ll put an example to see it more clear.

 "entities" : {
            "hashtags" : [
                    {
                            "**text**" : "whatever",
                            "indices" : [
                                    59,
                                    69
                            ]
                    },
                    {
                            "**text**" : "whatever",
                            "indices" : [
                                    82,
                                    95
                            ]
                    }
            ],
            "urls" : [ ],
            "user_mentions" : [ ]
    },

The value of the text is the one I want to process.

So I have developed a program in Java and it reports me the following error in the mapper class:

java.lang.ClassCastException: com.mongodb.BasicDBObject cannot be cast to java.lang.String
    at HashTagsMapper.map(HashTagsMapper.java:27)
    at HashTagsMapper.map(HashTagsMapper.java:18)
    at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
    at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:415)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
    at org.apache.hadoop.mapred.Child.main(Child.java:249)

This is the mapper class –>

public class HashTagsMapper extends Mapper<Object, BSONObject, Text, IntWritable>
{   
    public void map(Object key, BSONObject value, Context context) throws IOException, InterruptedException
    {

        ArrayList <String> name = new ArrayList<String>();
        BSONObject entities = (BSONObject) value.get("entities");
        BasicDBList hashtags = (BasicDBList) entities.get("hashtags");
        for(int index = 0; index < hashtags.size(); index++){
            name.add((String) hashtags.get(index));
        }
        try{
           FileWriter fw = new FileWriter("/home/jonrodriguez/Hashtags.txt");
           PrintWriter escribirListaRedundantes = new PrintWriter(fw);

           escribirListaRedundantes.println(name);

           fw.close();

        }

           catch(java.io.IOException ioex){}
        for(int i = 0; i < name.size(); i++){
            context.write(new Text(name.get(i)), new IntWritable(1));
        }
    }

Can anyone help me? Thanks!

Answer 1

The problem is the class cast exception. Why don’t you try to write not

(String)hashtags.get(index)

, but

hashtags.get(index).toString()

The problem could be that the basicDbList is a List of BDBObjects and you cannot cast the parent to a child.