I am using Jackson library to parse a large JSON response from server. Size of the json is around 7-8 mb.
I am getting the outOfMemoryError on this piece of code:
ObjectMapper mapper = new ObjectMapper();
JsonNode rootParser = mapper.readValue(is, JsonNode.class);
and this is the exception that I am getting:
01-14 13:13:20.103: E/AndroidRuntime(25468): FATAL EXCEPTION: Thread-13
01-14 13:13:20.103: E/AndroidRuntime(25468): java.lang.OutOfMemoryError
01-14 13:13:20.103: E/AndroidRuntime(25468): at java.util.ArrayList.add(ArrayList.java:123)
01-14 13:13:20.103: E/AndroidRuntime(25468): at org.codehaus.jackson.node.ArrayNode._add(ArrayNode.java:722)
01-14 13:13:20.103: E/AndroidRuntime(25468): at org.codehaus.jackson.node.ArrayNode.add(ArrayNode.java:203)
01-14 13:13:20.103: E/AndroidRuntime(25468): at org.codehaus.jackson.map.deser.std.BaseNodeDeserializer.deserializeArray(JsonNodeDeserializer.java:224)
01-14 13:13:20.103: E/AndroidRuntime(25468): at org.codehaus.jackson.map.deser.std.BaseNodeDeserializer.deserializeObject(JsonNodeDeserializer.java:200)
01-14 13:13:20.103: E/AndroidRuntime(25468): at org.codehaus.jackson.map.deser.std.BaseNodeDeserializer.deserializeArray(JsonNodeDeserializer.java:224)
01-14 13:13:20.103: E/AndroidRuntime(25468): at org.codehaus.jackson.map.deser.std.BaseNodeDeserializer.deserializeObject(JsonNodeDeserializer.java:200)
01-14 13:13:20.103: E/AndroidRuntime(25468): at org.codehaus.jackson.map.deser.std.BaseNodeDeserializer.deserializeArray(JsonNodeDeserializer.java:224)
01-14 13:13:20.103: E/AndroidRuntime(25468): at org.codehaus.jackson.map.deser.std.BaseNodeDeserializer.deserializeObject(JsonNodeDeserializer.java:200)
01-14 13:13:20.103: E/AndroidRuntime(25468): at org.codehaus.jackson.map.deser.std.BaseNodeDeserializer.deserializeObject(JsonNodeDeserializer.java:197)
01-14 13:13:20.103: E/AndroidRuntime(25468): at org.codehaus.jackson.map.deser.std.BaseNodeDeserializer.deserializeArray(JsonNodeDeserializer.java:224)
01-14 13:13:20.103: E/AndroidRuntime(25468): at org.codehaus.jackson.map.deser.std.BaseNodeDeserializer.deserializeObject(JsonNodeDeserializer.java:200)
01-14 13:13:20.103: E/AndroidRuntime(25468): at org.codehaus.jackson.map.deser.std.BaseNodeDeserializer.deserializeObject(JsonNodeDeserializer.java:197)
01-14 13:13:20.103: E/AndroidRuntime(25468): at org.codehaus.jackson.map.deser.std.JsonNodeDeserializer.deserialize(JsonNodeDeserializer.java:58)
01-14 13:13:20.103: E/AndroidRuntime(25468): at org.codehaus.jackson.map.deser.std.JsonNodeDeserializer.deserialize(JsonNodeDeserializer.java:15)
01-14 13:13:20.103: E/AndroidRuntime(25468): at org.codehaus.jackson.map.ObjectMapper._readMapAndClose(ObjectMapper.java:2732)
01-14 13:13:20.103: E/AndroidRuntime(25468): at org.codehaus.jackson.map.ObjectMapper.readValue(ObjectMapper.java:1909)
01-14 13:13:20.103: E/AndroidRuntime(25468): at com.sarla.smartglance.communication.JsonDecoder.decodeResponse(JsonDecoder.java:87)
01-14 13:13:20.103: E/AndroidRuntime(25468): at com.sarla.smartglance.communication.JsonDecoder.decode(JsonDecoder.java:68)
01-14 13:13:20.103: E/AndroidRuntime(25468): at com.sarla.smartglance.communication.MHttpManager$1.run(MHttpManager.java:86)
I tried everything but couldn’t find any solution to parse such a large amount of data on android.
With 7-8 megs of JSON, tree model that you are using will typically use 20 – 50 megs of memory (dom models are 3-5x as big, both for XML and JSON). There isn’t much you can do about that, regardless of library used: they all build trees using
Lists andMaps, which is a heavy-weight way of doing it.Instead you should consider using Plain Old Java Objects (POJOs), which will use much less memory. For this you need to model POJO that matches your JSON structure; without knowing structure I can’t give an example (if you add sample on question, I can), but code to parse is similar to GSON code referenced by another answer:
This will work on Jackson as well as many other Java JSON libs (Gson, Genson at least), and will also be faster method to use. JSON trees are inherently expensive and heavy-weight, and not to be used for multi-megatbyte content.
Finally, if your input consists of a sequence of items, there are even better ways to slice it up (which can be done regardless of whether individual items would be
JsonNodes or POJOs!). But I don’t know if your content is like that.