Question

I am trying to read several big files(over 100MB). And by so far it always cracks down in the middle with the OutofMemory Error. Is there any solutions to it?

           FileInputStream fstream = new FileInputStream(f);
          // Get the object of DataInputStream
        DataInputStream dain = new DataInputStream(fstream);
        //  BufferedReader br = new BufferedReader(new InputStreamReader(in));

        BufferedReader in = new BufferedReader(new InputStreamReader(dain));
        String text = in.readLine();
        while(text != null) {
            stat(text);
            text = in.readLine();
        }

The Exception is like this:

    Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOfRange(Arrays.java:2694)
at java.lang.String.<init>(String.java:234)
at java.io.BufferedReader.readLine(BufferedReader.java:349)
at java.io.BufferedReader.readLine(BufferedReader.java:382)

Here is what Stat does:

    public void stat(String text) {
    String postTypeId = this.getXmlValue(text, "PostTypeId");
    String viewCountStr = this.getXmlValue(text, "ViewCount");
    String answerCountStr = this.getXmlValue(text, "AnswerCount");
    String userId = this.getXmlValue(text, "OwnerUserId");
    String postId = this.getXmlValue(text, "Id");
    String parentId = this.getXmlValue(text, "ParentId");
    String backUpId = this.getXmlValue(text, "LastEditorUserId");
    //Add post rel
    if(parentId==null) {
        if(!postTable.containsKey(postId)) 
            postTable.put(postId, new PostRel());
    } else {
        try{
        postTable.get(parentId).addAnswer(postId);
        }catch(Exception exp) {
        }
    }
              generalCount(postTypeId,viewCountStr,answerCountStr,userId,postId,parentId,backUpId);

}

And In generalCount, I tried to insert another table:

            if(userTable.containsKey(userId)) {
        userTable.get(userId).addPost(postId);
        if(parentId!=null)
            userTable.get(userId).addAnswer(parentId);
    } else{
        UserPostInfo newInfo = new UserPostInfo();
        newInfo.addPost(postId);
        if(parentId!=null)
            newInfo.addAnswer(parentId);
        userTable.put(userId, newInfo);

Answer 1

1. Give the JVM more memory to work with
2. Use less memory while reading the files (can you work with streaming data instead?)
3. Work with memory-mapped files