Question

I have a JSON file which has 40k documents, each document contains a date field. I need to query within Java with dates to retrieve data, so I stored the date in numberLong format.

Date dt = new Date();
SimpleDateFormat format = new SimpleDateFormat("MM/dd/yyyy");
dt = format.parse(nextLine[j]);
document.put(ColumnNameAsKey[j], dt.getTime());

where the above code is done in for loop in an API to store data to mongo.
But after entering all those data, and then when I queried numberlong changes automatically for the same date, so that I am unable to retrieve all data for the required date. My query to retrieve is

querygraph.put("Complaint Date (MM/DD/YYYY)", new 
    BasicDBObject("$gte",startdate.getTime()).append("$lte",EndDate.getTime()));

for eg : if the date 08/01/2012 contains large number of document, the correct numberlong for the date 08/01/2012 is replaced in the date field in mongo. this will continue, but aftr some number of documents, the numberlong keep on changing.. ie if NumberLong is 134353300000 for the date 08/01/2012, then after 6 or more document the numberlong will be different from the former one.. causing unable to retrieve exact data for the date 08/01/2012..
What makes the difference here?

Answer 1

I think see your problem, because you store the millisecond precision of time you are actually getting problems with the long ints representing parts of a day making it impossible to query past, say, midnight.

This is because MongoDB querying does not take this sort of contextual querying into account.

First off a hint, don’t store as number longs, store as the $date BSON type using only the lines:

Date dt = new Date();
SimpleDateFormat format = new SimpleDateFormat("MM/dd/yyyy");

You will get more functionality from using the specified BSON date type and the querying ability is the same across the board.

You have the right idea about querying your records:

querygraph.put("Complaint Date (MM/DD/YYYY)", new 
    BasicDBObject("$gte",startdate.getTime()).append("$lte",EndDate.getTime())); 

But I got a feeling you are doing something wrong. When you create the start date and the end date you are actually looking for the 00:00:00 time of the start date and the 23:59:59 time of the end date. This is due to your getTime() function, UNIX timestamp does not return partial times as such it will just return the default which is effectively now().

One way around that could make your life easier is to standardise times on this field so that you specify a time of 00:00:00 for all dates allowing you pick out ranges correctly.