Question

I need to do the following in my code:

• Read data from a text file
• Convert the data into Json
• Upsert the data into MongoDB

Here is an example of how the content of text file is going to look:

{
   "S": "someString" <- Type String when inerted in mongodb
   "N": 123          <- Type Int32
   "F": 12.3         <- Type Double
   "D": ?            <- Need to be Type DateTime when inerted in mongodb
}

I don’t know what I am suppose to have in the place of the “?” so when I use bson.json_util.loads function in python it can properly convert the text file into Json which later can be inserted into mongoDB.

Here is the code that does the load and insert:

with open('data.txt') as f:
    data = json_util.loads(f.read())
    db[dbName][colName].update({'_id': id}, data, upsert=True,safe=True)

I would appreciate it if someone can give an example of how the file should be formatted. (If your example can include even more complex Bson types like type “binary” or “code” that would be nice too 🙂 )

Answer 1

Mongo’s representation of datetimes is {"$date": number-of-milliseconds-since-epoch}. In your example:

{
   "S": "someString",
   "N": 123,
   "F": 12.3,
   "D": {"$date": 1352540684243}
}

D will generate a datetime field when written to mongo.

See the documentation for mongo json extensions.

You can also easily extend json_util to program your own extensions, for example, for ISO-formatted datetimes:

import json, dateutil.parser, bson.json_util

a = """{
    "mydate": {"$isodate": "2012-11-01T20:19:55.782Z"}
}"""

def my_hook(dct):
    if '$isodate' in dct:
        return dateutil.parser.parse(dct['$isodate'])
    return bson.json_util.object_hook(dct)

obj = json.loads(a, object_hook=my_hook)