Answered: MongoDB + PyMongo – Searching within a date range.

I wrote a short blog post last night when I felt that I couldn’t get any further independently with searching within a Mongo database for data between a date and time range.

I was highlighting a question that I had posed on Stack overflow.

Fortunately today when I got home from work very late I found there were two very short and very useful responses.
Turns out that I was over complicating my approach. The date is actually represented in the Mongo shell as an ISo date but it’s actually a BSON date type.

So by converting it to a string I was causing the problem in the first place.

Here’s what I was trying:

date1 = datetime.datetime.utcnow()-datetime.timedelta(minutes=15)
date1 = date1.strftime(“%Y-%m-%dT%H:%M:%S.000Z”)
for cursor in sensors.find({‘Date’:{“$gte”:’ISODate(“‘date1’)”‘}}):

First line is defining the date and time minus 15 minutes.
Second line is converting it into the right format.
Third line is creating the cursor and adding the ISODate(“”) component.

However, here’s what I should have been doing.

date1 = datetime.datetime.utcnow()-datetime.timedelta(minutes=15)
for cursor in sensors.find({‘Date’:{“$gte”: date1}}):

That simple!

However, on my travels I found that you can profile a Mongo database.
Learn about profiling MOngo here
This turns on profiling:

db.setProfilingLevel(2)db.setProfilingLevel(0)

This shows you the entries written to the profile table:

db.system.profile.find()

This is the kind of output you can expect from profiling:

“query”, “ns” : “DatabaseName.CollectionName”, “query” : { “Date” : { “$gte” : “ISODate(\”2014-12-31T12:30:09.000Z\”)” } }, “ntoreturn” : 0, “ntoskip” : 0, “nscanned” : 0, “keyUpdates” : 0, “nreturned” : 0, “responseLength” : 20, “millis” : 1, “client” : “127.0.0.1”, “user” : “” }

My huge thanks to the people on Stack overflow who helped out.