• Category Archives Python
  • Answered: MongoDB + PyMongo – Searching within a date range.

    I wrote a short blog post last night when I felt that I couldn’t get any further independently with searching within a Mongo database for data between a date and time range.

    I was highlighting a question that I had posed on Stack overflow.

    Fortunately today when I got home from work very late I found there were two very short and very useful responses.
    Turns out that I was over complicating my approach. The date is actually represented in the Mongo shell as an ISo date but it’s actually a BSON date type.

    So by converting it to a string I was causing the problem in the first place.

    Here’s what I was trying:

    date1 = datetime.datetime.utcnow()-datetime.timedelta(minutes=15)
    date1 = date1.strftime(“%Y-%m-%dT%H:%M:%S.000Z”)
    for cursor in sensors.find({‘Date’:{“$gte”:’ISODate(“‘date1’)”‘}}):

    First line is defining the date and time minus 15 minutes.
    Second line is converting it into the right format.
    Third line is creating the cursor and adding the ISODate(“”) component.

    However, here’s what I should have been doing.

    date1 = datetime.datetime.utcnow()-datetime.timedelta(minutes=15)
    for cursor in sensors.find({‘Date’:{“$gte”: date1}}):

    That simple!

    However, on my travels I found that you can profile a Mongo database.
    Learn about profiling MOngo here
    This turns on profiling:

    db.setProfilingLevel(2)db.setProfilingLevel(0)

    This shows you the entries written to the profile table:

    db.system.profile.find()

    This is the kind of output you can expect from profiling:

    “query”, “ns” : “DatabaseName.CollectionName”, “query” : { “Date” : { “$gte” : “ISODate(\”2014-12-31T12:30:09.000Z\”)” } }, “ntoreturn” : 0, “ntoskip” : 0, “nscanned” : 0, “keyUpdates” : 0, “nreturned” : 0, “responseLength” : 20, “millis” : 1, “client” : “127.0.0.1”, “user” : “” }

    My huge thanks to the people on Stack overflow who helped out.


  • Question: MongoDB + PyMongo – Searching within a date range.

    Here we go again. I blogged about this before but now I’m trying to do it using Python. I don’t know why this is so difficult!

    I had to ask the question on Stack overflow because I’m no closer to solving it.

    When I eventually find the answer or if someone helps, I’ll of course post the solution here. It’s funny, when researching this, I kept finding the blog post I wrote before. Funny but ever so slightly irritating as well because I really don’t need to read wheat I’ve written!

    Updated on 8th January 2015 at 11PM: I have written a second blog post with the answer to this question here.


  • Turning on and off your Christmas tree lights from the Raspberry Pi.

    Thanks to a suggestion from my wife last week, I bought a Energenie socket controller for the Raspberry Pi. This little gadget allows wireless communication to a special socket that plugs into an electrical outlet. When working, this allows you to easily write code that will turn on or off that socket. The pack comes with one transmitter and two receiving sockets but actually, I bought two kits as each socket may be controlled by up to two transmitters and one transmitter may control up to four sockets.

    The reason that I bought this is quite simple. I have a new Raspberry Pi B+ in our living room with RaspBMC to allow us to use the XBMC media center software. we also put up our Christmas tree last weekend so we have the yearly problem of having to reach around quite a large christmas tree to reach the socket to turn off power to the lights every night. As they would say on twitter, it’s a perfect example of a first world problem! I hate that term but I’m getting off the point. Thanks to the RP and the Energenie I can turn on and off our Christmas tree lights remotely without going anywhere near that hard to reach socket.

    The problem of course was, on Wednesday after I spent a short amount of time getting this set up the night before, my wife posted a status to facebook saying, “It’s bad when I’m not technical enough to turn on the christmas tree lights”. Point taken. I set about creating a web interface to allow us to do this from our phones.

    I have never used the Flask Python web micro framework so this was a very new venture for me. the code you see before is my second version. The first one didn’t use views, a config file, templates or flash messages. It worked but it wasn’t as clean. I like to learn to do things properly so I scrapped it and read about how to do this properly. If you are interested in Flask, I really suggest you read this incredibly helplful tutorial by guel Grinberg.
    Here are all the steps. Hopefully I haven’t left anything out. To make this a little more conveenient for you, I’ve also included a package of all the code and graphics you will need to get this running on your own system.

    What you’ll need

    • A Raspberry Pi
    • An Energenie socket

    Getting your environment ready.

    I’m installing this on RaspBMC. This is the Raspberry Pi distribution for XBMC use primarily.
    Update the aptitude repository
    sudo apt-get update
    Install build dependencies.
    apt-get install gcc python-dev
    install the Python GPIO package to gain control of the pins on the Pi.
    cd /tmp
    wget https://pypi.python.org/packages/source/R/RPi.GPIO/RPi.GPIO-0.5.8.tar.gz

    Extract the contents of the archive.
    tar xzvf RPi.GPIO-0.5.8.tar.gz
    cd RPi.GPIO-0.5.8
    sudo python setup.py install

    Change to your home directory and get the Energenie install software from the following address.
    cd
    wget https://energenie4u.co.uk/res/software/ENER002-2PI.py

    Now install this.
    sudo python ENER002-2PI.py
    Make a directory that will hold your project
    cd
    mkdir

    ProjectName}
    cd {ProjectName}

    Get the really fantastic library created by Amy Mather’s. More information can be found here.
    wget https://github.com/MiniGirlGeek/energenie-demo/raw/master/energenie.py
    Get the Python setup tools package.
    sudo apt-get install python-setuptools
    get the Flask micro web framework.
    sudo easy_install flask
    Get the Jinja2 Python template engine.
    sudo easy_install Jinja2
    You now have all the components required to get coding.

    create your app file and directory structure

    The structure is as follows:
    {ProjectName}/
    {ProjectName}/run.py
    {ProjectName}/config.py
    {ProjectName}/energenie.py
    {ProjectName}/apple-touch-icon-120x120-precomposed.png
    {ProjectName}/apple-touch-icon-120x120.png
    {ProjectName}/favicon.png
    {ProjectName}/app/
    {ProjectName}/app/views.py
    {ProjectName}/app/__init__.py
    {ProjectName}/app/static/
    {ProjectName}/app/static/apple-touch-icon-120x120.png
    {ProjectName}/app/static/christmastree.png
    {ProjectName}/app/static/style.css
    {ProjectName}/app/templates/
    {ProjectName}/app/templates/index.html

    The code

    The following section has the code for each file along with a description of what that file is used for.

    run.py

    You will use this file to launch your application.

    from app import app
    app.run(host='0.0.0.0', debug=True) # Set to be accessible over the network with debugging enabled.

    config.py

    The config.py file does as you would expect. It is used to store config variables for the application.
    SECRET_KEY = 'YourSecretKey'
    The secret key is what ever you define. This is used by the flash messaging component.

    app/views.py

    This is the main part of your application. all of the processing happens here.

    # Import statements.
    from flask import render_template, flash
    from app import app
    from energenie import switch_on, switch_off

    # Main page. accessible from http://yourIPAddress:5000/
    @app.route('/')
    def index():
    return render_template('index.html', title='Christmas')

    # Code that is called with http://yourIPAddress:5000/on. This turns on the lights and adds a message to say the lights are on.
    @app.route('/on')
    def on():
    switch_on(1)
    flash('Christmas tree lights on.')
    return render_template('index.html', title='Christmas - Lights On')

    # Code that is called with http://yourIPAddress:5000/off. This turns off the lights and adds a message to say the lights are off.
    @app.route('/blue')
    def off():
    switch_off(1)
    flash('Christmas tree lights off.')
    return render_template('index.html', title='Christmas - Lights Off')

    app/__init__.py

    App initialization. Also includes the definition of the config file and tells flask that we are using views.py.

    from flask import Flask
    app = Flask(__name__)
    app.config.from_object('config')
    from app import views

    app/static/style.css

    You need to put static files into the static directory. The following style sheet definition defines some basic page layout options.

    body {
    margin-top:100px;
    font-size: 30px;
    }

    .red-button-link {
    text-decoration: none;
    padding: 15px 20px;
    background: red;
    color: #FFF;
    -webkit-border-radius: 6px;
    -moz-border-radius: 6px;
    border-radius: 6px;
    border: solid 2px #20538D;
    text-shadow: 0 -2px 0 rgba(0, 0, 0, 0.4);
    -webkit-box-shadow: inset 0 2px 0 rgba(255, 255, 255, 0.4), 0 2px 2px rgba(0, 0, 0, 0.2);
    -moz-box-shadow: inset 0 2px 0 rgba(255, 255, 255, 0.4), 0 2px 2px rgba(0, 0, 0, 0.2);
    box-shadow: inset 0 2px 0 rgba(255, 255, 255, 0.4), 0 2px 2px rgba(0, 0, 0, 0.2);
    -webkit-transition-duration: 0.2s;
    -moz-transition-duration: 0.2s;
    transition-duration: 0.2s;
    -webkit-user-select:none;
    -moz-user-select:none;
    -ms-user-select:none;
    user-select:none;
    }
    .red-button-link:hover {
    background: red;
    border: solid 2px #2A4E77;
    text-decoration: none;
    }
    .red-button-link:active {
    text-decoration: none;
    -webkit-box-shadow: inset 0 2px 6px rgba(0, 0, 0, 0.6);
    -moz-box-shadow: inset 0 2px 6px rgba(0, 0, 0, 0.6);
    box-shadow: inset 0 2px 6px rgba(0, 0, 0, 0.6);
    background: red;
    border: solid 2px #23E5F;
    }

    .green-button-link {
    text-decoration: none;
    padding: 15px 20px;
    background: green;
    color: #FFF;
    -webkit-border-radius: 6px;
    -moz-border-radius: 6px;
    border-radius: 6px;
    border: solid 2px #20538D;
    text-shadow: 0 -2px 0 rgba(0, 0, 0, 0.4);
    -webkit-box-shadow: inset 0 2px 0 rgba(255, 255, 255, 0.4), 0 2px 2px rgba(0, 0, 0, 0.2);
    -moz-box-shadow: inset 0 2px 0 rgba(255, 255, 255, 0.4), 0 2px 2px rgba(0, 0, 0, 0.2);
    box-shadow: inset 0 2px 0 rgba(255, 255, 255, 0.4), 0 2px 2px rgba(0, 0, 0, 0.2);
    -webkit-transition-duration: 0.2s;
    -moz-transition-duration: 0.2s;
    transition-duration: 0.2s;
    -webkit-user-select:none;
    -moz-user-select:none;
    -ms-user-select:none;
    user-select:none;
    }
    .green-button-link:hover {
    background: green;
    border: solid 2px #2A4E77;
    text-decoration: none;
    }
    .green-button-link:active {
    text-decoration: none;
    -webkit-box-shadow: inset 0 2px 6px rgba(0, 0, 0, 0.6);
    -moz-box-shadow: inset 0 2px 6px rgba(0, 0, 0, 0.6);
    box-shadow: inset 0 2px 6px rgba(0, 0, 0, 0.6);
    background: green;
    border: solid 2px #203E5F;
    }

    PNG files

    Add the graphics from the attached zip file if you like but if you would rather use your own, add them to the static folder.

    templates/index.html

    This is the template file. The UI that you will see is defined in this file.





    {{ title }}


    Picture of the Christmas Tree.
    Turn the lights on      Turn the lights off

    {% with messages = get_flashed_messages() %}
    {% if messages %}
    {% for message in messages %}
    {{ message }} {% endfor %}
    {% endif %}
    {% endwith %}




    Download the files required in an archive

    You may download the archive here

    if you would rather not copy and paste the code.