Overriding module functions in Python

Overview

Today I will describe a way of overriding functions from imported modules in Python which can be useful for profiling function calls or changing the functionality of a function you’re using without modifying the original source code.

We will be using a real world example for this tutorial. Let’s say you want to keep a count of the number of queries run by SQLObject. This is simple if you’re always calling the query function directly. But if the function is being called deeply within the module (as is the case with SQLObject) you have to get more crafty.

The problem

Let’s say you have the following bit of code:

import sqlobject

class SomeTable(sqlobject.SQLObject):
  some_column = sqlobject.StringCol()

for i in xrange(0, 5):
    s = SomeTable(some_column="Test %d" % i)
    print s.some_column

We know that somewhere behind the scenes, SQLObject is running real SQL queries, but we’re not using that code directly, nor is it presented to us. It would be ideal if SQLObject kept an internal query counter, but it doesn’t so we have to be enterprising and find a way to do it ourselves.

A little bit of digging through the SQLObject code leads us to an interesting bit of code in sqlobject.dbconnection:

class DBAPI:
    def _runWithConnection(self, meth, *args):
        conn = self.getConnection()
        try:
            val = meth(conn, *args)
        finally:
            self.releaseConnection(conn)
        return val

This code gets called each time SQLObject executes a query.

The solution

So we’ve found the code, but now we need a way of keeping track of the number of calls to that function. One way to do this is to simply override that attribute with one that is more to our liking. First let’s see what such a function would look like:

QUERY_COUNT = 0

original_function = sqlobject.dbconnection.DBAPI._runWithConnection

def wrapper(self, meth, *args):
    global QUERY_COUNT
    QUERY_COUNT += 1
    return original_function(self, meth, *args)

sqlobject.dbconnection.DBAPI._runWithConnection = wrapper

This works for most cases, but what if there were subclasses of DBAPI that had their own _runWithConnection methods? We need a way to replace those as well.

You can probably already see where this is going. We need to get all subclasses of the DBAPI class and recursively all of their subclasses, then replace the _runWithConnection method in each of them with our wrapper function.

The roadblock

This is relatively trivial with classes that inherit from ‘object’ (also known as ‘2.4 style’ classes), unfortunately DBAPI is not one of them! To make a long story short, things quickly get more complicated than they need to be. Wouldn’t it be nice if someone had written a module that could do this for us?

Introducing wraptools

You may be pleased to know that someone has in fact created such a module. That person happens to be me, which is why it’s being plugged here ๐Ÿ™‚

Let’s go back to the original solution and modify it slightly to use wraptools.

from wraptools import wraps

QUERY_COUNT = 0

@wraps(sqlobject.dbconnection.DBAPI._runWithConnection)
def wrapper(original_function, self, meth, *args):
    global QUERY_COUNT
    QUERY_COUNT += 1
    return original_function(self, meth, *args)

So in this example @wraps does all of the dirty work of replacing every instance of the function (including subclasses) via the sweet syntactic sugar of a decorator. The original function and any arguments it was called with gets passed along to your wrapper, allowing you to munge the input or output to your liking.

Putting it all together

import sqlobject
from wraptools import wraps

class SomeTable(sqlobject.SQLObject):
  some_column = sqlobject.StringCol()

QUERY_COUNT = 0

@wraps(sqlobject.dbconnection.DBAPI._runWithConnection)
def wrapper(original_function, self, meth, *args):
    global QUERY_COUNT
    QUERY_COUNT += 1
    return original_function(self, meth, *args)

for i in xrange(0, 5):
    s = SomeTable(some_column="Test %d" % i)

print "Total queries:", QUERY_COUNT

And there you have it: An elegant, simple way of profiling SQLObject queries! This concept can be easily extended to profile and override SQLAlchemy as well as most other Python modules.

You can access the module documentation here.

The damn code

If you’re psyched, you can grab the source code from my code repository.

Easy GUI applications with Python, GTK+ and Glade

Foreword

This tutorial is not intended to be a detailed guide on using Glade or programming in Python, but rather an introduction to using GTK+/Glade in conjunction with Python to create elegant, maintainable GUI applications for the Gnome desktop.

This guide assumes the capability to hunt down and install packages on your distribution of choice as well as some degree of development experience in Python.

If you want to create GUI driven applications, it’s in your best interest to use an interface builder. For GTK, the best such tool is Glade. Glade is a WYSIWYG interface builder which generates XML files describing the user interface and callbacks for your application. This allows you to completely separate your view (user interface) from the rest of your code and focus on the underlying logic of your program rather than needlessly adjusting the padding of a widget for hours. In most cases, you won’t have to write user interface drawing code at all.

Let’s get started.

Prerequisites

  • Python (2.4 or higher recommended)
  • Python Glade library (python-glade2)
  • Glade interface builder, Glade3 recommended (glade-3)

Building the user interface

Open the Glade interface builder and familiarize yourself with the interface. The ‘Palette’ contains all of your toolbox options, analogous to brush tools in GIMP. The ‘Properties’ panel is your one stop shop for adjusting metadata about your widgets such as display options (‘General’, ‘Common’) and callbacks (‘Signals’). Finally, the main panel is the canvas on which you will be drawing your application GUI.

For this example, we’ll be creating a window with a single button that says “Hello, world!” Pay special attention to the ‘Name’ properties of each widget, as we will be referencing them from the Python code, and they must correlate.

Step one: In the Palette, click the ‘Window’ icon under ‘Top Levels’. It’s the one that looks like a blank window, but you can also mouse over each icon to see its description.

Palette window

This will add a checkered box to your canvas.

Canvas - Window

Step two: In the properties window, set the ‘Window Title’ to what should appear in the title bar (in this case, we’ll use “Hello, world!”) and change the ‘Name’ to ‘main_window’

Properties window

Step three: Switch to the ‘Signals’ tab in the Properties window and set up a handler for the ‘destroy’ signal of the main window. Set the handler to ‘on_main_window_destroy’. This signal will fire when the user closes the application window.

Main window signals

Step four: Back in the Palette, click the ‘Button’ icon under ‘Control and Display’ and then click anywhere in the gray checkered window we created a moment ago. In properties, change the Label to ‘Hello, world!’ and the Name to ‘hello_button’

Canvas
Button properties

Step five: Add a handler for the ‘clicked’ signal of the hello button. Name the callback ‘on_hello_button_clicked’. This signal will fire whenever the user clicks on the button.

Button signals

Now just save the project as ‘gtk_hello_world.glade’ and exit the application. Congratulations, you’ve successfully built a GUI for your application. Give yourself a pat on the back.

Writing the code

The code is rather simple, we don’t have to mess with any specifics about drawing the widgets, we just tell them when to display and what to do when different events are called. If you don’t understand how classes in Python work, go read up on them before continuing.

import gtk
import gtk.glade


GLADE_FILE = 'gtk_hello_world.glade'


# A decorator that defines a function as a callback to a Glade signal
def glade_callback(function):
    function.__glade_callback__ = True
    return function

# Retrieves a dictionary of functions decorated as Glade callbacks
def get_glade_callbacks(input):
    attributes = dir(input)

    result = {}
    for key in attributes:
        attr = getattr(input, key)
        if callable(attr) and hasattr(attr, '__glade_callback__'):
            result[key] = attr

    return result


class HelloWorld(object):
    def run(self):
        self.interface = gtk.glade.XML(GLADE_FILE)

        self.callbacks = get_glade_callbacks(self)
        self.interface.signal_autoconnect(self.callbacks)

        self.main_window = self.interface.get_widget('main_window')
        self.main_window.show()

        gtk.main()


    @glade_callback
    def on_hello_button_clicked(self, data):
        print 'Hello, world!'

    @glade_callback
    def on_main_window_destroy(self, data):
        print 'Main window destroyed, exiting...'
        gtk.main_quit()


if __name__ == "__main__":
    hello = HelloWorld()
    hello.run()

The glade_callback function is a decorator which sets an attribute on a function defining it as a Glade callback. The get_glade_callbacks function looks for all functions with that attribute and returns them as a dictionary. I use these two helper functions in all of my GTK applications.

The run method is where the meat of the action happens. First we tell Glade to load the GUI that we created earlier in the tutorial:

    self.interface = gtk.glade.XML(GLADE_FILE)

Then we connect all of our Glade callbacks (all functions decorated with glade_callback) to their respective Glade signals as we defined them in the interface builder using introspection.

    self.callbacks = get_glade_callbacks(self)
    self.interface.signal_autoconnect(self.callbacks)

Now we get a handle on the main window and tell it to display. The name comes directly from the ‘Name’ property we set in Glade.

    self.main_window = self.interface.get_widget('main_window')
    self.main_window.show()

Finally, we tell GTK to start handling events for the program and passing them off to our custom signal handlers.

    gtk.main()

That little bit of scaffolding takes care of all of the GUI aspects of the application, and we’re now free to just worry about what we do when a user performs a specific action. That’s where our callbacks come in, which are the two functions on_hello_button_clicked which is called whenever the user clicks our button and on_main_window_destroy which is called when the window is closed.

You can grab the full source code and Glade file here.

Further reading material

Hopefully this guide was helpful. Comments or suggestions welcome.

Happy hacking!