Introducing uberparse


uberparse is a Python library which reduces the burden of handling command line and configuration file parsing for modules and applications, making it trivial for imported modules to recursively expose configuration options and command line arguments.

In essence, uberparse is a thin wrapper around ConfigParser and optparse, but it presents them in a coherent and very simple way. Options are defined as attributes of a class, and a single function call transparently handles all of the configuration and argument parsing.

A simple example

Below is the simplest possible uberparse example:

import uberparse

class ModuleFlags(uberparse.FlagClass):
test_flag = dict(default="value", help="a test flag", short="-t")

options, args = uberparse.ParseOptions()

print "Test flag:", options["test_flag"]

This gets you the following for free:

$ ./ --help
Usage: [options]

-h, --help show this help message and exit

Options for application:
-t TEST_FLAG, --test_flag=TEST_FLAG
a test flag

Options for builtin:
-c CONFIG, --config=CONFIG
path to configuration file
--genconf dump configuration options and exit

$ ./ --test_flag=hello
Test flag: hello

$ ./ -t hello
Test flag: hello

It’s also trivial to create custom configuration files:


test_flag = config value

Builtin flags

You get –help, –help-xxx, –config, and –genconf options for free when using uberparse. The more precise –help options are useful for managing very large, nested sets of flags. The –genconf option will dump a valid configuration file with the current options, comments and values to the console.


The operation of uberparse can be configured on a per-application basis using another settings class. For example, you could configure the default config file using the following in your module:

class FlagSettings(uberparse.SettingsClass):
config = "test.ini"
include = ('some_test_module',)

Any class name can be used, so long as it inherits from uberparse.SettingsClass.

Module flags

As mentioned briefly earlier, any imported modules will also have their flags parsed. The unified configuration file is capable of configuring options from imported modules as well as the main application. Flags are defined in modules the same way they are defined in the main application.

The include setting from SettingsClass can be used to selectively import flags from specific modules. The default behavior is to use flags defined in any imported module.


You can grab the source code and a more comprehensive example here.

Automation made easy

Arguably one of the most common uses for shell scripting is the automation of mundane tasks. These tasks generally fall into two categories:

  1. Simple, fully automated processes
  2. Potentially dangerous, step-through processes

In the simple case, it is often desirable to log the success or failure of each step for later review in the event that something goes awry. In the latter case, it is common to prompt the user before each step and branch appropriately based on their response.

Both of these cases can be rather cumbersome to do in a scripting language like Bash, especially if you are unfamiliar with the argument passing and quoting rules of the language.

Many moons ago I created a little Bash include script which had some handy functions for handling these common tasks, and today I would like to introduce its standalone, grown up replacement: “Runt”.

Runt makes it easy to run commands and give useful output when it’s appropriate to do so (such as in error conditions). It also supports prompting the user interactively before executing a step. Let’s look at a few examples.

The most simple case is to run the command with a friendly description, and notify upon completion:

% runt -d "Updating APT repository" -- apt-get update
[INFO] Updating APT repository ...
[OKAY] Updating APT repository completed successfully!

For more dangerous commands, prompting the user may be appropriate. The user is given the chance to abort, or simply skip this particular step (while simulating success):

% runt -vi -d "Performing an APT upgrade" -- apt-get -yy upgrade
[INFO] Performing an APT upgrade ...
[INFO] Will execute: apt-get -yy upgrade

  Press ENTER to continue.
  Type "skip" and press ENTER to skip step.
  Type "info" and press ENTER for details.
  Or press Ctrl-C to abort.


[OKAY] Performing an APT upgrade completed successfully!
[OKAY] The output of the command was:

Reading package lists...
Building dependency tree...
Reading state information...
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

There are a number of options provided by Runt, including “terse” init-style output:

% runt -t -d "Checking for passwd" -- test -f /etc/passwd
 * Checking for passwd ...                                               [ ok ]

Of course, if something goes wrong, you’ll get more useful output:

% runt -t -d "Simulating an error" -- ls /not/a/real/path
 * Simulating an error ...                                               [ !! ]

[ERROR] Simulating an error exited with error (2)
[ERROR] The command executed was 'ls /not/a/real/path'
[ERROR] The output of the command was:

ls: cannot access /not/a/real/path: No such file or directory

[ERROR] Execution failed!

If no description is supplied (using -d), one will be made for you:

% runt -- test -f /etc/passwd
[INFO] Running 'test -f /etc/passwd' ...
[OKAY] Running 'test -f /etc/passwd' completed successfully!

Runt also supports executing commands on a remote server (via SSH):

% runt -s "" -- true
[INFO] Running 'true' ...
[INFO] Executing on server:
[OKAY] Running 'true' completed successfully!

Setting a working directory works on local and remote hosts:

% runt -v -w /etc -- ls passwd
[INFO] Running 'ls passwd' ...
[INFO] Command to execute: ls passwd
[INFO] Working directory: /etc

[OKAY] Running 'ls passwd' completed successfully!
[INFO] The output of the command was:


Detailed in-line help is also available:

% runt --help

Usage: ./runt [options] (command)


Hopefully someone is able to get a bit of use out of the tool. It’s proven useful to me in its various forms on a number of occasions during my career as a sysadmin.

If you’re interested, grab Runt 1.2 here. There is also a Runt 1.2 Debian package if that’s your poison.

Update: Runt has been rewritten as a Python script and additional functionality has been added (as of version 1.2).

Happy hacking!

The sanctity of marriage

In the late 1800’s, people fought to keep marriage defined as the joining of a man and woman of the same skin color. It wasn’t long ago that some people felt interracial marriage would ruin the “sanctity of marriage”.

Yesterday, that same spirit of intolerance, using the same flawed arguments, succeeded in passing an amendment to the California state constitution to retroactively invalidate the matrimony of any two people of the same gender.

The first amendment and article six of the constitution of the United States of America guarantee the separation of church and state by saying that no religion shall be unfairly recognized by the government.

Yesterday, the people of California ensured that the Christian bible’s supposed definition of marriage is the only one that can be recognized by the state of California by voting “yes” on Proposition 8. Yesterday, a blind eye was turned to the constitution by pandering to the desires the world’s largest, most influential cult.

1) This is an obvious repeat of the controversy surrounding interracial marriages. The same arguments being used then are being used now.

Just like the joining of a black man and a white woman was considered impure prior to Loving v. Virginia, the joining of a woman and another woman is being construed as impure today.

Only the word we use to describe this perceived impurity has changed; “miscegenation” has become “abomination”.

2) The majority of the funding for the Prop 8 campaign came from outside of the state of California.

Over $40 million in campaign funding came from the state of Utah alone (want to guess which institution?). It’s clear that the desires of the people of California are being trumped by a tyrannical outside party with deep pockets.

3) It is not the state’s place to define marriage.

If two consenting individuals want to make a contract between themselves to join in matrimony, that is their contract to make and the government’s job to uphold. If they wish to call that contract “marriage”, it is their right to do so, and the government’s job to recognize it.

4) The wedding of two people of the same gender doesn’t directly impact the rights of heterosexual couples.

Inevitably when I speak to supporters of Prop 8, they feel that they are being slighted. I’ve yet to hear a compelling argument for how gay marriage directly impacts their lives. Just a bunch of fear mongering that the “moral fabric” of our country will be destroyed.

Let me get this straight. Getting married to a stripper of the opposite sex in Vegas… OKAY. Getting married to someone you dearly love of the same sex… BAD. Loud and clear.

Gay people will continue to be gay regardless of if you let them marry the people they love. If ever there were an incredible uprising of gay pride, it will be in the aftermath of this unconstitutional marriage of church and state; the only marriage that can be considered a true abomination.

I’m hopeful that the U.S. Supreme Court’s tradition of curtailing the state’s unholy affairs with religion in favor of civil liberties will hold strong when presented with the opportunity to overturn Prop 8 and all equivalent state amendments.

Two logs, one logger.

Though the Python logging documentation doesn’t make it entirely clear, it is possible to log to two different places with a single logger object. It’s even possible to dynamically adjust the log level of each output handler.

The trick is to get a handle on a single logger (you can use a named logger if you don’t want to use “root”. We’ll kick off the process by making sure we have the logging module imported.

import logging

We must set the root logger level to at least as low as all other loggers or it will choke the logging of the other handlers.

root_logger = logging.getLogger("")

You can then add as many log handlers to that logger as you like, each with its own log level.

file_logger = logging.FileHandler(....)

console_logger = logging.StreamHandler()

Then elsewhere in your code, you simply use:"This will go to both handlers.")
logging.debug("This will only go to the file logger.")

Hopefully that’s been educational. A working example can be found in my boilerplate repository as usual.

Xen “host-only” networking

Xen has a very flexible network configuration system, but getting what you want out of it can take a fair amount of Linux networking-fu and some time poking around in the Xen network scripts. Today I’m going to show you how to set up “host only” networking in Xen, similar to that available in VMware, with minimal knowledge and effort.

Host only networking provides network connectivity among Xen guests (domU) while preventing access to the physical network and the Xen host (dom0). Let’s jump right into the meat of the HOWTO.

To set up host-only networking, do the following:

  1. Copy my network-hostonly script to /etc/xen/scripts.
  2. Set execute bit on /etc/xen/scripts/network-hostonly
  3. In /etc/xen/xend-config.sxp change “(network-script network-xxx)” to “(network-script network-hostonly)”
  4. In /etc/xen/xend-config.sxp change “(vif-script vif-xxx)” to “(vif-script vif-bridge)”
  5. Reboot your dom0 (restarting Xen *may* be sufficient)


This can be accomplished by running the following (as root):

wget -o /etc/xen/scripts/network-hostonly \
chmod +x /etc/xen/scripts/network-hostonly
sed -i 's/^\(network-script /\(network-script network-hostonly\)/' \
sed -i 's/^\(vif-script /\(vif-script vif-bridge\)/' \

To give a little bit of background, the network-hostonly script is a derivative of the network-bridge script that is distributed with Xen. The primary difference is that the hostonly script does not add your physical network interface to the Xen bridge, leaving only the virtual interfaces which are used by domU hosts. Your physical network interface (and by proxy, the physical network) is left completely untouched.


Free circuit simulation software

There are many great tools for analog circuit simulation and diagramming, but most of them are non-trivial to set up and have a steep a learning curve. I wanted to point out a great piece of software that I found a while back but have never seen linked from popular software lists.

Some things that make this this simulator stand out from the crowd:

  • Extremely simple and intuitive diagramming interface
  • Real time circuit simulation while you’re making changes to the circuit
  • Clear visual indicators of circuit state and electron flow
  • Scopes can be added to monitor up to three components simultaneously

Paul Falstad has an extensive collection of mathematics and physics related Java applets on his website, but one that really grabbed my attention is his analog circuit simulator. It comes packed with all of the basic electronics components you’d expect as well as some more advanced ones like boolean logic gates, MOSFET chips and JFETs.

For budding electrical engineers, there is a comprehensive set of example circuits with brief explanations to get you started on the right direction.

The source code is available (though the license is unclear) and the applet can be trivially embedded into web pages with a particular circuit, making it a great tool for displaying schematics.

It’s as simple as copying the JAR file and an exported circuit to your web server, and then using the following HTML to embed the applet:

    Sorry, you need a Java-enabled browser to see the simulation.

Happy hacking!

Overriding module functions in Python


Today I will describe a way of overriding functions from imported modules in Python which can be useful for profiling function calls or changing the functionality of a function you’re using without modifying the original source code.

We will be using a real world example for this tutorial. Let’s say you want to keep a count of the number of queries run by SQLObject. This is simple if you’re always calling the query function directly. But if the function is being called deeply within the module (as is the case with SQLObject) you have to get more crafty.

The problem

Let’s say you have the following bit of code:

import sqlobject

class SomeTable(sqlobject.SQLObject):
  some_column = sqlobject.StringCol()

for i in xrange(0, 5):
    s = SomeTable(some_column="Test %d" % i)
    print s.some_column

We know that somewhere behind the scenes, SQLObject is running real SQL queries, but we’re not using that code directly, nor is it presented to us. It would be ideal if SQLObject kept an internal query counter, but it doesn’t so we have to be enterprising and find a way to do it ourselves.

A little bit of digging through the SQLObject code leads us to an interesting bit of code in sqlobject.dbconnection:

class DBAPI:
    def _runWithConnection(self, meth, *args):
        conn = self.getConnection()
            val = meth(conn, *args)
        return val

This code gets called each time SQLObject executes a query.

The solution

So we’ve found the code, but now we need a way of keeping track of the number of calls to that function. One way to do this is to simply override that attribute with one that is more to our liking. First let’s see what such a function would look like:


original_function = sqlobject.dbconnection.DBAPI._runWithConnection

def wrapper(self, meth, *args):
    global QUERY_COUNT
    QUERY_COUNT += 1
    return original_function(self, meth, *args)

sqlobject.dbconnection.DBAPI._runWithConnection = wrapper

This works for most cases, but what if there were subclasses of DBAPI that had their own _runWithConnection methods? We need a way to replace those as well.

You can probably already see where this is going. We need to get all subclasses of the DBAPI class and recursively all of their subclasses, then replace the _runWithConnection method in each of them with our wrapper function.

The roadblock

This is relatively trivial with classes that inherit from ‘object’ (also known as ‘2.4 style’ classes), unfortunately DBAPI is not one of them! To make a long story short, things quickly get more complicated than they need to be. Wouldn’t it be nice if someone had written a module that could do this for us?

Introducing wraptools

You may be pleased to know that someone has in fact created such a module. That person happens to be me, which is why it’s being plugged here 🙂

Let’s go back to the original solution and modify it slightly to use wraptools.

from wraptools import wraps


def wrapper(original_function, self, meth, *args):
    global QUERY_COUNT
    QUERY_COUNT += 1
    return original_function(self, meth, *args)

So in this example @wraps does all of the dirty work of replacing every instance of the function (including subclasses) via the sweet syntactic sugar of a decorator. The original function and any arguments it was called with gets passed along to your wrapper, allowing you to munge the input or output to your liking.

Putting it all together

import sqlobject
from wraptools import wraps

class SomeTable(sqlobject.SQLObject):
  some_column = sqlobject.StringCol()


def wrapper(original_function, self, meth, *args):
    global QUERY_COUNT
    QUERY_COUNT += 1
    return original_function(self, meth, *args)

for i in xrange(0, 5):
    s = SomeTable(some_column="Test %d" % i)

print "Total queries:", QUERY_COUNT

And there you have it: An elegant, simple way of profiling SQLObject queries! This concept can be easily extended to profile and override SQLAlchemy as well as most other Python modules.

You can access the module documentation here.

The damn code

If you’re psyched, you can grab the source code from my code repository.

Easy GUI applications with Python, GTK+ and Glade


This tutorial is not intended to be a detailed guide on using Glade or programming in Python, but rather an introduction to using GTK+/Glade in conjunction with Python to create elegant, maintainable GUI applications for the Gnome desktop.

This guide assumes the capability to hunt down and install packages on your distribution of choice as well as some degree of development experience in Python.

If you want to create GUI driven applications, it’s in your best interest to use an interface builder. For GTK, the best such tool is Glade. Glade is a WYSIWYG interface builder which generates XML files describing the user interface and callbacks for your application. This allows you to completely separate your view (user interface) from the rest of your code and focus on the underlying logic of your program rather than needlessly adjusting the padding of a widget for hours. In most cases, you won’t have to write user interface drawing code at all.

Let’s get started.


  • Python (2.4 or higher recommended)
  • Python Glade library (python-glade2)
  • Glade interface builder, Glade3 recommended (glade-3)

Building the user interface

Open the Glade interface builder and familiarize yourself with the interface. The ‘Palette’ contains all of your toolbox options, analogous to brush tools in GIMP. The ‘Properties’ panel is your one stop shop for adjusting metadata about your widgets such as display options (‘General’, ‘Common’) and callbacks (‘Signals’). Finally, the main panel is the canvas on which you will be drawing your application GUI.

For this example, we’ll be creating a window with a single button that says “Hello, world!” Pay special attention to the ‘Name’ properties of each widget, as we will be referencing them from the Python code, and they must correlate.

Step one: In the Palette, click the ‘Window’ icon under ‘Top Levels’. It’s the one that looks like a blank window, but you can also mouse over each icon to see its description.

Palette window

This will add a checkered box to your canvas.

Canvas - Window

Step two: In the properties window, set the ‘Window Title’ to what should appear in the title bar (in this case, we’ll use “Hello, world!”) and change the ‘Name’ to ‘main_window’

Properties window

Step three: Switch to the ‘Signals’ tab in the Properties window and set up a handler for the ‘destroy’ signal of the main window. Set the handler to ‘on_main_window_destroy’. This signal will fire when the user closes the application window.

Main window signals

Step four: Back in the Palette, click the ‘Button’ icon under ‘Control and Display’ and then click anywhere in the gray checkered window we created a moment ago. In properties, change the Label to ‘Hello, world!’ and the Name to ‘hello_button’

Button properties

Step five: Add a handler for the ‘clicked’ signal of the hello button. Name the callback ‘on_hello_button_clicked’. This signal will fire whenever the user clicks on the button.

Button signals

Now just save the project as ‘’ and exit the application. Congratulations, you’ve successfully built a GUI for your application. Give yourself a pat on the back.

Writing the code

The code is rather simple, we don’t have to mess with any specifics about drawing the widgets, we just tell them when to display and what to do when different events are called. If you don’t understand how classes in Python work, go read up on them before continuing.

import gtk


# A decorator that defines a function as a callback to a Glade signal
def glade_callback(function):
    function.__glade_callback__ = True
    return function

# Retrieves a dictionary of functions decorated as Glade callbacks
def get_glade_callbacks(input):
    attributes = dir(input)

    result = {}
    for key in attributes:
        attr = getattr(input, key)
        if callable(attr) and hasattr(attr, '__glade_callback__'):
            result[key] = attr

    return result

class HelloWorld(object):
    def run(self):
        self.interface =

        self.callbacks = get_glade_callbacks(self)

        self.main_window = self.interface.get_widget('main_window')


    def on_hello_button_clicked(self, data):
        print 'Hello, world!'

    def on_main_window_destroy(self, data):
        print 'Main window destroyed, exiting...'

if __name__ == "__main__":
    hello = HelloWorld()

The glade_callback function is a decorator which sets an attribute on a function defining it as a Glade callback. The get_glade_callbacks function looks for all functions with that attribute and returns them as a dictionary. I use these two helper functions in all of my GTK applications.

The run method is where the meat of the action happens. First we tell Glade to load the GUI that we created earlier in the tutorial:

    self.interface =

Then we connect all of our Glade callbacks (all functions decorated with glade_callback) to their respective Glade signals as we defined them in the interface builder using introspection.

    self.callbacks = get_glade_callbacks(self)

Now we get a handle on the main window and tell it to display. The name comes directly from the ‘Name’ property we set in Glade.

    self.main_window = self.interface.get_widget('main_window')

Finally, we tell GTK to start handling events for the program and passing them off to our custom signal handlers.


That little bit of scaffolding takes care of all of the GUI aspects of the application, and we’re now free to just worry about what we do when a user performs a specific action. That’s where our callbacks come in, which are the two functions on_hello_button_clicked which is called whenever the user clicks our button and on_main_window_destroy which is called when the window is closed.

You can grab the full source code and Glade file here.

Further reading material

Hopefully this guide was helpful. Comments or suggestions welcome.

Happy hacking!

Ewige Blumenkraft!

I’ve created a new category for this blog titled “Adventures in Anarchy”. The name isn’t particularly important, but I plan to use this category as a placeholder for my philosophical, theological, technological, or just plain nonsensical links, rants, and drunken musings.

As most people close to me know, I’m a huge fan of the late anarchist, writer and occasional time traveler Robert Anton Wilson whose life and works are the true inspiration for today’s posting. I’m going to kick off this blog with a bit of musing over what I see as his most compelling philosophy.

Nothing “is”

RAW strongly believed that we can never know truth, but only our own perception of truth. He understood that all human comprehension is filtered through prejudice, fear, hate, love, and paranoia (“reality tunnels”) and that anything we experience will be naturally masked by those ancient primate emotions. RAW believed that the most dangerous concept is certainty, because it shuts minds off from alternate views, manufacturing ignorance which, in turn, commonly leads to war, misunderstanding and injustice.

I’ll leave you with some of my favorite RAW quotes:

“Nobody sees the obvious, nobody observes the ordinary. There are more miracles in a square yard of earth than in all the fables of the Church.”

“Belief is the death of intelligence.”

“It only takes 20 years for a liberal to become a conservative without changing a single idea.”

“Certitude is seized by some minds, not because there is any philosophical justification for it, but because such minds have an emotional need for certitude.”

“Animals outline their territories with their excretions, humans outline their territories by ink excretions on paper.”

“The function of Theology? The recitation of the incomprehensible by the unspeakable to pick the pockets of the unthinking.”

“I used to be an atheist, until I realized I had nothing to shout during blow jobs. “Oh Random Chance! Oh Random Chance!” just doesn’t cut it!.”

“I just preach that we’d all think and act more sanely if we had to use “maybe” a lot more often. Can you imagine a world with Jerry Falwell hollering “Maybe Jesus ‘was’ the son of God and maybe he hates Gay people as much as I do” — or every tower in Islam resounding with “There ‘is’ no God except maybe Allah and maybe Mohammed is his prophet”?”

“Reversing a sentimental error of the ’60s, the new anti-war slogan should be MAKE ASSASSINATIONS, NOT WARS. And, best of all, if this idea catches on internationally we can expect at least 50 contracts on George Bush the first week.”

“The Snafu law holds that, the greater your power to punish, the less factual feedback you will receive. If you can fire people for telling you what you don’t want to hear, you will only hear what you want. This law seems to apply to all authoritarian contraptions, especially governments and corporations. Concretely, I suspect [Bush] knows factually less about the world than any dogcatcher in Biloxi.”

“You are precisely as big as what you love and precisely as small as what you allow to annoy you.”

“You know, I have found a new way to get high and stay spaced out for hours on end, and the government can’t stop me… It’s called senility.”

“I think I got off on the wrong planet. Beam me up Scotty, there’s no rational life here.”

Personal rants as well as book exerpts are available on his website. Additionally he authored portions of the Principia Discordia.