Last Friday I gave a lightning talk at the Dutch Python Users meetup in Utrecht, entitled “Python for those little throwaway scripts (that you end up not throwing away)”. Here are the slides (pdf).

The first half of the talk is arguing that you can, and maybe should, write your small utility scripts in Python (rather than in say bash). This is based on my observation, that anything worth putting in a script in the first place (as opposed to just cracking out on the commandline) will tend to hang around for years, getting more complicated all the time. That means it’s worth the kind of consideration we give all code: source control, proper logging and error handling (including calling out to somewhere when things to catastrophically wrong), a comfortable interface (commandline options and output formats), and so on. If your major coding language is Python, chances are the utilities you make the most use of are for Python also; write your script in Python and you get the benefit of that experience.

I showed (a tidied version of) a little utility we use at Buzzcapture: a decorator that you wrap around the Python function that is the entrypoint to your script, that adds error handling, standardised logging (including commandline control of things like log level and where to log to), and similar basic functionality. Using it looks a bit like this:

from in_house.toolbox import script
 
@script(name='defrobnicator') # wraps with exception handling
def main(script_utils):
    args_parser = script_utils.args_parser() # already with -q, 
                                             # --log-file, etc
    args_parser.add_argument('--token', dest='token')
 
    args = args_parser.parse_args() # monkey-patched for magic!
 
    script_utils.logger.info(u'started') # logger uses name

The nice thing about having a utility like this is that it gives you a place to accumulate “in-house” knowledge and patterns. We have a pattern at Buzzcapture that got repeated three times by three different devs, with slightly different commandline options each time; the third time it was folded into the @script utility, and just today I saw it get used again, but this time just by turning on an option in the @script decorator.

At this point I ran out of time (a lightning talk is only five minutes long, which is even shorter than you think it is), but bravely forged on in the face of a hostile audience (no, not really) to show brief glimpses of a couple of tools that might make writing bash-style scripts in Python slightly more comfortable:

  • sh is a library that exposes your shell utilities as Python functions. It overrides the import mechanism so that from sh import ifconfig will go looking on your path for a shell executable called ifconfig — if it finds one (which it should), then ifconfig('eth0') will run the shell command and give you its output as a string. This is scary magic which I have not yet dared to use in production, but if nothing else it would be highly instructive to read the source and figure out how they do it.
  • plumbum has similar import magic, but it attempts to mimic shell syntax (e.g., by using Python’s bitwise OR operator | to compose shell commands into a shell pipeline). More magic madness, but again instructive; I also like the design philosophy, that separates defining the shell commands from executing them.
  • blessings is a Pythonic answer to curses: an interface to the terminal which is pleasant and intuitive to use. It makes it extremely easy to add some colour to textual output, but also has more advanced features like controlling cursor position.

I used blessings recently in a quick script I threw together to check Pivotal Tracker (a task management tool we use at Buzzcapture) for progress –and potential slippage– during the course of the two-week sprints we work with. It’s tentatively called slippage, and you can find it on Bitbucket.