An Introduction to Compassionate Screen Scraping

August 11, 2008 One of the most common quickie projects on the web is to screenscrape a website and play around with its data. These projects are a lot of fun, and can allow for inventive mashups, but often the screepscraping scripts cause unnecessary load on the site's servers due to inconsiderate technique. This is an introduction to the art of compassionate screenscraping.

Filed under pythonscreen-scraping

Deploying Django with Fabric

November 4, 2008 A look at how to manage deployment complexity with Django using Fabric. Something of a continuation on the post from yesterday.

Filed under djangopythonfabric

An Epic Introduction to PyObjC and Cocoa

August 22, 2008 PyObjC is one of the most helpful projects I have ever used, but a number of individuals have been having trouble getting started with PyObjC on Leopard because the documentation is in a bit of a disarray. In particular, there didn't seem to be a comprehensive tutorial that could introduce a newcomer to all the important aspects of PyobjC, and that was completely up to date. Here is my attempt to fill that void. With a vengeance.

Filed under pythoncocoapyobjc

Huge CSV and XML Files in Python

January 22, 2009 Quick walkthrough of my code for converting a very large CSV file into a very large XML file using the Python standard libraries. Despite a few issues along the way, was a very pleasant experience.

Filed under python

Using Threadpools in Python

February 10, 2009 This article takes a look at creating a threadpool in Python. Specifically it takes a stab at iteratively processing CSV and XML files and farming out the parsed data for processing by a threadpool. The Python logging, csv and ElementTree modules make cameo appearances.

Filed under python

Using Optional Parameters in Django Urls

February 4, 2008 A simple but helpful trick for using optional parameters in Django views to allow one view to serve multiple urls with varying parameters.

Filed under djangopython

War Card Game in Python

December 3, 2012 A simple implementation of the war card game in Python, made for an interview some time back.

Filed under python

Python Datastructures Backed by Redis

September 5, 2010 Part of my day's experiment was to play with implementing Python datastructures which are implemented ontop of Redis. Here we take a look at dictionaries and lists, but it should be straightforward to extend this idea to sets as well.

Filed under pythonredis

Storing Bounded Timeboxes in Redis

April 8, 2011 If you're doing analytics, reports or dealing with memory constraints in Redis, you're probably dealing with keeping two sorted-sets mutually consistent. This article also takes a look at using multi/exec to keep it fresh.

Filed under pythonredis

Parallel HTTP Requests in Python

December 2, 2008 Last week I was doing parallel HTTP Requests in PHP, and it seemed like a crying shame I was doing something in PHP that I had never figured out in Python. Like it often happens, it only takes a couple dozen lines to teach Python this new trick.

Filed under python

Tailing in Python

May 16, 2010 A quick and pointless look at implementing tail in Python. Something of a koan.

Filed under python