Run, Python, Run

RunSnakeRun is a nice tool to view the pstats data the python profiler generates. But for me, the squaremap is hard to read and does not really add any value. Also there is no way to 'watch' a pstats file, or combine many files into one. Generally I prefer working with the commandline, even for starting gui applications.<p>My current setup to profile a python cgi application consists of a set of scripts to mock the cgi runtime, invoke the application with cprofile and accumulate profiling data in a single pstats file. To get a grasp of the hot spots, I'm using the magnificent gprof2dot script (<a href="http://code.google.com/p/jrfonseca/wiki/Gprof2Dot" rel="nofollow">http://code.google.com/p/jrfonseca/wiki/Gprof2Dot</a>) to generate a callgraph with profiling data (example: <a href="http://wiki.jrfonseca.googlecode.com/git/gprof2dot.png" rel="nofollow">http://wiki.jrfonseca.googlecode.com/git/gprof2dot.png</a>). Gprof2dot hides all unimportant (by default less than 1% runtime) calls and colors functions according to the time spend executing them. To view absolute numbers, I wrote a small python script to print the first view entries of a pstats file to the terminal and use that with watch (<a href="http://linux.die.net/man/1/watch" rel="nofollow">http://linux.die.net/man/1/watch</a>) to keep an eye on absolute runtimes.<p>Overall this is a nice setup and I'm very impressed by the rich amount tools available for python, though sometimes the APIs are kind of strange to me (for example cProfile, pstats).

just to be clear - runsnakerun is a viewer that presents data from the standard python profiler package. you can use the profiler without it - see <a href="http://docs.python.org/library/profile.html" rel="nofollow">http://docs.python.org/library/profile.html</a><p>also, when you're using a new api, it's worth paying attention to the default arguments for functions. since .pop() has an implicit argument of -1 you shouldn't be surprised that popping from the end of an array is most efficient (this is related to the O(n^2) v O(n) issue mentioned in the article).

I've been very happy with yappi: <a href="http://code.google.com/p/yappi/" rel="nofollow">http://code.google.com/p/yappi/</a> . The output is a little clunky to work with at first, but it tells you what you need to know.

Since the post mentions it, can someone provide a summary of the asymptotic runtime of different python list operations? I have no idea how python lists are implemented.