Python's subprocess module is great. It is definitely the cleanest wrapping of a system program of any of the scripting languages I've ever used. When you just want output, you call .communicate() and be done with it. When you want to interact, the file descriptors are there for you to play with. It's scripting heaven.<p>There is one feature that he could have touched on more:<p>shell=False is the only easy, safe way to put user-supplied input in a command line. This stuff is just too damned easy to get wrong.<p>I've seen <i>heroic</i> code that tried to work around shell quoting and escaping vulnerabilities. One colleague was trying to write a wrapper around the ldapsearch binary, and decided that filtering all "|" would do it. He'd completely forgotten that ` also triggers arbitrary command execution.<p>Don't be a hero. Keep bash away from your input.
I had to use os.popen (Maya, MotionBuilder, and outside) (obsolete by now, but still available), and it seemed much more easier than subprocess.Popen because I simply wanted a file filter.<p>Basically a coworker had ways to read/write custom ASCII 3D animation and model format, and another one introduced a binary form for it. All I did was to replace his "open" function with "open_model" and "open_anim" (no monkey patching), with something like this:<p><pre><code> def open_anim(name, mode, *args, **kwargs):
if name.upper().endswith( ".BIN_ANIM" ):
if 'r' in mode:
return os.popen( "anim_bin2text <" + name, mode, *args, **kwargs )
if 'w' in mode:
return os.popen( "anim_text2bin >" + name, mode, *args, **kwargs )
return open(name, mode, *args, **kwargs)</code></pre>
Great article. Chaining pipes together in Python with subprocess can get a little wordy, but the power is there to do anything you need to do.<p>It's especially excellent when compared to Ruby. Ruby has a bunch of popen calls, but each one has strange quirks and inflexibility to make it not good for building pipelines.<p><a href="http://whynotwiki.com/Ruby_/_Process_management#How_do_I_execute_an_external_program.3F" rel="nofollow">http://whynotwiki.com/Ruby_/_Process_management#How_do_I_exe...</a><p>I'm finding myself using a ton more shell (bash) for pipelines, as this is what it was built to do, but when I need to add more logic its Python and subprocess.
The python parts seemed OK. But the data flow diagram in the OP at:<p><a href="http://jimmyg.org/blog/2009/working-with-python-subprocess.html#what-happens-when-you-execute-a-command" rel="nofollow">http://jimmyg.org/blog/2009/working-with-python-subprocess.h...</a><p>(I.e., the first blue box in the article, "what happens when you execute a command")<p>is wrong. It implies that the command (ls -l) is read by the terminal emulator, which gives it to bash, which gives it to the kernel.<p>This is wrong in several ways. The terminal emulator starts bash (or whatever shell) and it is bash that directly reads the string "ls -l", figures out what it means, finds the ls program in PATH, and runs the ls binary. It is ls that calls kernel functions when it's doing its job.<p>And of course, the kernel is not really of importance here. The selected command does whatever it does, which might not even access the kernel ("Hello, world").<p>And bash can and sometimes does directly call kernel functions without an intermediate binary (e.g., unlink(2) to overwrite a file for output redirection).