TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

The Mighty Named Pipe

385 点作者 vsbuffalo大约 10 年前

16 条评论

aidos大约 10 年前
Nice article. Really easy to follow introduction.<p>I only discovered process substitution a few months ago but it&#x27;s already become a frequently used tool in my kit.<p>One thing that I find a little annoying about unix commands sometimes is how hard it can be to google for them. &#x27;&lt;()&#x27;, nope, &quot;command as file argument to other command unix,&quot; nope. The first couple of times I tried to use it, I knew it existed but struggled to find any documentation. &quot;Damnit, I know it&#x27;s something like that, how does it work again?...&quot;<p>Unless you know to look for &quot;Process Substitution&quot; it can be hard to find information on these things. And that&#x27;s once you even know these things exist....<p>Anyone know a good resource I should be using when I find myself in a situation like that?
评论 #9184731 未加载
评论 #9184382 未加载
评论 #9183591 未加载
评论 #9183352 未加载
评论 #9183632 未加载
评论 #9185067 未加载
评论 #9185121 未加载
unhammer大约 10 年前
Once you discover &lt;() it&#x27;s hard not to (ab)use it everywhere :-)<p><pre><code> # avoid temporary files when some program needs two inputs: join -e0 -o0,1.1,2.1 -a1 -a2 -j2 -t$&#x27;\t&#x27; \ &lt;(sort -k2,2 -t$&#x27;\t&#x27; freq&#x2F;forms.${lang}) \ &lt;(sort -k2,2 -t$&#x27;\t&#x27; freq&#x2F;lms.${lang}) # gawk doesn&#x27;t care if it&#x27;s given a regular file or the output fd of some process: gawk -v dict=&lt;(munge_dict) -f compound_translate.awk &lt;in.txt # prepend a header: cat &lt;(echo -e &quot;${word}\t% ${lang}\tsum&quot; | tr [:lower:] [:upper:]) \ &lt;(coverage ${lang})</code></pre>
评论 #9183456 未加载
评论 #9185112 未加载
larsf大约 10 年前
Pipes are probably the original instantiation of dataflow processing (dating back to the 1960s). I gave a tech talk on some of the frameworks: <a href="https://www.youtube.com/watch?v=3oaelUXh7sE" rel="nofollow">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=3oaelUXh7sE</a><p>And my company creates a cool dataflow platform - <a href="https://composableanalytics.com" rel="nofollow">https:&#x2F;&#x2F;composableanalytics.com</a>
评论 #9183811 未加载
Malarkey73大约 10 年前
Vince Buffalo is author of the best book on bioinformatics: Bioinformatics Data Skills (O&#x27;Reilly). It&#x27;s worth a read for learning unix&#x2F;bash style data science of any flavour.<p>Or even if you think you know unix&#x2F;bash and data there are new and unexpected snippets every few pages that surprise you.
dbbolton大约 10 年前
In zsh, =(cmd) will create a temporary file, &lt;(cmd) will create a named pipe, and $(cmd) creates a subshell. There are also fancy options that use MULTIOS. For example:<p><pre><code> paste &lt;(cut -f1 file1) &lt;(cut -f3 file2) | tee &gt;(process1) &gt;(process2) &gt;&#x2F;dev&#x2F;null </code></pre> can be re-written as:<p><pre><code> paste &lt;(cut -f1 file1) &lt;(cut -f3 file2) &gt; &gt;(process1) &gt; &gt;(process2) </code></pre> <a href="http://zsh.sourceforge.net/Doc/Release/Expansion.html#Process-Substitution" rel="nofollow">http:&#x2F;&#x2F;zsh.sourceforge.net&#x2F;Doc&#x2F;Release&#x2F;Expansion.html#Proces...</a><p><a href="http://zsh.sourceforge.net/Doc/Release/Redirection.html#Redirection" rel="nofollow">http:&#x2F;&#x2F;zsh.sourceforge.net&#x2F;Doc&#x2F;Release&#x2F;Redirection.html#Redi...</a>
amelius大约 10 年前
If you like pipes, then you will love lazy evaluation. It is unfortunate, though, that Unix doesn&#x27;t support that (operations can block when &quot;writing&quot; only, not when &quot;nobody is reading&quot;).
评论 #9183809 未加载
评论 #9184038 未加载
评论 #9183712 未加载
baschism大约 10 年前
AFAIK process substitution is a bash-ism (not part of POSIX spec for &#x2F;bin&#x2F;sh). I recently had to go with the slightly less wieldy named pipes in a dash environment and put the pipe setup, command execution and teardown in a script.
mhax大约 10 年前
I&#x27;ve used *nix for ~15 years and never used a named pipe or process substitution before. Great to know about!
评论 #9183755 未加载
评论 #9183544 未加载
anateus大约 10 年前
In fish shell the canonical example is this:<p><pre><code> diff (sort a.txt|psub) (sort b.txt|psub) </code></pre> The psub command performs the process substitution.
评论 #9189026 未加载
AndrewSB大约 10 年前
Does anyone have a working link to Gary Bernhardt&#x27;s The Unix Chainsaw, as mentioned in the article?
评论 #9184366 未加载
评论 #9194463 未加载
评论 #9183522 未加载
frankerz大约 10 年前
How does the &gt; process substitution differ from simply piping the output with | ?<p>For example (from Wikipedia)<p>tee &gt;(wc -l &gt;&amp;2) &lt; bigfile | gzip &gt; bigfile.gz<p>vs<p>tee &lt; bigfile | wc -l | gzip &gt; bigfile.gz
评论 #9183388 未加载
评论 #9183330 未加载
评论 #9183444 未加载
评论 #9183325 未加载
评论 #9184878 未加载
chuckcode大约 10 年前
Anybody know of a way to increase the buffer size of pipes? I&#x27;ve experienced cases where piping a really fast program to a slow one caused them both to go slower as the OS pauses first program writing when pipe buffer is full. This seemed to ruin the caching for the first program and caused them both to be slower even though normally pipes are faster as you&#x27;re not touching disk.
评论 #9184853 未加载
jamesrom大约 10 年前
Is this guy a bioinformatician? I think he&#x27;s a bioinformatician.<p>Can&#x27;t be sure if he is a bioinformatician because he never really mentions that he is a bioinformatician.
评论 #9184217 未加载
评论 #9183913 未加载
leni536大约 10 年前
moreutils [1] has some really cool programs for pipe handling.<p>pee: tee standard input to pipes sponge: soak up standard input and write to a file ts: timestamp standard input vipe: insert a text editor into a pipe<p>[1] <a href="https://joeyh.name/code/moreutils/" rel="nofollow">https:&#x2F;&#x2F;joeyh.name&#x2F;code&#x2F;moreutils&#x2F;</a>
hitlin37大约 10 年前
i heard somewhere that go follows unix pipe link interfaces.
Dewie大约 10 年前
Pipes are very cool and useful, but it&#x27;s hard for me to understand this common <i>worship</i> of something like that. Yes, it&#x27;s useful and elegant, but is it really the best thing since Jesus Christ?
评论 #9183479 未加载
评论 #9183994 未加载