TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

Pipes: A spiritual successor to Yahoo Pipes

466 点作者 sea-gold10 个月前

25 条评论

onli10 个月前
This is my project :) Nice to see this here again! If there are questions, I&#x27;m ready to answer (I have a rather young baby here though that wanted attention while writing this, but she sleeps now, so I think we are good).<p>Pipes saw some internal updates recently. It was fighting with some instability. My first measure was an internal re-architeture, on a small scale. Before, the data was transported as text from block to block. (Almost) Every block parsed the input, created an RSS feed object and then worked with that on the data, only to then output the created RSS feed as string. This was changed to just work with the RSS object directly and move that from block to block. That had some consequences for some existing pipes, mainly because it also changed when the input feeds gets normalized.<p>As that did not help, next step was a server upgrade. I&#x27;m not sure whether it was directly the additional processing capacity or the dependency upgrades, but that one helped so far. Knock on wood. Also reconfigured the amount of threads and puma workers, that might have solved a bottleneck somewhere. I sadly never pinpointed exactly what made the server process stop before.<p>I had played with a proper split between web frontend and pipes processing, but at least my approach was not viable. Worked nicely while developing it, immediately crashed with the production workload. And Pipes is not all that huge... I&#x27;ll have to try again some time.
评论 #40843330 未加载
RyLuke10 个月前
I see a lot of folks here asking about what happened to the original Yahoo Pipes.<p>We had the same question, so we went and talked with a lot of the original team and wrote up the story[0]. We also made a fun mini-site that contains a lot of easter eggs (e.g. if you click on the &quot;Memory Pipes&quot; folder on the desktop, you&#x27;ll see a bunch of candid photos of the original team circa 2007)<p>[0] <a href="https:&#x2F;&#x2F;retool.com&#x2F;pipes" rel="nofollow">https:&#x2F;&#x2F;retool.com&#x2F;pipes</a>
评论 #40846912 未加载
评论 #40853131 未加载
dang10 个月前
Related:<p><i>The architecture and software behind Pipes</i> - <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=22959013">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=22959013</a> - April 2020 (12 comments)<p><i>Show HN: Early-stage Yahoo Pipes spiritual successor</i> - <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=14440993">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=14440993</a> - May 2017 (126 comments)
评论 #40861170 未加载
评论 #40843625 未加载
theshrike7910 个月前
n8n is a similar tool and can be self-hosted: <a href="https:&#x2F;&#x2F;n8n.io" rel="nofollow">https:&#x2F;&#x2F;n8n.io</a><p>And NodeRED is big in the home automation space: <a href="https:&#x2F;&#x2F;nodered.org" rel="nofollow">https:&#x2F;&#x2F;nodered.org</a>
评论 #40844392 未加载
评论 #40851941 未加载
klaussilveira10 个月前
Finally! Yahoo Pipes was so ahead of its time. The whole concept of microdata, RSS and wrangling data around in pipes was amazing and it is sad that it died. The web was so promising at that time.
smetj10 个月前
I have no experience with this particular implementation but I do have experience with some important business process developed in Azure Logic Apps, which is kind of the same in spirit I believe.<p>From what I have seen, these kind of frameworks facilitates the worst possible combination of factors: A solution designed, created and implemented by non-programmers, not using a programming language ending up in a production environment.<p>One could argue this is a success in its own but I have only seen these kind of things hit their limits almost immediately after the initial POC and evolve into terrible tech debt. Just learn to program or script already.<p>That being said, I do not want to disrespect the effort into building this and perhaps there is a place for these kind of solutions I yet have to experience.
评论 #40843445 未加载
评论 #40843428 未加载
lolive10 个月前
At work, we use extensively Palantir Foundry and its [excellent] PipelineBuilder. I was reluctant at first with this low-code no-code stuff. But after porting custom python code, I must admit it covers 99% of the transformation needs we have. Plus it keeps a semantic all along, that is much more difficult to understand&#x2F;maintain by code [especially in Python].
akshayrajp10 个月前
Can Zapier and similar tools be considered a spiritual successor to Yahoo Pipes too?
评论 #40842806 未加载
harinijan10 个月前
Interesting to see the evolution of Visual programming across many tools. We are working on something similar[0] for low-code&#x2F;no-code devs focusing on creation of APIs, backend tasks, and AI workflows.<p>[0] <a href="https:&#x2F;&#x2F;buildship.com" rel="nofollow">https:&#x2F;&#x2F;buildship.com</a>
anothername1210 个月前
Back in the day I had hundreds of pipes, sub-pipes for all kinds of stuff from porn to price gathering, comparisons, to banking. When Yahoo Pipes died so did my will to port all that to something else.<p>If I had to do it all again in 2024, what’s a robust self hosted project like this?
评论 #40853874 未加载
ninetyninenine10 个月前
every form of visual programming I&#x27;ve ever seen is a singular style of some of a box that does processing and a line that connects that box to another box.
评论 #40842599 未加载
评论 #40842800 未加载
评论 #40842416 未加载
评论 #40843126 未加载
评论 #40843399 未加载
评论 #40842398 未加载
评论 #40843442 未加载
评论 #40842444 未加载
评论 #40844181 未加载
评论 #40851223 未加载
评论 #40850886 未加载
评论 #40845513 未加载
freeqaz10 个月前
This is the code for a YC startup I ran in this space. Very similar. Build on AWS Lambda: <a href="https:&#x2F;&#x2F;github.com&#x2F;refinery-labs&#x2F;refinery">https:&#x2F;&#x2F;github.com&#x2F;refinery-labs&#x2F;refinery</a><p>I need to fix the docs because they are dead: <a href="https:&#x2F;&#x2F;web.archive.org&#x2F;web&#x2F;20200819221527&#x2F;https:&#x2F;&#x2F;docs.refinery.io&#x2F;getting-started&#x2F;" rel="nofollow">https:&#x2F;&#x2F;web.archive.org&#x2F;web&#x2F;20200819221527&#x2F;https:&#x2F;&#x2F;docs.refi...</a><p>Yahoo Pipes was an inspiration though. It was similar to Heroku in many ways.
ReadCarlBarks10 个月前
It is on GitHub: <a href="https:&#x2F;&#x2F;github.com&#x2F;pipes-digital&#x2F;pipes">https:&#x2F;&#x2F;github.com&#x2F;pipes-digital&#x2F;pipes</a>.
jumploops10 个月前
&gt; The default output format of a pipe is RSS.<p>Do people still use RSS feeds?<p>I work on a similar product[0], conceptually at least, to Pipes, and many of our users have asked for RSS feeds as output.<p>Until now I’ve rationalized that as low priority, but maybe I’m wrong?<p>[0] <a href="https:&#x2F;&#x2F;magicloops.dev">https:&#x2F;&#x2F;magicloops.dev</a>
评论 #40843117 未加载
评论 #40843057 未加载
评论 #40843297 未加载
评论 #40845014 未加载
评论 #40843213 未加载
评论 #40843474 未加载
评论 #40843691 未加载
评论 #40843459 未加载
评论 #40844429 未加载
评论 #40844873 未加载
gabigrin10 个月前
Very cool project. A great demonstration that when you&#x27;re useful you don&#x27;t need a fancy interface!<p>If anyone is interested in a more generic visual-programming language, check out Flyde - <a href="https:&#x2F;&#x2F;www.flyde.dev" rel="nofollow">https:&#x2F;&#x2F;www.flyde.dev</a> - an open-source visual-programming language I&#x27;m working on
BaudouinVH10 个月前
tiny bit of feedback : I tried to sign-in with a non gmail address and did not receive the confirmation email. Switched to a gmail address and everything was smooth in the sign-in process.
jzemeocala10 个月前
I wonder if we could pull off some wackiness with custom RSS feeds. Might have some utility in keeping an LLM constantly up to date as well.
ozten10 个月前
Yahoo pipes was amazing!<p>My vague memory of how this ended last time was processing cost and lack of a business model.
dbacar10 个月前
You know, there is also Apache Nifi.
shouyatf10 个月前
rss-funnel[0] is a self-hosted feed filtering pipeline with similar goal.<p>[0]: <a href="https:&#x2F;&#x2F;github.com&#x2F;shouya&#x2F;rss-funnel">https:&#x2F;&#x2F;github.com&#x2F;shouya&#x2F;rss-funnel</a><p>disclaimer: rss-funnel is my own project.
cranberryturkey10 个月前
Yahoo pipes was awesome. Way ahead of its time. I don’t know why yahoo shut it down.
_gtly10 个月前
I want to support this idea, though If you use their service it&#x27;s free to play around with 3 feeds, but then it gets more pricey. Maybe worth it for some?<a href="https:&#x2F;&#x2F;www.pipes.digital&#x2F;pricing" rel="nofollow">https:&#x2F;&#x2F;www.pipes.digital&#x2F;pricing</a>
评论 #40843410 未加载
nsonha10 个月前
kinda related but does anyone know anything similar to yql, in the sense that you can query everything using the same syntax?
nhggfu10 个月前
the twitter example link goes nowhere.
评论 #40843018 未加载
评论 #40843167 未加载
yalogin10 个月前
I think this has some value potentially now, but they should not use “pipes” to describe it. Yahoo pipes is dead and there is no point associating one self with that brand anymore. The concept is easy enough to explain without using that word.
评论 #40845141 未加载
评论 #40842814 未加载
评论 #40842925 未加载