TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

Faster Parallel Python Without Python Multiprocessing

93 点作者 Cyranix将近 6 年前

4 条评论

heyflyguy将近 6 年前
I am a terrible python scriptwriter. Still, I use my own scripts that I have written through trial and error and reading lots of stack overflow. I process about 500 images per day and each one takes about 30 seconds each. Adrian Rosenbrock has been a real lifesaver. I have a machine dedicated to this. I tried using multiprocessing once and could not get it to work. Being able to process in a parallel fashion would be a gamechanger for me.<p>The beauty of python for someone like me is that I can get my job done without actually having to do it. I free up my own time to leverage more of my creativity and have a multiplicative effect on my productivity.<p>The reason I bring all of this up is that so many of the examples for advanced libraries I see are geared towards seasoned software engineers. The examples include false arrays of data so that you can &quot;just see&quot; how to use it. I don&#x27;t think anyone realizes how confusing this is to the guy who is a restaurant manager, or the gal who is a researcher that just needs to know how to make this work for them.<p>If it&#x27;s an image, how about putting image = cv2.imread(&quot;C:\image.jpg&quot;) or whatever?<p>Anyway, a bit of a rant but there are people who are very thankful that smart people in this world like yourself write libraries we can use to make our daily lives better. Including example code that is stupid simple would make me so much happier.
mike_mg将近 6 年前
While I appreciate the efforts of authors and believe in long term mission, they seem to not mention anywhere some key shortcomings of Ray, while marketing it pretty hard (eg see the paper).<p>I have used ray (a year ago) in one of the advertised basic applications: parallelising the environments for RL. It was unusable back then, as it was clogging up the memory.<p>The plasma store which is backend for arrow was never cleaned which made the computation stop after 3 hours<p>Here’s the issue:<p><a href="https:&#x2F;&#x2F;github.com&#x2F;ray-project&#x2F;ray&#x2F;issues&#x2F;2128" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;ray-project&#x2F;ray&#x2F;issues&#x2F;2128</a><p>Or perhaps this has been fixed already?
htfy96将近 6 年前
Every time I see a performance issue&#x2F;solution of Python, I was wondering why there is no company maintaining a distribution with a high-performance Python JIT compiler with patched, GIL-free packages. Given the prevalence of Python and what have succeeded in JVM, it seems like a fruitful business.
评论 #20066877 未加载
评论 #20066636 未加载
评论 #20067148 未加载
评论 #20067164 未加载
评论 #20066858 未加载
tech_tuna将近 6 年前
I love Python but they best way to do do parallel Python is to use Go.