TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

Show HN: Torch Lens Maker – Differentiable Geometric Optics in PyTorch

179 点作者 fouronnes3大约 2 个月前
Hello HN! For the past 6 months I&#x27;ve been working on an open source python library that implements differentiable geometric optics in PyTorch. It&#x27;s very experimental still, but eventually the goal is to use it to design optical systems with a state of the art optimization framework and a beautiful code based API. Think OpenSCAD, but for optical systems.<p>Not only is PyTorch&#x27;s autograd an amazing general purpose optimizer, but torch.nn (the neural network building blocks) can be used pretty much out of the box to model an optical system. This is because there is a strong analogy to be made between layers of a neural network, and optical elements in a so-called sequential optical system. So the magic is that we can stack lenses as if we were stacking Conv2D and ReLu layers and everything works out. Instead of Conv2D you have ray-surface collision detection, instead of ReLu you have the law of refraction. Designing lenses is surprisingly like training a neural network.<p>Check out the docs for examples of using the API. My favorite one is the rainbow :) <a href="https:&#x2F;&#x2F;victorpoughon.github.io&#x2F;torchlensmaker&#x2F;examples&#x2F;rainbow" rel="nofollow">https:&#x2F;&#x2F;victorpoughon.github.io&#x2F;torchlensmaker&#x2F;examples&#x2F;rain...</a><p>You should be able to `pip install torchlensmaker` to try it out, but I just set it up so let me know if there&#x27;s any trouble.<p>I was part of the Winter 1&#x27;24 batch at the Recurse Center (<a href="https:&#x2F;&#x2F;www.recurse.com&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.recurse.com&#x2F;</a>) working on this project pretty much full time. I&#x27;m happy to talk about that experience too!

23 条评论

etik大约 2 个月前
Great work! Here&#x27;s some prior art in the (torch) space: <a href="https:&#x2F;&#x2F;github.com&#x2F;vccimaging&#x2F;DiffOptics" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;vccimaging&#x2F;DiffOptics</a><p>A few notes, though paraxial approximations are &quot;dumb&quot;, they are very useful tools for lens designers and understanding&#x2F;constraining the design space - calculating the F&#x2F;#, aperture stop, principal planes and is critical in some approaches. This pushes what autodiff tools are capable of because you need to get Hessians of your surface. There&#x27;s also a rich history in objective function definition and quadrature integration techniques thereof which you can work to implement, and you may like to have users be able to specify explicit parametric constraints.
评论 #43436560 未加载
评论 #43537095 未加载
cbarrick大约 2 个月前
Neat!<p>I&#x27;ve been working off and on on a similar hobby project, working through the book _Computational Fourier Optics: A MATLAB Tutorial_, and implementing it in Jax.<p>My main interest is adaptive optics, but I&#x27;m only a hobbyist (limited physics background) and honestly haven&#x27;t had much time to put into it.
评论 #43436919 未加载
评论 #43436579 未加载
skwb大约 2 个月前
I&#x27;m a avid (hobbyist) photographer and I&#x27;ve noticed a TON of genuinely good 3rd party lenses (primarily Sigma and Tamron) and even &#x27;fine&#x27; lenses at rock bottom prices (Viltrox, 7Artisans, TTArtisans, etc) for like $250. The conventional wisdom I&#x27;ve heard is that computer aided design has totally revolutionized this field.<p>I can only hope that projects like these help build better lenses for the future.
mhalle大约 2 个月前
It&#x27;s really awesome that you&#x27;ve taken a widely available tool like PyTorch and used it out of domain to provide a library like this, especially one focused on exact solutions and not approximations.<p>Any plans to include diffractive optics as well? (A totally self-serving question, given that refractive optics is much more common.) In a past life I taught holography and wrote interactive programs to visualize the image forming properties of holograms.
aaclark大约 2 个月前
This is very cool and crosses paths with a few projects I&#x27;ve been working on recently. - implementing a ReLU network in Blender, mostly for visualization - applying the Riemann-Schwarz mapping theorem to discrete radiance fields - solving a spherical-elliptical optics dilemma in perspective projection Your project dovetails spectacularly with this yet you&#x27;ve tackled the core chain of geometry problems &quot;in the opposite direction&quot;. It seems I&#x27;ll have to pick a different thesis topic, but I&#x27;d love to pick your brain about it.
评论 #43439002 未加载
Scipio_Afri大约 2 个月前
Very cool. This is somewhat naive question considering I actually have an EE background, and I think I know the answer but considering their shared EM theory, do you see any parallels of this thinking tangentially applicable to radio frequency system design?
评论 #43436892 未加载
MITSardine大约 2 个月前
Looks very interesting. What would you say are the main technical challenges in the way of your development?<p>By the way, the roadmap link on the Github repo page is broken (though I did find it in the project&#x27;s website).<p>I saw you&#x27;re interested in handling Bezier splines. This might be premature generalization, but would you want to support whole BREPs? Is that something that is used in applications? I imagine lenses are generally simple geometries, so perhaps not.
评论 #43445959 未加载
bee_rider大约 2 个月前
I will ask a dumb question as someone who knows nothing about this stuff (since you already have good questions by smart people):<p>How close is something like this to being competitive with ray-tracing (as featured in video game engines, or as featured in something like Blender)? I guess, since it is using Torch it should be… surprisingly performant, right? You get some hardware acceleration at least.
评论 #43436833 未加载
qoez大约 2 个月前
As an expert in this: What&#x27;s your opinion on using optics like this as actual neural networks? Any big drawbacks or big real benefits
num3ric大约 2 个月前
Potential similarities with Mitsuba&#x27;s inverse rendering functionality?<a href="https:&#x2F;&#x2F;mitsuba.readthedocs.io&#x2F;en&#x2F;stable&#x2F;src&#x2F;inverse_rendering&#x2F;caustics_optimization.html" rel="nofollow">https:&#x2F;&#x2F;mitsuba.readthedocs.io&#x2F;en&#x2F;stable&#x2F;src&#x2F;inverse_renderi...</a>
pixelpoet大约 2 个月前
Surprised no one has mentioned Mitsuba renderer, in particular the caustic design demo: <a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=eTHL3W2NUn0&amp;list=PLI9y-85z_Po6da-pyTNGTns2n4fhpbLe5&amp;index=4" rel="nofollow">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=eTHL3W2NUn0&amp;list=PLI9y-85z_P...</a>
makizar大约 2 个月前
Could you ELI5 what the applications would be ? Could a render engine be built on top of this and hooked up to a DCC like Blender ? Or is this a way to do computational photography, say correct the depth of field of an image of &quot;denoise&quot; it ?
评论 #43438991 未加载
isgb大约 2 个月前
Is there any way to simulate (maybe even interactively) things like focus and zoom? It would be cool to have some way to shift lenses (or lens groups) along the optical axis and visualize how light rays get projected onto the image plane.
评论 #43437928 未加载
viraptor大约 2 个月前
What would this be used for in practice? I understand what it does, but have little experience in the area and thought we know what shapes we need for almost all applications. Who would go as far as a complete shape design?
RobotToaster大约 2 个月前
This looks really great.<p>Do you have any plans to add stock lens catalogue matching? To make it easier for hobbyists to manufacture lens assemblies.
评论 #43436581 未加载
turnsout大约 2 个月前
Really cool! I haven’t peered into the internals yet, so forgive the ignorant question: are the calculations spectral?
评论 #43441516 未加载
Evidlo大约 2 个月前
Very cool. Do you thinking supporting diffractive optics is possible, or too much of a deviation?
TeeMassive大约 2 个月前
I wonder if anyone tried making lens out of its outputs using transparent 3D printer resin?
评论 #43441626 未加载
gtsnexp大约 2 个月前
How far are we from completely replacing Zemax?
评论 #43437743 未加载
stormfather大约 2 个月前
Oh fuck me. Can you use this to make an analog transformer out of tiny lenses that&#x27;s pre-trained? Like, take a digital PyTorch transformer model and spec out the lenses to recreate the computations? And build that? If so, you will cover yourself in glory. Intelligence too cheap to meter.
guy234大约 2 个月前
it begs the question of using optics for ML?
评论 #43435900 未加载
GistNoesis大约 2 个月前
Will there be some diffraction optic in the future, can we just add some phase somewhere or will it need a complete rewrite ? I&#x27;d like to experiment with photon sieves and holograms.
mentalgear大约 2 个月前
Very innovative application of NN architecture in a different (physics&#x2F;optics) domain !<p>&gt; The key idea is that there is a strong analogy to be made between layers of a neural network, and optical elements in a so-called sequential optical system. If we have a compound optical system made of a series of lenses, mirrors, etc., we can treat each optical element as the layer of a neural network. The data flowing through this network are not images, sounds, or text, but rays of light. Each layer affects light rays depending on its internal parameters (surface shape, refractive material...) and following the very much non‑linear Snell&#x27;s law. Inference, or the forward model, is the optical simulation where given some input light, we compute the system&#x27;s output light. Training, or optimization, is finding the best shapes for lenses to focus light where we want it.
评论 #43435842 未加载