TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

Discovering algorithms by enumerating terms in Haskell

143 点作者 agomez3149 个月前

16 条评论

bmc75059 个月前
Maciej Bendkowski has some related work [1] on generating random lambda terms, but was unable to overcome what he calls the asymptotic sparsity problem:<p><pre><code> Sampling simply-typed terms seems notoriously more challenging than sampling closed ones. Even rejection sampling, whenever applicable, admits serious limitations due to the imminent asymptotic sparsity problem — asymptotically almost no term, be it either plain or closed, is at the same time (simply) typeable. [...] Asymptotic sparsity of simply-typed λ-terms is an impenetrable barrier to rejection sampling techniques. As the term size tends to infinity, so does the induced rejection overhead. In order to postpone this inevitable obstacle, it is possible to use dedicated mechanisms interrupting the sampler as soon as it is clear that the partially generated term cannot be extended to a typeable one. The current state-of-the-art samplers take this approach, combining Boltzmann models with modern logic programming execution engines backed by highly-optimised unification algorithms. Nonetheless, even with these sophisticated optimisations, such samplers are not likely to generate terms of sizes larger than one hundred. </code></pre> I would be curious to see a more rigorous analysis of the sample complexity of generating well-typed expressions in, e.g., the STLC. Maybe there is a way to avoid or reduce the rejection rate before evaluation.<p>[1]: <a href="https:&#x2F;&#x2F;arxiv.org&#x2F;pdf&#x2F;2005.08856" rel="nofollow">https:&#x2F;&#x2F;arxiv.org&#x2F;pdf&#x2F;2005.08856</a>
评论 #41149270 未加载
评论 #41150430 未加载
grnnja9 个月前
What the author is getting at is a pretty cool research area called program synthesis, where the goal is to create a program that satisfies a specification.<p>Most techniques are essentially brute force enumeration with tricks to improve performance, so they tend to struggle to find larger programs. A lot of active research is in improving performance.<p>Compared to asking a LLM to write a program, program synthesis approaches will guarantee that a solution will satisfy the specification which can be very powerful.<p>In particular, as the author has discovered, one area where program synthesis excels is finding small intricate bitwise operator heavy programs that can be hard to reason about as a human.<p>The most famous example of program synthesis is Microsoft&#x27;s FlashFill, which is used in Excel. You give it a few input output examples and FlashFill will try to create a small program to generalize them, and you can apply the program to more inputs, which saves you a bunch of time. An example from the paper is:<p><pre><code> Input -&gt; Output International Business Machines -&gt; IBM Principles Of Programming Languages -&gt; POPL International Conference on Software Engineering -&gt; ICSE String Program: Loop(\w : Concatenate(SubStr2(v_1, UpperTok, w))) </code></pre> Here&#x27;s a few papers:<p>EUSOLVER: <a href="https:&#x2F;&#x2F;www.cis.upenn.edu&#x2F;~alur&#x2F;Tacas17.pdf" rel="nofollow">https:&#x2F;&#x2F;www.cis.upenn.edu&#x2F;~alur&#x2F;Tacas17.pdf</a><p>FlashFill: <a href="https:&#x2F;&#x2F;www.microsoft.com&#x2F;en-us&#x2F;research&#x2F;wp-content&#x2F;uploads&#x2F;2016&#x2F;12&#x2F;popl11-synthesis.pdf" rel="nofollow">https:&#x2F;&#x2F;www.microsoft.com&#x2F;en-us&#x2F;research&#x2F;wp-content&#x2F;uploads&#x2F;...</a><p>BlinkFill: <a href="https:&#x2F;&#x2F;www.vldb.org&#x2F;pvldb&#x2F;vol9&#x2F;p816-singh.pdf" rel="nofollow">https:&#x2F;&#x2F;www.vldb.org&#x2F;pvldb&#x2F;vol9&#x2F;p816-singh.pdf</a><p>Synquid: <a href="https:&#x2F;&#x2F;cseweb.ucsd.edu&#x2F;~npolikarpova&#x2F;publications&#x2F;pldi16.pdf" rel="nofollow">https:&#x2F;&#x2F;cseweb.ucsd.edu&#x2F;~npolikarpova&#x2F;publications&#x2F;pldi16.pd...</a>
评论 #41150152 未加载
评论 #41150197 未加载
评论 #41150765 未加载
dleslie9 个月前
Might help to read some of the author&#x27;s earlier work, like:<p><a href="https:&#x2F;&#x2F;medium.com&#x2F;@maiavictor&#x2F;solving-the-mystery-behind-abstract-algorithms-magical-optimizations-144225164b07" rel="nofollow">https:&#x2F;&#x2F;medium.com&#x2F;@maiavictor&#x2F;solving-the-mystery-behind-ab...</a><p>And the Omega documentation:<p><a href="https:&#x2F;&#x2F;hackage.haskell.org&#x2F;package&#x2F;control-monad-omega-0.3&#x2F;docs&#x2F;Control-Monad-Omega.html" rel="nofollow">https:&#x2F;&#x2F;hackage.haskell.org&#x2F;package&#x2F;control-monad-omega-0.3&#x2F;...</a><p>But I&#x27;ll admit, I want to see a paper or code.
kccqzy9 个月前
I don&#x27;t think the original submission has enough details for us to reproduce or even understand what&#x27;s being done. Omega monad is just a diagonal search monad that supports infinity. I don&#x27;t understand the syntax in the screenshot. What&#x27;s the type of the terms being enumerated? I can see lambda, but what&#x27;s @inc or the pluses and minuses?
评论 #41152715 未加载
mordechai90009 个月前
&quot;Perhaps I&#x27;m being really naive and an exponential wall is about to humble (and embarrass) me.&quot;<p>There is something fascinating and awe-inspiring about searching very large, enumerative datasets for useful results. Like wandering Borge&#x27;s Library of Babel in search of not just meaning but something true and useful.
diwank9 个月前
More on the Omega monad from its author:<p><a href="http:&#x2F;&#x2F;web.archive.org&#x2F;web&#x2F;20210616105135&#x2F;http:&#x2F;&#x2F;lukepalmer.wordpress.com&#x2F;2008&#x2F;05&#x2F;02&#x2F;enumerating-a-context-free-language&#x2F;" rel="nofollow">http:&#x2F;&#x2F;web.archive.org&#x2F;web&#x2F;20210616105135&#x2F;http:&#x2F;&#x2F;lukepalmer....</a>
tadfisher9 个月前
I&#x27;m confused, what is the search space? All possible functions from Int -&gt; Int? And how do you verify a result is optimal across inputs?
评论 #41149584 未加载
评论 #41149255 未加载
LightMachine9 个月前
Uhm author here. Not sure why this tweet is on Hacker News, as it is just a non-technical &quot;blog post&quot;. But I&#x27;ve posted a follow-up today with some code and details, if you&#x27;re curious:<p><a href="https:&#x2F;&#x2F;x.com&#x2F;VictorTaelin&#x2F;status&#x2F;1819774880130158663" rel="nofollow">https:&#x2F;&#x2F;x.com&#x2F;VictorTaelin&#x2F;status&#x2F;1819774880130158663</a><p>That&#x27;s as much as I can share for now though
评论 #41149128 未加载
评论 #41150129 未加载
cvoss9 个月前
Did you know that there is an efficient algorithm for learning (guessing) a regular language from a series of examples&#x2F;non-examples? Basically, you directly construct the smallest automaton that correctly discriminates the training inputs. It&#x27;s Dana Angluin&#x27;s L* algorithm.<p>The technique generalizes to finite tree automata, meaning you can discover generators of simple expression grammars, given examples&#x2F;non-examples.<p>So if you can construe your problem as a labeled tree recognition problem, assuming it&#x27;s solvable using a finite tree automaton, then you can discover the algorithm for it efficiently.<p>For example, if you have three strings of bits (2 inputs and 1 output) and line them up as a single string of triplets of bits, it does not surprise me that it is easy to discover the automaton state transition rules that give you the carry bit for each triple and tell you when to reject the triple because the output bit is not correct for the two input bits.<p>The author has arranged the problem this way when sketching the ADC template and has also jump-started the search by assuming the solution space includes exactly one output bit for each pair of input bits. (That may seem like an obvious necessity, but that is a constraint which is not required by the tree automaton formulation, which need not differentiate &quot;inputs&quot; and &quot;outputs&quot;.)
treyd9 个月前
I&#x27;m confused about what this could be, with a title as technical as this but being posted somewhere with the signal:noise of Twitter, also where I can&#x27;t view it.
评论 #41148479 未加载
sporkl9 个月前
This kind of reminds me of what happens when you implement an interpreter in a relational programming language, which lets you do cool stuff like generating quines by specifying that the program and it’s output should be the same.<p>Quick search lead to this paper which is exactly what I’m talking about: <a href="http:&#x2F;&#x2F;webyrd.net&#x2F;quines&#x2F;quines.pdf" rel="nofollow">http:&#x2F;&#x2F;webyrd.net&#x2F;quines&#x2F;quines.pdf</a>
评论 #41150114 未加载
verdverm9 个月前
Super interesting. Sounds like the algo I created Prioritized Grammar Enumeration.<p><a href="https:&#x2F;&#x2F;seminars.math.binghamton.edu&#x2F;ComboSem&#x2F;worm-chiu.pge_gecco2013.pdf" rel="nofollow">https:&#x2F;&#x2F;seminars.math.binghamton.edu&#x2F;ComboSem&#x2F;worm-chiu.pge_...</a><p>PGE is essentially a BFS&#x2F;DFS traversal of the space of all formulas, by using local enumeration of the AST. The biggest gains where from eliminating duplicate work (commutativity &#x2F; associativity) and not going down bad branches (too much complexity added for no meaningful change to output). A lot of overlap in ideas here, and a lot of open research questions that could be worked on (like can use use RL to help guide the search like A*). There&#x27;s definitely an exponential explosion or wall as the AST gets wider &#x2F; deeper.<p>At one point I wrote the core algorithm in Haskell, which made it so much more concise and beautiful, but eventually landed on python <a href="https:&#x2F;&#x2F;github.com&#x2F;verdverm&#x2F;pypge">https:&#x2F;&#x2F;github.com&#x2F;verdverm&#x2F;pypge</a><p>In all of Genetic Programming &#x2F; Symbolic Regression, everyone starts by trying to generate computer code and then switches to just math formula. They are different classes of problems because code has more &quot;genes&quot; and is order sensitive, whereas math is not
garyrob9 个月前
Maybe this could be used with the Evolutionary Algorithm subset known as Genetic Programming, originally popularized by John Koza in the 80s?
anamax9 个月前
Doug Lenat&#x27;s AM (<a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Automated_Mathematician" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Automated_Mathematician</a>) seems relevant.
rowanG0779 个月前
My intuition tells me you will quickly run out of memory on anything that isn&#x27;t just a small function. Your state space will just be so big that your super positions will become too large.
jmull9 个月前
It&#x27;s kind of vague, but... it sounds like this is essentially a breadth first search (where the graph being searched is represented by a grammar)?<p>Hard to tell.