TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

Tiny Pointers

163 点作者 levzettelin3 个月前

7 条评论

judofyr3 个月前
I looked a bit into this a few years back and found it quite interesting. Despite them calling them &quot;Tiny Pointers&quot; I would say it&#x27;s closer to a open addressing hash map. You have a specific key, and then you can &quot;allocate&quot; an entry in the hash map. This gives you back a &quot;pointer&quot;. You can then later use the original key <i>and</i> the pointer together to determine the index of the entry. There&#x27;s also a slight chance that the allocation will fail (similar to a hash collision in a hash map). The &quot;trick&quot; here is that two different keys can end up having the <i>exact</i> same pointer (because we&#x27;re always dereferencing with the key). This makes them more compact.<p>I was struggling a bit to come up with good use cases for it. Their examples are all around combining them with existing data structures and they show that the space complexity is smaller, but it wasn&#x27;t completely clear to me how feasible this would actually be in practice.
评论 #43027057 未加载
评论 #43029246 未加载
评论 #43027384 未加载
评论 #43031048 未加载
mont_tag3 个月前
Didn&#x27;t Python&#x27;s compact dictionary implementation do this a decade ago?<p>&quot;The dict type now uses a “compact” representation based on a proposal by Raymond Hettinger which was first implemented by PyPy. The memory usage of the new dict() is between 20% and 25% smaller compared to Python 3.5.&quot; -- <a href="https:&#x2F;&#x2F;docs.python.org&#x2F;3.6&#x2F;whatsnew&#x2F;3.6.html#whatsnew36-compactdict" rel="nofollow">https:&#x2F;&#x2F;docs.python.org&#x2F;3.6&#x2F;whatsnew&#x2F;3.6.html#whatsnew36-com...</a><p>&quot;Note, the sizeof(index) can be as small as a single byte for small dicts, two bytes for bigger dicts and up to sizeof(Py_ssize_t) for huge dict.&quot; -- <a href="https:&#x2F;&#x2F;mail.python.org&#x2F;pipermail&#x2F;python-dev&#x2F;2012-December&#x2F;123028.html" rel="nofollow">https:&#x2F;&#x2F;mail.python.org&#x2F;pipermail&#x2F;python-dev&#x2F;2012-December&#x2F;1...</a><p>The &quot;tiny pointers&quot; are in the _make_index method in the proof of concept code. -- <a href="https:&#x2F;&#x2F;code.activestate.com&#x2F;recipes&#x2F;578375-proof-of-concept-for-a-more-space-efficient-faster&#x2F;" rel="nofollow">https:&#x2F;&#x2F;code.activestate.com&#x2F;recipes&#x2F;578375-proof-of-concept...</a><p><pre><code> @staticmethod def _make_index(n): &#x27;New sequence of indices using the smallest possible datatype&#x27; if n &lt;= 2**7: return array.array(&#x27;b&#x27;, [FREE]) * n # signed char if n &lt;= 2**15: return array.array(&#x27;h&#x27;, [FREE]) * n # signed short if n &lt;= 2**31: return array.array(&#x27;l&#x27;, [FREE]) * n # signed long return [FREE] * n </code></pre> The logic is still present today in CPython. -- <a href="https:&#x2F;&#x2F;raw.githubusercontent.com&#x2F;python&#x2F;cpython&#x2F;3e222e3a15959690a41847a1177ac424427815e5&#x2F;Objects&#x2F;dictobject.c" rel="nofollow">https:&#x2F;&#x2F;raw.githubusercontent.com&#x2F;python&#x2F;cpython&#x2F;3e222e3a159...</a><p><pre><code> dk_indices is actual hashtable. It holds index in entries, or DKIX_EMPTY(-1) or DKIX_DUMMY(-2). Size of indices is dk_size. Type of each index in indices varies with dk_size: * int8 for dk_size &lt;= 128 * int16 for 256 &lt;= dk_size &lt;= 2**15 * int32 for 2**16 &lt;= dk_size &lt;= 2**31 * int64 for 2**32 &lt;= dk_size</code></pre>
评论 #43027022 未加载
评论 #43026809 未加载
评论 #43026770 未加载
kragen3 个月前
Note that this is not the paper by Krapivin that <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=43002511">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=43002511</a> is about.
评论 #43027310 未加载
kazinator3 个月前
&gt; <i>How large do the pointers need to be? The natural answer is that each pointer uses log nbits. However, the fact that each pointer has a distinct owner makes it possible to compress the pointers to o(log n) bits.</i><p>What if you have to <i>debug</i> the whole situation, such that you don&#x27;t always know who is the owner of a pointer you are looking at?<p>&gt; <i>A user k can call Allocate(k) in order to get a tiny pointer p; they can dereference the tiny pointer pby computing a function Dereference(k,p) whose value depends only on k, p, and random bits; and they can free a tiny pointer p by calling a function Free(k,p).</i><p>That is tantamount ot saying that the pointer is not actually <i>p</i> but the tuple <i>&lt;k, p&gt;</i>, and so its size consists of the number of bits in <i>k</i>, the number of bits in <i>p</i> plus an indication of where the division between these bits lie: where <i>k</i> ends and <i>p</i> begins.<p>We can abbreviate <i>&lt;k, p&gt;</i> to <i>p</i> in contexts where <i>k</i> can be implicitly understood.
BenoitEssiambre3 个月前
Here&#x27;s a somewhat related post where I argue that logic that _can_ have small pointers has lower entropy and is more optimal as bayesian model of the domain: <a href="https:&#x2F;&#x2F;benoitessiambre.com&#x2F;entropy.html" rel="nofollow">https:&#x2F;&#x2F;benoitessiambre.com&#x2F;entropy.html</a>
levzettelin3 个月前
Can someone ELI5?
评论 #43026376 未加载
评论 #43026198 未加载
评论 #43028007 未加载
kittikitti3 个月前
Is there a peer reviewed version of this or does Hacker News exclusively post non peer reviewed works?
评论 #43029373 未加载
评论 #43029579 未加载