TE
TechEcho
Home24h TopNewestBestAskShowJobs
GitHubTwitter
Home

TechEcho

A tech news platform built with Next.js, providing global tech news and discussions.

GitHubTwitter

Home

HomeNewestBestAskShowJobs

Resources

HackerNews APIOriginal HackerNewsNext.js

© 2025 TechEcho. All rights reserved.

What are the lesser known but useful data structures?

347 pointsby mck-over 11 years ago

22 comments

tikhonjover 11 years ago
There&#x27;s a whole set of interesting data structures that are not very well known: succinct data structures[1]. The idea is simple: we want to store data in a compressed form, but also perform certain operations quickly without uncompressing.<p>These can be very useful for certain applications. The article on &quot;Cramming 80,000 Words into a JavaScript File&quot;[2] is a nice example. It shows you how you can store a compressed trie in memory but still use it. I also like this[3] series of blog posts leading up to wavelet trees.<p>These certainly count as obscure data structures, unlike many of the ones listed on SO. I had never even considered the <i>idea</i> of compressing data in memory like this, much less encountered actual examples of succinct data structures! I have to thank Edward Kmett for introducing me to the whole field.<p>These data structures are important not just because they&#x27;re neat themselves, but because they got me to think a new way. In particular, I realized that using pointers all over the place--to represent things like trees--is not always efficient. Instead of parsing data, it might be better to store it as a blob of some sort with a binary index. Just starting to consider details like that is valuable all on its own.<p>[1]: <a href="http://en.wikipedia.org/wiki/Succinct_data_structure" rel="nofollow">http:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Succinct_data_structure</a><p>[2]: <a href="http://stevehanov.ca/blog/index.php/?id=120" rel="nofollow">http:&#x2F;&#x2F;stevehanov.ca&#x2F;blog&#x2F;index.php&#x2F;?id=120</a><p>[3]: <a href="http://alexbowe.com/rrr/" rel="nofollow">http:&#x2F;&#x2F;alexbowe.com&#x2F;rrr&#x2F;</a> and <a href="http://alexbowe.com/wavelet-trees/" rel="nofollow">http:&#x2F;&#x2F;alexbowe.com&#x2F;wavelet-trees&#x2F;</a>
评论 #7079933 未加载
评论 #7080668 未加载
评论 #7080666 未加载
kintamanimattover 11 years ago
&gt; This question exists because it has historical significance, but <i>it is not considered a good, on-topic question for this site</i>, so please do not use it as evidence that you can ask similar questions here.<p>Yet it&#x27;s one of the best questions on SO. Something&#x27;s very wrong with SO if this isn&#x27;t considered a good, on-topic question for a programming Q&amp;A site.
评论 #7079944 未加载
teddyhover 11 years ago
Even though they are included in the GNU C library, most people do not seem to know about Obstacks:<p><i>An &quot;obstack&quot; is a pool of memory containing a stack of objects. You can create any number of separate obstacks, and then allocate objects in specified obstacks. Within each obstack, the last object allocated must always be the first one freed, but distinct obstacks are independent of each other.<p>Aside from this one constraint of order of freeing, obstacks are totally general: an obstack can contain any number of objects of any size. They are implemented with macros, so allocation is usually very fast as long as the objects are usually small. And the only space overhead per object is the padding needed to start each object on a suitable boundary.</i><p><a href="https://www.gnu.org/software/libc/manual/html_node/Obstacks.html" rel="nofollow">https:&#x2F;&#x2F;www.gnu.org&#x2F;software&#x2F;libc&#x2F;manual&#x2F;html_node&#x2F;Obstacks....</a><p>Sure, they’re not very <i>interesting</i>, but the point is that you get them <i>for free</i> in the GNU C standard library.
评论 #7096615 未加载
bazzarghover 11 years ago
It&#x27;s not highlighting one thing, but Chris Okasaki&#x27;s book on Purely Functional Data Structures, and this brilliant top answer to a question about functional data structures published since the book will keep you in reading material for a while: <a href="http://cstheory.stackexchange.com/a/1550" rel="nofollow">http:&#x2F;&#x2F;cstheory.stackexchange.com&#x2F;a&#x2F;1550</a><p>(it was all &#x27;lesser known&#x27; to me when I started using haskell not so long ago)
评论 #7081888 未加载
batbombover 11 years ago
I use a Hierarchical Triangular Mesh for indexing gamma ray events from the universe. The data is partitioned in the database according to it&#x27;s HTM id.<p><a href="http://arxiv.org/pdf/cs/0701164.pdf" rel="nofollow">http:&#x2F;&#x2F;arxiv.org&#x2F;pdf&#x2F;cs&#x2F;0701164.pdf</a><p>Currently I use this for indexing ~11 billion gamma ray events. Researchers typically supply a region in the sky, a search radius, and some cuts (energy, event quality, etc...)
评论 #7080100 未加载
hyperpapeover 11 years ago
Looking at these lists, I strongly suspect that people upvote based on whether they personally recognize the data structure.<p>It goes against the intent of the original question, but iIt&#x27;s almost ideally designed to make you feel good--you get the rush of knowledge then nerd sniped as you head to wikipedia.
评论 #7079753 未加载
VexXtremeover 11 years ago
I love how the question was locked because &quot;it is not considered a good, on-topic question for this site&quot;. It&#x27;s crazy. Unless an extremely specific concrete answer can be given, a question immediately gets killed. SO has turned into such a turd of a website.
评论 #7080762 未加载
评论 #7080850 未加载
评论 #7081747 未加载
评论 #7082226 未加载
评论 #7080625 未加载
chasover 11 years ago
I&#x27;m happy to see finger trees got mentioned. Finger trees[0] are extremely useful and general data structure that can be used to implement persistent sequences, priority queues, search trees and priority search queues. (Haskell&#x27;s Data.Sequence[1] uses specialized 2-3 finger trees internally) They can form the basis of all sorts of interesting custom structures by supplying the appropriate monoid[3], but this does make them harder to approach if you are not familiar with the abstractions.<p>[3] A monoid is any structure that has members that can combine associatively. In addition, it must have an element that can combine with any other element and result in the other element. Some examples: (strings, string concatenation, the empty string); (integers, addition, 0); (natural numbers, max, 0); (booleans, and, True); (functions, composition, the identity function). The functional pearl[2] that describes the design of Haskell&#x27;s diagrams library[4] goes into much more detail if you are interested in their application to programming.<p>[0] <a href="http://apfelmus.nfshost.com/articles/monoid-fingertree.html" rel="nofollow">http:&#x2F;&#x2F;apfelmus.nfshost.com&#x2F;articles&#x2F;monoid-fingertree.html</a><p>[1] <a href="http://hackage.haskell.org/package/containers-0.5.4.0/docs/Data-Sequence.html" rel="nofollow">http:&#x2F;&#x2F;hackage.haskell.org&#x2F;package&#x2F;containers-0.5.4.0&#x2F;docs&#x2F;D...</a><p>[2] <a href="http://www.cis.upenn.edu/~byorgey/pub/monoid-pearl.pdf" rel="nofollow">http:&#x2F;&#x2F;www.cis.upenn.edu&#x2F;~byorgey&#x2F;pub&#x2F;monoid-pearl.pdf</a><p>[4] <a href="http://projects.haskell.org/diagrams/" rel="nofollow">http:&#x2F;&#x2F;projects.haskell.org&#x2F;diagrams&#x2F;</a>
评论 #7081924 未加载
jbogganover 11 years ago
Bloom filters and count-min sketch are awesome.<p><a href="http://en.wikipedia.org/wiki/Bloom_filter" rel="nofollow">http:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Bloom_filter</a> <a href="http://en.wikipedia.org/wiki/Count%E2%80%93min_sketch" rel="nofollow">http:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Count%E2%80%93min_sketch</a>
nilknover 11 years ago
I don&#x27;t really consider tries and bloom filters all that poorly known. These commonly come up in interviews for fresh graduates at Google&#x2F;Facebook. Zippers, skip lists, ropes, round-robin databases, etc. are more genuinely not known I think.
abcd_fover 11 years ago
XOR linked list: <a href="http://en.wikipedia.org/wiki/XOR_linked_list" rel="nofollow">http:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;XOR_linked_list</a><p>It&#x27;s a double-linked list with just one link per node. However, to start traversing it you have to know at least two adjacent nodes.<p>PS. May not be <i>useful</i> per se, but interesting nonetheless.
评论 #7081512 未加载
krisgeeover 11 years ago
I was going to say Trie but it was the first response to the SO thread so I guess it wasn&#x27;t as little known as I thought.<p>I implemented it because I was making a game that had scrabble elements in it and needed to check ahead to see if the player had a word that could still take letters (a prefix) or if they&#x27;d hit a dead end. Fit the whole SOWPODS into a remarkably tiny space with millisecond lookups. Probably my favourite part of the project.
lifthrasiirover 11 years ago
I found the following page in the Concatenative wiki particularly interesting: <a href="http://concatenative.org/wiki/view/Exotic%20Data%20Structures" rel="nofollow">http:&#x2F;&#x2F;concatenative.org&#x2F;wiki&#x2F;view&#x2F;Exotic%20Data%20Structure...</a> (Note that the page itself is not related to the concatenative languages.)
nlover 11 years ago
Bloom Filters: <a href="http://en.wikipedia.org/wiki/Bloom_filter" rel="nofollow">http:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Bloom_filter</a><p>Hamming Codes: <a href="http://en.wikipedia.org/wiki/Hamming_code" rel="nofollow">http:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Hamming_code</a>
ww520over 11 years ago
Extendible hashing is amazing in space utilization while retaining the performance of hashing.
shurcooLover 11 years ago
Does anyone know of an implementation of rope in golang? Something a little more feature complete than <a href="https://github.com/christianvozar/rope" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;christianvozar&#x2F;rope</a>.
swahover 11 years ago
Let me just drop this video that is on my watchlist: <a href="http://www.youtube.com/watch?v=-sEdiFMntMA&amp;feature=share&amp;list=PLFDnELG9dpVxEpbyL53CYebmLI58qJhlt" rel="nofollow">http:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=-sEdiFMntMA&amp;feature=share&amp;lis...</a> (Erik Dermaine is the lecturer)
pnathanover 11 years ago
I recently learned about the spatial index tree family in connection with data mining. I hope to implement a data-mining centric X tree (n-dimensional) solution for a data analytics package I&#x27;m writing soon. That family is is how you efficiently handle KNN lookups, afaict.
WWKongover 11 years ago
Me and my friend were pretty serious about creating a new data structure called &quot;drum&quot;. A drum is a one way store. You write to it but can&#x27;t read from it. We put it off till we figured a practical use.
评论 #7080644 未加载
评论 #7083884 未加载
评论 #7080500 未加载
keefeover 11 years ago
imho most DS &amp; Algo are like kihon - the basic principles t&#x27;werk just fine, it&#x27;s about augments and applications.
halfdeadcatover 11 years ago
Cup-a-bits
serge2kover 11 years ago
It&#x27;s mentioned in the link, but circular&#x2F;ring buffers.<p>I&#x27;ve been grappling with decoding&#x2F;playing back an audio stream and wouldn&#x27;t have gotten it working if I hadn&#x27;t found out about boosts lockfree ring buffer.
评论 #7083507 未加载