Tagger News takes a subset of HN articles, analyzes using ML, and applies tags

Direct URL to project details: <a href="https:&#x2F;&#x2F;devpost.com&#x2F;software&#x2F;tagger-news" rel="nofollow">https:&#x2F;&#x2F;devpost.com&#x2F;software&#x2F;tagger-news</a><p>A few comments:<p>1. To other commenters, as with the HN Vue demo a week ago (<a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=14284877" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=14284877</a>), the project is a technical proof-of-concept; the aesthetics aren&#x27;t the primary focus.<p>2. The Algolia API is better for scraping because it allows for bulk requests, unlike the official API (my old 2014 script still works I think: <a href="https:&#x2F;&#x2F;github.com&#x2F;minimaxir&#x2F;get-all-hacker-news-submissions-comments" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;minimaxir&#x2F;get-all-hacker-news-submissions...</a>)<p>3) How much time did it take to manually label the training&#x2F;test set before training the RF classifier? Even with topic modeling for <i>extrapolating</i> tags, accurate labeling for 20,000 submissions is a task.

I think the biggest value proposition of this is the ability to do sub-reddit like filtering on specific tags. As Hacker News grows I think dealing with the number of new submissions would become a bottleneck. During high-traffic times, new submissions sometimes drop of the first &#x27;new&#x27; page in around 10 minutes. Of course there is more traffic during these times to upvote good content, but I&#x27;m not sure that is better than letting a smaller number of people have a longer period of time to filter a smaller collection of content.

Strength of Hacker News is the network effect of a diverse, intelligent crowd. It would be hard to replace. A supplemental site tagging it and aiding search has value. Biggest problem I have searching HN, though, is Google mixing up stories and comments. The fix might be as simple as two domains that contain stories and comments cloned from HN, one domain for each, followed by Google Searches within those domains. Not sure if Google would automatically crawl it, though.

Congratulations to the team, although it seems the algorithm isn&#x27;t very accurate, since this article[0] from 1997 was tagged with &quot;Blockchain&quot;.<p>[0]: <a href="https:&#x2F;&#x2F;www.gnu.org&#x2F;philosophy&#x2F;right-to-read.html?source=techstories.org" rel="nofollow">https:&#x2F;&#x2F;www.gnu.org&#x2F;philosophy&#x2F;right-to-read.html?source=tec...</a>

I did similar project few months ago, it does automatic tagging + summarization of HN largely using scipy and numpy, you can see it in action: <a href="http:&#x2F;&#x2F;hntop.org" rel="nofollow">http:&#x2F;&#x2F;hntop.org</a> here github link <a href="https:&#x2F;&#x2F;github.com&#x2F;bexp&#x2F;textai" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;bexp&#x2F;textai</a>

Not to hijack, but this is similar to a small ML project a friend and I built. It takes news headlines from a bunch of sources and classifies them by common topic. We took a lot longer than a day to build it, though. ;)<p>It refreshes with new stories every few hours. You can check it out here: <a href="http:&#x2F;&#x2F;headlinr.herokuapp.com&#x2F;" rel="nofollow">http:&#x2F;&#x2F;headlinr.herokuapp.com&#x2F;</a><p>EDIT: click on the bubbles to see individual headlines. Also, here&#x27;s the GitHub page: <a href="https:&#x2F;&#x2F;github.com&#x2F;dgarrick&#x2F;headliner" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;dgarrick&#x2F;headliner</a>

Personally, I find the visual weight that the tags have exceed their value to me.

I think this is great, especially being able to click a tag and see a top 30 list of that tag.<p>Obvious suggestions that would make it usable as a primary HN interface:<p>• Login and voting (not sure the HN API supports this though)<p>• Tag suggestions to feed into the model

Link to try it out <a href="http:&#x2F;&#x2F;www.taggernews.com&#x2F;" rel="nofollow">http:&#x2F;&#x2F;www.taggernews.com&#x2F;</a>

Here is my take on it few years ago. I tried to make it more like magazine, and get article text and photo, and there is a section with only articles that reached top position. <a href="http:&#x2F;&#x2F;www.hnzine.com" rel="nofollow">http:&#x2F;&#x2F;www.hnzine.com</a>

Possibly off-topic, but: this was done at the &quot;Disrupt NYC&quot; hackathon, and somehow &quot;add tags to HN&quot; feels like about the least disruptive thing ever.

Wow I was thinking to make something similar: an experimental HN fork where submissions are tagged (collaboratively) <i>but</i> without titles, as these are rarely useful to predict the content of an article. And of course there is also the convenience of categorization.

This article on book publishing: <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=14334845" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=14334845</a> is tagged with &quot;blockchain&quot; only. Any idea why?

Tried <a href="http:&#x2F;&#x2F;www.taggernews.com&#x2F;tags&#x2F;aws&#x2F;" rel="nofollow">http:&#x2F;&#x2F;www.taggernews.com&#x2F;tags&#x2F;aws&#x2F;</a> and didn&#x27;t find any results. Is it because it is listing only few tags for now?

This is what @twitter needs to do to make discovering worthwhile information better.<p>Hashtag spam makes #hashtags mostly useless as a method of discovery.

Using ML? Why not using just Bayesian filters?

Can you also add a way to add TLDRs to everything, pls? :D

Stop trying to remake Hacker News. It&#x27;s pretty much perfect the way it is.