Hello HN!<p>TLDR;<p>- Quality News is a Hacker News client that provides additional data and insights on submissions, notably, the upvoteRate metric.<p>- We propose that this metric could be used to improve the Hacker News ranking score.<p>- In-depth explanation: <a href="https://github.com/social-protocols/news#readme">https://github.com/social-protocols/news#readme</a><p>The Hacker News ranking score is directly proportional to upvotes, which is a problem because it creates a feedback loop: higher rank leads to more upvotes leads to higher rank, and so on...<p><pre><code> →
↗ ↘
Higher Rank More Upvotes
↖ ↙
←
</code></pre>
As a consequence, success on HN depends almost entirely on getting enough upvotes in the first hour or so to make the front page and get caught in this feedback loop. And getting these early upvotes is largely a matter of timing, luck, and moderator decisions. And so the best stories don't always make the front page, and the stories on the front page are not always the best.<p>Our proposed solution is to use upvoteRate instead of upvotes in the ranking formula. upvoteRate is an estimate of how much more or less likely users are to upvote a story compared to the average story, taking account how much attention the story as received, based on a history of the ranks and times at which it has been shown. You can read about how we calculate this metric in more detail here: <a href="https://github.com/social-protocols/news#readme">https://github.com/social-protocols/news#readme</a><p>About 1.5 years ago, we published an article with this basic idea of counteracting the rank-upvotes feedback loop by using attention as negative feedback. We received very valuable input from the HN community (<a href="https://news.ycombinator.com/item?id=28391659" rel="nofollow">https://news.ycombinator.com/item?id=28391659</a>). Quality News has been created based largely on this feedback.<p>Currently, Quality News shows the upvoteRate metric for live Hacker News data, as well as charts of the rank and upvote history of each story. We have not yet implemented an alternative ranking algorithm, because we don't have access to data on flags and moderator actions, which are a major component of the HN ranking score.<p>We'd love to see the Hacker News team experiment with the new formula, perhaps on an alternative front page. This will allow the community to evaluate whether the new ranking formula is an improvement over the current one.<p>We look forward discussing our approach with you!<p>Links:<p>Site: <a href="https://news.social-protocols.org/" rel="nofollow">https://news.social-protocols.org/</a><p>Readme: <a href="https://github.com/social-protocols/news#readme">https://github.com/social-protocols/news#readme</a><p>Previous Blog Post: <a href="https://felx.me/2021/08/29/improving-the-hacker-news-ranking-algorithm.html" rel="nofollow">https://felx.me/2021/08/29/improving-the-hacker-news-ranking...</a><p>Previous Discussion: <a href="https://news.ycombinator.com/item?id=28391659" rel="nofollow">https://news.ycombinator.com/item?id=28391659</a>
When my RSS reader shows me an arXiv paper about ML with ‘fair’ in the title I hit the reject button. What is ‘fair’ is subjective and what I want is a feed relevant to my interests (also subjective.)<p>This is 2023 and text classification problems that I struggled with at a startup 5 years ago are now <i>easy</i> and the power of transformer models is obscured by the ChatGPT hype. It is time that we turn our back in the collaborative filtering algorithms that made social media a hellscape and embrace content-based filtering.<p>I have a model that predicts if an article will front page or get a high ratio of comments/votes. It has a terrible ROCAUC because it is such a fuzzy problem but it is well calibrated and just today my RSS reader told me a story I thought was a nothingburger would succeed on both metrics and… It did!<p>I did make an attempt to take into account the factors you’re concerned about and I was surprised that the AUC didn’t go up. Probably I did it wrong though.<p>Look up my profile, I’d love to chat about it.
There's a related question of "what's the <i>purpose</i> of the ranking algorithm?"<p>Is it to ensure that the #1 article is strictly "better" (via whatever function) than the #2 and the #2 better than the #3?<p>Or is it to ensure that at least N of the top 30 (page 1) submissions will tend to be interesting to many users on the site (driving engagement and discussion)?<p>As a user, I'm a lot more interested in the second goal than I am the first goal. This change seems to serve the first goal much more than it serves the second goal. The reinforcement loop of "on the front page => gets more votes" is a property that supports the second goal more than it supports the first. Looking at the top 30 on social-protocols (this algorithm) vs the front page on HN, I saw 1 additional story on HN that would motivate me to click through (5 vs 4), so not a massive difference.
I think it would be interesting if users could spend their karma on performing moderator actions, perhaps with some sort of algorithmic exchange rate that converts acquired karma into modcoins.<p>For example, it might cost 1000 modcoins to pin an article to the top of the page for ten minutes. Or perhaps more of a bold change: 2000 modcoins to make the text of your comment glow with a golden hue to make it more noticeable. 5000 modcoins to display an image of Paul Graham at the top of the thread, smiling beatifically at all the comments below Him. And so on.<p>This would of course be of no interest to users such as myself who habitually generate throwaway accounts and discard them, but I would be curious to see how high karma users would use such a feature.
Idea: if you know all of a user's votes, you can estimate that they at least glanced over the items up to the lowest-placed one they voted on on the same page. This is a bit more information than "users tend to read higher-placed items following a known distribution" like the formula from your readme. I guess you'd have to be HN to implement this.
Are you sure / do you have info that HN doesn't use some kind of "holding pen" for stories to have a fixed amount of time to see if they get a certain % of votes before being kicked down the list?<p>This is a classic problem with forums, and I wonder if HN already has something in place that you might not have factored in (which could then just be tuned better).
I'd like to see these alternate algorithms implemented. The API exists - and isn't that really the only way to ultimately judge if it's better or worse?<p>Another random idea: have the parameters affecting rankings be visible and adjustable with interactive sliders- so you could customize the various weights to try to attain the ideal mix of stories for you.<p>Or does that defeat the purpose. Is the joy of HN in knowing that when a story reaches the front page, you know it's on everyone's front page...
I guess I don't understand what the problem is with the current way of doing things? Like, HN is one of the few communities online now where the content and the conversation seem interesting, varied, and polite.<p>Whats the core problem you're trying to solve here?
How does your model compare to using:<p>(Users who upvoted a given submission / Users who saw a page that includes the submission and its vote icon)<p>This would be a percent between 0 (no one who saw a page containing a given submission upvoted it) and 100 (everyone who saw it upvoted it). Receiving more impressions wouldn’t change that percentage.<p>Weaknesses: It can only be calculated by HN itself. On pages that list lots of submissions (like the home page), it need may to compensate for relative position on the page. These pages may already randomize position enough for this not to be an issue, or to only be an issue for the first 3-5 items on the home page.
See also <a href="https://news.ycombinator.com/item?id=23286140" rel="nofollow">https://news.ycombinator.com/item?id=23286140</a>
Lots of discussion here about this approach and alternatives. Not sure how feasible this is but I think it would be even cooler to turn this into a site where users can define their own custom ranking algorithm and/or select from a set of available algorithms (including the one you're currently using). Maybe even provide a meta-ranking of the most popular ranking algorithms?
Can I somehow see the historical data for some of mine old submits? I seems to me that on <a href="https://news.social-protocols.org/" rel="nofollow">https://news.social-protocols.org/</a> I can get the data for past 24 hours.<p>I think that you have point with feedback loop.
HN needs to separate emotion from reason in upvotes and downvotes. I bet that many readers here confuse the upvote button with "I like it" and the downvote button with "I dislike it" while it should be about "is this comment truthful and informative, does it add something novel to the discussion?"<p>HN could implement this with a cosmetic change: all upvotes and downvotes would show a form to provide explanation. Those explanations will be reviewed, randomly, to spot emotional users and suspend their voting power for a month or two. As for those who can't be botheted with explaining their voting decision, they shouldn't be able to influence the global ranking. Rage downvoting and hive-mind upvoting will be gone very quickly.
I would like the HN UI to have a "favorite" button equally accessible to the "upvotes" button.<p>I know favorites are a feature, but they require clicking into the comments. I end up using upvote as a bookmark function, not as a method of approving of a post, because that's easier.<p>As it relates to this post, the HN UI encourages the feedback loop this submission is trying to fix.<p>Put a bookmark icon next to the upvote icon. Provide a unified view of upvotes+bookmarked for a user so they can see everything that got their interest.
As long as we're talking about redoing this, let me suggest letting authors see the names of upvoters (and only upvoters).<p>Quora had this and it did a fair bit to create positive community feelings for me. It also let people signal agreement/support without having to create a comment to do so, which I would find handy.