How Hacker News ranking really works: scoring, controversy and penalties

A 2013 analysis by Ken Shirriff still gets cited whenever Hacker News users wonder why a story abruptly disappeared from the front page. Shirriff crawled the /news and /news2 pages every minute for several days, staying under the site's guideline of two pages per minute, parsed the HTML with Beautiful Soup and charted the results with matplotlib.

Scoring and the reranking loop

The published formula, taken from an available version of the Arc server code, ranks a story by its upvotes, its age in minutes and a set of multipliers: gravity is 1.8, the time base is 120 minutes, and there are separate factors for stories without a URL, lightweight posts and jokes. Because age carries a larger exponent than votes, every score eventually decays toward zero, which is why nothing stays on the front page for long.

Ranking is not recomputed for every story on every visit. A story is reranked when it is upvoted and moves to its appropriate spot. On top of that, every 30 seconds one of the top 50 stories is randomly selected and reranked, and pages can be cached for 90 seconds. A story that stops collecting votes can therefore sit in the wrong place for minutes.

The penalty system

The published code turned out to be mostly accurate, but the real system applies far more penalties than users assume. About 20% of the articles on the front page had been penalized, and 38% of those on the second page. At the time, anything with NSA in the title received an automatic penalty of 0.4; the author tested words such as awesome, bitcoin and bubble and found no such effect. Many domains also carried automatic penalties between 0.25 and 0.8, among them github.com, medium.com, reddit.com, youtube.com, arstechnica.com and theguardian.com. Shirriff notes the theory that popular sources attract parallel submissions and more upvotes than a story merits.

A penalty factor is severe in practice. A factor of 0.4 makes each vote count as 0.3, or a story fall 66% faster than normal; 0.1 makes each vote count as 0.05.

Controversy is the most visible penalty. In the published code the contro-factor kicked in above 20 comments when comments outnumbered upvotes, scaling an article by the square of votes over comments. In practice Shirriff saw it activate at 40 comments and suspects the exponent is 3 rather than 2. The effect is sudden: a post sitting at number 5 dropped to 22 the moment it hit 40 comments, and another vanished from the top 60 entirely.

What it means, and an ironic postscript

The front page is not a pure meritocracy of upvotes: the top spot often belongs to an article that was not the highest scoring one, and the list of penalized front-page articles on a single day ran longer than the author expected. Shirriff's own article was hit with a 0.2 penalty minutes after reaching the front page; without it, he writes, the post would have reached number 5. He was later told that the voting-ring detector had fired erroneously. His closing advice is blunt: if an article has more comments than votes, adding a comment may kill it off entirely.