Hard Negative Mining: Teaching an LLM What "Almost Right" Looks Like

A model learns surprisingly little from examples that are obviously wrong. In a technical write-up on dev.to, engineer Shrijith Venkatramana explains hard negative mining with a retrieval example. Pair the query "How do I rotate an AWS IAM access key?" with a text about resizing a Kubernetes pod, and the model separates them effortlessly. Pair it instead with "How do I create an AWS IAM access key?" and the same vocabulary and the same object now carry a different answer.

Easy negatives are cheap, and often useless

In contrastive training the model is rewarded when a relevant document (d+) scores higher than an irrelevant one (d-). An easy pair, 0.82 against 0.12, carries almost no signal, because the model already knows the two differ. A hard pair, 0.82 against 0.76, forces it to find the distinguishing feature. Retrieval systems rarely fail by returning something unrelated; they fail on the wrong library version, the wrong API endpoint, or an installation guide instead of a configuration guide. The same logic matters wherever a system must rank or distinguish: a reranker shown four PostgreSQL passages must learn why one answers the query while another is merely related.

An idea older than language models

The technique took shape in computer vision. FaceNet, published in 2015 by Florian Schroff, Dmitry Kalenichenko and James Philbin, mapped faces into an embedding space and used online triplet mining, raising the difficulty of triplets as the network improved. Dense Passage Retrieval by Karpukhin and colleagues carried the same logic into text search in 2020, and ANCE from Microsoft researchers including Lee Xiong and Chenyan Xiong went further: an approximate-nearest-neighbour index finds difficult negatives with the model itself. RocketQA added denoised hard negatives for dense passage retrieval.

Mining in practice

A basic procedure retrieves the top k candidates for each query, removes the known positive, and keeps the rest. The trap is that hard does not mean wrong: a highly similar document may still be relevant, and pushing it away damages the retrieval space. Production pipelines therefore pass candidates through a cross-encoder or an LLM judge to remove probable false negatives before training. Cost is the other constraint. One million training queries with 20 hard negatives each give roughly 20 million candidate relationships. The author advises cheap first-stage retrieval, periodic mining, and teacher scores whose useful band sits in the middle, roughly 0.4 to 0.8.

Why it produces more gradient

In a contrastive objective the model is rewarded when the positive takes most of the probability mass. When a negative scores 0.87 against a positive of 0.90, the loss produces a far larger gradient than when the best negative sits at 0.20. Easy negatives teach broad regions of the space, hard ones teach the decision boundary. Re-mining after each training round closes the loop: a better model produces better mistakes, and those mistakes become better training data.