A lawyer asking for a citation is not asking for a similar paragraph. They are asking for a case that actually supports the proposition they want to make, in a court whose decisions bind the one they're in front of. Similarity is the easy part. Correctness is the job.
I learned this building a retrieval system over Indian court judgments. Here's the pipeline that came out of it and why each piece exists.
Where pure vector search breaks
Embeddings capture meaning, and legal language is full of near-identical meaning with opposite outcomes. "The petition is allowed" and "the petition is dismissed" embed close together. So do a majority opinion and the dissent that follows it. And section numbers, the thing lawyers search for most, are tokens an embedding model barely distinguishes: Section 138 and Section 148 look alike to it.
Vector search gives you the neighbourhood. It cannot tell you which house.
Chunk by the document's own structure
Judgments have paragraphs, and Indian judgments number them. I chunk on those boundaries rather than on a fixed token count. A chunk is a numbered paragraph, or a run of short ones, with the case name, court, date and paragraph number attached as metadata. When a result comes back, it is already a citable unit: case, paragraph, pinpoint.
Hybrid retrieval, fused
Every query runs through two retrievers:
- Lexical (BM25) over the raw text, which nails section numbers, statute names and exact phrases.
- Dense (embeddings) over the chunks, which handles paraphrase and concept queries.
The two ranked lists are merged with reciprocal rank fusion. It's a one-line formula and it works better than any weighting I tuned by hand:
score(doc) = Σ 1 / (k + rank_r(doc)) for each retriever r
k ≈ 60
A document that both retrievers like floats to the top. A document only one of them likes still surfaces, lower down.
Filter by what binds
Before ranking, filter. The user's forum determines which courts bind it: Supreme Court over everything, the relevant High Court over its subordinate courts, and larger benches over smaller ones within a court. These are metadata filters, not ranking hints. A persuasive but non-binding judgment should be labelled as such, not silently ranked above a binding one.
Date matters the same way. A judgment that was overruled later is still in the corpus. The index carries an overruled-by field, populated from a citation graph built during ingestion, and results that were overruled are flagged in the output rather than hidden.
Rerank with a cross-encoder
Fusion gives a decent top fifty. A cross-encoder reranker reads the query and each candidate together and re-scores them. It's slow, so it only runs on the shortlist, and it's the step that most improved the quality of the top five. The bi-encoder finds; the cross-encoder judges.
Verify before you cite
This is the step that turns search into something a lawyer can trust. For each top candidate, a model is asked one narrow question: does this paragraph support the proposition in the query, contradict it, or merely mention the same topic? The answer is a label with a one-line justification, and only "supports" results are presented as citations. The rest are shown separately as related reading.
That verification is a classification task, not a generation task. It's cheap, it's testable, and it's where the system earns the word "correct".
Show the paragraph
Every citation in the interface is the actual paragraph, with the pinpoint, next to the proposition it supports. Lawyers do not trust summaries and they are right not to. The system's job is to bring them to the right paragraph fast, then get out of the way.
What made the difference, in order
- Chunking on paragraph boundaries with citation metadata attached.
- Lexical plus dense retrieval, fused rather than weighted.
- Hard filters for binding authority and overruled status.
- Cross-encoder reranking on the shortlist.
- A verification step that labels support explicitly.
Each one on its own is a modest improvement. Together they change what the tool is: from "find similar text" to "find the case that supports this, and prove it".