Query Rewriting and Multi-Query Retrieval

Retrieval quality depends heavily on the query. Users write short, ambiguous or conversational questions, while documents use their own vocabulary. Query transformation closes that gap before search happens. The simplest form is rewriting: an LLM turns the user's message into a clearer search query. This is essential in multi-turn chat, where follow-ups like "and in 2023?" mean nothing without history, so the system condenses the conversation into a standalone query. Decomposition splits a complex question into sub-questions that can be retrieved separately, useful for comparisons or multi-hop questions. Step-back prompting, described by Google DeepMind researchers in 2023, asks a more general question first to retrieve background principles. Multi-query retrieval generates several variants of the question, runs a search for each, and merges the results. LangChain popularized this with its MultiQueryRetriever. RAG-Fusion combines multiple queries with reciprocal rank fusion (RRF), a method described by Cormack, Clarke and Buettcher in 2009, which scores each document by summing 1/(k + rank) across result lists. Documents that appear high in several lists rise to the top without needing comparable raw scores. A related technique, HyDE (Hypothetical Document Embeddings, Gao and colleagues, 2022), has the model write a hypothetical answer and embeds that instead of the question, because an answer-shaped text often sits closer to real answer passages in embedding space. Misconceptions: more queries are not always better. Each variant adds latency and cost, and poorly generated variants can drift from the user's intent, pulling in irrelevant documents. Rewriting can also drop important constraints like dates or product names. Measure recall and answer quality on real questions before adopting these steps.