[Basic best practices from a search engineer at Reddit.](https://softwaredoug.com/blog/2024/06/25/what-ai-engineers-need-to-know-search) Concepts I can currently grasp and seem relevant: 1. Use an evaluation framework to test the quality of search results 2. The whole retrieval solution ([[Information Retrieval (IR) System]])—[[Lexical Search]], [[Semantic Search]], [[Re-ranker]], [[Query Classification]], [[Intent Classification]], [[Relevance Feedback]] etc.—is more important than the [[Vector Database]] vendor. 3. Many ML / AI practitioners feel search tools like Solr and Elasticsearch are esoteric and reach for a vector search solution, then, only after their app is stood up, realize they need to sprinkle in some degree of lexical search, and stumble into problems like [[Intent Classification]]. 4. Measure the quality of a queries judgments with statistics like (n)DCG, ERR, MAP, Precision, Recall, and F-Score. 5. Beware of Precision/Recall tradoff. 6. Users have diverse information needs, expressed with multiple queries, that should have a unique treatment in ranking and UX. 7. [[Okapi BM25]] is just one of a bazillion lexical similarity scores 8. Combining, weighting, and scoring different search fields, like a match in the title vs buried in text body, is non-trivial. 9. Tokenization—n-gram, word-piece, whole tokenization, and related topics like stemming, handling punctuation, etc.—can dramatically change how well search performs. 10. Phrase matching, not only term matching, becomes more important in more technical disciplines where text is has specific meaning. 11. [[Query Relaxation]] 12. A good query parser should treat some terms as phrases, like a search for "USA" becomes a search for "USA" OR "United States of America" 13. Eliminate false positives from top N by applying filter queries to express what is certainly not relevant. 14. LambdaMART is one of those things like BM25 that's old school, but works (it's great for reranking hundreds or thousands of results relatively inexpensively), and you probably should learn. 15. It's common to have multiple reranking stages, starting with dumb rerankers optimized for top 1K up to fancy cross encoders that rerank only a few seaerch results. 16. There's no one size fits all solution to search. 17. Scoring the first pass using techniques like genetic algorithms or Bayesian optimization can help you get by without a reranker. 18. You can build a collection that memorizes highly engaging results for a query that you can boost, or vise versa. 19. Gather [[Relevance Feedback]] to improve the search system. 20. Use SearchArray or [[Okapi BM25]]S to prototype. Next, read this: https://x.com/atroyn/status/1819395539156557892: > it is far too early to spend time adopting the abstractions imposed by existing tools and frameworks. until you understand what you want to build and how to build it, these can only slow you down > ai applications are uniquely iterative in both the way they're built, and the way they're used. think about how you might use [@midjourney](https://x.com/midjourney) to refine an image generation that isn't quite right from the start, until it's what you want. because we cannot pre-define what output we expect from a given input, we find that we must iterate toward our goal and then make sure that the solution we converged to generalizes across our use-case Watch [his interview](https://www.youtube.com/watch?v=FJPA0IG9rHM)