How Search Engines Can Help Spot the Most Dangerous Vulnerabilities?

In cybersecurity, experts must decide where to focus their limited time and energy: the attack surface is massive. A new method called SiftRank turns this challenge into an information‑retrieval problem, much like searching on Google.

SiftRank uses large language models (LLMs) as rankers to order thousands of code functions in seconds. It works in batches of about ten items, scoring and re‑ranking them by the likelihood of containing a vulnerability. When the score distribution stabilizes, the process stops automatically, saving resources.

In a real test, SiftRank examined 2,197 changed functions in a firmware patch and identified the fixing function in under two minutes, costing only $0.82 in inference. No special infrastructure or security‑specific fine‑tuning is needed—just a standard LLM API.

This lets researchers with modest resources prioritize the most critical threats and speed up responses to emerging bugs. The open‑source code is on GitHub, ready for anyone to try this new way of “sifting” code for bugs.

SiftRank relies on three core mechanisms. First, listwise ranking: an LLM evaluates small groups (≈10) of documents, producing a local ranking. This reduces complexity to O(n) because each document is processed only once per batch. Second, inflection‑based convergence detection: the system monitors score distribution and stops iterating when variation falls below a predefined threshold, avoiding redundant evaluations. Third, iterative refinement: high‑scoring documents are resampled in subsequent batches, focusing computational power on the most promising candidates.

To mitigate typical LLM instability, SiftRank performs multiple randomized evaluations per document, aggregating results with weighted averages. This reduces bias from prompt or context variability. Moreover, the method does not require a preliminary retrieval stage (like BM25) because it operates directly on sets of thousands of items, leveraging LLMs' ability to understand code context and semantics.

In N‑day analysis, quickly identifying the fixing function is crucial: it reveals which firmware part was altered to address a vulnerability, aiding exploit or patch generation. Inference cost is calculated based on processed tokens; with 10‑item batches and medium‑sized models, consumption stays low.

SiftRank is released under the MIT license on GitHub, with scripts for integration with OpenAI, Anthropic, or other APIs. Developers can tune convergence thresholds, batch size, and aggregation methods to suit domains such as malware analysis, open‑source code review, or security patch assessment.