How to improve prompts with two gradients?

In recent years, large language models (LLMs) have shown impressive abilities in many tasks, from translation to answering questions. However, when dealing with more complex classification tasks – such as analyzing legal documents or detecting subtle intentions – LLMs can still make mistakes. To overcome these limits, researchers have explored prompt engineering, i.e., the careful design of the instructions given to the model. Manually optimizing prompts takes time and intuition, so an automatic method has been sought. This led to the PO2G framework (Prompt Optimization with Two Gradients), which uses two gradient signals to guide the search for the best prompt. In tests on nine natural‑language‑processing benchmarks – three from the original ProTeGi study and six general‑purpose tasks – PO2G reached nearly 89% accuracy after just three optimization cycles, whereas ProTeGi needed six cycles to achieve a comparable level. The authors also evaluated both frameworks on seven legal‑domain classification tasks, confirming that PO2G’s advantage holds in specialist settings. These results suggest that automating prompt search can make LLMs more reliable and faster, paving the way for safer applications in fields such as law, medicine, and finance.

The PO2G framework treats the prompt as an optimizable parameter in the space of token sequences. Unlike random‑search or evolutionary methods, PO2G computes two distinct gradients with respect to the classification loss. The first gradient comes from the standard cross‑entropy loss, indicating how to modify the prompt to increase the probability of the correct class. The second gradient stems from a regularization loss that penalizes excessive similarity to previously tried prompts, encouraging exploration of different regions of the sequence space. By combining these signals with a weighted sum, the algorithm updates the prompt via a gradient‑descent step in the embedding space, followed by a projection that maps the resulting vector to the nearest token sequence (usually via nearest‑neighbor lookup in the vocabulary). This keeps the prompt readable and coherent, avoiding meaningless tokens. The authors also introduced an annealing schedule for the weight of the second gradient, starting high to favor exploration and gradually decreasing it to focus on precision. In experiments, using two gradients reduced the variance of gradient estimates compared to a single stochastic gradient, leading to faster convergence: three PO2G iterations yielded an improvement equivalent to six ProTeGi iterations, which rely on a single policy‑gradient signal. Ablation studies showed that removing the diversity gradient drops accuracy to about 80% after three steps, while removing the loss gradient collapses to generic, uninformative prompts. These results highlight that the synergy between performance optimization and prompt‑space exploration is crucial to surpass the limits of manual tuning and single‑gradient methods.