Latent-GEPA: Accelerating Prompt Optimization via Embedding Inversion and Latent Semantic Guidance in Large Language Models

Modern language models expose many representations besides their final text: embeddings, hidden states, logits, residual streams, and other latent signals are now used for retrieval, interpretability, inversion, and automatic evalua- tion. A central question is whether methods that map these representations back to natural language preserve the semantic content of the original input, or whether they merely produce fluent and plausible text. This distinction matters especially for inputs involving negation, logical polarity, counterfactual statements, or violations of commonsense, where a reconstruction can remain lexically similar while changing the meaning. The analysis focuses on semantic fidelity in latent-to-text methods and on whether activation-derived evidence can improve prompt optimization for LLM- as-a-judge evaluation. It combines the construction of a new Semantic-Fidelity Corpus, diagnostic experiments on inversion and verbalization methods, and an LLM-as-a-judge evaluation pipeline optimized with GEPA. The method can be summarized as a two-track analysis: first, latent representations are inverted or verbalized and checked for semantic preservation; second, perplexity and activation verbalizations are tested as signals for improving judge prompts. The experimental setup uses semantic-stress examples, hidden-state inver- sion diagnostics, activation verbalizations from a base language model, and LLM-as-a-judge datasets evaluated with correlation metrics against human judgments. Early results show that perplexity feedback can improve GEPA prompt optimization in this setting. Together, the findings characterize seman- tic fidelity empirically and test how activation information can be made useful for downstream evaluation.