Overcoming Long-Context Limitations in Large Language Models through Retrievable and Composable Doc-to-LoRA Parametric Memories

While Large Language Models excel at document understanding, scaling input context increases latency, VRAM, attention requirements and triggers the Lost in the Middle phenomenon. This thesis investigates parametric document memories as an alternative to context-based conditioning, converting documents into Doc-to-LoRA adapters so textual prompts can be removed. We evaluate memory fidelity, robustness, storage, retrieval, and composition across multiple adapters under various stress tests such as interference, semantic grouping and active-memory scaling. Results indicate that the primary challenge lies in retrieving correct evidence and composing only query-relevant memories. In the end-to-end 2WikiMul- tihopQA evaluation, combining semantic prototype-based retrieval with two active adapters and weighted-delta composition achieves a Token F1 of 0.284, outperforming member-level retrieval with a token of 0.252. Retrieval com- pleteness strongly dictates answer quality because retrieving both required gold documents yields a Token F1 of 0.417, dropping to 0.073 when partially retrieved. Composition choice drastically impacts efficiency. In a ten-adapter setting, factor averaging reduced guarded generation time by 92.7 %, from 14.14s to 1.03s per query and peak VRAM by 35.8% from 21.5 to 13.8 GiB compared to weighted-delta, while retaining 96.9% of its Token F1. Finally, a conventional Qdrant retrieval scaling study on HotpotQA shows hybrid retrieval scaling from 10.6K to 1M documents with Accuracy@1 moving from 1.00 to 0.91, Joint Success@20 from 0.95 to 0.767, and increasing median latency only by 31.2%, 5,03 ms. At FullWiki scale with 5.23M documents, BM25 achieves a 11.97ms median latency with Recall@20 at 0.683 and Joint Success@20 at 0.460. This highlights two distinct challenges. External corpus search scales easily, whereas evidence coverage and efficient adapter composition represent the true bottlenecks for active parametric memories.