Enhancing Entity Representations through Soft Prompt Tuning: An Efficient Approach to Biomedical Named Entity Recognition

Named Entity Recognition (NER) has evolved in recent years from closed-vocabulary systems, constrained by fixed label sets, to zero-shot models capable of identifying entities dynamically through semantic descriptions. Models such as GLiNER exemplify this paradigm shift, making entity recognition generalizable to unseen entity types without additional supervision. Such models use natural language descriptions and Transformer-based architectures to generalize across diverse domains without requiring retraining for new entity types. However, their performance degrades in highly specialized domains such as biomedicine, where complex nomenclature and technical terminology demand domain-specific tuning. While full fine-tuning remains the prevailing method for domain adaptation in NER, the extent to which parameter-efficient fine-tuning (PEFT) techniques can provide comparable specialization has yet to be investigated. We present a comparative analysis of five distinct adaptation techniques applied to GLiNER for biomedical entity recognition: (1) Label Descriptions as Semantic Anchors, a description-driven bi-encoder baseline where entity types are grounded via fine-tuning of the label encoder; (2) Soft Prompting, comprising three sub-approaches: (2.1) Base Approach, using learnable embedding matrices; (2.2) Embedding Injection, inserting trainable prompts into Transformer layers; (2.3) Custom GLiNER Extension, combining Embedding Injection with native fine-tuning; and (3) Baseline Fine-tuning, included as reference method for comparative evaluation. Experiments are conducted on JNLPBA and BC5CDR, two biomedical benchmarks, with evaluation based on Macro and Micro F1 scores. Our Custom GLiNER Extension, training only 13% of model parameters, achieves competitive performance (JNLPBA Macro F1: 0.7016, BC5CDR Macro F1: 0.8526), closely matching standard fine-tuning (0.7164 and 0.8833) at a fraction of the trainable parameters, offering empirical guidance for PEFT approaches.