a runtime guardrail framework for secure and governable ai agents

Software systems increasingly delegate tasks to components based on large language models (LLMs), especially agents that maintain context, use tools, and act over multiple steps. Deployed agents must satisfy behavioural requirements, often expressed as natural-language constraints. However, these constraints are not enforced deterministically. An agent might deviate from them while still producing a fluent, plausible response rather than a conventional software failure. Monitoring the agent's observable behaviour makes such deviations visible. This thesis presents a monitoring framework to assess whether LLM-based agents comply with user-specified natural-language constraints. The architecture separates the monitored agent from a dedicated evaluation layer, allowing the agent to operate unmodified. This layer uses three LLM-based judges from different model families to independently assess the agent’s responses. Their evaluations are retained during execution and combined into a synthesis report. An observability layer records all information required for post-session inspection. The framework is implemented using the Google Agent Development Kit (ADK) for orchestration and LangSmith for tracing. The framework is evaluated through a controlled proof-of-concept experiment. Results demonstrate that the three judges consistently assign lower scores to non-compliant responses and explicitly identify behavioural violations. The recorded execution successfully preserves the data needed to inspect the assessments for each evaluated turn. These findings validate the monitoring approach and motivate further investigation with richer behavioural constraints, diverse agent and judge configurations, and stronger audit mechanisms.