A position paper by NVIDIA researchers outlines a vision for securing AI agents against indirect prompt injection attacks, where malicious instructions embedded in untrusted data trigger dangerous actions. The authors argue that effective defense requires moving beyond isolated model fixes to comprehensive system-level architectures.
- Dynamic replanning and security policy updates are necessary for handling dynamic tasks and realistic environments.
- Context-dependent security decisions should rely on LLMs only within designs that strictly constrain what the model can observe and decide.
- Personalization and human interaction must be core design considerations for inherently ambiguous cases.
- Existing benchmarks often create a false sense of utility and security due to their limitations.
System-level defenses serve as the structural skeleton for agentic systems by integrating rule-based and model-based checks, enabling more targeted research on robustness.