Self-adaptive anomaly detection for connected vehicles uses RL and human feedback

This paper presents an online anomaly detection framework for autonomous cyber-physical systems that integrates reinforcement learning with human-in-the-loop retraining to handle evolving system behaviors.

The framework employs a factorized deep Q-network with self-attention to select the optimal detector from a candidate pool based on inter-service dependencies. It utilizes an ensemble of three statistical drift detectors that prioritize precision by raising alarms only when all agree. A pending transition buffer and 60/40 prioritized replay strategy allow operators to incorporate expert knowledge without catastrophic forgetting.

Evaluated on a connected-vehicle testbed for automated valet parking, the attention-augmented agent achieved an F1 score of 0.69, significantly outperforming single detectors which scored at most 0.11. After concept drift from a software update reduced performance to 0.52, operator-triggered retraining recovered the F1 score to 0.65 on the new distribution.