media Hugging Face Forums · 11h ago · open_models

Independent researcher evaluates Kavach against IssueTrojanBench's 42 real prompt-injection attacks

from English

An independent researcher published KavachBench, a benchmark evaluating the AI coding agent guardrail Kavach against IssueTrojanBench’s corpus of 42 malicious tool-call actions derived from GitHub issue payloads.

The evaluation utilized IssueTrojanBench's real prompt-injection corpus containing 42 attack actions across 4 categories.
Results demonstrated that coverage is achieved through adapter canonicalization under a default-deny policy rather than benchmark-specific policy tuning.
The research harness and paper are available on Zenodo and GitHub for further review.

The findings highlight the importance of distinguishing between canonicalization-based enforcement and specific tuning when evaluating similar guardrails.

Importance 1/3 New harness with differentiators Trust 1/3 Hugging Face Forums Evaluation & benchmarks Safety & alignment

Read original

arxiv arXiv cs.CL · 2d ago · 1 view

Study reveals epistemic policy divergence in multi-turn LLM contamination

A study titled "Epistemic Policy Divergence in Multi-Turn LLM Contamination: A Protocol-Gradient Investigation" evaluates how large language models handle false premises injected into conversation history, a failure mode termed session-level contamination. The researchers tested GPT-5.4 Mini, Gemini-3.1 Flash-Lite, and GLM-4.5-Air across ten knowledge domains using 22,500 turns at temperature zero.

media Hugging Face Forums · 3d ago · 1 view

SecFathy rejects specialized 8B XSS model due to adversarial failures

Developer SecFathy open-sourced a research prototype called XSS Specialist, an 8B security model designed for XSS vulnerability analysis. The project initially aimed to push specialization limits using retrieval and LoRA but ultimately rejected the model because it failed adversarial near-miss tests where small identifier changes caused incorrect safety judgments.

media Hugging Face Forums · 3d ago · 9 views

Claude Opus 5 and Tetsu find six hollow CI checks in their own project

On September 27, Claude Opus 5 and the 3B model Tetsu identified six separate continuous integration checks in their public repository that reported green or passed despite failing to verify what they were supposed to measure. The issues ranged from a deployment gate refusing valid restarts due to stale digests to timing benchmarks with hollow thresholds that accepted negligible performance differences.

media Hugging Face Forums · 6d ago · 3 views

Researcher seeks arXiv cs.CR endorsement for RedSOC LLM adversarial evaluation

Independent AI security researcher Krishnakaanth Reddy Yeduguru is requesting an endorsement for the cs.CR category on arXiv for a paper titled "RedSOC — An Adversarial Evaluation Framework for LLM-Integrated Security Operations Centers."

media Hugging Face Forums · 7d ago · 12 views

Analysis of engineered overconfidence in AI systems trained via RLHF

Modern large language models are systematically overconfident, expressing high confidence in incorrect answers due to training incentives rather than technical calibration failures. During Reinforcement Learning from Human Feedback (RLHF), human raters reward responses that sound authoritative and definitive, causing the model to learn that uncertainty is penalized.