media Hugging Face Forums · 3d ago · open_models

SecFathy rejects specialized 8B XSS model due to adversarial failures

from English

Developer SecFathy open-sourced a research prototype called XSS Specialist, an 8B security model designed for XSS vulnerability analysis. The project initially aimed to push specialization limits using retrieval and LoRA but ultimately rejected the model because it failed adversarial near-miss tests where small identifier changes caused incorrect safety judgments.

The developer redesigned the system to separate reasoning from truth establishment, moving confirmation authority to an independent execution layer.
A potential XSS finding is only confirmed when a real headless browser executes a benign sentinel.
On a frozen 102-case live benchmark, the resulting architecture achieved 94.1% precision and 100% recall with zero false negatives.
The code, architecture, benchmarks, and negative results are available on GitHub under SecFathy/xss-specialist.

The author argues that reliable AI systems should be designed to remain functional even when the model is wrong, rather than depending solely on making the model trustworthy.

Importance 1/3 Trust 1/3 Hugging Face Forums Evaluation & benchmarks Safety & alignment

Related articles

media Hugging Face Forums · 11h ago · 1 view

Independent researcher evaluates Kavach against IssueTrojanBench's 42 real prompt-injection attacks

An independent researcher published KavachBench, a benchmark evaluating the AI coding agent guardrail Kavach against IssueTrojanBench’s corpus of 42 malicious tool-call actions derived from GitHub issue payloads.

arxiv arXiv cs.CL · 2d ago · 1 view

Study reveals epistemic policy divergence in multi-turn LLM contamination

A study titled "Epistemic Policy Divergence in Multi-Turn LLM Contamination: A Protocol-Gradient Investigation" evaluates how large language models handle false premises injected into conversation history, a failure mode termed session-level contamination. The researchers tested GPT-5.4 Mini, Gemini-3.1 Flash-Lite, and GLM-4.5-Air across ten knowledge domains using 22,500 turns at temperature zero.

media Hugging Face Forums · 3d ago · 9 views

Claude Opus 5 and Tetsu find six hollow CI checks in their own project

On September 27, Claude Opus 5 and the 3B model Tetsu identified six separate continuous integration checks in their public repository that reported green or passed despite failing to verify what they were supposed to measure. The issues ranged from a deployment gate refusing valid restarts due to stale digests to timing benchmarks with hollow thresholds that accepted negligible performance differences.

media Hugging Face Forums · 6d ago · 3 views

Researcher seeks arXiv cs.CR endorsement for RedSOC LLM adversarial evaluation

Independent AI security researcher Krishnakaanth Reddy Yeduguru is requesting an endorsement for the cs.CR category on arXiv for a paper titled "RedSOC — An Adversarial Evaluation Framework for LLM-Integrated Security Operations Centers."

media Hugging Face Forums · 7d ago · 12 views

Analysis of engineered overconfidence in AI systems trained via RLHF

Modern large language models are systematically overconfident, expressing high confidence in incorrect answers due to training incentives rather than technical calibration failures. During Reinforcement Learning from Human Feedback (RLHF), human raters reward responses that sound authoritative and definitive, causing the model to learn that uncertainty is penalized.