media Hugging Face Forums · 2h ago · open_models

Jeanbosange releases LIMEN Runtime Audit prototype for inspecting layer-wise activation trajectories

from English

Jeanbosange has published the first public prototype of LIMEN Runtime Audit, an open-source toolkit designed to inspect layer-wise activation trajectories in open-weight language models. The tool treats the sequence of hidden states across layers as a measurable trajectory, providing a reproducible observability layer for comparing internal runtime behavior.

Version 0.1 accepts exported NumPy arrays containing hidden states and optional logits.
It calculates metrics including path length, displacement, tortuosity, mean speed, speed variability, mean acceleration, turning angle, entropy, and top-1/top-2 margin.
The toolkit validates array dimensions, records extraction metadata, generates SHA-256 checksums, and outputs machine-readable JSON and Markdown audit reports.
It runs automated tests on Python 3.10, 3.11, and 3.12.

The author emphasizes that this is a descriptive audit tool rather than a predictor of reasoning or semantic states, aiming to help users compare model architectures, detect regressions, and study dynamic differences across prompt families.

Importance 1/3 New harness with differentiators Trust 1/3 Hugging Face Forums Evaluation & benchmarks Safety & alignment

Related articles

arxiv arXiv cs.CL · 9d ago

UTS bypasses ELOQUENT 2026 Voight-Kampff detectors with structural shifts

Researchers from UTS developed evasion strategies that sweep the top 5 positions on the ELOQUENT 2026 Voight-Kampff leaderboard by exploiting a fundamental asymmetry in detector vulnerability. The study finds that pushing generated text out of the detector's training distribution reliably defeats adversarial detection, whereas mimicking human data fails.

arxiv arXiv cs.CL · 9d ago

Study finds graded entity-familiarity readouts in language models

A study examines activations at the final prompt token in twelve instruction-tuned models from the Bielik, PLLuM, Gemma-4, and Qwen3 families to determine if they can estimate familiarity with an entity before generating an answer. Using a dataset of 1,440 Polish entities, the researchers found that familiarity-probe scores effectively separate real from fabricated entities across all model families.

media Hugging Face Forums · 9d ago · 2 views

Vulcora launches $51.2K challenge to recover hidden trigger in seven SmolLM2-based models

Vulcora has released a public challenge offering a $51,200 bounty for recovering the exact trigger sentence of a hidden backdoor embedded in one of seven open-source language models. The models are derived from HuggingFaceTB/SmolLM2-135M-Instruct and include one teaching model that confesses the mechanism's shape, five decoys, and one with a live undeclared trigger.

arxiv arXiv cs.AI · 10d ago

Win by Silence: Deletion Non-Monotonicity and Autonomous Exploitation in LLM Plan Evaluation

This paper investigates a failure mode in staged expected-value scorers for LLM-generated venture routes, where strategic plans can achieve higher scores by becoming less explicit through the deletion of interior transitions.

arxiv arXiv cs.LG · 10d ago

Mechanistic interpretability reveals LLM-as-Judge bias in hidden state geometry

This study presents a representation-level account of scoring bias in LLM-as-Judge models, analyzing the judge's hidden states rather than just input-output perturbations. The authors report findings across seven judges, seven bias types, and nine benchmarks.