arxiv arXiv cs.LG · 17h ago · research

Researchers exploit cross-session encryption reuse to steal reasoning traces from LLM APIs

from English

Researchers have identified a vulnerability in how leading large language model providers handle step-by-step reasoning traces, which are returned as encrypted blocks for client-side use. They found that these encrypted blocks are fully compatible and interchangeable across different sessions, users, and models within the same provider's ecosystem.

The team developed a scalable decryption jailbreak by injecting an encrypted reasoning trace from a capable model into a weaker, less safeguarded model to force it to output the trace in plaintext.
This method circumvents anti-distillation mechanisms, allowing the extraction of proprietary reasoning traces from providers including Anthropic, OpenAI, and Google.
The vulnerability enables large-scale private data extraction; decoding 315,320 reasoning blocks scraped from public repositories revealed 367 PII artifacts and 182 credentials.
The flaw inadvertently reveals hazardous information hidden within the reasoning process, even when the model's final output safely rejects a malicious request.
Attackers can use this flaw to execute invisible prompt injections by embedding malicious payloads within encrypted blocks to poison public agentic rollouts.

Following responsible disclosure, the authors propose concrete cryptographic and system-level mitigations to secure client-side reasoning.

Importance 2/3 arXiv cs.LG Anthropic Google DeepMind OpenAI Reasoning models Safety & alignment

Related articles

arxiv arXiv cs.AI · 56d ago

LegalHalluLens: Auditing Hallucinations in Legal AI

LegalHalluLens introduces a framework to audit AI hallucinations in legal contexts by analyzing typed hallucination profiles across four claim categories. It reveals a 38-40 point gap between obligation/numeric and temporal claims, and shows two systems with identical 52% hallucination rates can have opposite risk directions. The framework uses a Risk Direction Index and calibrated debate pipelines to reduce fabricated detections by 45% and improve accountability in legal AI deployment.

arxiv arXiv cs.AI · 18h ago · 3 views

Attackers steal reasoning traces from proprietary LLM APIs via encrypted block vulnerability

Researchers have identified an architectural vulnerability in how leading large language model providers handle step-by-step reasoning traces. Providers return these traces as encrypted blocks to clients, but the study finds these blocks are fully compatible and interchangeable across different sessions, users, and models within a provider's ecosystem.

arxiv arXiv cs.AI · 43d ago · 1 view

When Helpfulness Overrides Causal Caution: Context-Dependent Suppression and Recovery in LLMs

A study reveals that large language models systematically suppress 'Causal Caution'—the tendency to refrain from causal judgment without sufficient evidence—when shifting from academic to practical advisory contexts. This suppression occurs despite the models retaining the underlying capability, as evidenced by the ability to restore cautious reasoning through specific prompts.

media r/LocalLLaMA · 50d ago · 4 views

EU AI Act mandates AI-generated text watermarking from August 2024

The EU AI Act requires all AI systems generating synthetic text to include machine-readable, detectable watermarks using robust, interoperable technical solutions with two layers. This applies to all AI models, including open-source ones, and extends to any service accessible by EU citizens, regardless of location. Non-compliance risks fines of up to 35 million euros or a percentage of annual income, with providers of 'systemic risk' AI models facing heightened liability.

arxiv arXiv cs.AI · 53d ago · 6 views

NRT-Bench: Multi-turn Red-teaming of LLM Agents in Safety-Critical Systems

NRT-Bench introduces a benchmark for multi-turn red-teaming of LLM agents operating in a simulated nuclear power plant. Across four frontier operator models, 8.7% to 12.1% of attack sessions result in loss of a critical safety function, with vulnerabilities largely disjoint across models. The effectiveness of defences varies significantly by model, showing strong model dependence.