lab xAI News · just now · releases

Grok 4.6 leads in biosecurity refusal while maintaining routine biological performance

from English

LatchBio published an independent analysis of Grok 4.6 on its BioSecBench-Refusal and BioSecBench-Surveillance benchmarks, finding the model to be the strongest at refusing disguised hazardous tasks among tested frontier systems.

On BioSecBench-Refusal, Grok 4.6 achieved a trial-weighted harmonic mean of 62.1%, refusing 59.2% of red-team tasks while completing 64.8% of routine ones.
It is the only system tested to score above 50% on both refusal and routine compliance measures.
On BioSecBench-Surveillance, Grok 4.6 averaged a success rate of 53.5%, placing it behind Opus 5 but ahead of GPT-5.6 Sol.
The model demonstrated the ability to detect intent discrepancies in high-risk content disguised by filenames and encryption.

These results indicate that Grok 4.6 is well-calibrated for routine biological work while providing robust safeguards against adversarial use, supporting SpaceXAI's goal of serving frontier intelligence safely.

Importance 2/3 Trust 3/3 xAI News xAI Benchmark results Safety & alignment

Read original

media r/LocalLLaMA · 32d ago · 12 views

DeepSeek V4 Flash GA ranks same as Sonnet 5 and Grok 4.5 on DeepSWE

DeepSeek claims that its newly generally available model, DeepSeek V4 Flash, achieves performance parity with Anthropic's Sonnet 5 and xAI's Grok 4.5 on the DeepSWE benchmark.

arxiv arXiv cs.CL · 43d ago · 7 views

Intern-BioBreaker reveals physical biosecurity risks in frontier LLMs

Researchers developed Intern-BioBreaker, a specialized bio-red-teaming model, to assess the biological risks of frontier large language models by coupling computational stress testing with wet-lab validation. The study found that aligned models can be induced to provide operational guidance for safety-sensitive tasks and generate sequence-level outputs with harmful properties.

media r/LocalLLaMA · 52d ago · 4 views

Grok Build CLI v0.2.93 uploads full git history and .env secrets to xAI

An analysis of Grok Build CLI version 0.2.93 reveals that it uploads the entire repository, including full git history and sensitive environment files like .env, to xAI's cloud infrastructure. This data transmission occurs regardless of which files are actively opened or whether the "Improve the model" setting is disabled.

media r/LocalLLaMA · 55d ago · 15 views

Press coverage highlights cybersecurity risks of open-source GLM-5.2

Security firms Semgrep and Graphistry have identified that the open-source model GLM-5.2 is proficient at identifying software bugs and performing cybersecurity tasks. This capability raises concerns because the model can be downloaded and run on virtually any hardware without a vendor intermediary, potentially allowing hackers to bypass existing defenses.

media r/LocalLLaMA · 55d ago · 10 views

xAI releases Grok 4.5, GLM-5.2 trails on SWE Bench Pro

xAI has released Grok 4.5, a closed-weight model trained on tens of thousands of GB300s alongside Cursor. The launch highlights token efficiency as a key feature, claiming approximately 16k output tokens per task compared to Opus's 67k.