blog Simon Willison · 1h ago · agents

Qwen 3.8 27B defaults to xhigh reasoning, causing excessive overthinking

from English

Alibaba's Qwen research lab released Qwen 3.8 27B, a vision-capable LLM that outperforms its predecessor and the closed-weight Qwen 3.7-Plus in self-reported benchmarks.

The model defaults to an 'xhigh' reasoning effort level, which causes it to overthink simple prompts and consume excessive context tokens.

Testing shows that disabling or lowering this reasoning setting significantly improves speed and output relevance for local inference tasks.

Importance 2/3 Simon Willison Alibaba (Qwen) Multimodal Reasoning models

Read original

arxiv arXiv cs.LG · 59d ago

De-biased VLM-as-3D-Judge Protocol for Furniture Generation

A de-biased VLM-based judge protocol specializes TRELLIS on furniture generation using lightweight adaptation. The protocol addresses failure modes like image overload and geometry-hiding, with calibration showing 0.83–1.0 win rates and base-vs-base symmetry at 0.5. Among six adaptation methods, conditioner repair under severe degradation achieves parity with the base model, while no method exceeds a 65% win-rate target.

arxiv arXiv cs.CL · 60d ago · 1 view

OmniAgent: Native Active Perception for Omni-Modal Understanding

OmniAgent introduces a POMDP-based iterative Observation-Thought-Action cycle for video understanding, enabling on-demand action execution to selectively distill audio-visual cues into persistent textual memory. It achieves state-of-the-art performance on ten benchmarks, with a 7B agent outperforming a 10× larger Qwen2.5-VL-72B model on LVBench (50.5% vs. 47.3%).

arxiv arXiv cs.CL · 61d ago · 1 view

Visuals Lie, Consistency Speaks: Disentangling Spatial Attention from Reliability in Vision-Language Models

A study challenges the assumption that visual attention signals reliability in vision-language models. It finds near-zero correlation between spatial attention and accuracy, showing instead that self-consistency across reasoning paths is a stronger predictor of truth. Reliability is better explained by generation dynamics and internal state distributions, not visual attention patterns.

arxiv arXiv cs.CL · 61d ago · 2 views

ChLogic: Testing Logical Reasoning Robustness in Chinese Expressions

ChLogic evaluates how well large language models maintain logical reasoning when English logical structures are expressed in Chinese. It reveals a persistent English-Chinese performance gap, with back-translation improving results on general items but harming performance on difficult problems. The benchmark highlights the impact of surface realization, translation artifacts, and model-specific behaviors on multilingual reasoning.

media r/LocalLLaMA · 13d ago · 3 views

Qwen3.8-Max oneshots across 35 prompts, including aquarium break simulation

The article highlights Qwen3.8-Max's performance in a one-shot evaluation context, noting its ability to handle complex physical simulations.