A de-biased VLM-based judge protocol specializes TRELLIS on furniture generation using lightweight adaptation. The protocol addresses failure modes like image overload and geometry-hiding, with calibration showing 0.83–1.0 win rates and base-vs-base symmetry at 0.5. Among six adaptation methods, conditioner repair under severe degradation achieves parity with the base model, while no method exceeds a 65% win-rate target.
arxiv
arXiv cs.LG
·
51d ago
·
research
De-biased VLM-as-3D-Judge Protocol for Furniture Generation
from English
Importance 2/3
New harness with differentiators
arXiv cs.LG
Alibaba (Qwen)
Google DeepMind
Mistral AI
Evaluation & benchmarks
Multimodal
Reasoning models
Related articles
media
r/LocalLLaMA
·
52d ago
·
2 views
Benchmarking small LLMs on hard HTML data extraction
A user tested models from 2B to 35B parameters on 29 difficult HTML data extraction pages, finding that smaller models like gemma4 e2b and e4b outperform larger ones. Qwen3.6 27B led in performance, while all MOE models scored poorly, highlighting the importance of task-specific benchmarking.
arxiv
arXiv cs.CL
·
53d ago
·
1 view
RubricsTree: Scalable Evaluation Framework for Personal Health Agents
RubricsTree introduces a hierarchical taxonomy of over 100 clinically-verifiable Boolean rubrics, evolved from 4,000 real user queries via human-in-the-loop curation. It enables scalable, expert-aligned evaluation of personal health agents by dynamically routing queries to relevant rubrics and outperforms baseline methods in alignment, context sensitivity, and model performance gains of up to 66% on HealthBench.
arxiv
arXiv cs.AI
·
53d ago
·
2 views
RubricsTree: Scalable Evaluation Framework for Personal Health Agents
RubricsTree introduces a hierarchical taxonomy of over 100 clinically-verifiable Boolean rubrics, evolved from 4,000 real user queries via human-in-the-loop curation. It enables scalable, expert-aligned evaluation of personal health agents by dynamically routing queries to relevant rubrics and outperforms baseline methods in alignment, context degradation detection, and model performance gains of up to 66% on HealthBench.
arxiv
arXiv cs.CL
·
53d ago
Visuals Lie, Consistency Speaks: Disentangling Spatial Attention from Reliability in Vision-Language Models
A study challenges the assumption that visual attention signals reliability in vision-language models. It finds near-zero correlation between spatial attention and accuracy, showing instead that self-consistency across reasoning paths is a stronger predictor of truth. Reliability is better explained by generation dynamics and internal state distributions, not visual attention patterns.
arxiv
arXiv cs.CL
·
53d ago
MambaCount: Efficient Text-guided Object Counting
MambaCount introduces a spatial sparse state space duality block to enable efficient text-guided open-vocabulary object counting. It addresses causal modeling limitations and high entropy in spatial token responses, achieving state-of-the-art results on FSC-147 with a test MAE of 12.23 while maintaining linear complexity.