A new method uses program synthesis to generate Python programs that reproduce attention patterns in transformer models. These programs achieve over 75% average Intersection-over-Union similarity on held-out data and can replace up to 25% of attention heads with minimal impact on model performance, increasing perplexity by only 16% on average.
arxiv
arXiv cs.LG
·
52d ago
·
research
Reverse-Engineering Transformer Attention with Executable Programs
from English
Importance 2/3
New harness with differentiators
arXiv cs.LG
OpenAI
Meta AI
Mistral AI
AI agents
Open weights
Reasoning models
Related articles
arxiv
arXiv cs.CL
·
54d ago
·
5 views
LOGOS: A General-Purpose Generative Model for Natural Sciences
LOGOS is a unified generative language model that represents scientific objects and their interactions as token sequences in a shared grammar. It achieves consistent or superior performance across diverse natural science tasks, demonstrating the feasibility of a single model serving multiple domains. The model scales positively with parameter count, and its design suggests that AI for Science should align deeply with large language models through shared architectures and training.
media
AI News (smol.ai)
·
2d ago
Meta's Muse Spark 1.2 and OpenAI's GPT-5.6 Sol updates
Meta announced that its Muse Spark 1.2 model achieved gold-medal performance in five STEM Olympiads and entered the top 5 on the Vals Index at $0.69/test, reportedly 3x cheaper than Kimi. Meta attributed these gains to multi-agent orchestration with parallel reasoning, noting perfect theory scores at APhO and IPhO without external tools.
arxiv
arXiv cs.CL
·
47d ago
·
6 views
Test-Time Steering Resolves Temporal Fact Conflicts in LLMs
Researchers identify parametric temporal conflicts in language models where outdated facts persist in parameters. They introduce Temporal Attractor Steering (TAS), a test-time method that resolves 29-57% of such conflicts without retraining, maintaining 85-99% accuracy on non-conflict queries and outperforming a baseline on three of four models.
media
r/LocalLLaMA
·
51d ago
·
4 views
Ohio State University releases open-source Deep Research agent QUEST-35B
Ohio State University's NLP team has released QUEST-35B, an open-source Deep Research agent trained on approximately 32 H100 GPUs using 8,000 synthetic samples. The team open-sourced the training recipe, code, weights, and datasets, with benchmark results showing competitive performance compared to leading closed-source Deep Research systems.
media
r/LocalLLaMA
·
51d ago
Ohio State University releases open-source Deep Research agent QUEST-35B
Researchers at Ohio State University trained QUEST-35B, a Deep Research agent, using approximately 32 H100 GPUs and 8,000 synthetic samples. They open-sourced the training recipe, code, weights, and datasets, with benchmark results showing competitive performance compared to leading closed-source Deep Research systems.