CircuitLasso proposes a scalable method for learning sparse circuits in large language models using sparse linear regression. It achieves structural accuracy comparable to state-of-the-art intervention-based methods at significantly lower computational cost, while enabling efficient discovery of semantic feature propagation and improving performance on domain-generalization tasks with reduced cost.
arxiv
arXiv cs.AI
·
54d ago
·
research
CircuitLasso: Scalable Circuit Learning for LLM Interpretability
from English
Importance 2/3
New harness with differentiators
arXiv cs.AI
Allen AI
Hugging Face
OpenAI
Evaluation & benchmarks
Open weights
Reasoning models
Related articles
arxiv
arXiv cs.CL
·
51d ago
·
3 views
Semantic Clusters Pre-Train Tsetlin Machine for Interpretability
A new framework pre-trains the Tsetlin Machine using semantic clusters from language models, avoiding embeddings. The method groups text samples into coherent clusters via K-means or Top2Vec, then uses cluster-sample pairs to train a non-negated TM with Type I feedback. Results show superior performance across five datasets, matching BERT-level accuracy while maintaining full interpretability.
arxiv
arXiv cs.LG
·
54d ago
CircuitLasso: Scalable Circuit Learning for LLM Interpretability
CircuitLasso enables scalable circuit learning in large language models by using sparse linear regression. It recovers circuits with structural accuracy matching state-of-the-art methods at significantly lower computational cost, and demonstrates human-interpretable semantic propagation through model components. The learned circuits achieve comparable performance on a domain-generalization task with reduced cost.
arxiv
arXiv cs.CL
·
51d ago
·
1 view
H-RePlan: Hierarchical Recovery for Cross-Device Agent Systems
H-RePlan introduces a hierarchical replanning framework that separates device-local strategy recovery from global orchestrator replanning. It outperforms existing baselines by achieving higher completion and instruction adherence, with reduced token cost, through scope-aware recovery in multi-device agent systems.
arxiv
arXiv cs.AI
·
51d ago
·
1 view
Learnable Global Merging for Variable-Length Tokenization in Diffusion Transformers
A novel variable-length tokenizer uses learnable global merging to enable cross-length representation alignment in diffusion models. This data-independent approach overcomes position-dependent semantics and improves the quality-compute trade-off on ImageNet 256×25-6 generation compared to prior methods.
arxiv
arXiv cs.AI
·
51d ago
Hidden Evolution of Disguised Visual Context in VLMs
Visual tokens enter large language models as raw, unstructured signals. Their internal transformation and integration depend on architecture—either as in-context prompts or injected into intermediate layers—leading to distinct evolution paths in visual representation and frequency characteristics. We find that attention alone is insufficient; performance is driven by the quality of visual representations at each layer across different integration paradigms.