OpenBioRQ introduces a benchmark of 12,553 unsolved biomedical research questions across 12 domains, designed to test agentic models' faithfulness and abstention. It evaluates models in a tool-using setting without answer keys, using real follow-up evidence rather than parametric knowledge, and reveals significant agentic collapse on the hardest questions where tools are no longer used despite being critical.
Mateusz has developed a full pre-trained language model, Project Inkblot's Titan v1, combining Mamba SSM, Multi-Head Attention, and 32-expert MoE in a single decoder-only architecture under 1B parameters. The model, trained on a single NVIDIA L4 GPU for ~$50, achieves 27.5 validation perplexity and demonstrates efficient scaling via a single-line config update, with all components implemented from scratch in PyTorch. Titan v2's first training cycle is now complete, and dataset expansion is underway.
A new dataset, IFLLM, collects mouse trajectories and eye gazing data from users interacting with LLMs. It shows that implicit feedback significantly improves LLM alignment, boosting text-based reward model accuracy from 55% to 64% and nearly tripling response quality improvements after DPO training on eight LLMs.
A new dataset, IFLLM, collects mouse trajectories and eye gazing data from users interacting with LLMs. It shows that implicit feedback significantly improves LLM alignment, boosting text-based reward model accuracy from 55% to 64% and nearly tripling response quality improvements after DPO training on eight LLMs.
ScaffoldAgent introduces a utility-guided framework for dynamic outline optimization in open-ended deep research. It models outline evolution through Expansion, Contraction, and Revision operations, guided by a feedback mechanism that evaluates retrieval gain, structural coherence, and generation quality. Experiments show it improves long-form report generation and factual grounding compared to existing agents.
As enterprise agent tool catalogs scale from 10 to 110 agents, routing accuracy drops 16--23 percentage points on under-specified requests. An oracle analysis identifies retrieval and confusion gaps, with embedding-based shortlisting recovering +10--11pp F1. A human-annotated study of 1,435 utterances confirms real-world recovery of +10--17pp despite lower absolute performance.
KDoS introduces knowledge density to guide synthetic data generation through a three-stage feedback mechanism. Experiments on models from 0.6B to 16B and data scales from 1B to 5B tokens show that an optimal knowledge distribution consistently maximizes knowledge boundary expansion, is stable across model backbones, and outperforms baselines on six knowledge benchmarks.
CTC-internal scoring shows no WER improvement over greedy decoding on LibriSpeech, with acoustic confidence failing to correlate with linguistic plausibility. MBR decoding using RoBERTa PLL achieves a 5.42% WER, outperforming greedy decoding by 0.535 pp, demonstrating that linguistic information can overcome CTC's saturation limit.
Tmax presents the strongest open RL recipe for terminal agents, achieving 27% on Terminal-Bench 2.0 with only 9B parameters. It uses a novel data taxonomy to generate over 2.5x more terminal environments than prior datasets, enabling efficient training with a simple, outcome-only recipe. The dataset, models, and code are open-sourced at https://github.com/hamishivi/tmax.
WaveDetect introduces a signal processing approach using continuous wavelet transforms to detect machine-generated text by identifying spectral fingerprints. It outperforms existing methods in accuracy and robustness across adversarial attacks, domain shifts, and evolving LLMs, demonstrating strong generalization on RAID, EvoBench, and Domain-Shift datasets.
Pretrained LLM embeddings show measurable alignment with expert-defined mental-health symptom structure. Fine-tuning enhances this alignment, especially at fine category levels, with larger model sizes improving both zero-shot performance and supervised gains. Residual alignment persists after controlling for linguistic and stylistic confounds, indicating expert structure recovery is level-dependent and requires explicit confound testing.
Between 2010 and 2025, militaristic terms in scientific abstracts increased by 48% in OpenAlex and 32% in PubMed, with a sharp rise after 2019. The use of such language is aligned with global conflict levels and grows fastest in Global South publications, particularly in social sciences and engineering. A controlled experiment shows that war-framing reduces perceived credibility, funding willingness, and policy support, with a minor increase in urgency.
SVD-Surgeon is a training-free method that applies the Optimal Brain Surgeon framework to singular-value decomposition. It computes a closed-form update for retained singular values to compensate for truncation, improving the perplexity-compression trade-off on OPT and LLaMA 2-7B models without retraining.
Tapered Language Models (TLMs) allocate more parameters to earlier layers and fewer to later ones, reducing perplexity and boosting benchmark performance across architectures. This depth-aware capacity allocation improves language model outputs without adding compute or parameters, offering a simple, universal design principle.
NL2Scratch introduces an executable benchmark with 311,648 parser-valid NL-program pairs derived from real Scratch projects. It proposes Semantic Alignment Consistency (SAC) to measure semantic agreement, validating 23,594 examples and creating an 800-slot-balanced diagnostic benchmark. Experiments show a significant gap between lexical similarity and semantic alignment, with models achieving high token-level F1 often failing to reach perfect SAC, especially on longer examples.
A new method uses medical-term density filtering and signal-amplifying rephrasing to enhance French medical encoder pretraining. The approach outperforms educational quality filters and yields FineMed and DoctoBERT, achieving state-of-the-art results on DrBenchmark and a clinical NER task.
The paper argues that cultural alignment in NLP requires plural epistemologies, not just diverse data. It proposes a socio-technical model to analyze how multiple, locally grounded ways of knowing can be integrated into language technology, emphasizing that current approaches often fail to address deeper issues of power and governance.
BioMatrix integrates sequences, structures, and language for molecules and proteins in a single decoder-only architecture. It achieves state-of-the-art or competitive performance on 77 out of 80 downstream tasks, demonstrating effective multimodal generalist capabilities without external components.
A study finds that artificial agents learn visual word meanings best when concepts are perceptually close, with acquisition accuracy strongly predicted by perceptual distance (partial R² = 0.245). Bidirectional evaluations reveal that retrieval performance depends on exemplar-based memory, not prototype matching, and frozen visual embeddings enable grounding while limiting learning without representational changes.
Evaluations show Fongbe translations achieve poor quality (1.0-2.2/5) compared to Hausa's acceptable scores (4.0-4.5/5), with a consistent 3x BLEU gap. Automatic metrics like BERTScore show embedding collapse and weak human correlation, especially for Hausa, while Gemini outperforms others for Fongbe and GPT-4o for Hausa in human judgments. Minimum sample sizes of 2,500 sentences are needed for stable model rankings.