Research paper — korshunov.ai

Topic · Research paper

We present a framework for evaluating agentic skills by constructing realistic tasks and assessing skill utility through task execution. Applied to 500 real-world skills, it generates 1,000 tasks and scoring rubrics, evaluating 19 agent-model configurations across proprietary and open-source models. Results show significant variation in instruction adherence and performance gains, with skills substantially altering model behavior compared to no-skill setups.

arxiv arXiv cs.CL · 9d ago

Non-negative Elastic Net Decoding for Information Retrieval

NNN decoding selects documents as a joint set that jointly reconstructs the query embedding via a sparse non-negative linear combination. It strictly extends dense retrieval by handling queries that dense retrieval fails on, especially in corpora with correlated documents, and achieves superior performance through end-to-end training of embeddings.

media r/LocalLLaMA · 9d ago

Glimmer 1: A 10,000-parameter foundational language model

Glimmer 1 is a 10,000-parameter language model trained on 500K tokens from FineWeb-Edu. It features a 512-token context window, a standard Llama architecture with 16 hidden dimensions, 2 layers, 4 attention heads, and 1 KV head using GQA, and is available on Hugging Face.

arxiv arXiv cs.CL · 10d ago

Post-Hoc Operators Fail to Improve Accuracy in Small Code Models

A measurement study finds that 26 semantic post-hoc operators do not improve held-out accuracy over Best-of-N in frozen small code models. While two operators—expression-layer recovery and adaptive consensus early-stop—offer benefits in compute efficiency or program recovery, none outperform BoN in accuracy. The results highlight systemic limitations in error detection and coverage, suggesting that model harnesses and error coverage must be improved before post-hoc reasoning is considered.

arxiv arXiv cs.AI · 10d ago

MA-SBI: Calibration-Free SBI via Side-Channel Guidance

MA-SBI introduces a calibration-free simulation-based inference framework that uses side-channel text, like regime labels or instructions, to correct for simulator misspecification. It employs a learned corrector to apply observation-space shifts before posterior inference, without needing ground-truth parameter pairs or retraining. On hide-the-calibration benchmarks, MA-SBI matches the oracle posterior with text alone, outperforming RoPE under limited data, and shows robustness on real-world epidemiological and cognitive-science datasets.

arxiv arXiv cs.AI · 10d ago

AI research documentation improves over decade

Analysis of 56,800 AI conference papers shows documentation practices improved from 2014 to 2024. Papers sharing both code and data increased from 11% to 64%, and estimated reproducibility rose from 28% to 64%. These improvements predate formal reproducibility checklists, indicating a broader shift toward open science.

arxiv arXiv cs.AI · 10d ago

Bayesian Audits Reveal Inconsistent AI Evaluation Timelines

Public AI evaluation archives show that a single terminal result can arise from two distinct pre-terminal histories, with estimated times to reach 95% of performance ceilings at 23.03 or 75.13. A candidate selection-aware frontier model fails synthetic recovery and uncertainty calibration, and is rejected by fixed audit gates. An archive-and-adjudication protocol verifies timing boundaries and falsifies unsupported frontier claims.

arxiv arXiv cs.LG · 10d ago

DP-FL Backdoor Attacks: RING Exploits Privacy for Malicious Signals

A new attack, RING, exploits differential privacy in federated learning to conceal backdoor signals while maximizing impact. It achieves 90.3% attack success against state-of-the-art defenses, up to 26.08x over baseline methods, and reveals a critical security gap in DP-FL due to inherent masking of malicious updates.

arxiv arXiv cs.CL · 9d ago

Bilingual fine-tuning improves low-resource ASR with language identification

A study finds bilingual fine-tuning enhances automatic speech recognition in low-resource languages when language identification is accurate. Including a language identification token at inference improves ASR performance when identification accuracy is low, especially in diverse language pairs across different families and writing systems.

arxiv arXiv cs.AI · 10d ago

IMPACTeen Dataset Released with English and Polish Versions

IMPACTeen is a dataset of 1,021 texts annotated from five perspectives—teenagers, parents, psychologists, communication experts, and teachers. It includes 5,100 annotation records covering social influence techniques, intentions, consequences, and resistance, with annotations validated through human editing. The dataset, created using LLM generation and human validation, is available in both Polish and English and supports research on social influence and language model training.

arxiv arXiv cs.AI · 10d ago

AI-Enabled Progress in Stable Menus of Public Goods

Experiments on EC 2025's 'Stable Menus of Public Goods' show that human-intuition prompts improve LLM performance and multi-turn interactions enhance ambitious steps. However, when compared to a first-year PhD student using an unpublished manuscript, the LLM is found to be slightly less effective.

arxiv arXiv cs.AI · 10d ago

HAMON: Passive Optical Forecasting for Long-Horizon Time-Series

HAMON uses passive optical components to perform long-horizon time-series forecasting, outperforming top digital models on ETTm2 across all horizons and on ETTh2 at all but the longest horizon. It achieves up to 14% lower MSE and relies on physical optical propagation without trainable digital layers, demonstrating that passive optical mixing can produce competitive forecasts.

arxiv arXiv cs.LG · 10d ago

A nonparametric two-sample test using PReLU-IPM

The study introduces PReLU-IPM, a new integral probability metric based on a neural network discriminator with a single node. The resulting PReLU-TST test is nonparametric, consistent, and asymptotically equivalent to standard IPM-based tests, showing higher power or competitive performance on simulated and real datasets.

arxiv arXiv cs.LG · 10d ago

Fixed-Size Neural Networks Achieve Arbitrary Sobolev Approximation

A new activation function enables fixed-size neural networks to approximate any function in Sobolev spaces $W^{s,\infty}((a,b)^d)$ with arbitrary accuracy in the $W^{s-1,\infty}$-norm. The results use elementary activations like EUAF and DUAF$_\infty$, with explicit width and depth bounds, and extend to sigmoidal variants $\widetilde{\mathrm{DUAF}}_n$ preserving accuracy for all $1\leq s\leq n$.

arxiv arXiv cs.LG · 10d ago

Analytic Torsion and Spectral Gap Capture Persistent-Laplacian Performance

A compact spectral representation using Betti numbers, spectral gap, and analytic torsion distills persistent Laplacians into three mathematically grounded invariants. This approach captures essential predictive signals from the full spectrum, outperforms it in some cases, and reduces computational overhead on datasets like MNIST, QM-3D, and SKEMPI WT.

arxiv arXiv cs.LG · 10d ago

Multi-Center Benchmark for Abdominal Disease Diagnosis from Non-Contrast CT

A new multi-center benchmark enables abdominal disease diagnosis and report generation from non-contrast CT by synthesizing contrast-enhanced findings. The dataset includes paired NCCT-CECT studies and reports from two centers, showing NCCT achieves average multi-organ AUCs of 69.1% internally and 63.1% externally. The benchmark and code are publicly released to support research into safer, contrast-free abdominal imaging workflows.

arxiv arXiv cs.LG · 10d ago

HAMON: Passive Optical Forecasting Core

HAMON uses passive optical diffraction to generate forecasts, outperforming digital baselines on ETTm2 at all horizons and ETTh2 at all but the longest horizon. It achieves up to 14% lower MSE and operates without trainable digital mixing, relying instead on physical optical propagation.

A Framework for Evaluating Agentic Skills at Scale

Non-negative Elastic Net Decoding for Information Retrieval

Glimmer 1: A 10,000-parameter foundational language model

Post-Hoc Operators Fail to Improve Accuracy in Small Code Models

MA-SBI: Calibration-Free SBI via Side-Channel Guidance

AI research documentation improves over decade

Bayesian Audits Reveal Inconsistent AI Evaluation Timelines

DP-FL Backdoor Attacks: RING Exploits Privacy for Malicious Signals

Bilingual fine-tuning improves low-resource ASR with language identification

IMPACTeen Dataset Released with English and Polish Versions

AI-Enabled Progress in Stable Menus of Public Goods

HAMON: Passive Optical Forecasting for Long-Horizon Time-Series

A nonparametric two-sample test using PReLU-IPM

Fixed-Size Neural Networks Achieve Arbitrary Sobolev Approximation

Analytic Torsion and Spectral Gap Capture Persistent-Laplacian Performance

Multi-Center Benchmark for Abdominal Disease Diagnosis from Non-Contrast CT

HAMON: Passive Optical Forecasting Core