arxiv arXiv cs.LG · 53d ago · src: 54d ago · research

LoopCoder-v2 Achieves Optimal Two-Loop Performance

from English

LoopCoder-v2, a parallel loop Transformer model, achieves superior code generation and reasoning performance with two loops, improving SWE-bench Verified from 43.0 to 64.4 points and Multi-SWE from 14.0 to 31.0 points. Variants with three or more loops perform worse, indicating a non-monotonic loop-count effect due to growing positional mismatch and diminishing returns.

Importance 2/3 arXiv cs.LG Hugging Face Code generation Evaluation & benchmarks

Benchmarks

Benchmark	Model	Score
SWE-bench Verified	LoopCoder-v2	64.4%

Read original

arxiv arXiv cs.CL · 47d ago · 1 view

CAT-Translate: Compact Japanese-English Models Outperform Multilingual Ones in Real-World Tasks

CAT-Translate introduces a family of small, open-source models specialized for Japanese-English translation. Using synthetic parallel corpora and a two-stage fine-tuning approach, the models achieve superior performance on real-world benchmarks across business, legal, medical, financial, and patent domains, outperforming large multilingual models in practical applications.

arxiv arXiv cs.CL · 51d ago · 2 views

JAMER: Project-Level Code Framework Dataset and Benchmark

JAMER introduces JamSet and JamBench, the first project-level game code dataset and benchmark on a professional game engine. Built from 8,133 verified Game Jam projects, it enables deterministic evaluation and reveals a capability cliff in AI models as project scale increases, with runtime pass rates dropping from 80.4% to 5.7%.

arxiv arXiv cs.AI · 52d ago · 1 view

Data Intelligence Agents Enable Autonomous Data Querying

Data Intelligence Agents (DIA) deploy autonomous coding agents to streamline enterprise data workflows. The Query Generator matches or exceeds top published results on seven SQL benchmarks across four dialects, showing generalization through natural-language instructions and execution-based architecture.

arxiv arXiv cs.AI · 52d ago · 1 view

CAPRA: Multi-Agent LLM System for Software Architecture Feedback

CAPRA is a multi-agent LLM system that generates personalized, template-compliant LaTeX feedback on software architecture deliverables. It uses specialized agents, PyMuPDF, and gpt-4o to extract and analyze text and UML diagrams, with evidence anchoring and consistency management to ensure reliability. A preliminary evaluation of 10 student reports shows CAPRA met 88.8% of eight criteria and achieved moderate inter-rater agreement (kappa = 0.582), with each report processed in under 4 minutes.

media Hugging Face Forums · 12d ago

Calibra: Robot Dataset Observability for LeRobot

Calibra is an open-source toolkit designed to audit robot datasets and identify quality issues before training. It provides tools for building quality-aware coresets and estimating training outcomes.