arxiv arXiv cs.CL · 47d ago · src: 49d ago · research

P4IR Framework Improves LLM-Based Code Compliance Accuracy

from English

P4IR, a two-stage framework, uses supervised fine-tuning and Group Relative Policy Optimization to enhance large language model-based automated code compliance systems. It reduces tree edit and token-level Levenshtein distances by up to 23.8% and 38.6% respectively, outperforming leading LLMs like Claude Opus, GPT-5.2, and GLM-4.7 in zero-shot settings with few-shot prompting, and reduces false positives by a statistically significant margin.

Importance 2/3 Beats a top-lab benchmark New feature vs. leaders arXiv cs.CL Mistral AI Anthropic Meta AI Code generation Evaluation & benchmarks Reasoning models

Read original

arxiv arXiv cs.AI · 52d ago

Reverse-Engineering Transformer Attention with Executable Programs

A new method uses program synthesis to generate Python programs that reproduce attention patterns in transformer models. Fewer than 1,000 such programs achieve over 75% intersection-over-union similarity on TinyStories, and replacing 25% of attention heads with these programs increases perplexity by only 16% while preserving performance on question-answering tasks.

arxiv arXiv cs.CL · 52d ago · 2 views

LLM-as-Interface, ML-as-Predictor for Pediatric Appendicitis

ClaMPAPP, a hybrid system, uses an LLM to extract structured clinical features from free-text notes and passes them to an XGBoost classifier for diagnosis. It outperformed end-to-end LLMs in both internal and external validation, with better stability and fewer missed appendicitis cases, demonstrating superior diagnostic performance and safety in pediatric triage.

arxiv arXiv cs.AI · 46d ago

LLMs Benchmarked for Web Vulnerability Detection

A study evaluates six LLMs on detecting real-world web vulnerabilities in WordPress plugins, finding detection rates vary by model and prompt design. Claude Opus 4.6 achieved the highest detection rate at 63%, while Qwen 3.5 only reached 35%, and no model consistently identified all baseline vulnerabilities across iterations.

arxiv arXiv cs.AI · 46d ago · 1 view

Benchmark Evaluation of Small Language Models for Arabic NLP

A benchmark of 240 Arabic test items across eight domains and ten skills assesses twelve small language models in zero-shot settings. Gemma 3 (12B) achieved the highest overall score (4.548/5), followed by Aya and C4AI Command Arabic, with performance linked more to Arabic alignment and instruction-following than model size. Common failure modes include prompt leakage, hallucination, and weak task adherence.

arxiv arXiv cs.CL · 47d ago · 1 view

ORBIT: Training-Free Multi-Attribute Behavioral Steering

ORBIT enables training-free, simultaneous control of multiple behavioral attributes by using orthogonal subspace rotation. It achieves balanced, coherent steering across attributes without retraining, outperforming existing baselines on TraitFactory and ToneBank benchmarks.