korshunov.ai
English
Open weights Page 1 / 11
arxiv arXiv cs.LG · 8d ago

Baseline Evaluation of Open-Source LLMs for Multi-Label ATT&CK Classification

A ground-truth dataset of 2,076 human-annotated sentences from 83 complex CTI reports was constructed and mapped to 114 ATT&CK techniques with \k{appa} = 0.68 inter-annotator agreement. Seven open-source LLMs ranging from 8B to 236B parameters were evaluated, achieving a maximum micro-averaged F1 score of 0.22. Parameter size showed a statistically significant positive correlation with F1 score, while prompt strategy and temperature did not yield significant improvements, indicating current open-source LLMs are insufficient for production-grade ATT&CK classification.
arxiv arXiv cs.CL · 8d ago

RubricsTree: Scalable Evaluation Framework for Personal Health Agents

RubricsTree introduces a hierarchical taxonomy of over 100 clinically-verifiable Boolean rubrics, evolved from 4,000 real user queries via human-in-the-loop curation. It enables scalable, expert-aligned evaluation of personal health agents by dynamically routing queries to relevant rubrics and outperforms baseline methods in alignment, context sensitivity, and model performance gains of up to 66% on HealthBench.
arxiv arXiv cs.CL · 8d ago

Encoding Al-Mawrid Dictionary with ISO LMF and TEI Lex-0

The paper details a methodology for digitizing the Al-Mawrid Arabic-English dictionary using ISO LMF and TEI Lex-0. It achieves 91% structural parsing accuracy and demonstrates 85% precision and 98% recall for synonyms, with 88% precision for morpho-semantic features, based on a sample of the letter Ayn. The study highlights TEI Lex-0 limitations in capturing Arabic semantic and morphological nuances and proposes a scalable prefix-based system for LLOD integration.
arxiv arXiv cs.CL · 8d ago

Darshana Graph: A Corpus for Comparative Indian Philosophy

Darshana Graph presents a corpus of over 125,000 text records from Hindu, Buddhist, and Jain philosophical sources. It includes a unique subset of 8,500 aligned records from 18 commentators across five schools, enabling cross-commentator comparison. The corpus supports stylometric analysis and a large language model pipeline that extracts philosophical concept relationships, revealing disagreement patterns and extraction limitations.
arxiv arXiv cs.LG · 8d ago

KANLib: A Modular and Efficient Kolmogorov-Arnold Network Framework

KANLib introduces a modular, extensible, and computationally efficient framework for Kolmogorov-Arnold Networks. It unifies core concepts from PyKAN, EfficientKAN, and FastKAN, supporting adaptive grid rescaling and fine-grained architectural customization while maintaining PyTorch compatibility. Experiments on the California Housing dataset show KANLib achieves competitive efficiency and reproduces established KAN performance.
arxiv arXiv cs.AI · 8d ago

IUU+DB: LLM-Driven Database for Illegal Fishing and Supply Chain Crimes

IUU+DB is a large language model-driven system that tracks illegal, unreported, and unregulated fishing, seafood fraud, and labor abuse. It extracts key data elements from diverse documents, classifies relevant incidents, and enables trend analysis to identify geographic and behavioral hotspots. The system supports research, risk assessments, and policy enforcement in fisheries and supply chains.
arxiv arXiv cs.AI · 8d ago

RubricsTree: Scalable Evaluation Framework for Personal Health Agents

RubricsTree introduces a hierarchical taxonomy of over 100 clinically-verifiable Boolean rubrics, evolved from 4,000 real user queries via human-in-the-loop curation. It enables scalable, expert-aligned evaluation of personal health agents by dynamically routing queries to relevant rubrics and outperforms baseline methods in alignment, context degradation detection, and model performance gains of up to 66% on HealthBench.