korshunov.ai — ML news

Daily ML news, summarized.

• Beyond Standard LLMs: Alternative Architectures Emerge

  While most large language models remain autoregressive decoder-style transformers, recent years have seen the rise of alternatives such as text diffusion models and linear attention hybrids. These approaches aim to improve efficiency or modeling performance, with some like code world models targeting specific use cases. The article highlights a range of open-weight models and notes that non-traditional architectures deserve deeper exploration in future coverage.

  Ahead of AI · Source

• DeepSeek V3.2: Sparse Attention and RL Updates

  DeepSeek released V3.2, an updated flagship model featuring sparse attention and reinforcement learning improvements. The model demonstrates strong performance comparable to GPT-5 and Gemini 3.0 Pro, and is available as an open-weight model, expanding access to advanced language capabilities.

  Ahead of AI · Source

• LLM Research Papers: The 2025 List (July to December)

  A curated list of research papers on large language models from July to December 2025 is presented, categorized by topics such as reasoning models, reinforcement learning, inference strategies, model releases, architectures, and multimodal systems. The list was created as a separate post to improve readability and accessibility, alongside the publication of the annual "State of LLMs 2025" review article.

  Ahead of AI · Source

• Categories of Inference-Time Scaling for Improved LLM Reasoning

  The article categorizes inference-time scaling techniques to improve large language model reasoning without altering model weights. It covers methods like Chain-of-Thought Prompting, Self-Consistency, Best-of-N Ranking, Rejection Sampling, Self-Refinement, and Search Over Solution Paths, with a focus on practical implementations and recent academic work.

  Ahead of AI · Source

• DeepSeek R1 Introduces RLVR and GRPO for LLM Reasoning

  DeepSeek R1, released in January 2025, introduced Reinforcement Learning with Verifiable Rewards (RLVR) using the GRPO algorithm to enable reasoning in large language models. The open-weight model performed comparably to top proprietary models and suggested training costs of around $5 million—significantly lower than previous estimates—though it does not account for research and development expenses.

  Ahead of AI · Source

• A Visual Guide to Attention Variants in Modern LLMs

  The article presents a visual gallery of 45 LLM architectures with model cards, covering recent attention variants used in open-source models. It includes a poster version available on Redbubble, designed for print with clear visuals, though smaller text elements may be hard to read at reduced sizes.

  Ahead of AI · Source

• My Workflow for Understanding LLM Architectures

  The author outlines a workflow for reverse-engineering LLM architectures by starting with technical papers and then using available config files and reference implementations from Hugging Face and the transformers library. This approach is effective for open-weight models, as working code provides concrete, verifiable details that papers often lack.

  Ahead of AI · Source

• Six Components of a Coding Agent

  The article outlines six key components of a coding agent, emphasizing the role of agent harnesses in managing code context, tool use, memory, and iterative feedback. It distinguishes between LLMs, reasoning models, and agents, explaining that agents operate as control loops using models and tools, with coding harnesses being specialized for software engineering tasks.

  Ahead of AI · Source

• Recent LLM Architectures Focus on Long-Context Efficiency

  Newer large language model architectures emphasize long-context efficiency by reducing KV-cache size and attention costs. Key innovations include KV sharing in Gemma 4, layer-wise attention budgeting in Laguna XS.2, compressed convolutional attention in ZAYA1-8B, and mHC with compressed attention in DeepSeek V4.

  Ahead of AI · Source

• LLM Research Papers: The 2026 List (January to May)

  A curated list of research papers from January to May 2026 focuses on reasoning models, reinforcement learning, agent systems, and long-context capabilities. It highlights hybrid architectures like Nemotron 3 and Qwen3.6, with emphasis on efficiency and practical applications in real-world agent systems.

  Ahead of AI · Source