OpenAI has released initial performance results for Jalapeño, its first custom inference chip, demonstrating significant gains in speed and power efficiency compared to existing commercial systems. The company states that the new architecture allows for higher throughput and lower latency simultaneously, addressing a common trade-off in current hardware.
Microsoft Research has released GigaPath-Flash and GigaTIME-Flash, efficient variants of its pathology foundation models designed to enable population-scale discovery by drastically reducing computational requirements.
Parallel has integrated OpenAI's GPT-6 Astra into its AI agent infrastructure, achieving a 50% reduction in both research time and code costs compared to prior models. The company reports that the new model allows agents to complete complex knowledge work significantly faster while maintaining high-quality output.
Researchers introduce Parallel Decoding Distillation (PDD), a simplified trajectory-based method to accelerate inference in diffusion and flow matching models. Unlike current approaches that rely on hard-to-optimize variational score distillation and adversarial losses, PDD predicts multiple denoising steps per network evaluation.
Researchers propose Onyx, a cost-efficient approach to disk-oblivious approximate nearest neighbor (ANN) search that balances bandwidth and access count by inverting the design of state-of-the-art ORAM-ANN systems.
Researchers from NVIDIA have introduced GPIR, a system that accelerates Private Information Retrieval (PIR) on GPUs by addressing the high server-side computation and memory traffic inherent in lattice-based protocols. The system employs a stage-aware hybrid execution model that dynamically switches between operation-level and stage-level kernels to maximize on-chip data reuse.
A new study presents a unified system-level characterization of secure multiparty computation (MPC) and fully homomorphic encryption (FHE) for privacy-preserving machine learning inference. The work evaluates two MPC variants—arithmetic/binary sharing conversion and function secret sharing—alongside FHE across multiple CNN and Transformer models.
OpenAI has scaled its internal online storage platform, Habitat, to handle over 70 million requests per second for more than one billion weekly ChatGPT users across nearly 40 geographic regions. The system now serves more than 500 petabytes of data, evolving from a simple Python client-side library into a complex distributed service.
Researchers have introduced Twill, a system that formulates software pipelining (SWP) and warp specialization (WS) as a joint optimization problem solvable by off-the-shelf constraint solvers. This approach automates the derivation of optimal execution schedules for iterative programs on GPU architectures.
DeepSeek has released V4.1-Flash, a new open-weight flagship model designed for extreme inference efficiency and low cost. The model utilizes a causal encoder-decoder architecture to significantly reduce active compute and KV/cache costs.
DeepSeek has launched DeepSeek v4.1-Flash, a new open-weight flagship model featuring a novel causal encoder-decoder architecture designed to maximize inference efficiency and reduce costs.
DeepSeek AI has released DeepSeek-V4.1-Flash, a multimodal Mixture-of-Experts model featuring a 1-million-token context window and an FP4 KV cache that reduces global cache footprint to 890 bytes per token. The model utilizes a causal encoder-decoder architecture and Compressed Sparse Attention 2 (CSA2) to significantly lower memory requirements while maintaining performance.
The vLLM project released version 0.29.0, which includes a core bugfix regarding the dense prefix cache.
The v0.29.0rc5 release updates the default value of the `prefix_cache_retention_interval` parameter to `dense` specifically for Mamba models.
Multiverse Computing introduces Quantization-Aware Healing (QAH), a method that distills compressed, quantized large language models directly from their original pre-compression teachers. Applied to a GPT-OSS 120B model compressed to 60B parameters and quantized to MXFP4, the approach produces a model that beats its own full-precision bfloat16 version on 7 of 9 benchmarks.
The llama.cpp project released version b11120, which addresses a critical bug in the Vulkan backend by hiding internal symbols. This change prevents duplicate-dlopen state destruction issues that occurred when internal symbols were exported.
Researchers introduce CliffCompaction, an autocompaction technique designed to reduce costs for long-horizon coding agents by up to 50% under a bounded context. The method maintains or improves performance on Terminal-Bench and achieves state-of-the-art results on KernelBench by keeping compacted information faithful through truncation rather than rephrasing.
Researchers introduce CliffCompaction, an autocompaction technique designed for agents handling millions of tokens, which cuts costs by up to 50% under bounded context. The method maintains or improves performance on Terminal-Bench and achieves state-of-the-art results on KernelBench by keeping information faithful through truncation rather than rephrasing.
The llama.cpp project has released build b11118, which includes a new hex-dma feature introducing a direct-mapped DMA cache better suited for Hexagon Vector eXtended (HVX) FA mask handling.
The llama.cpp project released build b11111, which introduces Vulkan-based Intel Xe flash attention optimization kernels for split k paths on Xe-LPG Plus, Xe2, and Xe3 architectures.