LLaMA.cpp release b9684 introduces a new 3D convolution operation (conv_3d) and includes optimized implementations. The release provides prebuilt binaries for macOS, Linux, Android, Windows, and openEuler across various architectures and hardware acceleration options, including SYCL, Vulkan, CUDA, and OpenVINO.
llama.cpp version b9682 introduces Vulkan support for Linux and Windows, enabling GPU acceleration. The release includes binaries for macOS, Linux, Android, Windows, and openEuler across multiple architectures, with CPU and GPU options including CUDA, OpenVINO, SYCL, and ROCm.
llama.cpp version b9675 enables FP16 support for operations like SQR, SQRT, LOG, SIN, COS, and CLAMP. The release includes binaries for macOS, Linux, Android, Windows, and openEuler across multiple architectures, with support for Vulkan, ROCm, OpenVINO, SYCL (FP16 and FP32), and CUDA 12.4 and 13.3.
llama.cpp releases version b9680 with updated Vulkan support and new binaries for macOS, Linux, Android, Windows, and openEuler. The release includes CPU and GPU variants for multiple architectures, with support for Vulkan, CUDA, OpenVINO, SYCL, and ROCm.
AoiZora is a compiler-mediated topology planner that improves low-latency video diffusion inference on TPU sub-slices. By aligning logical sharding with physical placement through the compilation flow, it reduces one-step denoising latency by up to 1.42x on TPU v5e sub-slices compared to existing methods.
llama.cpp version b9669 adds backend sampling support for Eagle3. The release includes binaries for macOS, Linux, Android, Windows, and openEuler across multiple architectures and hardware acceleration options, including Vulkan, CUDA, ROCm, OpenVINO, and SYCL.
llama.cpp release b9670 includes fixes for NVFP4 edge cases in llama-graph, such as moving post-GEMM MUL operations and restricting build_ffn to supported combinations. The release provides binaries for macOS, Linux, Android, Windows, and openEuler across multiple architectures and backend options, including CUDA, Vulkan, SYCL, and OpenVINO.
llama.cpp release b9667 introduces Vulkan support with S_v=16 via gated_delta_net. It includes binaries for macOS, Linux, Android, Windows, and openEuler across multiple architectures, with options for Vulkan, CUDA 12.4 and 13.3, ROCm, OpenVINO, and SYCL.
LLaMA.cpp release b9663 adds support for OP EXPM1 and all unit test cases for FLOOR, TRUNC, and ROUND. It includes updated binaries for macOS, Linux, Android, Windows, and openEuler, with support for SYCL (FP32 and FP16), Vulkan, CUDA 12.4 and 13.3, and ROCm 7.2, along with an updated UI.
The sycl update extends support for reordered expert tensor handling in MoE MUL_MAT_ID to Q4_K, Q5_K, and Q6_K. Unsupported 3D reorder cases now fallback instead of aborting.
The llama.cpp release b9661 adds GGML_OP_COL2IM_1D support for Vulkan, using a bounded gather loop instead of full-K scan with modulo. It returns nullptr for unsupported types and includes builds for macOS, Linux, Android, Windows, and openEuler across CPU, Vulkan, CUDA, and SYCL.
The DGX Spark is being unfairly criticized despite its strong scalability and usable local AI performance. Its ConnectX technology allows lossless expansion, and at 240W power, it enables running agentic DS4Flash locally for around $9k with 256GB of CUDA memory.
HAMON uses passive optical components to perform long-horizon time-series forecasting, outperforming top digital models on ETTm2 across all horizons and on ETTh2 at all but the longest horizon. It achieves up to 14% lower MSE and relies on physical optical propagation without trainable digital layers, demonstrating that passive optical mixing can produce competitive forecasts.
HAMON uses passive optical diffraction to generate forecasts, outperforming digital baselines on ETTm2 at all horizons and ETTh2 at all but the longest horizon. It achieves up to 14% lower MSE and operates without trainable digital mixing, relying instead on physical optical propagation.