media r/LocalLLaMA · 1h ago · open_models

Custom code makes DeepSeek V4.1 run faster than official API on A100 GPUs

from English

A user has developed a custom implementation that allows the DeepSeek V4.1 model to run on NVIDIA A100 GPUs with performance exceeding the official API.

The project is available via the GitHub repository shi3z/deepseekv4.1-A100-custom.
This achievement is notable because the A100 hardware does not natively support FP4 precision, which is typically required for such optimizations.
The custom code enables faster inference speeds compared to the standard official API endpoint.

Importance 2/3 r/LocalLLaMA DeepSeek Hardware & chips Inference efficiency

Read original

media Latent Space · 5h ago · 1 view

DeepSeek releases v4.1-Flash with 763B-P8B-D16B architecture for efficient inference

DeepSeek has launched DeepSeek v4.1-Flash, a new open-weight flagship model featuring a novel causal encoder-decoder architecture designed to maximize inference efficiency and reduce costs.

media Hugging Face Forums · 10h ago · 1 view

Proposal for vendor-neutral AI runtime with memory-budget-aware execution

A proposal on the Hugging Face forums outlines a common GPU execution and memory-management layer designed to decouple application code from specific hardware vendors like NVIDIA, AMD, and Intel. The goal is to allow users to specify a memory budget rather than manually configuring backend-specific flags, device maps, or offloading strategies.

lab NVIDIA Research · 14h ago · 4 views

Onyx achieves 1.7-9.9x lower cost and 2.3-12.3x lower latency for disk-oblivious ANN search

Researchers propose Onyx, a cost-efficient approach to disk-oblivious approximate nearest neighbor (ANN) search that balances bandwidth and access count by inverting the design of state-of-the-art ORAM-ANN systems.

lab NVIDIA Research · 14h ago · 2 views

NVIDIA releases GPIR for GPU-accelerated private information retrieval

Researchers from NVIDIA have introduced GPIR, a system that accelerates Private Information Retrieval (PIR) on GPUs by addressing the high server-side computation and memory traffic inherent in lattice-based protocols. The system employs a stage-aware hybrid execution model that dynamically switches between operation-level and stage-level kernels to maximize on-chip data reuse.

lab NVIDIA Research · 14h ago · 4 views

NVIDIA characterizes MPC and FHE performance and cost for PPML

A new study presents a unified system-level characterization of secure multiparty computation (MPC) and fully homomorphic encryption (FHE) for privacy-preserving machine learning inference. The work evaluates two MPC variants—arithmetic/binary sharing conversion and function secret sharing—alongside FHE across multiple CNN and Transformer models.