media r/LocalLLaMA · 19h ago · open_models

gtrg55 releases INT4 quantized NVIDIA Cosmos3 64B for local image generation

from English

A user has published code and weights for running NVIDIA's 64-billion parameter Cosmos3 model locally using INT4 quantization. The implementation supports both Text-to-Image and Image-to-Video tasks on Apple Silicon via MLX as well as CUDA.

The repository provides GitHub code and Hugging Face weights for the Cosmos3-Super-Text2Image-4Step-INT4-G64-BF16 model.
Quantization allows the large 64B parameter model to run on consumer hardware, with a single clip generation taking approximately five minutes on an M4 Max Mac with 128 GB of memory.
The project includes a comparison with Grok to demonstrate performance capabilities.

This release enables local inference of state-of-the-art image generation models on devices that typically lack the VRAM required for full-precision weights.

Importance 2/3 r/LocalLLaMA NVIDIA Image generation Inference efficiency

Read original

lab NVIDIA Research · 27d ago · 34 views

Nvidia presents FusionRelight for real-time portrait relighting via hybrid domain knowledge fusion

Researchers from Nvidia have introduced FusionRelight, a method for portrait relighting that combines physically plausible illumination transfer with identity preservation and compact real-time inference. The approach utilizes Hybrid Domain Knowledge Fusion (HDKF), a training framework that distills physics, reflectance, and realism priors from synthetic, One-Light-at-a-Time (OLAT), and in-the-wild data into a student model.

lab NVIDIA Research · 44d ago · 25 views

Clustered Codebook Quantization for 2D Gaussian-based Image Compression

Researchers present Cluster-Guided Vector Quantization (CGVQ), a method designed to improve the rate-distortion performance of Gaussian primitive-based image compression. The approach partitions Gaussian parameters into homogeneous groups prior to quantization, addressing the inefficiency caused by storing large numbers of floating-point parameters per primitive.

arxiv arXiv cs.LG · 85d ago · 7 views

AoiZora: Topology-Aware Auto-Parallel Optimization for Video Diffusion Inference

AoiZora is a compiler-mediated topology planner that improves low-latency video diffusion inference on TPU sub-slices. By aligning logical sharding with physical placement through the compilation flow, it reduces one-step denoising latency by up to 1.42x on TPU v5e sub-slices compared to existing methods.

arxiv arXiv cs.LG · 6d ago · 10 views

FlashAttention-4 introduces Direct-P for hardware-aware FP4 inference and training

FlashAttention-4 addresses the performance limitations of Blackwell's 4-bit floating-point (FP4) tensor cores by introducing new paths for noncausal inference and causal training. The method, called Direct-P, maps scores directly to FP4 probabilities to bypass softmax conversion overheads that typically dominate once matrix products shrink.

github llama.cpp · 7d ago · 10 views

llama.cpp adds support for NVIDIA Nemotron-3-Puzzle-75B-A9B

llama.cpp introduces support for the NVIDIA Nemotron-3-Puzzle-75B-A9B model, enabling inference for this Mixture of Experts architecture. The update implements infrastructure for variable-per-layer expert feed-forward sizes and top-k routing, which are required because the model has 40 MoE layers with distinct n_ff_exp and top-k values.