The article proposes Symphony, a hybrid programmable and specialized architecture designed to address memory system inefficiencies in current high-performance architectures like GPUs. It focuses on orchestrating data throughout the memory hierarchy to reduce unnecessary data movement and distance.
Researchers address temporal discontinuities in long-horizon video relighting by reframing the task as temporally conditioned latent domain translation. The framework enforces cross-chunk continuity by propagating target-domain latents across boundaries and utilizes masked target-domain self-conditioning to make this behavior learnable.
Researchers from Nvidia have introduced FusionRelight, a method for portrait relighting that combines physically plausible illumination transfer with identity preservation and compact real-time inference. The approach utilizes Hybrid Domain Knowledge Fusion (HDKF), a training framework that distills physics, reflectance, and realism priors from synthetic, One-Light-at-a-Time (OLAT), and in-the-wild data into a student model.
A new paper explores using foundational large language models (LLMs) to automate hyperparameter optimization (HPO), a process that typically relies on manual approaches in limited-budget settings. The authors developed a methodology where LLMs suggest hyperparameter configurations based on dataset and model descriptions, which are then iteratively refined according to model performance.
Researchers introduce Source, a new training data attribution (TDA) method that combines the computational efficiency of influence functions with the accuracy of unrolling-based approaches. By using an influence-function-like formula to approximate unrolled differentiation, Source addresses limitations in implicit differentiation methods, such as their failure to account for optimization bias or multi-stage pipelines.
NVIDIA has introduced OmniDreams, a foundation generative world model designed to address the bottlenecks in evaluating autonomous driving policies within closed-loop simulations. Mid- and post-trained from the Cosmos diffusion model on 21k hours of driving scenarios, it autoregressively generates action-conditioned videos in real time.
The paper introduces JacNet, a neural network architecture that directly learns the Jacobian of an input-output function rather than the mapping itself. This approach allows for precise control over derivative properties, enabling the enforcement of structural priors such as invertibility and k-Lipschitz continuity.
The article introduces a thesis by jlorraine that addresses the challenges of applying bilevel or nested optimization to large-scale deep learning setups. While gradient-based optimization typically updates a single set of parameters, many applications require updating subsets of parameters on different objectives nested inside each other.
Researchers propose Masters, a mask-progressive reinforcement learning framework for distilling large-scale vision-language models into compact versions suitable for edge deployment. The method addresses the instability caused by the size gap between teacher and student models.
Researchers identify Relevant Visual Information Shift (RVIS) during decoding as the primary cause for the failure of existing visual token pruning methods in Multimodal Large Language Models (MLLMs). To address this, they propose Decoding-stage Shift-aware Token Pruning (DSTP), a training-free framework that aligns visual tokens with shifting reasoning requirements.
Researchers present GenRecal, a general-purpose distillation framework that transfers knowledge from large vision-language models (VLMs) to smaller, more efficient counterparts. The method addresses the challenge of heterogeneous VLM architectures by using a Recalibrator to align and adapt feature representations across different model types.
Researchers introduce Zone of Proximal Policy Optimization (ZPPO), a method that injects teacher knowledge into prompts rather than policy gradients to address brittleness in knowledge distillation and drift in reinforcement learning. ZPPO constructs Binary Candidate-included Questions (BCQ) and Negative Candidate-included Questions (NCQ) for hard questions, recirculating them via a replay buffer until the student's accuracy improves or they are evicted.
Researchers propose a novel think-answer distillation framework that improves compact vision-language models' ability to utilize visual evidence by masking salient reasoning prefixes. This approach addresses the "visual forgetting" issue common in long think-answer traces, where students lose focus on visual cues during extended reasoning.
Researchers propose SpatialClaw, a training-free framework that adopts code as the action interface to improve open-ended 3D and 4D spatial reasoning in vision-language models. Unlike existing agents that rely on single-pass execution or rigid tool calls, SpatialClaw maintains a stateful Python kernel pre-loaded with input frames and perception primitives.
Researchers propose Agent Explorative Policy Optimization (AXPO) to address the "Thinking-Acting Gap" in vision-language models that require external tools. This gap causes standard RL methods like GRPO to attempt tool use on only ~30% of rollouts, with high failure rates suppressing learning signals.
Researchers propose ROSA, a robotics foundation model serving system designed for robot factories that moves beyond single-robot, edge-computing assumptions. The system utilizes shared GPU-pool serving to allow fleets of robots to access server-class GPUs over the network.
Researchers at NVIDIA have introduced Spatial-IQ, a diagnostic framework designed to deconstruct the spatial intelligence of multimodal large language models (MLLMs). Unlike existing benchmarks that treat models as black boxes, this approach decomposes object counting in stacked 3D structures into nine perceptual and cognitive sub-tasks aligned with human developmental stages.
Researchers present Cluster-Guided Vector Quantization (CGVQ), a method designed to improve the rate-distortion performance of Gaussian primitive-based image compression. The approach partitions Gaussian parameters into homogeneous groups prior to quantization, addressing the inefficiency caused by storing large numbers of floating-point parameters per primitive.
Researchers analyze gameplay videos from the competitive shooter game Delta Force to extract and understand emergent non-verbal communication, such as gestures and movement patterns. This work addresses a gap in prior studies that have largely focused on MOBA games or verbal coordination in other shooters.
Researchers at NVIDIA are addressing the difficulty of scheduling competitive group play and recreating lab environments by adopting field experiment methodology. This approach integrates experimentation directly into tournament structures, creating hybrid tournament-experiments.