ByteDance has announced a commitment to developing its next-generation AI model without relying on AI distillation techniques.
The company intends to pursue an independent development path for its future models.
media
r/LocalLLaMA
·
1h ago
·
open_models
ByteDance vows to avoid AI distillation, develop new model its own way
from English
Related articles
media
r/LocalLLaMA
·
7h ago
·
3 views
MiniMax releases open-weight H3 video generation model with native stereo audio
MiniMax released the open-weight H3 omni-modal video generation model on Hugging Face, featuring native stereo audio that can drive video in a single forward pass. The model supports 2K resolution at 24fps for clips lasting 5 to 15 seconds and accepts up to nine reference images, three videos, and three audio clips per generation.
arxiv
arXiv cs.LG
·
3d ago
U-OPSD enables unsupervised on-policy self-distillation for LLMs
Researchers propose Unsupervised On-Policy Self-Distillation (U-OPSD), a method that achieves post-training improvement for large language models using only the model's own generations without external supervision. The approach samples multiple rollouts to construct a pseudo-solution via majority vote, then distills a teacher distribution into prefixes of the model's longest incorrect completion.
arxiv
arXiv cs.CL
·
6d ago
·
1 view
Douyin releases DME multimodal embedding model with state-of-the-art performance
Douyin has released the technical report for its Multimodal Embedding (DME) model, which combines large-scale contrastive pre-training with latent reasoning to achieve strong discrimination and efficiency.
media
r/LocalLLaMA
·
6d ago
·
5 views
Apple's AFM3 20B uses instruction-following pruning to activate ~20% of layers
Apple has introduced a new architecture for its third-generation foundation models (AFM3) that utilizes "Instruction-Following Pruning" to significantly reduce active parameter count. The model is designed to activate approximately 20% of its MLP layers by making routing decisions once per prompt rather than per token.
media
MarkTechPost
·
8d ago
·
4 views
NVIDIA NeMo releases Molt, a compact PyT-native agentic RL framework
NVIDIA's NeMo team has released Molt, an open-source agentic reinforcement learning framework designed to reduce the complexity of algorithm iteration for researchers. The codebase is approximately 8.6K lines of RL code, significantly smaller than comparable frameworks like verl (62K lines) and slime (25K lines).