media Hugging Face Forums · 1h ago · open_models

mlx-smolvla enables native MLX execution of SmolVLA checkpoints on Apple Silicon

from English

The open-source project mlx-smolvla provides a Python runtime for executing SmolVLA checkpoints using MLX on Apple Silicon, featuring inference, a LeRobot protocol server, and a preview for LoRA training. Benchmarks on an Apple M5 Pro show the MLX path achieves 110.75 ms median latency for 50-action chunks, which is 1.85× faster than PyTorch-MPS.

Runtime is native MLX without Torch or Transformers dependencies.
Strict CPU compatibility lanes ensure deterministic comparison with PyTorch, while Metal uses statistical checks.
The project includes a policy server implementing LeRobot’s async-inference gRPC protocol.
LoRA training is currently a research preview with limited parity validation.

The author notes that raw base checkpoints lack effective physical-action interfaces and requires reviewed checkpoints with matching robot statistics for motion clients.

Importance 1/3 Trust 1/3 Hugging Face Forums Hardware & chips Local inference

Read original

media r/LocalLLaMA · 16h ago · 1 view

NVIDIA launches PAIR for personal AI clusters

NVIDIA has introduced PAIR, a tool designed to help users connect old hardware into a personal AI cluster. The initiative is aimed at leveraging existing devices to run AI workloads locally.

media Hugging Face Forums · 26d ago · 17 views

RunSnack enables P2P GPU sharing via a single link

RunSnack is a tool that allows users to share GPU compute directly with others through a single link, eliminating the need for accounts or cloud platforms. It targets ML researchers and students who need short-term access to hardware like A100s or 4090s without traditional infrastructure.

media Hugging Face Forums · 14h ago · 1 view

Heshware explores graph-based memory layer for local AI

Heshware is investigating whether a dynamically activated graph of memory nodes can serve as an interpretable long-term memory layer for a local language model. This proposal stems from the Marven Neural Node Network, originally an interactive visualization of cognitive processing.

github llama.cpp · 19h ago · 1 view

llama.cpp b10816 adds M3 fa-vec tunings and new binaries

The llama.cpp project has released version b10816, which includes updates to the Metal backend for Apple Silicon devices. The release focuses on adding remaining fast-vector tunings specifically for the M3 chip architecture.

github llama.cpp · 1d ago · 2 views

llama.cpp b10796 adds n_expert_used_max to fix Nemotron-3-Puzzle loading

The llama.cpp project released build b10796, which includes a change to the model loader to support models with varying numbers of experts per layer.