media Hugging Face Forums · just now · open_models

RunSnack enables P2P GPU sharing via a single link

from English

RunSnack is a tool that allows users to share GPU compute directly with others through a single link, eliminating the need for accounts or cloud platforms. It targets ML researchers and students who need short-term access to hardware like A100s or 4090s without traditional infrastructure.

Hosts run a script to generate a shareable link for their GPU.
Renters open the link in a browser to access an instant xterm.js terminal with GPU access.
Compute runs inside a sandboxed Docker container with read-only, privilege-dropped settings.
Data transfer uses WebRTC for encrypted, peer-to-peer communication without central server visibility.
The marketplace is organic, allowing users to negotiate prices directly via Discord or other channels.

The author seeks feedback from the ML community regarding security concerns, missing features like persistent storage, and pricing expectations compared to existing services.

Importance 1/3 Trust 1/3 Hugging Face Forums Hardware & chips Local inference

Read original

github llama.cpp · 1h ago Live

llama.cpp adds Granite-Switch architecture with in-graph router for per-token LoRA selection

The llama.cpp project introduces support for the Granite-Switch architecture, a dense model variant featuring N embedded LoRA adapters that are selected per-token via control tokens. This implementation replaces a previous proof-of-concept global sticky index with an in-graph causal "router" attention mechanism to resolve adapter indices.

media r/LocalLLaMA · 3h ago

Meta open sources on-device model Muse Glimmer

Meta has released the on-device model Muse Glimmer as an open-source project. The company also announced that a variant named Muse Spark 1.2 is scheduled for release soon.

github llama.cpp · 5h ago

llama.cpp b10336 release with ggml-webgpu refactor

The llama.cpp project has released version b10336, which includes a refactor of several wgsl files and simplification of flash_attn wgsl within the ggml-webgpu component.

media Hugging Face Forums · 1d ago · 8 views

vLLM Launcher provides a Windows desktop workbench for local LLM inference

A new project called vLLM Launcher has been released as a Windows desktop application designed to manage local large language model inference through WSL2. This tool allows users to control multiple engine backends, including vLLM, SGLang, and llama.cpp, from a single graphical interface.

github llama.cpp · 2d ago · 1 view

llama.cpp b10330 fuses rms_norm, mul, and rope on CUDA

The llama.cpp project has released version b10330, introducing a CUDA optimization that fuses the rms_norm, multiplication, and RoPE operations into a single kernel. This change is accompanied by memory range checks for the fused operations and new test cases for broadcast weights.