media r/LocalLLaMA · 9h ago

I reverse engineered Windows Copilot into a free OpenAI-compatible API

A user has created a local API that replicates OpenAI-compatible GPT-4 functionality using Microsoft's free Copilot service. The tool logs into a Microsoft account once, runs locally on a Windows device, and exposes a server at http://localhost:8000/v1 that supports streaming and multi-turn conversations without requiring an API key or billing. It is designed for personal and educational use, and available via GitHub at https://github.com/sums001/Windows-Copilot-API.

lab Google DeepMind Blog · 9h ago

Gemini 3.5 Flash Adds Computer Use Capability

Google has introduced computer use in Gemini 3.5 Flash, enabling the model to execute code and interact with external tools. This feature allows users to run programming tasks and access real-time information through integrated computing functions.

media r/LocalLLaMA · 10h ago

Has anyone else found vLLM outputs worse than llama.cpp?

A user reports noticing less reliable outputs from vLLM compared to llama.cpp, including formatting errors, context forgetting, and lower code quality. They ask whether such differences stem from quantization, chat templates, parser issues, or configuration errors, and seek confirmation if others have observed similar quality discrepancies between inference backends.

media r/LocalLLaMA · 11h ago

Build a LLM from Scratch using MLX

A developer created a Nano LLM with 20.2M parameters on a MacBook Air using the MLX framework. The project demonstrates that building a large language model from scratch is feasible with minimal hardware and basic Python knowledge.

media r/LocalLLaMA · 14h ago

llama.cpp web UI adds optional JavaScript execution via Web Workers

llama.cpp's web UI now supports executing JavaScript generated by language models in the browser using Web Workers, enabled via an opt-in setting. The code runs in a sandboxed iframe with security restrictions, though network requests appear disabled and the allowed sandbox capabilities lack clear documentation.

media r/LocalLLaMA · 15h ago

Dual GPU Sanity Check: Is This a Smart Buy?

A user asks whether adding a GTX 5060 Ti 16GB to their existing RTX 5090 setup is worth it for better VRAM to run larger LLMs and extend ComfyUI video generation. The upgrade would allow using Qwen 3.6 with 256K context and improve 1440p video generation, though performance gains in ComfyUI are limited due to current software constraints.

media r/LocalLLaMA · 15h ago

Qwen-AgentWorld-35B-A3B for Coding?

The Qwen-AgentWorld-35B-A3B model shows strong performance in coding tasks, with a 65.63% score on Software Writing Evaluation and 65.92% overall benchmark. It outperforms Qwen3.5-35B-A3B and rivals larger models in agent-based tasks, with a first impression noting superior accuracy in long-term agent workflows.

media r/LocalLLaMA · 15h ago

Gemma 4 26BA4B Surprisingly Usable at IQ3_S

A user reports that Gemma 4 26B quantized to Q3 runs at 25 tokens per second on a MacBook Air, performing nearly as well as bf16 for non-coding, tool-calling tasks. They question whether this performance reflects confirmation bias or if small quantized models are genuinely usable.

arxiv arXiv cs.AI · 16h ago

Text2DSL: LLM-Based Code Generation for Domain-Specific Languages

This paper introduces Text2DSL, a distinct task of generating domain-specific language code from natural language. Using the PolkitBench dataset of 4,204 validated pairs, it shows that structured context—such as BNF grammar and API specs—boosts syntactic and structural validity and CodeBLEU scores by 60% to 95% across different LLM models, without fine-tuning.

media r/LocalLLaMA · 16h ago

Qwen3.6 27B more dumb in vLLM compared to llama.cpp

A user reports that Qwen3.6-27B runs significantly less intelligently in vLLM than in llama.cpp, exhibiting issues like ignoring messages, hallucinating tool calls, and failing to recognize prior conversation context. Despite proper configuration and prompt templates, the model appears to lose coherence and misinterprets its own tool usage, with errors occurring consistently rather than sporadically.

github llama.cpp · 17h ago

vulkan-shaders-gen now fails build on shader compilation errors

The vulkan-shaders-gen tool now detects and fails the build when shader compilation fails, preventing the creation of a broken libggml-vulkan. This fix addresses a prior issue where build success masked runtime failures, and includes improvements to error handling and atomic flag management across platforms.

arxiv arXiv cs.LG · 18h ago

TRIZ-Inspired Text-to-CAD Framework Enhances Creative Design

A TRIZ-inspired text-to-CAD framework uses large language models to generate creative, editable 3D CAD models by integrating inventive principles from patent data. In a chair design case study, it achieved 4.0-14.7% mass reduction while preserving structural integrity through principles like segmentation and composite materials.

arxiv arXiv cs.LG · 18h ago

CAT-Translate: Compact Japanese-English Translation Models

CAT-Translate introduces a family of small, open-source models (0.8B to 7B parameters) specialized for Japanese-English bidirectional translation. Using synthetic parallel corpora and a two-stage fine-tuning approach with Multi-Objective GRPO, the models outperform multilingual models on real-world benchmarks across business, legal, medical, financial, and patent domains.

arxiv arXiv cs.LG · 19h ago

ASCII Art Enables Text-Only LLMs to Control VLA Systems

A text-only large language model can be adapted into a Vision--Language--Action controller by using ASCII-rendered visual observations. This approach allows LLMs to interpret visual states through text, enabling them to follow natural-language instructions and generate executable actions in both simulation and on physical manipulators.

media Hugging Face Forums · 19h ago

I Built an MCP Server in Go for AI Agents - 200 Lines Tutorial

A 200-line Go tutorial demonstrates building a lightweight Model Context Protocol server using Go's concurrency and simplicity. The server enables AI agents like Claude to access structured data and Go applications, potentially making them 10x more useful.

media Hugging Face Forums · 19h ago

Best model for local usage and working on Unity with MCP at 12 GB VRAM

A user is seeking a lightweight LLM tailored for Unity 6.5 with MCP, operating within 12 GB VRAM. They currently rely on free tiers of Cursor and Claude but find them insufficient, asking if any specialized models exist or alternative solutions are available.

media Hugging Face Forums · 19h ago

Wav2vec2 and WavLM Audio Classifier Stuck at 33% Accuracy

A user reports that fine-tuning wav2vec2-base or wavlm-base-plus for 3-class audio classification achieves only 33% accuracy, matching chance levels. The model is trained with only the classification head updated, using padded clips of 1.0s duration without attention masks, and with a learning rate of 1e-3, leading to poor performance despite class imbalance and short input clips.

media r/LocalLLaMA · 20h ago

llama-server crashes with 'pasted as file' prompt for image data extraction

llama-server crashes when a user pastes a long prompt as a text file alongside an image, treating it as a file attachment. The server works fine when the prompt is sent in smaller batches but fails when the full prompt is combined into one text block and submitted with the image.

media r/LocalLLaMA · 20h ago

Qwen releases 35B-parameter MoE for agent environment simulation

Qwen has launched Qwen-AgentWorld-35B-A3B, a 35B-parameter MoE model with only about 3B active parameters per token. It is trained to simulate responses from MCP, terminal, software engineering, Android, web, and OS GUI environments by predicting next observations after agent actions, enabling efficient agent training and environment simulation without real tool execution.

arxiv arXiv cs.CL · 21h ago

SHERLOC: Structured Diagnostic Localization for Code Repair Agents

SHERLOC introduces a training-free framework that pairs a reasoning LLM with compact repository tools and self-recovery. It achieves state-of-the-art localization accuracy and recall on SWE-Bench, improving repair agents' resolve rate by 5.95 percentage points while reducing localization and total token usage by 36.7% and 23.1% respectively.