A user has released a fork of llama.cpp featuring an experimental loop detection sampler designed to identify and break repetitive token cycles during inference. This new composable sampler monitors the generated token stream in real time, detects exact repeating patterns, and temporarily increases sampling temperature to disrupt the loop.
- The sampler uses parameters like last_n (default 64), min_pattern_len (default 3), and min_reps (default 3) to define detection sensitivity.
- When a loop is detected, it applies a temperature multiplier (temp_factor) to increase randomness and push the model out of the cycle.
- Configuration is available via CLI flags, JSON API for llama-server, or programmatically through the C API sampler chain.
- The implementation allows tuning to balance between catching loops early and avoiding false positives on legitimate repeated phrases.