A causal investigation of vocabulary-logit steering on the Qwen2-1.5B model reveals that the largest coordinates in the steering vector are not necessarily responsible for the observed behavioral effects. The study, conducted by ntrillard, analyzes how sparse ranked logit steering influences token generation and identifies the specific structural components that drive these changes.
- Transport (generation of target words) appears around K≈150 and persists through K≈300, with K=200 being the maximum-alignment point but not a unique behavioral optimum.
- Pure row(W) projection (λ=1) shows zero transport across 30 seeds, while every λ<1 condition retains a nonzero out-of-row component and shows some transport.
- The effect is identified as lexical forcing rather than semantic transport; boosting top-200 lexical neighbors yields 27/30 transport, whereas unboosted semantic neighbors yield 0/30.
- Deleting the single largest coordinate does not eliminate the effect, indicating no single load-bearing coordinate exists in the steering vector.
- Recomputing the vector dynamically per prefix or self-context barely changes the vector and does not improve performance over static vectors.
The findings suggest that the interaction between coordinate identity and ranked magnitude is critical for steering success, challenging assumptions about which vector components drive model behavior.