A user applies norm-preserving biprojection abliteration to the Qwen3.6-35B-A3B model, achieving a 0% refusal rate on a held-out test set while keeping math and code benchmarks intact.
The technique orthogonalizes weight rows against the refusal direction and rescales them to their original L2 norm, preventing the benchmark degradation caused by vanilla abliteration. The author addresses specific challenges in the hybrid Mixture of Experts architecture, including handling mixed self-attention and linear attention layers, and applying projections via einsum to 3D expert tensors.
The work includes an open-source enriched harmful dataset with 7,356 prompts across 35 categories and provides model checkpoints and code for reproducing the results.