Survival of the Fitted: Qwen3.6-27B’s Jacobian lens reads and steers Qwen3.8-27B with zero refitting [R]

Interpretability lenses get fitted to one exact checkpoint, and as far as I can tell nobody had tested what a version update does to one. So this was my question:

when a model line updates, does the fitted instrument survive, or do you refit every release?

I tested the published Jacobian lens for Qwen3.6-27B (Neuronpedia, from Anthropic’s July workspace paper) applied unchanged to Qwen3.8-27B.

Setup: 3.8-27B shipped 113 days after 3.6-27B. Same 64 layers, same hidden dim, same tokenizer, training relationship undocumented. One protocol, both models, two readouts each: the transported Jacobian readout and the raw logit lens as baseline. bf16, greedy, single seed.

Reading result: the main task is 40 two-hop prompts where the middle entity is never stated. Example: “Fact: The currency used in the country shaped like a boot is”, where the target is Italy and Italy appears nowhere in the prompt. The transferred lens keeps the latent entity near the top of the 248,320-token vocab. Median rank at layer 48 is 4 on the home model vs 17 transferred. At layer 24 it’s 121 vs 38, so the successor is actually better at mid-depth (paired sign tests, p Steering result: I took pullback directions for “ paradox” / “ paradoxical” / 悖论 / 矛盾 from the 3.6 lens, orthogonalized within layer, and projected them out of 3.8’s residual stream at layers 18 to 47 during generation. Prompt: “Describe Escher’s impossible staircase”. The word paradox disappears from the output in all cells, on both models, while the description stays coherent (lithograph, closed loop, illusion all intact). Directions derived entirely from the old checkpoint still find the concept in the new one. Scope: one lens family, one model line, one version step, matched architecture and tokenizer. The design can’t fully separate lens misfit from model change, and I make no claim about cross-family transfer or larger gaps. The practical upshot is that cross-checkpoint transfer is measurable, so a monitoring pipeline can test its lens instead of assuming refit is required.

Eval code, the 40-prompt set, per-layer rank tables for all four model-by-readout cells, and the ablation captures: