Quy-Anh Dang

Quy-Anh Dang

Researcher, Knovel Engineering


← All work

LLM Interpretability, Layer by Layer — Part 6: Steering and Editing

November 7, 2026 · 7 min read · Interpretability, Large Language Models, Machine Learning

In Part 4 we changed internal activations to test a hypothesis. This post changes them to achieve something: make the model behave differently, on purpose, without retraining it.

This is the control rung of the ladder. If we really understand how a concept is represented, we should be able to write it into the model ourselves. By the end you'll know:

  1. steering vectors: adding a direction to the residual stream,
  2. how to flip a fact (Kobe Bryant → golf) and why the choice of direction matters,
  3. how to steer open-ended writing (sentiment), and what it costs in fluency,
  4. how steering relates to ablation and model editing, and how to check for side effects.

1. Steering vectors#

The idea follows from the linear representation hypothesis in Part 2. If "plays golf rather than basketball" is a direction in the residual stream, then adding that direction should make the model "believe" it more:

Here is the layer, the position, and the steering strength. Nothing else changes: same weights, same prompt. We just add a vector during the forward pass.

The direction usually comes from a contrast:

  • Activation addition (ActAdd; Turner et al., 2023): the difference between activations on two prompts, such as "Love" minus "Hate".
  • Contrastive activation addition (CAA; Rimsky et al., 2024) and other mean-difference methods: the average difference over many pairs. This is exactly Part 2's difference of means.
  • SAE features (Part 3): a feature's decoder column is a ready-made direction.
  • Probe weights: the direction a linear probe uses (often less effective for steering than mean differences).

ELI5: Part 0's office corridor again. Steering is slipping an extra sticky note into every folder at one desk, one that says "think golf." Every worker further down reads it along with everything else.

2. Flipping a fact#

We take the 24 basketball players GPT-2 gets right, and at layer 6, at the athlete's last-name token (where Part 4 found the sport is carried), we add

and watch and as grows. We try two ways of building the golf average, plus a random direction of the same size as a control:

Steering 24 basketball players toward golf at layer 6. With the vector built from the two golfers GPT-2 knows, P(basketball) falls from 0.44 to almost 0 and P(golf) rises from 0.03 to 0.51. With the vector built from all 30 golfers, P(basketball) falls to 0.10 but P(golf) barely moves. A random direction changes little.

With golfers the model knows (orange). GPT-2 small clearly prefers golf for only two golfers, Tiger Woods and Jack Nicklaus. A direction built from just those two works well. At , P(basketball) has dropped from 0.44 to 0.05 and P(golf) has risen from 0.03 to 0.43. It peaks at 0.51 around . Kobe Bryant plays the sport of … golf.

With all 30 golfers (green). This direction removes basketball (0.44 → 0.10) but installs nothing: P(golf) stays at 0.04. Most of the 30 golfers are unknown to GPT-2 small, so their average isn't "golf." It's closer to "an athlete I don't recognize." Adding it makes Kobe Bryant less familiar, not more golf-like. This is Part 2's lesson in a new form: a direction is only as good as the examples it was built from, and examples the model doesn't understand produce a direction that means something else.

Random direction (grey, dashed). Small pushes do nothing. Large ones slowly reduce P(basketball) (to 0.31 at ), because pushing the stream that hard in any direction degrades it. That's why a random control matters: it tells you how much of an effect is just "we disturbed the model."

3. Side effects#

Steering edits a representation, not a single fact. The "golf" vector at Kobe Bryant's name should change his sport, but what else does it change? We ask two other questions about him, with the same push ():

PromptUnsteered top answersSteered top answers
Kobe Bryant played for the Los AngelesLakers 0.94, Clippers 0.05Lakers 0.57, Dodgers 0.24, Rams 0.04
Kobe Bryant was born in the city ofLos 0.06, Kobe 0.03, Chicago 0.03Los 0.03, Chicago 0.02, St 0.02

The team question shows the side effect clearly. The Lakers still win, but the model now gives 24% to the Dodgers, a baseball team. The vector didn't just say "golf." It said "not basketball," and that bled into other basketball-linked facts. This is the general rule: steering at an entity changes everything the model associates with that entity, more or less, depending on how cleanly the concept is represented.

4. Steering open-ended writing#

Facts are easy to score. What about the tone of free text? We build a sentiment direction from 10 pairs of short sentences, such as "This is wonderful." vs "This is terrible." and "I love it." vs "I hate it.", by averaging the difference of their layer-6 residual streams at the last token. Then we add at every position while GPT-2 writes 25 tokens after neutral prompts like "I went to the store and".

To score the results, a separate sentiment classifier (DistilBERT fine-tuned on movie reviews) labels each continuation, and the unsteered GPT-2 measures how natural it is (perplexity: lower means more fluent):

Left: the share of continuations judged positive rises from 4% at strength -8 to 73% at +8, with 42% unsteered. Right: perplexity is lowest unsteered (52) and rises steeply in both directions, to 394 at +8 and 1,645 at -8.

Strength −8−4−20+2+4+8
Judged positive4%15%17%42%54%60%73%
Perplexity1,645241705277138394

The steering works in both directions: from 42% positive unsteered to 15% at and 60% at . Two continuations of "I went to the store and", with the same random seed:

  • : "…found "surprising" and I was glad to see lots of items, the shipping was easy."
  • : "…tried to hit him."

The cost shows on the right. Perplexity rises quickly with strength: 70 at , 241 at , and the text falls apart into word salad at . The direction's length is about 12, while the residual stream at layer 6 has a typical length of about 82. At we're adding a vector longer than the stream itself, so it's no surprise the model loses the plot.

The practical rule: there is a window of strengths that changes the behaviour before fluency collapses. Always report a quality measure (perplexity, task accuracy, or human ratings) next to the steering effect, as with interval width next to coverage in the conformal series.

5. In code#

layer = 6

def name_rep(who):
    _, cache = model.run_with_cache(f"{who} plays the sport of")
    pos = len(model.to_str_tokens(who)) - 1
    return cache[f"blocks.{layer}.hook_resid_pre"][0, pos]

golf = torch.stack([name_rep(w) for w in ["Tiger Woods", "Jack Nicklaus"]]).mean(0)
basketball = torch.stack([name_rep(w) for w in ["LeBron James", "Stephen Curry", "Kevin Durant",
                                                 "Larry Bird", "Tim Duncan", "Steve Nash"]]).mean(0)
d = golf - basketball

who = "Kobe Bryant"
pos = len(model.to_str_tokens(who)) - 1
b, g = model.to_single_token(" basketball"), model.to_single_token(" golf")
for c in [0.0, 0.5, 1.0, 1.5]:
    def steer(act, hook):
        act[:, pos] += c * d                       # add the direction at the name only
        return act
    probs = model.run_with_hooks(f"{who} plays the sport of",
                                 fwd_hooks=[(f"blocks.{layer}.hook_resid_pre", steer)])[0, -1].softmax(-1)
    print(f"c = {c}: P(basketball) = {probs[b]:.2f}, P(golf) = {probs[g]:.2f}")
# c = 0.0: P(basketball) = 0.60, P(golf) = 0.01
# c = 0.5: P(basketball) = 0.39, P(golf) = 0.05
# c = 1.0: P(basketball) = 0.12, P(golf) = 0.20
# c = 1.5: P(basketball) = 0.04, P(golf) = 0.30

This version uses only six basketball players for the average, so its numbers differ a little from the figure, which used all of them.

6. Steering, ablation and editing#

Steering is one of three related ways to intervene:

  • Steering adds a direction at inference time. It's cheap and reversible, and it can be switched on per request. But it touches every computation that reads the stream at that point, which causes side effects like the Dodgers above.
  • Directional ablation removes a direction: project it out of the stream, . A well-known result in chat models found a single "refusal direction": removing it made models stop refusing harmful requests, and adding it made them refuse harmless ones (Arditi et al., 2024). GPT-2 small has no safety training to refuse with, so we can't reproduce that here, but Part 7 uses ablation to remove a hidden behaviour.
  • Model editing changes the weights so a fact is permanently different. ROME (Meng et al., 2022) uses the same insight as Part 4, that facts are retrieved by MLPs at the subject's last token, and makes a targeted low-rank update to one MLP layer so that, for example, "Kobe Bryant" maps to golf. Edits face the same side-effect question, and evaluations check both specificity (other athletes unchanged) and generalization (the edit holds across paraphrases).

All three are evidence for understanding only to the extent they're specific: the behaviour you aimed at changes, and little else does. Report the side effects you measured, and the random-direction control, alongside the effect.

Recap#

  • A steering vector is a direction added to the residual stream at chosen layers and positions: . Contrast pairs and differences of means are the usual way to build one.
  • Adding a "golf minus basketball" direction at layer 6 flips GPT-2 small's answer for basketball players (P(golf) 0.03 → 0.51), but only when the direction is built from golfers the model knows. A direction from mostly unknown golfers removes basketball without adding golf.
  • Steering has side effects: Kobe Bryant's team drifted from the Lakers toward the Dodgers.
  • A sentiment direction steers free text both ways (15% to 60% positive at ), but fluency drops quickly. Always report quality alongside the effect, and compare to a random direction.
  • Ablation removes directions; editing (ROME) changes weights. All three should be judged by how specific they are.

Next up — Part 7: Auditing, and a Checklist. Everything so far started from a behaviour we already knew about. The real test of interpretability is finding one we don't. We plant a hidden behaviour in GPT-2 ourselves, then try to find, explain and remove it using only the tools from this series.


© 2026 Quy-Anh Dang. All rights reserved.