Skip to content

Projects

Projects
Research2026★ 19

AXON

Watch a language model think, token by token

Hooks GPT-2’s residual stream at layer 8, runs each token’s hidden state through a sparse autoencoder, and streams the top-K firing features into a live 3D force graph in the browser.

//The problem

Mechanistic interpretability results normally arrive as static figures in a paper. You can read that a feature fired without ever developing an intuition for what the activation pattern looks like as text is actually being generated.

//What it does

  • Nodes are SAE features, labelled with human-readable concepts pulled from Neuronpedia.
  • Edges connect features that co-activated within the same token step.
  • Node size and brightness track activation magnitude, breathing with a decay animation.
  • The whole graph streams over WebSocket as generation happens, so the structure evolves live.

//Measured

Hook point
GPT-2 residual stream, layer 8
Feature labels
Neuronpedia
Transport
WebSocket, per-token streaming
Stars
19

//Built with

  • TransformerLens
  • SAELens
  • FastAPI
  • Three.js
  • WebSocket
  • Neuronpedia

status: Active · research tool