用于特征提取的稀疏自编码器

A neural activation vector can carry several learned signals in overlapping directions, making an individual coordinate difficult to interpret. A sparse autoencoder learns another representation of selected activations, with an encoder, feature activations, and a decoder that reconstructs the original vector. An overcomplete dictionary has more feature directions than input dimensions. Sparsity limits how many of those features are active for a given input, rather than requiring every feature to be active simultaneously. The objective balances reconstruction with sparse feature usage. Scaling Monosemanticity studies this approach on Claude 3 Sonnet and reports interpretable features, including one strongly associated with the Golden Gate Bridge. Weaker activations can include related landmarks, so even a compelling label should not be treated as an exhaustive one-concept definition. Increasing that feature’s activation affected generated behavior in the reported intervention. This provides evidence about that intervention rather than a complete map of every causal mechanism in the model. The reconstruction is approximate: a sparse feature set can leave residual activation information unrepresented. Wider dictionaries, sparsity settings, and training choices can alter reconstruction and interpretation. Scaling and Evaluating Sparse Autoencoders studies these measurement questions, including sparse coding with a fixed number of selected activations. Importantly, fitting an SAE to a model’s collected activations is a separate training operation from training the underlying language model. Feature extraction can aid analysis, but cannot by itself certify that a model is safe, reveal every learned concept, or replace direct behavioral evaluation.