A Unifying Framework for Unsupervised Concept Extraction

Concept extraction methods, including sparse autoencoders and transcoders, seek symbolic concepts within learned representations. This work studies their theoretical guarantees by treating concept extraction as a generative-model identification problem. A general identifiability theorem reduces the analysis to characterizing the intersection of two sets. Applying the framework to several common methods simplifies existing arguments and provides a foundation for designing concept extraction approaches with provable guarantees.