Neural networks may implicitly realize symbolic structure in continuous vectors

This research proposes that the internal representations of neural networks implicitly encode symbolic structures like logic and language. Despite using continuous vectors, models may be performing structured computations similar to traditional symbolic AI.