Dimension of activity in random neural networks
| Title: | Dimension of activity in random neural networks |
|---|---|
| Authors: | Clark, David G., Abbott, L. F., Litwin-Kumar, Ashok |
| Source: | Phys. Rev. Lett. 131, 118401 (2023) |
| Publication Year: | 2022 |
| Collection: | Computer Science Condensed Matter Quantitative Biology |
| Subject Terms: | Quantitative Biology - Neurons and Cognition, Condensed Matter - Disordered Systems and Neural Networks, Computer Science - Neural and Evolutionary Computing |
| More Details: | Neural networks are high-dimensional nonlinear dynamical systems that process information through the coordinated activity of many connected units. Understanding how biological and machine-learning networks function and learn requires knowledge of the structure of this coordinated activity, information contained, for example, in cross covariances between units. Self-consistent dynamical mean field theory (DMFT) has elucidated several features of random neural networks -- in particular, that they can generate chaotic activity -- however, a calculation of cross covariances using this approach has not been provided. Here, we calculate cross covariances self-consistently via a two-site cavity DMFT. We use this theory to probe spatiotemporal features of activity coordination in a classic random-network model with independent and identically distributed (i.i.d.) couplings, showing an extensive but fractionally low effective dimension of activity and a long population-level timescale. Our formulae apply to a wide range of single-unit dynamics and generalize to non-i.i.d. couplings. As an example of the latter, we analyze the case of partially symmetric couplings. Comment: 8 pages, 6 figures; clarified derivation |
| Document Type: | Working Paper |
| DOI: | 10.1103/PhysRevLett.131.118401 |
| Access URL: | http://arxiv.org/abs/2207.12373 |
| Accession Number: | edsarx.2207.12373 |
| Database: | arXiv |
| DOI: | 10.1103/PhysRevLett.131.118401 |
|---|