On How Iterative Magnitude Pruning Discovers Local Receptive Fields in Fully Connected Neural Networks

Authors: Redman, William T., Wang, Zhangyang, Ingrosso, Alessandro, Goldt, Sebastian

Subject Terms: Computer Science - Machine Learning

Access URL: http://arxiv.org/abs/2412.06545

A distributional simplicity bias in the learning dynamics of transformers

Authors: Rende, Riccardo, Gerace, Federica, Laio, Alessandro, Goldt, Sebastian

Subject Terms: Computer Science - Computation and Language

Access URL: http://arxiv.org/abs/2410.19637

Sliding down the stairs: how correlated latent variables accelerate learning with neural networks

Authors: Bardone, Lorenzo, Goldt, Sebastian

Source: ICML 2024, PMLR 235:3024-3045

Subject Terms: Statistics - Machine Learning, Condensed Matter - Statistical Mechanics, Computer Science - Machine Learning, Mathematics - Probability, Mathematics - Statistics Theory

Access URL: http://arxiv.org/abs/2404.08602

Fine-tuning Neural Network Quantum States

Authors: Rende, Riccardo, Goldt, Sebastian, Becca, Federico, Viteritti, Luciano Loris

Source: Phys. Rev. Research 6, 043280 (2024)

Subject Terms: Condensed Matter - Disordered Systems and Neural Networks, Condensed Matter - Strongly Correlated Electrons

Access URL: http://arxiv.org/abs/2403.07795

Learning from higher-order statistics, efficiently: hypothesis tests, random features, and neural networks

Authors: Székely, Eszter, Bardone, Lorenzo, Gerace, Federica, Goldt, Sebastian

Source: Advances in Neural Information Processing Systems 37 (NeurIPS 2024)

Subject Terms: Statistics - Machine Learning, Condensed Matter - Statistical Mechanics, Computer Science - Machine Learning

Access URL: http://arxiv.org/abs/2312.14922

Transformer Wave Function for two dimensional frustrated magnets: emergence of a Spin-Liquid Phase in the Shastry-Sutherland Model

Authors: Viteritti, Luciano Loris, Rende, Riccardo, Parola, Alberto, Goldt, Sebastian, Becca, Federico

Subject Terms: Condensed Matter - Strongly Correlated Electrons, Condensed Matter - Disordered Systems and Neural Networks

Access URL: http://arxiv.org/abs/2311.16889

A simple linear algebra identity to optimize Large-Scale Neural Network Quantum States

Authors: Rende, Riccardo, Viteritti, Luciano Loris, Bardone, Lorenzo, Becca, Federico, Goldt, Sebastian

Source: Communications Physics 7, 260 (2024)

Subject Terms: Condensed Matter - Strongly Correlated Electrons, Condensed Matter - Disordered Systems and Neural Networks

Access URL: http://arxiv.org/abs/2310.05715

Quantifying lottery tickets under label noise: accuracy, calibration, and complexity

Authors: Arora, Viplove, Irto, Daniele, Goldt, Sebastian, Sanguinetti, Guido

Source: Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence, PMLR 216:88-98 (2023)

Subject Terms: Computer Science - Machine Learning

Access URL: http://arxiv.org/abs/2306.12190

The RL Perceptron: Generalisation Dynamics of Policy Learning in High Dimensions

Authors: Patel, Nishil, Lee, Sebastian, Mannelli, Stefano Sarao, Goldt, Sebastian, Saxe, Andrew

Subject Terms: Computer Science - Machine Learning, Condensed Matter - Disordered Systems and Neural Networks

Access URL: http://arxiv.org/abs/2306.10404

Attacks on Online Learners: a Teacher-Student Analysis

Authors: Margiotta, Riccardo Giuseppe, Goldt, Sebastian, Sanguinetti, Guido

Source: NeurIPS 2023

Subject Terms: Statistics - Machine Learning, Condensed Matter - Disordered Systems and Neural Networks, Condensed Matter - Statistical Mechanics, Computer Science - Cryptography and Security, Computer Science - Machine Learning

Access URL: http://arxiv.org/abs/2305.11132

Mapping of attention mechanisms to a generalized Potts model

Authors: Rende, Riccardo, Gerace, Federica, Laio, Alessandro, Goldt, Sebastian

Source: Phys. Rev. Research 6, 023057 (2024)

Subject Terms: Condensed Matter - Disordered Systems and Neural Networks, Condensed Matter - Statistical Mechanics, Computer Science - Computation and Language, Statistics - Machine Learning

Access URL: http://arxiv.org/abs/2304.07235

Perspectives on adaptive dynamical systems

Authors: Sawicki, Jakub, Berner, Rico, Loos, Sarah A. M., Anvari, Mehrnaz, Bader, Rolf, Barfuss, Wolfram, Botta, Nicola, Brede, Nuria, Franović, Igor, Gauthier, Daniel J., Goldt, Sebastian, Hajizadeh, Aida, Hövel, Philipp, Karin, Omer, Lorenz-Spreen, Philipp, Miehl, Christoph, Mölter, Jan, Olmi, Simona, Schöll, Eckehard, Seif, Alireza, Tass, Peter A., Volpe, Giovanni, Yanchuk, Serhiy, Kurths, Jürgen

Source: Chaos 33, 071501 (2023)

Subject Terms: Nonlinear Sciences - Adaptation and Self-Organizing Systems

Access URL: http://arxiv.org/abs/2303.01459

Neural networks trained with SGD learn distributions of increasing complexity

Authors: Refinetti, Maria, Ingrosso, Alessandro, Goldt, Sebastian

Source: ICML 2023

Subject Terms: Statistics - Machine Learning, Condensed Matter - Disordered Systems and Neural Networks, Condensed Matter - Statistical Mechanics, Computer Science - Machine Learning

Access URL: http://arxiv.org/abs/2211.11567

The impact of memory on learning sequence-to-sequence tasks

Authors: Seif, Alireza, Loos, Sarah A. M., Tucci, Gennaro, Roldán, Édgar, Goldt, Sebastian

Source: Mach. Learn.: Sci. Technol. 5 015053 (2024)

Subject Terms: Computer Science - Machine Learning, Condensed Matter - Statistical Mechanics, Statistics - Machine Learning

Access URL: http://arxiv.org/abs/2205.14683

Maslow's Hammer for Catastrophic Forgetting: Node Re-Use vs Node Activation

Authors: Lee, Sebastian, Mannelli, Stefano Sarao, Clopath, Claudia, Goldt, Sebastian, Saxe, Andrew

Source: Proceedings of the 39th International Conference on Machine Learning, PMLR 162:12455-12477 (2022)

Subject Terms: Statistics - Machine Learning, Computer Science - Machine Learning

Access URL: http://arxiv.org/abs/2205.09029

Data-driven emergence of convolutional structure in neural networks

Authors: Ingrosso, Alessandro, Goldt, Sebastian

Source: Proceedings of the National Academy of Science vol 119 (40) e2201854119 (2022)

Subject Terms: Condensed Matter - Disordered Systems and Neural Networks, Quantitative Biology - Neurons and Cognition, Statistics - Machine Learning

Access URL: http://arxiv.org/abs/2202.00565

The dynamics of representation learning in shallow, non-linear autoencoders

Authors: Refinetti, Maria, Goldt, Sebastian

Source: Proceedings of the 39th International Conference on Machine Learning, PMLR 162:18499-18519 (2022)

Subject Terms: Statistics - Machine Learning, Condensed Matter - Disordered Systems and Neural Networks, Condensed Matter - Statistical Mechanics, Computer Science - Machine Learning

Access URL: http://arxiv.org/abs/2201.02115

Continual Learning in the Teacher-Student Setup: Impact of Task Similarity

Authors: Lee, Sebastian, Goldt, Sebastian, Saxe, Andrew

Source: Proceedings of the 38th International Conference on Machine Learning, PMLR 139:6109-6119, 2021

Subject Terms: Statistics - Machine Learning, Condensed Matter - Statistical Mechanics, Computer Science - Machine Learning

Access URL: http://arxiv.org/abs/2107.04384

Redundant representations help generalization in wide neural networks

Authors: Doimo, Diego, Glielmo, Aldo, Goldt, Sebastian, Laio, Alessandro

Source: Advances in Neural Information Processing Systems 35 (2022)

Subject Terms: Statistics - Machine Learning, Computer Science - Computer Vision and Pattern Recognition, Computer Science - Machine Learning

Access URL: http://arxiv.org/abs/2106.03485

Bayesian reconstruction of memories stored in neural networks from their connectivity

Authors: Goldt, Sebastian, Krzakala, Florent, Zdeborová, Lenka, Brunel, Nicolas

Source: PLOS Computational Biology 19(1): e1010813 2023

Subject Terms: Quantitative Biology - Neurons and Cognition, Condensed Matter - Statistical Mechanics, Statistics - Machine Learning

Access URL: http://arxiv.org/abs/2105.07416