Strategy Coopetition Explains the Emergence and Transience of In-Context Learning

Authors: Singh, Aaditya K., Moskovitz, Ted, Dragutinovic, Sara, Hill, Felix, Chan, Stephanie C. Y., Saxe, Andrew M.

Subject Terms: Computer Science - Machine Learning

Access URL: http://arxiv.org/abs/2503.05631

SODA: Bottleneck Diffusion Models for Representation Learning

Authors: Hudson, Drew A., Zoran, Daniel, Malinowski, Mateusz, Lampinen, Andrew K., Jaegle, Andrew, McClelland, James L., Matthey, Loic, Hill, Felix, Lerchner, Alexander

Source: 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) CVPR Computer Vision and Pattern Recognition (CVPR), 2024 IEEE/CVF Conference on. :23115-23127 Jun, 2024

Relation: 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

Why transformers are obviously good models of language

Authors: Hill, Felix

Subject Terms: Computer Science - Computation and Language

Access URL: http://arxiv.org/abs/2408.03855

What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation

Authors: Singh, Aaditya K., Moskovitz, Ted, Hill, Felix, Chan, Stephanie C. Y., Saxe, Andrew M.

Subject Terms: Computer Science - Machine Learning

Access URL: http://arxiv.org/abs/2404.07129

Scaling Instructable Agents Across Many Simulated Worlds

Authors: SIMA Team, Raad, Maria Abi, Ahuja, Arun, Barros, Catarina, Besse, Frederic, Bolt, Andrew, Bolton, Adrian, Brownfield, Bethanie, Buttimore, Gavin, Cant, Max, Chakera, Sarah, Chan, Stephanie C. Y., Clune, Jeff, Collister, Adrian, Copeman, Vikki, Cullum, Alex, Dasgupta, Ishita, de Cesare, Dario, Di Trapani, Julia, Donchev, Yani, Dunleavy, Emma, Engelcke, Martin, Faulkner, Ryan, Garcia, Frankie, Gbadamosi, Charles, Gong, Zhitao, Gonzales, Lucy, Gupta, Kshitij, Gregor, Karol, Hallingstad, Arne Olav, Harley, Tim, Haves, Sam, Hill, Felix, Hirst, Ed, Hudson, Drew A., Hudson, Jony, Hughes-Fitt, Steph, Rezende, Danilo J., Jasarevic, Mimi, Kampis, Laura, Ke, Rosemary, Keck, Thomas, Kim, Junkyung, Knagg, Oscar, Kopparapu, Kavya, Lawton, Rory, Lampinen, Andrew, Legg, Shane, Lerchner, Alexander, Limont, Marjorie, Liu, Yulan, Loks-Thompson, Maria, Marino, Joseph, Cussons, Kathryn Martin, Matthey, Loic, Mcloughlin, Siobhan, Mendolicchio, Piermaria, Merzic, Hamza, Mitenkova, Anna, Moufarek, Alexandre, Oliveira, Valeria, Oliveira, Yanko, Openshaw, Hannah, Pan, Renke, Pappu, Aneesh, Platonov, Alex, Purkiss, Ollie, Reichert, David, Reid, John, Richemond, Pierre Harvey, Roberts, Tyson, Ruscoe, Giles, Elias, Jaume Sanchez, Sandars, Tasha, Sawyer, Daniel P., Scholtes, Tim, Simmons, Guy, Slater, Daniel, Soyer, Hubert, Strathmann, Heiko, Stys, Peter, Tam, Allison C., Teplyashin, Denis, Terzi, Tayfun, Vercelli, Davide, Vujatovic, Bojan, Wainwright, Marcus, Wang, Jane X., Wang, Zhengdong, Wierstra, Daan, Williams, Duncan, Wong, Nathaniel, York, Sarah, Young, Nick

Subject Terms: Computer Science - Robotics, Computer Science - Artificial Intelligence, Computer Science - Human-Computer Interaction, Computer Science - Machine Learning

Access URL: http://arxiv.org/abs/2404.10179

The Transient Nature of Emergent In-Context Learning in Transformers

Authors: Singh, Aaditya K., Chan, Stephanie C. Y., Moskovitz, Ted, Grant, Erin, Saxe, Andrew M., Hill, Felix

Subject Terms: Computer Science - Machine Learning, Computer Science - Artificial Intelligence, Computer Science - Computation and Language

Access URL: http://arxiv.org/abs/2311.08360

Vision-Language Models as Success Detectors

Authors: Du, Yuqing, Konyushkova, Ksenia, Denil, Misha, Raju, Akhil, Landon, Jessica, Hill, Felix, de Freitas, Nando, Cabi, Serkan

Subject Terms: Computer Science - Computer Vision and Pattern Recognition, Computer Science - Artificial Intelligence, Computer Science - Machine Learning

Access URL: http://arxiv.org/abs/2303.07280

The Edge of Orthogonality: A Simple View of What Makes BYOL Tick

Authors: Richemond, Pierre H., Tam, Allison, Tang, Yunhao, Strub, Florian, Piot, Bilal, Hill, Felix

Subject Terms: Computer Science - Machine Learning

Access URL: http://arxiv.org/abs/2302.04817

Collaborating with language models for embodied reasoning

Authors: Dasgupta, Ishita, Kaeser-Chen, Christine, Marino, Kenneth, Ahuja, Arun, Babayan, Sheila, Hill, Felix, Fergus, Rob

Subject Terms: Computer Science - Machine Learning, Computer Science - Artificial Intelligence, Computer Science - Computation and Language

Access URL: http://arxiv.org/abs/2302.00763

SemPPL: Predicting pseudo-labels for better contrastive representations

Authors: Bošnjak, Matko, Richemond, Pierre H., Tomasev, Nenad, Strub, Florian, Walker, Jacob C., Hill, Felix, Buesing, Lars Holger, Pascanu, Razvan, Blundell, Charles, Mitrovic, Jovana

Subject Terms: Computer Science - Computer Vision and Pattern Recognition, Computer Science - Artificial Intelligence, Computer Science - Machine Learning

Access URL: http://arxiv.org/abs/2301.05158

Transformers generalize differently from information stored in context vs in weights

Authors: Chan, Stephanie C. Y., Dasgupta, Ishita, Kim, Junkyung, Kumaran, Dharshan, Lampinen, Andrew K., Hill, Felix

Subject Terms: Computer Science - Computation and Language, Computer Science - Artificial Intelligence, Computer Science - Machine Learning

Access URL: http://arxiv.org/abs/2210.05675

Meaning without reference in large language models

Authors: Piantadosi, Steven T., Hill, Felix

Subject Terms: Computer Science - Computation and Language, Computer Science - Artificial Intelligence

Access URL: http://arxiv.org/abs/2208.02957

Language models show human-like content effects on reasoning tasks

Authors: Dasgupta, Ishita, Lampinen, Andrew K., Chan, Stephanie C. Y., Sheahan, Hannah R., Creswell, Antonia, Kumaran, Dharshan, McClelland, James L., Hill, Felix

Subject Terms: Computer Science - Computation and Language, Computer Science - Artificial Intelligence, Computer Science - Machine Learning

Access URL: http://arxiv.org/abs/2207.07051

Know your audience: specializing grounded language models with listener subtraction

Authors: Singh, Aaditya K., Ding, David, Saxe, Andrew, Hill, Felix, Lampinen, Andrew K.

Subject Terms: Computer Science - Machine Learning, Computer Science - Artificial Intelligence, Computer Science - Computation and Language

Access URL: http://arxiv.org/abs/2206.08349

Data Distributional Properties Drive Emergent In-Context Learning in Transformers

Authors: Chan, Stephanie C. Y., Santoro, Adam, Lampinen, Andrew K., Wang, Jane X., Singh, Aaditya, Richemond, Pierre H., McClelland, Jay, Hill, Felix

Subject Terms: Computer Science - Machine Learning, Computer Science - Artificial Intelligence, Computer Science - Computation and Language

Access URL: http://arxiv.org/abs/2205.05055

Semantic Exploration from Language Abstractions and Pretrained Representations

Authors: Tam, Allison C., Rabinowitz, Neil C., Lampinen, Andrew K., Roy, Nicholas A., Chan, Stephanie C. Y., Strouse, DJ, Wang, Jane X., Banino, Andrea, Hill, Felix

Subject Terms: Computer Science - Machine Learning, Computer Science - Artificial Intelligence

Access URL: http://arxiv.org/abs/2204.05080

Can language models learn from explanations in context?

Authors: Lampinen, Andrew K., Dasgupta, Ishita, Chan, Stephanie C. Y., Matthewson, Kory, Tessler, Michael Henry, Creswell, Antonia, McClelland, James L., Wang, Jane X., Hill, Felix

Subject Terms: Computer Science - Computation and Language, Computer Science - Artificial Intelligence, Computer Science - Machine Learning

Access URL: http://arxiv.org/abs/2204.02329

Zipfian environments for Reinforcement Learning

Authors: Chan, Stephanie C. Y., Lampinen, Andrew K., Richemond, Pierre H., Hill, Felix

Subject Terms: Computer Science - Machine Learning

Access URL: http://arxiv.org/abs/2203.08222

Feature-Attending Recurrent Modules for Generalization in Reinforcement Learning

Authors: Carvalho, Wilka, Lampinen, Andrew, Nikiforou, Kyriacos, Hill, Felix, Shanahan, Murray

Subject Terms: Computer Science - Machine Learning, Computer Science - Artificial Intelligence

Access URL: http://arxiv.org/abs/2112.08369

Creating Multimodal Interactive Agents with Imitation and Self-Supervised Learning

Authors: DeepMind Interactive Agents Team, Abramson, Josh, Ahuja, Arun, Brussee, Arthur, Carnevale, Federico, Cassin, Mary, Fischer, Felix, Georgiev, Petko, Goldin, Alex, Gupta, Mansi, Harley, Tim, Hill, Felix, Humphreys, Peter C, Hung, Alden, Landon, Jessica, Lillicrap, Timothy, Merzic, Hamza, Muldal, Alistair, Santoro, Adam, Scully, Guy, von Glehn, Tamara, Wayne, Greg, Wong, Nathaniel, Yan, Chen, Zhu, Rui

Subject Terms: Computer Science - Machine Learning

Access URL: http://arxiv.org/abs/2112.03763