International AI Safety Report

Authors: Bengio, Yoshua, Mindermann, Sören, Privitera, Daniel, Besiroglu, Tamay, Bommasani, Rishi, Casper, Stephen, Choi, Yejin, Fox, Philip, Garfinkel, Ben, Goldfarb, Danielle, Heidari, Hoda, Ho, Anson, Kapoor, Sayash, Khalatbari, Leila, Longpre, Shayne, Manning, Sam, Mavroudis, Vasilios, Mazeika, Mantas, Michael, Julian, Newman, Jessica, Ng, Kwan Yee, Okolo, Chinasa T., Raji, Deborah, Sastry, Girish, Seger, Elizabeth, Skeadas, Theodora, South, Tobin, Strubell, Emma, Tramèr, Florian, Velasco, Lucia, Wheeler, Nicole, Acemoglu, Daron, Adekanmbi, Olubayo, Dalrymple, David, Dietterich, Thomas G., Felten, Edward W., Fung, Pascale, Gourinchas, Pierre-Olivier, Heintz, Fredrik, Hinton, Geoffrey, Jennings, Nick, Krause, Andreas, Leavy, Susan, Liang, Percy, Ludermir, Teresa, Marda, Vidushi, Margetts, Helen, McDermid, John, Munga, Jane, Narayanan, Arvind, Nelson, Alondra, Neppel, Clara, Oh, Alice, Ramchurn, Gopal, Russell, Stuart, Schaake, Marietje, Schölkopf, Bernhard, Song, Dawn, Soto, Alvaro, Tiedrich, Lee, Varoquaux, Gaël, Yao, Andrew, Zhang, Ya-Qin, Albalawi, Fahad, Alserkal, Marwan, Ajala, Olubunmi, Avrin, Guillaume, Busch, Christian, de Carvalho, André Carlos Ponce de Leon Ferreira, Fox, Bronwyn, Gill, Amandeep Singh, Hatip, Ahmet Halit, Heikkilä, Juha, Jolly, Gill, Katzir, Ziv, Kitano, Hiroaki, Krüger, Antonio, Johnson, Chris, Khan, Saif M., Lee, Kyoung Mu, Ligot, Dominic Vincent, Molchanovskyi, Oleksii, Monti, Andrea, Mwamanzi, Nusu, Nemer, Mona, Oliver, Nuria, Portillo, José Ramón López, Ravindran, Balaraman, Rivera, Raquel Pezoa, Riza, Hammam, Rugege, Crystal, Seoighe, Ciarán, Sheehan, Jerry, Sheikh, Haroon, Wong, Denise, Zeng, Yi

Subject Terms: Computer Science - Computers and Society, Computer Science - Artificial Intelligence, Computer Science - Machine Learning

Access URL: http://arxiv.org/abs/2501.17805

From Efficiency Gains to Rebound Effects: The Problem of Jevons' Paradox in AI's Polarized Environmental Debate

Authors: Luccioni, Alexandra Sasha, Strubell, Emma, Crawford, Kate

Subject Terms: Computer Science - Computers and Society

Access URL: http://arxiv.org/abs/2501.16548

Generate to Discriminate: Expert Routing for Continual Learning

Authors: Byun, Yewon, Mehta, Sanket Vaibhav, Garg, Saurabh, Strubell, Emma, Oberst, Michael, Wilder, Bryan, Lipton, Zachary C.

Subject Terms: Computer Science - Machine Learning

Access URL: http://arxiv.org/abs/2412.17009

Hardware Scaling Trends and Diminishing Returns in Large-Scale Distributed Training

Authors: Fernandez, Jared, Wehrstedt, Luca, Shamis, Leonid, Elhoushi, Mostafa, Saladi, Kalyan, Bisk, Yonatan, Strubell, Emma, Kahn, Jacob

Subject Terms: Computer Science - Machine Learning, Computer Science - Distributed, Parallel, and Cluster Computing

Access URL: http://arxiv.org/abs/2411.13055

Gradient Localization Improves Lifelong Pretraining of Language Models

Authors: Fernandez, Jared, Bisk, Yonatan, Strubell, Emma

Subject Terms: Computer Science - Computation and Language

Access URL: http://arxiv.org/abs/2411.04448

Collage: Decomposable Rapid Prototyping for Information Extraction on Scientific PDFs

Authors: Gururaja, Sireesh, Zhang, Yueheng, Tang, Guannan, Zhang, Tianhao, Murphy, Kevin, Yi, Yu-Tsen, Seo, Junwon, Rollett, Anthony, Strubell, Emma

Subject Terms: Computer Science - Computation and Language, Computer Science - Human-Computer Interaction

Access URL: http://arxiv.org/abs/2410.23478

Scalable Data Ablation Approximations for Language Models through Modular Training and Merging

Authors: Na, Clara, Magnusson, Ian, Jha, Ananya Harsh, Sherborne, Tom, Strubell, Emma, Dodge, Jesse, Dasigi, Pradeep

Subject Terms: Computer Science - Computation and Language, Computer Science - Machine Learning

Access URL: http://arxiv.org/abs/2410.15661

Stereotype or Personalization? User Identity Biases Chatbot Recommendations

Authors: Kantharuban, Anjali, Milbauer, Jeremiah, Strubell, Emma, Neubig, Graham

Subject Terms: Computer Science - Computation and Language

Access URL: http://arxiv.org/abs/2410.05613

What is Your Data Worth to GPT? LLM-Scale Data Valuation with Influence Functions

Authors: Choe, Sang Keun, Ahn, Hwijeen, Bae, Juhan, Zhao, Kewen, Kang, Minsoo, Chung, Youngseog, Pratapa, Adithya, Neiswanger, Willie, Strubell, Emma, Mitamura, Teruko, Schneider, Jeff, Hovy, Eduard, Grosse, Roger, Xing, Eric

Subject Terms: Computer Science - Machine Learning, Computer Science - Artificial Intelligence, Computer Science - Computation and Language

Access URL: http://arxiv.org/abs/2405.13954

Carbon Connect: An Ecosystem for Sustainable Computing

Authors: Lee, Benjamin C., Brooks, David, van Benthem, Arthur, Gupta, Udit, Hills, Gage, Liu, Vincent, Pierce, Benjamin, Stewart, Christopher, Strubell, Emma, Wei, Gu-Yeon, Wierman, Adam, Yao, Yuan, Yu, Minlan

Subject Terms: Computer Science - Distributed, Parallel, and Cluster Computing, Computer Science - Hardware Architecture, Computer Science - Emerging Technologies, Computer Science - Machine Learning

Access URL: http://arxiv.org/abs/2405.13858

Source-Aware Training Enables Knowledge Attribution in Language Models

Authors: Khalifa, Muhammad, Wadden, David, Strubell, Emma, Lee, Honglak, Wang, Lu, Beltagy, Iz, Peng, Hao

Subject Terms: Computer Science - Computation and Language, Computer Science - Artificial Intelligence

Access URL: http://arxiv.org/abs/2404.01019

OLMo: Accelerating the Science of Language Models

Authors: Groeneveld, Dirk, Beltagy, Iz, Walsh, Pete, Bhagia, Akshita, Kinney, Rodney, Tafjord, Oyvind, Jha, Ananya Harsh, Ivison, Hamish, Magnusson, Ian, Wang, Yizhong, Arora, Shane, Atkinson, David, Authur, Russell, Chandu, Khyathi Raghavi, Cohan, Arman, Dumas, Jennifer, Elazar, Yanai, Gu, Yuling, Hessel, Jack, Khot, Tushar, Merrill, William, Morrison, Jacob, Muennighoff, Niklas, Naik, Aakanksha, Nam, Crystal, Peters, Matthew E., Pyatkin, Valentina, Ravichander, Abhilasha, Schwenk, Dustin, Shah, Saurabh, Smith, Will, Strubell, Emma, Subramani, Nishant, Wortsman, Mitchell, Dasigi, Pradeep, Lambert, Nathan, Richardson, Kyle, Zettlemoyer, Luke, Dodge, Jesse, Lo, Kyle, Soldaini, Luca, Smith, Noah A., Hajishirzi, Hannaneh

Subject Terms: Computer Science - Computation and Language

Access URL: http://arxiv.org/abs/2402.00838

Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Research

Authors: Soldaini, Luca, Kinney, Rodney, Bhagia, Akshita, Schwenk, Dustin, Atkinson, David, Authur, Russell, Bogin, Ben, Chandu, Khyathi, Dumas, Jennifer, Elazar, Yanai, Hofmann, Valentin, Jha, Ananya Harsh, Kumar, Sachin, Lucy, Li, Lyu, Xinxi, Lambert, Nathan, Magnusson, Ian, Morrison, Jacob, Muennighoff, Niklas, Naik, Aakanksha, Nam, Crystal, Peters, Matthew E., Ravichander, Abhilasha, Richardson, Kyle, Shen, Zejiang, Strubell, Emma, Subramani, Nishant, Tafjord, Oyvind, Walsh, Pete, Zettlemoyer, Luke, Smith, Noah A., Hajishirzi, Hannaneh, Beltagy, Iz, Groeneveld, Dirk, Dodge, Jesse, Lo, Kyle

Subject Terms: Computer Science - Computation and Language

Access URL: http://arxiv.org/abs/2402.00159

AboutMe: Using Self-Descriptions in Webpages to Document the Effects of English Pretraining Data Filters

Authors: Lucy, Li, Gururangan, Suchin, Soldaini, Luca, Strubell, Emma, Bamman, David, Klein, Lauren F., Dodge, Jesse

Subject Terms: Computer Science - Computation and Language

Access URL: http://arxiv.org/abs/2401.06408

Understanding the Effect of Model Compression on Social Bias in Large Language Models

Authors: Gonçalves, Gustavo, Strubell, Emma

Subject Terms: Computer Science - Computation and Language

Access URL: http://arxiv.org/abs/2312.05662

Power Hungry Processing: Watts Driving the Cost of AI Deployment?

Authors: Luccioni, Alexandra Sasha, Jernite, Yacine, Strubell, Emma

Source: ACM Conference on Fairness, Accountability, and Transparency (ACM FAccT '24), June 3--6, 2024, Rio de Janeiro, Brazil

Subject Terms: Computer Science - Machine Learning

Access URL: http://arxiv.org/abs/2311.16863

Energy and Carbon Considerations of Fine-Tuning BERT

Authors: Wang, Xiaorong, Na, Clara, Strubell, Emma, Friedler, Sorelle, Luccioni, Sasha

Subject Terms: Computer Science - Computation and Language, Computer Science - Machine Learning

Access URL: http://arxiv.org/abs/2311.10267

To Build Our Future, We Must Know Our Past: Contextualizing Paradigm Shifts in Natural Language Processing

Authors: Gururaja, Sireesh, Bertsch, Amanda, Na, Clara, Widder, David Gray, Strubell, Emma

Subject Terms: Computer Science - Computation and Language, Computer Science - Computers and Society

Access URL: http://arxiv.org/abs/2310.07715

Making Scalable Meta Learning Practical

Authors: Choe, Sang Keun, Mehta, Sanket Vaibhav, Ahn, Hwijeen, Neiswanger, Willie, Xie, Pengtao, Strubell, Emma, Xing, Eric

Subject Terms: Computer Science - Machine Learning, Computer Science - Artificial Intelligence

Access URL: http://arxiv.org/abs/2310.05674

Efficiency Pentathlon: A Standardized Arena for Efficiency Evaluation

Authors: Peng, Hao, Cao, Qingqing, Dodge, Jesse, Peters, Matthew E., Fernandez, Jared, Sherborne, Tom, Lo, Kyle, Skjonsberg, Sam, Strubell, Emma, Plessas, Darrell, Beltagy, Iz, Walsh, Evan Pete, Smith, Noah A., Hajishirzi, Hannaneh

Subject Terms: Computer Science - Computation and Language

Access URL: http://arxiv.org/abs/2307.09701