Thanos: A Block-wise Pruning Algorithm for Efficient Large Language Model Compression

Authors: Ilin, Ivan, Richtarik, Peter

Subject Terms: Computer Science - Machine Learning, Computer Science - Artificial Intelligence, Computer Science - Computation and Language, Computer Science - Performance, 68T07, 68Q32

Access URL: http://arxiv.org/abs/2504.05346

Collaborative Value Function Estimation Under Model Mismatch: A Federated Temporal Difference Analysis

Authors: Beikmohammadi, Ali, Khirirat, Sarit, Richtárik, Peter, Magnússon, Sindri

Subject Terms: Computer Science - Machine Learning

Access URL: http://arxiv.org/abs/2503.17454

BurTorch: Revisiting Training from First Principles by Coupling Autodiff, Math Optimization, and Systems

Authors: Burlachenko, Konstantin, Richtárik, Peter

Subject Terms: Computer Science - Machine Learning, Computer Science - Mathematical Software, 65Y10, I.2.6, I.2.8, C.1.3, C.5.3, G.4, C.3

Access URL: http://arxiv.org/abs/2503.13795

Smoothed Normalization for Efficient Distributed Private Optimization

Authors: Shulgin, Egor, Khirirat, Sarit, Richtárik, Peter

Subject Terms: Computer Science - Machine Learning, Computer Science - Cryptography and Security, Computer Science - Distributed, Parallel, and Cluster Computing, Mathematics - Optimization and Control, Statistics - Machine Learning

Access URL: http://arxiv.org/abs/2502.13482

A Novel Unified Parametric Assumption for Nonconvex Optimization

Authors: Riabinin, Artem, Khaled, Ahmed, Richtárik, Peter

Subject Terms: Computer Science - Machine Learning, Computer Science - Artificial Intelligence, Mathematics - Optimization and Control, Statistics - Machine Learning

Access URL: http://arxiv.org/abs/2502.12329

Double Momentum and Error Feedback for Clipping with Fast Rates and Differential Privacy

Authors: Islamov, Rustem, Horvath, Samuel, Lucchi, Aurelien, Richtarik, Peter, Gorbunov, Eduard

Subject Terms: Computer Science - Machine Learning, Mathematics - Optimization and Control, Statistics - Machine Learning

Access URL: http://arxiv.org/abs/2502.11682

Revisiting Stochastic Proximal Point Methods: Generalized Smoothness and Similarity

Authors: Tovmasyan, Zhirayr, Malinovsky, Grigory, Condat, Laurent, Richtárik, Peter

Subject Terms: Mathematics - Optimization and Control

Access URL: http://arxiv.org/abs/2502.03401

The Ball-Proximal (='Broximal') Point Method: a New Algorithm, Convergence Theory, and Applications

Authors: Gruntkowska, Kaja, Li, Hanmin, Rane, Aadi, Richtárik, Peter

Subject Terms: Mathematics - Optimization and Control, Computer Science - Machine Learning, Statistics - Machine Learning

Access URL: http://arxiv.org/abs/2502.02002

ATA: Adaptive Task Allocation for Efficient Resource Management in Distributed Machine Learning

Authors: Maranjyan, Artavazd, Saad, El Mehdi, Richtárik, Peter, Orabona, Francesco

Subject Terms: Computer Science - Machine Learning, Computer Science - Distributed, Parallel, and Cluster Computing, Mathematics - Optimization and Control, Statistics - Machine Learning

Access URL: http://arxiv.org/abs/2502.00775

Symmetric Pruning of Large Language Models

Authors: Yi, Kai, Richtárik, Peter

Subject Terms: Computer Science - Machine Learning, Computer Science - Artificial Intelligence

Access URL: http://arxiv.org/abs/2501.18980

Ringmaster ASGD: The First Asynchronous SGD with Optimal Time Complexity

Authors: Maranjyan, Artavazd, Tyurin, Alexander, Richtárik, Peter

Subject Terms: Computer Science - Machine Learning, Computer Science - Distributed, Parallel, and Cluster Computing, Mathematics - Optimization and Control, Statistics - Machine Learning

Access URL: http://arxiv.org/abs/2501.16168

On the Convergence of DP-SGD with Adaptive Clipping

Authors: Shulgin, Egor, Richtárik, Peter

Subject Terms: Computer Science - Machine Learning, Computer Science - Cryptography and Security, Mathematics - Optimization and Control, Statistics - Machine Learning

Access URL: http://arxiv.org/abs/2412.19916

MARINA-P: Superior Performance in Non-smooth Federated Optimization with Adaptive Stepsizes

Authors: Sokolov, Igor, Richtárik, Peter

Subject Terms: Computer Science - Machine Learning, Mathematics - Optimization and Control, Statistics - Machine Learning

Access URL: http://arxiv.org/abs/2412.17082

Differentially Private Random Block Coordinate Descent

Authors: Maranjyan, Artavazd, Sadiev, Abdurakhmon, Richtárik, Peter

Subject Terms: Mathematics - Optimization and Control, Computer Science - Cryptography and Security, Computer Science - Machine Learning, Statistics - Machine Learning

Access URL: http://arxiv.org/abs/2412.17054

Speeding up Stochastic Proximal Optimization in the High Hessian Dissimilarity Setting

Authors: Gasanov, Elnur, Richtárik, Peter

Subject Terms: Mathematics - Optimization and Control

Access URL: http://arxiv.org/abs/2412.13619

Methods with Local Steps and Random Reshuffling for Generally Smooth Non-Convex Federated Optimization

Authors: Demidovich, Yury, Ostroukhov, Petr, Malinovsky, Grigory, Horváth, Samuel, Takáč, Martin, Richtárik, Peter, Gorbunov, Eduard

Subject Terms: Mathematics - Optimization and Control, Computer Science - Machine Learning

Access URL: http://arxiv.org/abs/2412.02781

Pushing the Limits of Large Language Model Quantization via the Linearity Theorem

Authors: Malinovskii, Vladimir, Panferov, Andrei, Ilin, Ivan, Guo, Han, Richtárik, Peter, Alistarh, Dan

Subject Terms: Computer Science - Machine Learning

Access URL: http://arxiv.org/abs/2411.17525

Error Feedback under $(L_0,L_1)$-Smoothness: Normalization and Momentum

Authors: Khirirat, Sarit, Sadiev, Abdurakhmon, Riabinin, Artem, Gorbunov, Eduard, Richtárik, Peter

Subject Terms: Computer Science - Machine Learning

Access URL: http://arxiv.org/abs/2410.16871

Tighter Performance Theory of FedExProx

Authors: Anyszka, Wojciech, Gruntkowska, Kaja, Tyurin, Alexander, Richtárik, Peter

Subject Terms: Mathematics - Optimization and Control, Computer Science - Machine Learning, Statistics - Machine Learning

Access URL: http://arxiv.org/abs/2410.15368

Unlocking FedNL: Self-Contained Compute-Optimized Implementation

Authors: Burlachenko, Konstantin, Richtárik, Peter

Subject Terms: Computer Science - Machine Learning, Computer Science - Artificial Intelligence, Computer Science - Mathematical Software, Computer Science - Performance, Mathematics - Optimization and Control, G.4, C.3, I.2.11

Access URL: http://arxiv.org/abs/2410.08760