EigenVI: score-based variational inference with orthogonal function expansions

Authors: Cai, Diana, Modi, Chirag, Margossian, Charles C., Gower, Robert M., Blei, David M., Saul, Lawrence K.

Subject Terms: Statistics - Machine Learning, Computer Science - Machine Learning, Statistics - Computation

Access URL: http://arxiv.org/abs/2410.24054

Enhancing Policy Gradient with the Polyak Step-Size Adaption

Authors: Li, Yunxiang, Yuan, Rui, Fan, Chen, Schmidt, Mark, Horváth, Samuel, Gower, Robert M., Takáč, Martin

Subject Terms: Computer Science - Machine Learning

Access URL: http://arxiv.org/abs/2404.07525

Directional Smoothness and Gradient Methods: Convergence and Adaptivity

Authors: Mishkin, Aaron, Khaled, Ahmed, Wang, Yuanhao, Defazio, Aaron, Gower, Robert M.

Subject Terms: Computer Science - Machine Learning, Mathematics - Optimization and Control

Access URL: http://arxiv.org/abs/2403.04081

Level Set Teleportation: An Optimization Perspective

Authors: Mishkin, Aaron, Bietti, Alberto, Gower, Robert M.

Subject Terms: Computer Science - Machine Learning, Mathematics - Optimization and Control

Access URL: http://arxiv.org/abs/2403.03362

Batch and match: black-box variational inference with a score-based divergence

Authors: Cai, Diana, Modi, Chirag, Pillaud-Vivien, Loucas, Margossian, Charles C., Gower, Robert M., Blei, David M., Saul, Lawrence K.

Subject Terms: Statistics - Machine Learning, Computer Science - Artificial Intelligence, Computer Science - Machine Learning, Statistics - Computation

Access URL: http://arxiv.org/abs/2402.14758

Function Value Learning: Adaptive Learning Rates Based on the Polyak Stepsize and Function Splitting in ERM

Authors: Garrigos, Guillaume, Gower, Robert M., Schaipp, Fabian

Subject Terms: Computer Science - Machine Learning, Mathematics - Optimization and Control, 90C53, 74S60, 90C06, 62L20, 68W20, 15B52, 65Y20, 68W40, G.1.6

Access URL: http://arxiv.org/abs/2307.14528

A Model-Based Method for Minimizing CVaR and Beyond

Authors: Meng, Si Yi, Gower, Robert M.

Subject Terms: Mathematics - Optimization and Control, Computer Science - Machine Learning

Access URL: http://arxiv.org/abs/2305.17498

Improving Convergence and Generalization Using Parameter Symmetries

Authors: Zhao, Bo, Gower, Robert M., Walters, Robin, Yu, Rose

Subject Terms: Computer Science - Machine Learning, Mathematics - Optimization and Control

Access URL: http://arxiv.org/abs/2305.13404

MoMo: Momentum Models for Adaptive Learning Rates

Authors: Schaipp, Fabian, Ohana, Ruben, Eickenberg, Michael, Defazio, Aaron, Gower, Robert M.

Subject Terms: Computer Science - Machine Learning, Mathematics - Optimization and Control, 90C53, 74S60, 90C06, 62L20, 68W20, 15B52, 65Y20, 68W40, G.1.6

Access URL: http://arxiv.org/abs/2305.07583

Handbook of Convergence Theorems for (Stochastic) Gradient Methods

Authors: Garrigos, Guillaume, Gower, Robert M.

Subject Terms: Mathematics - Optimization and Control, 65K05, 68T99, G.1.6

Access URL: http://arxiv.org/abs/2301.11235

A Stochastic Proximal Polyak Step Size

Authors: Schaipp, Fabian, Gower, Robert M., Ulbrich, Michael

Subject Terms: Mathematics - Optimization and Control, Computer Science - Machine Learning, Statistics - Machine Learning, 90C26

Access URL: http://arxiv.org/abs/2301.04935

Linear Convergence of Natural Policy Gradient Methods with Log-Linear Policies

Authors: Yuan, Rui, Du, Simon S., Gower, Robert M., Lazaric, Alessandro, Xiao, Lin

Subject Terms: Computer Science - Machine Learning, Computer Science - Artificial Intelligence, Mathematics - Optimization and Control

Access URL: http://arxiv.org/abs/2210.01400

SP2: A Second Order Stochastic Polyak Method

Authors: Li, Shuang, Swartworth, William J., Takáč, Martin, Needell, Deanna, Gower, Robert M.

Subject Terms: Computer Science - Machine Learning, Mathematics - Optimization and Control

Access URL: http://arxiv.org/abs/2207.08171

Cutting Some Slack for SGD with Adaptive Polyak Stepsizes

Authors: Gower, Robert M., Blondel, Mathieu, Gazagnadou, Nidham, Pedregosa, Fabian

Subject Terms: Computer Science - Machine Learning, Mathematics - Optimization and Control, 90C53, 74S60, 90C06, 62L20, 68W20, 15B52, 65Y20, 68W40, G.1.6

Access URL: http://arxiv.org/abs/2202.12328

A general sample complexity analysis of vanilla policy gradient

Authors: Yuan, Rui, Gower, Robert M., Lazaric, Alessandro

Subject Terms: Computer Science - Machine Learning, Mathematics - Optimization and Control, Statistics - Machine Learning

Access URL: http://arxiv.org/abs/2107.11433

Stochastic Polyak Stepsize with a Moving Target

Authors: Gower, Robert M., Defazio, Aaron, Rabbat, Michael

Subject Terms: Computer Science - Machine Learning, Mathematics - Optimization and Control, 90C53, 74S60, 90C06, 62L20, 68W20, 15B52, 65Y20, 68W40, G.1.6

Access URL: http://arxiv.org/abs/2106.11851

SAN: Stochastic Average Newton Algorithm for Minimizing Finite Sums

Authors: Chen, Jiabin, Yuan, Rui, Garrigos, Guillaume, Gower, Robert M.

Source: Proceedings of The 25th International Conference on Artificial Intelligence and Statistics, PMLR 151:279-318, 2022

Subject Terms: Mathematics - Optimization and Control, Mathematics - Numerical Analysis

Access URL: http://arxiv.org/abs/2106.10520

$\texttt{RidgeSketch}$: A Fast sketching based solver for large scale ridge regression

Authors: Gazagnadou, Nidham, Ibrahim, Mark, Gower, Robert M.

Subject Terms: Mathematics - Optimization and Control, 65K10, 68T99, 90C06, 68W20, 65Y20, 62J07

Access URL: http://arxiv.org/abs/2105.05565

Variance-Reduced Methods for Machine Learning

Authors: Gower, Robert M., Schmidt, Mark, Bach, Francis, Richtarik, Peter

Subject Terms: Computer Science - Machine Learning, Mathematics - Optimization and Control, Statistics - Machine Learning, 65K05, 68T99, G.1.6

Access URL: http://arxiv.org/abs/2010.00892

A new framework for the computation of Hessians

Authors: Gower, Robert M., Mello, Margarida P.

Source: Optimization Methods and Software, 27(2):251--273, 2012

Subject Terms: Mathematics - Optimization and Control, Computer Science - Mathematical Software, 65K10, 65D25, 65F50

Access URL: http://arxiv.org/abs/2007.15040