Accelerated optimization methods and their role in modern large-model training

The report covers recent achievements related to the introduction of accelerated batch methods for retraining, accelerated tensor methods, accelerated methods under addend similarity conditions, accelerated methods for saddle point problems, and accelerated methods for distributed optimization with unbiased compression.