Titus Pința

Publications

1. A Newton-type Method for Non-smooth Under-determined Systems of Equations

   T. Pința

   Numer. Algorithms. (under review) arXiv

   We study a variant of Newton's algorithm applied to under-determined systems of non-smooth equations. The notion of regularity employed in our work is based on Newton differentiability, which generalizes semi-smoothness. The classic notion of Newton differentiability does not suffice for our purpose, due to the existence of multiple zeros and as such we extend it to uniform Newton differentiability. In this context, we can show that the distance between the iterates and the set of zeros of the system decreases super-linearly. For the special case of smooth equations, the assumptions of our algorithm are simplified. Finally, we provide some numerical examples to showcase the behavior of our proposed method. The key example is a toy model of complementarity constraint problems, showing that our method has great application potential across engineering fields.
2. A Stochastic Newton-type Method for Non-smooth Optimization

   T. Pința

   Math. Program. (under review) arXiv

   We introduce a new framework for analyzing (Quasi-)Newton type methods applied to non-smooth optimization problems. The source of randomness comes from the evaluation of the (approximation) of the Hessian. We derive, using a variant of Chernoff bounds for stopping times, expectation and probability bounds for the random variable representing the number of iterations of the algorithm until approximate first order optimality conditions are validated. As an important distinction to previous results in the literature, we do not require that the estimator is unbiased or that it has finite variance. We then showcase our theoretical results in a stochastic Quasi-Newton method for X-ray free electron laser orbital tomography and in a sketched Newton method for image denoising.
3. A semi-Bregman proximal alternating method for a class of nonconvex problems: local and global convergence analysis

   E. Cohen, D. R. Luke, T. Pința, S. Sabach, and M. Teboulle

   J. Global Optim. 89 (2024) pdf

   We focus on nonconvex and non-smooth block optimization problems, where the smooth coupling part of the objective does not satisfy a global/partial Lipschitz gradient continuity assumption. A general alternating minimization algorithm is proposed that combines two proximal-based steps, one classical and another with respect to the Bregman divergence. Combining different analytical techniques, we provide a complete analysis of the behavior—from global to local—of the algorithm, and show when the iterates converge globally to critical points with a locally linear rate for sufficiently regular (though not necessarily convex) objectives. Numerical experiments illustrate the theoretical findings.
4. An Extension of the Second Order Dynamical System that Models Nesterov's Convex Gradient Method

   S. C. László, C. Alecsa, and T. Pința

   Appl. Math. Optim. 84 (2021) arXiv

   In this paper we deal with a general second order continuous dynamical system associated to a convex minimization problem with a Fréchet differentiable objective function. We show that inertial algorithms, such as Nesterov’s algorithm, can be obtained via the natural explicit discretization from our dynamical system. Our dynamical system can be viewed as a perturbed version of the heavy ball method with vanishing damping, however the perturbation is made in the argument of the gradient of the objective function. This perturbation seems to have a smoothing effect for the energy error and eliminates the oscillations obtained for this error in the case of the heavy ball method with vanishing damping, as some numerical experiments show. We prove that the value of the objective function in a generated trajectory converges in order $𝒪(1/t^2)$ to the global minimum of the objective function. Moreover, we obtain that a trajectory generated by the dynamical system converges to a minimum point of the objective function.
5. Asymptotic analysis of a structure-preserving integrator for damped Hamiltonian systems

   A. Viorel, C. Alecsa, and T. Pința

   Discrete Contin. Dyn. Syst. 41 (2021)

   The present work deals with the numerical long-time integration of damped Hamiltonian systems. The method that we analyze combines a specific Strang splitting, that separates linear dissipative effects from conservative ones, with an energy-preserving averaged vector field (AVF) integrator for the Hamiltonian subproblem. This construction faithfully reproduces the energy-dissipation structure of the continuous model, its equilibrium points and its natural Lyapunov function. As a consequence of these structural similarities, both the convergence to equilibrium and, more interestingly, the energy decay rate of the continuous dynamical system are recovered at a discrete level. The possibility of replacing the implicit AVF integrator by an explicit Störmer-Verlet one is also discussed, while numerical experiments illustrate and support the theoretical findings.
6. New optimization algorithms for neural network training using operator splitting techniques

   C. Alecsa, T. Pința, and I. Boroș

   Neural Networks. 126 (2020) arXiv

   In the following paper we present a new type of optimization algorithms adapted for neural network training. These algorithms are based upon sequential operator splitting technique for some associated dynamical systems. Furthermore, we investigate through numerical simulations the empirical rate of convergence of these iterative schemes toward a local minimum of the loss function, with some suitable choices of the underlying hyper-parameters. We validate the convergence of these optimizers using the results of the accuracy and of the loss function on the MNIST, MNIST-Fashion and CIFAR 10 classification datasets.