The theory and practice of stochastic optimization has focused on stochastic gradient descent (SGD) in recent years, retaining the basic first-order stochastic nature of SGD while aiming to improve it via mechanisms such as averaging, momentum, and variance reduction.

Topology optimization has become progressively important as a tool for engineering structural designs due to advancing manufacturing technology and increasing computational power.