This talk covers several recent works that share a common theme of optimizing maps among a network of objects or domains. In this context, maps take the form of matrices or neural networks. A network of maps differs from standard networks and graphs in the sense that there are regularization constraints derived from map composition. Such constraints offer powerful tools for map denoising and to propagate and aggregate information through the network. We will discuss algebraic and combinatorial theories of these constraints and applications in geometry reconstruction,3D understanding, and scene synthesis.