In recent years, machine learning (ML) has emerged as a promising tool for dealing with the difficulty of representing high dimensional functions. This gives us an unprecedented opportunity to revisit theoretical foundations of various scientific fields and solve problems that were too complicated for conventional approaches to address. Here we identify a list of such problems in the context of multi-scale molecular and materials modeling and review ML-based strategies that boost simulations with ab initio accuracy to much larger scales than conventional approaches. Using examples at scales of many-electron Schrödinger equation, density functional theory, and molecular dynamics, we present two equally important principles: 1) ML-based models should respect important physical constraints in a faithful and adaptive way; 2) to build truly reliable models, efficient algorithms are needed to explore relevant physical space and construct optimal training data sets. Finally, we present our efforts on developing related software packages and high-performance computing schemes, which have now been widely used worldwide by experts and practitioners in the molecular and materials simulation community.