Human face is a distinctive characteristic of individual identification and recognition, showing extensive variations in shape, color, and spatial arrangement among facial features. As evidenced by the remarkably visual similarity between monozygotic twins, human face is believed to be a complex trait under strong genetic control. Nevertheless, heritability estimates of human face are often surprisingly difficult to replicate due to the complexity of facial phenotyping. Previous researches primarily focus on the face geometry derived from 3D facial image data, leading to limited growth of the sample size. Furthermore, although how our brain naturally perceives facial features remains largely a mystery, easily accessible natural face images certainly work better than 3D facial geometry in the context of face identification. Therefore, here we present a novel heritability study of human face by integrating artificial intelligence (AI)-driven phenotyping and a large-scale publicly available family image database. This new method not only enables us to effectively characterize the similarity of human faces, but also effectively increases the sample size.