Determining the number of factoArs is essential to factor analysis. In this paper, we propose an efficient cross validation (CV) method to determine the number of factors in the approximate factor model. The method applies CV twice, first along the direction of the observations and then the direction of the variables, and hence is referred to hereafter as double cross-validation (DCV). Unlike most CV methods, which are prone to overfitting, DCV is statistically consistent in determining the number of factors when both dimensions of variables and sample size are sufficiently large. Simulation studies show that DCV has outstanding performance in comparison to existing methods in selecting the number of factors, especially when the idiosyncratic error has heteroscedasticity, or heavy tail, or relatively large variance.