Hermitian PSD fixed rank constraint is used in many applications, e.g., it is also used for approximating the Hermitian PSD constraint. We study and compare three methodologies for minimizing f(X) with X being a Hermitian PSD fixed rank matrix. The first approach is the simplest factor-based Burer-Monteiro method, in which a PSD fixed rank matrix X is replaced by its low-rank decomposition YY^* thus an unconstrained minimization of f(YY^*) can be solved instead. The second approach is to regard the set of Hermitian PSD fixed rank matrices as an embedded manifold in the Euclidean space and consider the Riemannian optimization over the embedded manifold. The third approach is to regard it as a quotient manifold and consider the Riemannian optimization over the quotient manifold. For simplicity, we only consider the nonlinear conjugate gradient (CG) algorithm, which is an efficient algorithm in these methods. We show that CG in the first two methodolgies are equivalent to CG on the quotient manifold with suitably chosen metrics, retractions, and vector transports. We also analyze the condition number of the Riemannian Hessian under these different metrics. The difference in the condition number of the Riemannian Hessian under different metrics is consistent with the difference in the numerical performance of three methodologies for problems including matrix completion, phase retrieval, and interferometry recovery.