With the two-stage fourth-order temporal evolution of the gas distribution function and Weighted Essential Non-Oscillatory (WENO) reconstruction, a high-order finite difference gas-kinetic scheme is proposed. Different from the previous high-order finite volume gas-kinetic methods, which uses the discontinuous initial reconstruction at the cell interface, the present scheme is the finite difference method with a continuous flow distribution at the grid point. The primary 1d numerical tests has been obtained.