With the increasing usage of the electronic limit order book (LOB) in modern financial markets, high-frequency algorithmic trading has captured over 70 percent of the whole trading volume in various financial markets. Market making as the markets' liquidity provider by simultaneously posting bid and ask limit orders on the limit order book, serves as a fundamental problem in high-frequency trading, and attracts the attention from a lot of disciplines. Key issues in market making strategies include (1) how to design the goal of an optimal market making strategy, (2) how to dynamically adjust bid and ask orders to optimize the goal, and (3) how to control the risk during the execution such as inventory risks, adverse selection risks and model uncertainty risks. Therefore, market making is naturally framed as a Markov decision process (MDP) and reinforcement learning techniques provide a way to optimize these problems. In this study, we apply a deep reinforcement learning framework for market making strategies, especially for cryptocurrency markets. Different optimization algorithms for the reinforcement learning framework are compared based on the risk-adjusted return measure. This work contributes to our understanding on the effectiveness of reinforcement learning for market making problems.