Quantum error correction is indispensable for quantum technologies. The most promising route towards realizing full fault tolerance is to make use of the topological properties of many body systems, since as for such systems error correction consists of local Pauli corrections on all the physical quantum systems involved. Unfortunately, topological error correcting codes exhibit high code degeneracy: many error configurations give rise to the same error syndrome, and as a result the decoding procedure is highly non-trivial. We use a model-free deep reinforcement learning techniques in order to design decoders for the simplest topological quantum code, Kitaev’s toric code. For uncorrelated noise (i.e. bit or phase flip noise) we find that our model-free decoders exhibit improved performance, as compared to model-based reinforcement learning decoders, close to the percolation threshold of 11%. More importantly, because of our model-free approach, our decoder is capable of achieving high threshold even in the presence of correlated noise for which previous model-based decoders perform poorly.