Tractography is a tool that is often used to study structural brain connectivity from diffusion magnetic resonance imaging data. Despite its ability to visualize fibers in the white brain matter, it results in a high number of invalid streamlines. For the sake of research and clinical applications, it is of great interest to reduce this number and so improve the quality of tractography. Over the years, many solutions have been proposed, often with a need for ground truth data. As such data for tractography is very difficult to generate even with expertise, it is meaningful to instead use methods like reinforcement learning that does not have such a requirement. In 2021 a deep reinforcement learning tractography network was published: Track-To-Learn. There is however still room for improvement in the reward function of the framework and this is what we focused on in this thesis. Firstly we successfully reproduced some of the published results of Track-To-Learn and observed that almost 20 % of the streamlines were anatomically plausible. Continuously we modified the reward function by giving a reward boost to streamlines which started or terminated within a specified mask. This addition resulted in a small increase of plausible streamlines for a more realistic dataset. Lastly we attempted to include anatomical filtering in the reward function. The produced results were however not enough to draw any valid conclusions about the influence of the modification. Nonetheless, the work of this thesis showed that including further fiber specific anatomical constraints in the reward function of Track-To-Learn could possibly improve the quality of the generated tractograms and would be of interest in both research and clinical settings.