This masters thesis is focused around the comparison of time dependent video grayscale conversion methods based on a user experiment. The test methodology is based on the pairwise comparison method 2AFC. It is composed of two test variants: a test with a reference video and a test without a reference video. The coefficients of agreement, consistency and correlation are utilized in the result analysis. The testing was done on 60 subjects which have done 7200 pairwise comparisons. The test results show that the time dependent method Hu14 is the most universal. Time dependent method Kim09 was bested by some non-dependent methods. The results also indicate that there is some correlation between the two test variants and that the choice of the input video can have impact on the method performance. The main contribution of this thesis is that non-dependent methods can under certain circumstances rival the performance of dependent methods and that the addition of a reference video did not have a meaningful impact on the test subjects judgements.