We have compared the recently developed Multi-level binary replacement (MBR) design method for use in computer experiments, to the Latin hypercube design (LHD) and the Orthogonal array (OA) design. For means of comparison, we have suggested an algorithm for drawing permutations of the MBR design, so as to obtain what we have called a MBR based Latin hypercube design. In our comparison study, the main focus have been the design scores with respect to the root mean squared error (RMSE), Max and alias sum of squares criteria. We found that the MBR design generally performed good with respect to all criteria. It scored similarly to the OA design method and better than conventional Latin hypercube sampling. The score however varied with the number of samples and the set of design generators chosen for constructing the MBR design. The MBR design performed better for designs with a relatively high number of samples compared to the number of factors.