During the radio therapy treatment, it has been difficult to synchronize the radiation beam with the tumor position. Many compensation techniques have been used before. But all these techniques have some system latency, up to a few hundred milliseconds. Hence it is necessary to predict tumor position to compensate for the control system latency. In recent years, many attempts have been made to predict the position of a moving tumor during respiration. Analyzing external breathing signals presents a methodology in predicting the tumor position. Breathing patterns vary from very regular to irregular patterns. The irregular breathing patterns make prediction difficult. A solution is presented in this paper which utilizes neural networks as the predictive filter to determine the tumor position up to 500 milliseconds in the future. Two different neural network architectures, feedforward backpropagation network and recurrent network, are used for prediction. These networks are initialized in the same manner for the comparison of their prediction accuracies. The networks are able to predict well for all the 5 breathing cases used in the research and the results of both the networks are acceptable and comparable. Furthermore, the network parameters are optimized using a genetic algorithm to improve the performance. The optimization results obtained proved to improve the accuracy of the networks. The results of both the networks showed that the networks are good for prediction of different breathing behaviors.