Numerical methods to solve initial value problems of differential equations progressed quite a bit in the last century. We give a brief summary of how useful numerical methods are for ordinary differential equations of first and higher order. In this thesis both computational and theoretical discussion of the application of numerical methods on differential equations takes place. The thesis consists of an investigation of various categories of numerical methods for the solution of ordinary differential equations including the numerical solution of ordinary differential equations from a number of practical fields such as equations arising in population dynamics and astrophysics. It includes discussion what are the advantages and disadvantages of implicit methods over explicit methods, the accuracy and stability of methods and how the order of various methods can be approximated numerically. Also, semidiscretization of some partial differential equations and stiff systems which may arise from these semidiscretizations are examined.