Newton's second law, Schrödinger's equation and Maxwell's equations are all theories composed of at most second-time derivatives. Indeed, it is not often we need to take the time derivative of the acceleration. So why are we not seeing more higher-order derivative theories? Although several studies present higher derivatives' usefulness in quadratic gravity and scalar-field theories, one will eventually encounter a problem. In 1850, the physicist Mikhail Ostrogradsky presented a theorem that stated that a non-degenerate Lagrangian composed of finite higher-order time derivatives results in a Hamiltonian unbounded from below. Explicitly, it was shown that the Hamiltonian of such a system includes linearity in physical momenta, often referred to as the ''Ostrogradsky ghost''. This thesis studies how one can avoid the Ostrogradsky ghost by considering degenerate Lagrangians to put constraints on the momenta. The study begins by showing the existence of the ghost and later cover the essential Hamiltonian formalism needed to conduct Hamiltonian constraint analyses of second-order time derivative systems, both single-variable and systems coupled to a regular one. Ultimately, the degenerate second-order Lagrangians successfully eliminate the Ostrogradsky ghost by generating secondary constraints restricting the physical momenta. Moreover, an outline of a Hamiltonian analysis of a general higher-order Lagrangian is presented at the end.