Continuous and discrete-time Successive Backward Sweep (SBS) methods for solving nonlinear optimal control problems involving terminal and control constraints are proposed in this dissertation. They closely resemble the Neighboring Extremals and Differential Dynamic Programming algorithms, which are based on the successive solutions to a series of linear control problems with quadratic performance indices. The SBS methods are relatively insensitive to the initial guesses of the state and control histories, which are not required to satisfy the system dynamics. Hessian modifications are utilized, especially for non-convex problems, to avoid singularities during the backward integration of the gain equations. The SBS method requires the satisfaction of the Jacobi no-conjugate point condition and hence, produces optimal solutions. The standard implementation of the SBS method for continuous-time systems incurs terminal boundary condition errors due to an algorithmic singularity as well as numerical inaccuracies in the computation of the gain matrices. Alternatives for boundary error reduction are proposed, notably the aiming point and the switching between two forms of the sweep expansion formulae. Modification of the sweep formula expands the domain of convergence of the SBS method and allows for a rigorous testing for the existence of conjugate points. Numerical accuracy of the continuous-time formulation of the optimal control problem can be improved with the use of symplectic integrators, which generally are implicit schemes in time. A time-explicit group preserving method based on the Magnus series representation of the state transition is implemented in the SBS setting and is shown to outperform a non-symplectic integrator of the same order. Discrete-time formulations of the optimal control problem, directly accounting for a specific time-stepping method, lead to consistent systems of equations, whose solutions satisfy the boundary conditions of the discretized problem accurately. In this regard, the second-order, implicit mid-point averaging scheme, a symplectic integrator, is adapted for use with the SBS method. The performance of the mid-point averaging scheme is compared with other methods of equal and higher-order non-symplectic schemes to show its advantages. The SBS method is augmented with a homotopy- continuation procedure to isolate and regulate certain nonlinear effects for difficult problems, in order to extend its domain of convergence. The discrete-time SBS method is also extended to solve problems where the controls are approximated to be impulsive and to handle waypoint constraints as well. A variety of highly nonlinear optimal control problems involving orbit transfer, atmospheric reentry, and the restricted three-body problem are treated to demonstrate the performance of the methods developed in this dissertation.