This dissertation introduces a Dynamical Neural Network (DNN) model based adaptive inverse optimal control design for a class of nonlinear systems. A DNN structure is developed and stabilized based on a control Lyapunov function (CLF). The CLF must satisfy the partial Hamilton Jacobi-Bellman (HJB) equation to solve the cost function in order to prove the optimality. In other words, the control design is derived from the CLF and inversely achieves optimality when the given cost function variables are determined posterior. All the stability of the closed loop system is ensured using the Lyapunov-based analysis. In addition to structure stability, uncertainty/ disturbance presents a problem to a DNN in that it could degrade the system performance. Therefore, the DNN needs a robust control against uncertainty. Sliding mode control (SMC) is added to nominal control design based CLF in order to stabilize and counteract the effects of disturbance from uncertain DNN, also to achieve global asymptotic stability. In the next section, a DNN observer is considered for estimating states of a class of controllable and observable nonlinear systems. A DNN observer-based adaptive inverse optimal control (AIOC) is needed. With weight adaptations, an adaptive technique is introduced in the observer design and its stabilizing control. The AIOC is designed to control a DNN observer and nonlinear system simultaneously while the weight parameters are updated online. This control scheme guarantees the quality of a DNN's state and minimizes the cost function. In addition, a tracking problem is investigated. An inverse optimal adaptive tracking control based on a DNN observer for unknown nonlinear systems is proposed. Within this framework, a time-varying desired trajectory is investigated, which generates a desired trajectory based on the external inputs. The tracking control design forces system states to follow the desired trajectory, while the DNN observer estimates the states and identifies unknown system dynamics. The stability method based on Lyapunov-based analysis is guaranteed a global asymptotic stability. Numerical examples and simulation studies are presented and shown for each section to validate the effectiveness of the proposed methods.