Traffic congestion is a central problem in transportation systems, especially in urban areas. The rapid development of Connected and Automated Vehicles (CAVs) and new traffic infrastructure technologies provides a promising solution to solve this problem. This work focuses on the safety-critical optimal control of CAVs in autonomous traffic systems. The dissertation starts with the roundabout problem of controlling CAVs travelling through a roundabout so as to jointly minimize their travel time, energy consumption, and centrifugal discomfort while providing speed-dependent safety guarantees. A systematic approach is developed to determine the safety constraints for each CAV dynamically. The joint optimal control and control barrier function (OCBF) controller is applied, where the unconstrained optimal control solution is derived which is subsequently optimally tracked by a real-time controller while guaranteeing the satisfaction of all safety constraints. Secondly, the dissertation deals with the feasibility problem of OCBF. The feasibility problem arises when the control bounds conflict with the Control Barrier Function (CBF) constraints and is solved by adding a single feasibility constraint to the Quadratic Problem (QP) in the OCBF controller to derive the feasibility guaranteed OCBF. The feasibility guaranteed OCBF is applied in the merging control problem which provably guarantees the feasibility of all QPs derived from the OCBF controller. Thirdly, the dissertation deals with the performance loss of OCBF due to the improperly selected reference trajectory which deviates largely from the complete optimal solution especially when the vehicle limitations are tight. A neural network is used to learn the control policy from data retrieved by offline calculation from the complete optimal solutions. Tracking the learnt reference trajectory with CBF outperforms OCBF in simulation experiments. Finally, a hierarchical framework of modular control zones (CZ) is proposed to extend the safety-critical optimal control of CAV from a single CZ to a traffic network. The hierarchical modular CZ framework is developed consisting of a lower-level OCBF controller and a higher-level feedback flow controller to coordinate adjacent CZs which outperforms a direct extension of the OCBF framework to multiple CZs without any flow control in simulation.