Autonomous cyber-physical systems (CPSs) such as autonomous connected vehicles (ACVs), unmanned aerial vehicles (UAVs), critical infrastructure (CI), and the Internet of Things (IoT) will be essential to the functioning of our modern economies and societies. Therefore, maintaining the autonomy of CPSs as well as their stability, robustness, and security (SRS) in face of exogenous and disruptive events is a critical challenge. In particular, it is crucial for CPSs to be able to not only operate optimally in the vicinity of a normal state but to also be robust and secure so as to withstand potential failures, malfunctions, and intentional attacks. However, to evaluate and improve the SRS of CPSs one must overcome many technical challenges such as the unpredictable behavior of a CPS's cyber-physical environment, the vulnerability to various disruptive events, and the interdependency between CPSs. The primary goal of this dissertation is, thus, to develop novel foundational analytical tools, that weave together notions from machine learning, game theory, and control theory, in order to study, analyze, and optimize SRS of autonomous CPSs. Towards achieving this overarching goal, this dissertation led to several major contributions. First, a comprehensive control and learning framework was proposed to thwart cyber and physical attacks on ACV networks. This framework brings together new ideas from optimal control and reinforcement learning (RL) to derive a new optimal safe controller for ACVs in order to maximize the street traffic flow while minimizing the risk of accidents. Simulation results show that the proposed optimal safe controller outperforms the current state of the art controllers by maximizing the robustness of ACVs to physical attacks. Furthermore, using techniques from convex optimization and deep RL a joint trajectory and scheduling policy is proposed in UAV-assisted networks that aims at maintaining the freshness of ground node data at the UAV. The analytical and simulation results show that the proposed policy can outperform policies such discretized state RL and value-based methods in terms of maximizing the freshness of data. Second, in the IoT domain, a novel watermarking algorithm, based on long short term memory cells, is proposed for dynamic authentication of IoT signals. The proposed watermarking algorithm is coupled with a game-theoretic framework so as to enable efficient authentication in massive IoT systems. Simulation results show that using our approach, IoT messages can be transmitted from IoT devices with an almost 100% reliability. Next, a brainstorming generative adversarial network (BGAN) framework is proposed. It is shown that this framework can learn to generate real-looking data in a distributed fashion while preserving the privacy of agents (e.g. IoT devices, ACVs, etc). The analytical and simulation results show that the proposed BGAN architecture allows heterogeneous neural network designs for agents, works without reliance on a central controller, and has a lower communication over head compared to other state-of-the-art distributed architectures. Last, but not least, the SRS challenges of interdependent CI (ICI) are addressed. Novel game-theoretic frameworks are proposed that allow the ICI administrator to assign different protection levels on ICI components to maximizing the expected ICI security. The mixed-strategy Nash of the games are derived analytically. Simulation results coupled with theoretical analysis show that, using the proposed games, the administrator can maximize the security level in ICI components. In summary, this dissertation provided major contributions across the areas of CPSs, machine learning, game theory, and control theory with the goal of ensuring SRS across various domains such as autonomous vehicle networks, IoT systems, and ICIs. The proposed approaches provide the necessary fundamentals that can lay the foundations of SRS in CPSs and pave the way toward the practical deployment of autonomous CPSs and applications. / Doctor of Philosophy / In order to deliver innovative technological services to their residents, smart cities will rely on autonomous cyber-physical systems (CPSs) such as cars, drones, sensors, power grids, and other networks of digital devices. Maintaining stability, robustness, and security (SRS) of those smart city CPSs is essential for the functioning of our modern economies and societies. SRS can be defined as the ability of a CPS, such as an autonomous vehicular system, to operate without disruption in its quality of service. In order to guarantee SRS of CPSs one must overcome many technical challenges such as CPSs' vulnerability to various disruptive events such as natural disasters or cyber attacks, limited resources, scale, and interdependency. Such challenges must be considered for CPSs in order to design vehicles that are controlled autonomously and whose motion is robust against unpredictable events in their trajectory, to implement stable Internet of digital devices that work with a minimum communication delay, or to secure critical infrastructure to provide services such as electricity, gas, and water systems. The primary goal of this dissertation is, thus, to develop novel foundational analytical tools, that weave together notions from machine learning, game theory, and control theory, in order to study, analyze, and optimize SRS of autonomous CPSs which eventually will improve the quality of service provided by smart cities. To this end, various frameworks and effective algorithms are proposed in order to enhance the SRS of CPSs and pave the way toward the practical deployment of autonomous CPSs and applications. The results show that the developed solutions can enable a CPS to operate efficiently while maintaining its SRS. As such, the outcomes of this research can be used as a building block for the large deployment of smart city technologies that can be of immense benefit to tomorrow's societies.