<p dir="ltr">In recent years, advancements in robotics and computing power have enabled robots to master complex tasks. Nevertheless, merely executing tasks isn't sufficient for robots. To achieve higher robot autonomy, learning the objective function is crucial. Autonomous systems can effectively eliminate the need for explicit programming by autonomously learning the control objective and deriving their control policy through the observation of task demonstrations. Hence, there's a need to develop a method for robots to learn the desired objective functions. In this thesis, we address several challenges in objective learning for autonomous systems, enhancing the applicability of our method in real-world scenarios. The ultimate objective of the thesis is to create a universal objective learning approach capable of addressing a range of existing challenges in the field while emphasizing data efficiency and robustness. Hence, building upon the previously mentioned intuition, we present a framework for autonomous systems to address a variety of objective learning tasks in real-time, even in the presence of noisy data. In addition to objective learning, this framework is capable of handling various other learning and control tasks.</p><p dir="ltr">The first part of this thesis concentrates on objective learning methods, specifically inverse optimal control (IOC). Within this domain, we have made three significant contributions aimed at addressing three existing challenges in IOC: 1) learning from minimal data, 2) learning without prior knowledge of system dynamics, and 3) learning with system outputs. </p><p dir="ltr">The second part of this thesis aims to develop a unified IOC framework to address all the challenges previously mentioned. It introduces a new paradigm for autonomous systems, referred to as Online Control-Informed Learning. This paradigm aims to tackle various of learning and control tasks online with data efficiency and robustness to noisy data. Integrating optimal control theory, online state estimation techniques, and machine learning methods, our proposed paradigm offers an online learning framework capable of tackling a diverse array of learning and control tasks. These include online imitation learning, online system identification, and policy tuning on-the-fly, all with efficient use of data and computation resources while ensuring robust performance.</p>