In dieser Arbeit wird erstmalig der Einfluss stochastischer Schwankungen auf endoreversible Modelle untersucht. Hierfür wird die Novikov-Maschine mit drei verschieden Wärmetransportgesetzen (Newton, Fourier, asymmetrisch) betrachtet. Während die maximale verrichtete Arbeit und der dazugehörige Wirkungsgrad recht einfach im Falle konstanter Wärmebadtemperaturen hergeleitet werden können, ändern sich dies, falls die Temperaturen stochastisch fluktuieren können. Im letzteren Fall muss die stochastische optimale Kontrolltheorie genutzt werden, um das Maximum der zu erwartenden Arbeit und die dazugehörige Kontrollstrategie zu ermitteln. Im Allgemeinen kann die Lösung derartiger Probleme auf eine nichtlineare, partielle Differentialgleichung, welche an eine Optimierung gekoppelt ist, zurückgeführt werden. Diese Gleichung wird stochastische Hamilton-Jacobi-Bellman-Gleichung genannt. Allerdings können, wie in dieser Arbeit dargestellt, die Berechnungen vereinfacht werden, wenn man annimmt, dass die Fluktuationen unabhängig von der betrachteten Kontrollvariablen sind. In diesem Fall zeigen analytische Betrachtungen, dass die Gleichungen für die verrichtete Arbeit and den Wirkungsgrad ihre ursprüngliche Form behalten, aber manche Terme müssen durch entsprechende Zeitmittel bzw. Erwartungswerte ersetzt werden, jeweils abhängig von der betrachteten Art der Kontrolle. Basierend auf einer Analyse der Leistungsparameter im Falle einer Gleichverteilung der heißen Temperatur der Novikov-Maschine können Schlussfolgerungen auf deren Monotonieverhalten gezogen werden. Der Vergleich verschiedener, zeitunabhängiger, symmetrischer Verteilungen führt zu einer bis dato unbekannten Erweiterung des Curzon-Ahlborn-Wirkungsgrades im Falle kleiner Schwankungen. Weiterhin wird eine Analyse einer Novikov-Maschine mit asymmetrischen Wärmetransport, bei der das Verhalten der heißen Temperatur durch einen Ornstein-Uhlenbeck-Prozess beschrieben wird, durchgeführt. Abschließend wird eine Novikov-Maschine mit Fourierscher Wärmeleitung, bei der die Dynamik der heißen Temperatur von der Kontrollvariable abhängt, betrachtet. Durch das Lösen der Hamilton-Jacobi-Bellman-Gleichung können neuartige Schlussfolgerungen gezogen werden, wie derartige Systeme optimal zu steuern sind. / In this thesis, the influence of stochastic fluctuations on the performance of endoreversible engines is investigated for the first time. For this, a Novikov-engine with three different heat transport laws (Newtonian, Fourier, asymmetric) is considered. While the maximum work output and corresponding efficiency can be deduced easily in the case of constant heat bath temperatures, this changes, if these temperatures are allowed to fluctuate stochastically. In the latter case, stochastic optimal control theory has to be used to find the maximum of the expected work output and the corresponding control policy. In general, solving such problems leads to a non-linear, partial differential equation coupled to an optimization, called the stochastic Hamilton-Jacobi-Bellman equation. However, as presented in this thesis, calculations can be simplified, if one assumes that the fluctuations are independent of the considered control variable. In this case, analytic considerations show that the equations for performance measures like work output and efficiency keep their original form, but terms have to be replaced by appropriate time averages and expectation values, depending on the considered control type. Based on an analysis of the performance measures in the case of a uniform distribution of the hot temperature of the Novikov engine, conclusions on their monotonicity behavior are drawn. The comparison of several, time independent, symmetric distributions reveals a to date unknown extension to the Curzon-Ahlborn efficiency in the case of small fluctuations. Furthermore, an analysis of a Novikov engine with asymmetric heat transport, where the behavior of the hot temperature is described by an Ornstein-Uhlenbeck process, is performed. Finally, a Novikov engine with Fourier heat transport is considered, where the dynamics of the hot temperature depends on the control variable. By solving the corresponding Hamilton-Jacobi-Bellman equation, new conclusions how to optimally control such systems are drawn.

<p>It is predicted that there would be 41.6 billion IoT devices by 2025, which has kindled new interests on the timing coordination between sensors and controllers, i.e., how to use the waiting time to improve the performance. Sun et al. showed that a <i>controller</i> can strictly improve the data freshness, the so-called Age-of-Information (AoI), via careful scheduling designs. The optimal waiting policy for the <i>sensor</i> side was later characterized in the context of remote estimation. The first part of this work develops the jointly optimal sensor/controller waiting policy. It generalizes the above two important results in that not only do we consider joint sensor/controller designs, but we also assume random delay in both the forward and feedback directions. </p> <p> </p> <p>The second part of the work revisits and significantly strengthens the seminal results of Sun et al on the following fronts: (i) When designing the optimal offline schemes with full knowledge of the delay distributions, a new <i>fixed-point-based</i> method is proposed with <i>quadratic convergence rate</i>; (ii) When the distributional knowledge is unavailable, two new low-complexity online algorithms are proposed, which provably attain the optimal average AoI penalty; and (iii) the online schemes also admit a modular architecture, which allows the designer to <i>upgrade</i> certain components to handle additional practical challenges. Two such upgrades are proposed: (iii.1) the AoI penalty function incurred at the destination is unknown to the source node and must also be estimated on the fly, and (iii.2) the unknown delay distribution is Markovian instead of i.i.d. </p> <p> </p> <p>With the exponential growth of interconnected IoT devices and the increasing risk of excessive resource consumption in mind, the third part of this work derives an optimal joint cost-and-AoI minimization solution for multiple coexisting source-destination (S-D) pairs. The results admit a new <i>AoI-market-price</i>-based interpretation and are applicable to the setting of (i) general heterogeneous AoI penalty functions and Markov delay distributions for each S-D pair, and (ii) a general network cost function of aggregate throughput of all S-D pairs. </p> <p> </p> <p>In each part of this work, extensive simulation is used to demonstrate the superior performance of the proposed schemes. The discussion on analytical as well as numerical results sheds some light on designing practical network utility maximization protocols.</p>