Stochastic control theory is introduced and its importance relative to control science in general is discussed. It is noted that although most practical control problems are stochastic in some sense, stochastic control theory has found few successful applications. One of the major reasons for this dearth of practical applications is considered to be the difficulty of finding optimal solutions for most stochastic control problems. Consequently, suboptimal solutions are usually necessary and it is shown that the determination of useful suboptimal solutions depends largely on an understanding of the structure of stochastic control laws. It is observed that the basic result about the structure of stochastic control laws is that stochastic controllers perform two functions, estimation and control implementation. The mathematical form of the first function is updating equations while the mathematical form of the second function is the control policy. It is observed that it is the possible interaction between the two functions of estimation and control implementation which causes stochastic control policies to differ from deterministic control policies. It is shown that this interaction can occur in two ways: (i) the accuracy of the estimation can affect the control policy; (ii) the control policy can affect the rate at which uncertainty is reduced. The effects of the possible interaction on stochastic control policies is investigated by studying three classes of stochastic control problems, those that are certainty equivalent, neutral and separable. A consideration of the relationship between these three properties leads to the conclusion that stochastic control policies have three components: (i) the certainty equivalent policy; (ii) caution; (iii) probing and that the functions of these components are (i) to produce the required plant behaviour, (ii) to take into account the fact that the accuracy of the estimation can affect the control policy, (iii) to take advantage of the fact that the control policy can affect the rate at which uncertainty is reduced. Two of the components of stochastic control policies, caution and probing, describe the effect of the possible interaction between estimation and control implementation. The properties of these components are determined through several simple examples. It is noted that the caution component is more easily determined exactly than is the probing component; however, the exact form of the probing component is possibly not very important. Two simulations of simple stochastic control problems are reported. The first of these concerns the 'known gain problem' and clearly demonstrates that the advantages given by the application of stochastic control theory increase rapidly as the length of the process controlled increases. The second simulation is of the more complex 'unknown constant gain problem' and demonstrates the importance of caution. Caution is necessary if large and very costly errors are to be avoided while probing is necessary particularly for longer processes to reduce uncertainty. It is shown that a control policy called the neutral control policy which includes the optimal amount of caution but which has no probing component is more easily derived than is the optimal control policy. It is seen from the second simulation that the neutral control policy with a very simple additive probing term is a good suboptimal control policy. It is conjectured that neutral-plus-probing control policies may prove very useful as suboptimal policies for a large number of stochastic control problems.
| Identifer | oai:union.ndltd.org:bl.uk/oai:ethos.bl.uk:468359 |
| Date | January 1971 |
| Creators | Patchell, John W. |
| Publisher | University of Oxford |
| Source Sets | Ethos UK |
| Detected Language | English |
| Type | Electronic Thesis or Dissertation |
| Source | http://ora.ox.ac.uk/objects/uuid:b60c466a-7435-4a3d-b17b-53adb16ec5e4 |
Page generated in 0.0022 seconds