1 |
Approaches for Efficient Autonomous Exploration using Deep Reinforcement LearningThomas Molnar (8735079) 24 April 2020 (has links)
<p>For autonomous exploration of complex and unknown environments, existing Deep Reinforcement Learning (Deep RL) approaches struggle to generalize from computer simulations to real world instances. Deep RL methods typically exhibit low sample efficiency, requiring a large amount of data to develop an optimal policy function for governing an agent's behavior. RL agents expect well-shaped and frequent rewards to receive feedback for updating policies. Yet in real world instances, rewards and feedback tend to be infrequent and sparse.</p><p> </p><p>For sparse reward environments, an intrinsic reward generator can be utilized to facilitate progression towards an optimal policy function. The proposed Augmented Curiosity Modules (ACMs) extend the Intrinsic Curiosity Module (ICM) by Pathak et al. These modules utilize depth image and optical flow predictions with intrinsic rewards to improve sample efficiency. Additionally, the proposed Capsules Exploration Module (Caps-EM) pairs a Capsule Network, rather than a Convolutional Neural Network, architecture with an A2C algorithm. This provides a more compact architecture without need for intrinsic rewards, which the ICM and ACMs require. Tested using ViZDoom for experimentation in visually rich and sparse feature scenarios, both the Depth-Augmented Curiosity Module (D-ACM) and Caps-EM improve autonomous exploration performance and sample efficiency over the ICM. The Caps-EM is superior, using 44% and 83% fewer trainable network parameters than the ICM and D-ACM, respectively. On average across all “My Way Home” scenarios, the Caps-EM converges to a policy function with 1141% and 437% time improvements over the ICM and D-ACM, respectively.</p>
|
2 |
Advanced Methods for Simulation and Performance Analysis of Planetary Radar Sounder DataThakur, Sanchari 23 April 2020 (has links)
Radar sounders (RS) are low frequency remote sensing instruments that profile the shallow subsurface of planetary bodies providing valuable scientific information. The prediction of the RS performance and the interpretation of the target properties from the RS data are challenging due to the complex electromagnetic interaction between many acquisition variables. RS simulations address this issue by forward modeling this complex interaction and simulating the radar response. However, existing simulators require detailed and subjective modeling of the target in order to produce realistic radargrams. For less-explored planetary bodies, such information is difficult to obtain with high accuracy. Moreover, the high computational requirements of conventional electromagnetic simulators prohibit the simulation of a large number of radargrams. Thus, it is not possible to generate and analyze a database of simulated radargrams representative of the acquisition scenario that would be very useful for both the RS design and the data analysis phase. To overcome these difficulties and to produce realistic simulated radargrams, this thesis proposes two novel approaches to the simulation and analysis of the radar response. The first contribution is a simulation approach that leverages the data available over geological analogs of the investigated target and reprocesses them to obtain the simulated radargrams. The second contribution is a systematic approach to the generation and analysis of a database of simulated radargrams representing the possible scenarios during the RS acquisition. The database is analyzed to predict the RS performance, to design the instrument parameters, and to support the development of automatic target detection algorithms. To demonstrate the proposed techniques the thesis addresses their use in two future RS instruments, which are at different phases of development: (1) the Radar for Icy Moons Exploration (RIME) and (2) a RS for Earth observation of the polar ice caps. The first contribution focuses on the analysis of the detectability of complex tectonic targets on the icy moons of Jupiter by RIME by simulating the radar response of 3D target models. The second contribution presents a feasibility study for an Earth orbiting RS based on the proposed simulation approaches.
|
Page generated in 0.1802 seconds