<p>Stochastic Gradient Descent (SGD) is a widely-used iterative algorithm for solving stochastic optimization problems for a smooth (and possibly non-convex) objective function via queries from a first-order stochastic oracle.</p> <p>In this dissertation, we critically examine SGD's choice of executing a single step as opposed to multiple steps between subsample updates. Our investigation leads naturally to generalizing SG into Retrospective Approximation (RA) where, during each iteration, a deterministic solver executes possibly multiple steps on a subsampled deterministic problem and stops when further solving is deemed unnecessary from the standpoint of statistical efficiency. RA thus leverages what is appealing for implementation -- during each iteration, a solver, e.g., L-BFGS with backtracking line search is used, as is, and the subsampled objected function is solved only to the extent necessary. We develop a complete theory using relative error of the observed gradients as the principal object, demonstrating that almost sure and L1 consistency of RA are preserved under especially weak conditions when sample sizes are increased at appropriate rates. We also characterize the iteration and oracle complexity (for linear and sub-linear solvers) of RA, and identify two practical termination criteria, one of which we show leads to optimal complexity rates. The message from extensive numerical experiments is that the ability of RA to incorporate existing second-order deterministic solvers in a strategic manner is useful both in terms of algorithmic trajectory as well as from the standpoint of  dispensing with hyper-parameter tuning.</p>

Neural networks can perform an incredible array of complex tasks, but successfully training a network is difficult because it requires us to minimize a function about which we know very little. In practice, developing a good model requires both intuition and a lot of guess-and-check. In this dissertation, we study a type of fully-connected neural network that improves on standard rectifier networks while retaining their useful properties. We then examine this type of network and its loss function from a probabilistic perspective. This analysis leads to a new rule for parameter initialization and a new method for predicting effective learning rates for gradient descent. Experiments confirm that the theory behind these developments translates well into practice.

A scaled gradient descent method for competition of applications of conjugate gradient with priori estimations of the minimum value / This research proposes a novel method of improving the Gradient Descent method in an effort to be competitive with applications of the conjugate gradient method while reducing computation per iteration. Iterative methods for unconstrained optimization have found widespread application in digital signal processing applications for large inverse problems, such as the use of conjugate gradient for parallel image reconstruction in MR Imaging. In these problems, very good estimates of the minimum value at the objective function can be obtained by estimating the noise variance in the signal, or using additional measurements. The method proposed uses an estimation of the minimum to develop a scaling for Gradient Descent at each iteration, thus avoiding the necessity of a computationally extensive line search. A sufficient condition for convergence and proof are provided for the method, as well as an analysis of convergence rates for varying conditioned problems. The method is compared against the gradient descent and conjugate gradient methods. A method with a computationally inexpensive scaling factor is achieved that converges linearly for well-conditioned problems. The method is tested with tricky non-linear problems against gradient descent, but proves unsuccessful without augmenting with a line search. However with line search augmentation the method still outperforms gradient descent in iterations. The method is also benchmarked against conjugate gradient for linear problems, where it achieves similar convergence for well-conditioned problems even without augmenting with a line search. / Thesis / Master of Science (MSc) / This research proposes a novel method of improving the Gradient Descent method in an effort to be competitive with applications of the conjugate gradient method while reducing computation per iteration. Iterative methods for unconstrained optimization have found widespread application in digital signal processing applications for large inverse problems, such as the use of conjugate gradient for parallel image reconstruction in MR Imaging. In these problems, very good estimates of the minimum value at the objective function can be obtained by estimating the noise variance in the signal, or using additional measurements. The method proposed uses an estimation of the minimum to develop a scaling for Gradient Descent at each iteration, thus avoiding the necessity of a computationally extensive line search. A sufficient condition for convergence and proof are provided for the method, as well as an analysis of convergence rates for varying conditioned problems. The method is compared against the gradient descent and conjugate gradient methods. A method with a computationally inexpensive scaling factor is achieved that converges linearly for well-conditioned problems. The method is tested with tricky non-linear problems against gradient descent, but proves unsuccessful without augmenting with a line search. However with line search augmentation the method still outperforms gradient descent in iterations. The method is also benchmarked against conjugate gradient for linear problems, where it achieves similar convergence for well-conditioned problems even without augmenting with a line search.

No description available.

No description available.

No / We often complete step downs in the absence of visual feedback of the lower-limbs, and/or of the area on the ground where we intend to land (e.g. when descending a step whilst carrying a laundry basket). Therefore, the present study examined whether information from lower visual field (lvf) provides any advantage to the control of step descent. Ten healthy subjects (age 24.4 ± 9.4 years) completed repeated step downs over three-step heights with visual information available from either full or upper visual fields (lvf occluded), and for specific intervals relative to step initiation. Visuomotor control of step descent was assessed by determining pre-landing kinematic measures and landing mechanic variables for the initial landing period. Findings indicate that whilst there were only limited effects on pre-landing kinematic measures under lvf occlusion, individual’s ability to plan/control landing mechanics was significantly different in such conditions compared to when they had access to full field vision. These changes were consistent with participants being uncertain regarding precise floor height when access to lvf was restricted, and consequently led them to adapt their landing behaviour but without fundamentally altering their stepping strategy. Compared to when vision was available throughout, the occlusion of vision (full or upper visual field) from toe-off or mid-swing onwards caused very few differences in landing behaviour. This suggests that the contribution of information from lvf to the control of landing behaviour occurs predominantly prior to or during movement initiation and that ‘online’ vision is used only in the latter portion of the descent phase to subtly ‘fine tune’ landings.

The 2012 Decadal Survey has stated that there is a critical role for a Venus In-situ Explore (VISE) missions to a variety of important sites, specifically the Tessera terrain. This work aims to answer the Decadal Survey's call by developing a new comprehensive Entry, Descent, Landing, and Locomotion (EDLL) vehicle for in-situ exploration of Venus, especially in the Tessera regions. TANDEM, the Tension Adjustable Network for Deploying Entry Membrane, is a new planetary probe concept in which all of EDLL is achieved by a single multifunctional tensegrity structure. The concept uses same fundamental concept as the ADEPT (Adaptable Deployable Entry and Placement Technology) deployable heat shield but replaces the standard internal structure with the structure from the tensegrity-actuated rover to provide a combined aeroshell and rover design. The tensegrity system implemented by TANDEM reduces the mass of the overall system while enabling surface locomotion and mitigating risk associated with landing in the rough terrain of Venus's Tessera regions, which is otherwise nearly inaccessible to surface missions. TANDEM was compared to other state-of-the-art lander designs for an in-situ mission to Venus. It was shown that TANDEM provides the same scientific experimentation capabilities that were proposed for the VITaL mission, with a combined mass reduction for the aeroshell and lander of 52% (1445 kg), while eliminating the identified risks associated with entry loads and very rough terrain. Additionally, TANDEM provides locomotion when on the surface as well as a host of other maneuvers during entry and descent, which was not present in the VITaL design. Based on its unique multifunctional infrastructure and excellent crashworthiness for impact on rough surfaces, TANDEM presents a robust system to address some of the Decadal Survey's most pressing questions about Venus. / Ph. D.

To enable a crewed mission to Mars, precision landing capabilities of Entry, Descent, and Landing (EDL) systems must be improved. The need for larger payloads, higher landing sites, and controllability has motivated the National Aeronautics and Space Administration (NASA) to invest in new technologies to replace traditional rigid aeroshell systems, which are limited in size by the payload envelope of existing launch vehicles. A Hypersonic Inflatable Aerodynamic Decelerator (HIAD) is an emerging technology that provides an increased drag area by inflating the aeroshell to diameters not possible with rigid aeroshells, allowing the vehicle to decelerate higher in the atmosphere, offering access to higher landing sites with more timeline margin. To enable a crewed mission to Mars, future entry vehicles will require precision landing capabilities that go beyond heritage EDL guidance strategies that utilize fuel-intensive and error-prone bank reversals. A novel Direct Force Control (DFC) approach of independently controlling the lift and side force of a vehicle that utilizes a HIAD with an aerodynamic shape morphing capability is proposed. To date, the mechanisms and controls required to morph an inflatable structure to generate lift have not been explored. In this dissertation, novel morphing HIAD concepts are investigated and designed to satisfy mission requirements, aerodynamic tools are built to assess the aerodynamic performance of morphed blunt body shapes, and a structural feasibility study is performed using models correlated to test data to determine the forces required to generate the desired shape change based on a crewed mission to Mars. A novel control methodology is introduced by applying a unique DFC strategy to a morphing HIAD to enhance precision landing capabilities of EDL systems, and the ability of a morphing HIAD to safely land a vehicle on Mars is assessed by performing a closed-loop feedback simulation for a Mars entry trajectory. Finally, a control mechanism is demonstrated on a small-scale inflatable structure. Conclusions and contributions of this research are presented along with a discussion of future research opportunities of morphing HIADs. / PHD

A steepest descent method is constructed for the general setting of a linear differential equation paired with uniqueness-inducing conditions which might yield a generally overdetermined system. The method differs from traditional steepest descent methods by considering the conditions when defining the corresponding Sobolev space. The descent method converges to the unique solution to the differential equation so that change in condition values is minimal. The system has a solution if and only if the first iteration of steepest descent satisfies the system. The finite analogue of the descent method is applied to example problems involving finite difference equations. The well-posed problems include a singular ordinary differential equation and Laplace’s equation, each paired with respective Dirichlet-type conditions. The overdetermined problems include a first-order nonsingular ordinary differential equation with Dirichlet-type conditions and the wave equation with both Dirichlet and Neumann conditions. The method is applied in an investigation of the Tricomi equation, a long-studied equation which acts as a prototype of mixed partial differential equations and has application in transonic flow. The Tricomi equation has been studied for at least ninety years, yet necessary and sufficient conditions for existence and uniqueness of solutions on an arbitrary mixed domain remain unknown. The domains of interest are rectangular mixed domains. A new type of conditions is introduced. Ladder conditions take the uncommon approach of specifying information on the interior of a mixed domain. Specifically, function values are specified on the parabolic portion of a mixed domain. The remaining conditions are specified on the boundary. A conjecture is posed and states that ladder conditions are necessary and sufficient for existence and uniqueness of a solution to the Tricomi equation. Numerical experiments, produced by application of the descent method, provide strong evidence in support of the conjecture. Ladder conditions allow for a continuous deformation from Dirichlet conditions to initial-boundary value conditions. Such a deformation is applied to a class of Tricomi-type equations which transition from degenerate elliptic to degenerate hyperbolic. A conjecture is posed and states that each problem is uniquely solvable and the solutions vary continuously as the differential equation and corresponding conditions vary continuously. If the conjecture holds true, the result will provide a method of unifying elliptic Dirichlet problems and hyperbolic initial-boundary value problem. Numerical evidence in support of the conjecture is presented.

CoordenaÃÃo de AperfeiÃoamento de Pessoal de NÃvel Superior / A existÃncia da infÃncia à vivida em um espaÃo que està no interior de uma dinÃmica temporal. A pulsaÃÃo desse espaÃo à marcada pela potÃncia mÃltipla, complexa e irregular do encontro de experiÃncias e concepÃÃes singulares e coletivas de cada pessoa no decorrer do tempo. Inserida na perspectiva da cultura de matriz africana e afro-brasileira, o presente estudo buscou investigar como e quais experiÃncias sÃo produzidas e partilhadas por um grupo de crianÃas afrodescendentes no Arraial do Retiro, em Salvador-BA, buscando identificar experiÃncias educacionais que se configuram naquela territorialidade. A pesquisa foi norteada pela abordagem qualitativa da histÃria-sociolÃgica a qual encontra no espaÃo geogrÃfico construÃdo historicamente e na afrodescendÃncia os principais aportes teÃrico-metodolÃgicos que guiaram este estudo. Adotando uma perspectiva de dentro, as ruas do bairro de maioria afrodescendente foi o espaÃo social eleito para realizaÃÃo da pesquisa com um grupo de nove crianÃas entre 3 e 9 anos de idade. Foram realizados seis encontros com as crianÃas do grupo, nos quais foram utilizadas mÃltiplas linguagens: oralidade, contar histÃrias, desenhos, fotografias, dinÃmicas, cadernos de campo, observaÃÃo e o conviver como vista a alcanÃar os objetivos propostos. O extenso material qualitativo reunido foi organizado em dois nÃveis: 1) panorama geral da primeira infÃncia no bairro; 2) relato minucioso do grupo de crianÃas que participaram mais intensamente da pesquisa. A anÃlise do primeiro nÃvel possibilitou a identificaÃÃo de que hà uma diversidade de formas de viver a primeira infÃncia no Arraial do Retiro. No que diz respeito ao segundo nÃvel, se procedeu à configuraÃÃo do contexto espacial e das aÃÃes das crianÃas afrodescendentes do grupo. Assim, observamos que as mesmas participam da dinÃmica comunitÃria do Arraial Retiro produzindo, trocando, acessando, e fazendo circular experiÃncias educativas sociais, culturais e afetivos gestados nas relaÃÃes entre elas, jovens, adultos e idosos naquela territorialidade. A partir dos resultados e com base na articulaÃÃo entre teoria e empiria foi possÃvel compreender a importÃncia das singularidades da(s) infÃncia(s) soteropolitana(s) e brasileira(s). Nessa pesquisa se defendeu que essas especificidades se fazem a partir da conexÃo entre os pertencimentos Ãtnico, territorial, etÃrio, social e de gÃnero em meio a uma interpenetraÃÃo das dimensÃes individuais e coletivas. / Childhoodâs existence is experienced in a space which lies at the core of a temporal dynamics. The pulsation of that space is marked by the multiple, complex, and irregular potency of experiences and conceptions, both individual and collective, of each person along a timeline. Inserted in the perspective of the culture of African and Afro-Brazilian matrix, the present study aims at investigating how and which experiences are produced and shared by a group of Afro-descendant children from the Arraial do Retiro, a neighborhood in Salvador, trying to identify educational experiences that have been configured in that territory. The present study follows the qualitative approach of the Sociological History, which sets up Afro-descent and the geographical space that has been built historically as the main theoretical-methodological referentials. Adopting a perspective from inside, the streets of that neighborhood, where Afro-descendants constitute a numerical majority, were established as the social space where the research was carried out with a group of nine children, aged between three and nine. During the six meetings with the children of the group, multiple languages have been used: orality, storytelling, drawings, photographs, dynamic classroom-like methods, field notebooks, observation and togetherness, in order to reach the proposed aims. The comprehensive qualitative material that was collected was organized in two levels: 1) general panorama of the first childhood in the neighborhood; 2) and a thorough report of the group of children who took part of the research more intensely. The analysis of the first level has allowed us to identify that there is a diversity of forms of living the first childhood in Arraial do Retiro. Concerning the second level, we proceeded the configuration of the spatial context and the actions of the Afro-descendant children. Thus, it was observed that those children take part of the communityâs dynamics of Arraial do Retiro by producing, exchanging, accessing, and spreading social, cultural, and affectionate educational experiences, all of them being born from the relations between youngsters, adults and elderly people in that territory. Judging by the results and the linking between theory and empiricism, I affirm the importance of considering the singularities of the Brazilian and Soteropolitan childhood. I also claim that those specificities are made from the connection between the ethnic, territorial, social, age and gender-oriented senses of belonging, amidst an interpenetration of the collective dimensions.