Recently, Stochastic Gradient Descent (SGD) and its variants have become the dominant methods in the large-scale optimization of machine learning problems. A variety of strategies have been proposed for tuning the step sizes, ranging from adaptive step sizes (e.g., AdaGrad) to heuristic methods to change the step size in each iteration. Also, momentum has been widely employed in machine learning tasks to accelerate the training process. Yet, there is a gap in our theoretical understanding of them. In this work, we start to close this gap by providing formal guarantees to a few heuristic optimization methods and proposing improved algorithms if the theoretical results are suboptimal. First, we analyze a generalized version of the AdaGrad (Delayed AdaGrad) step sizes in both convex and non-convex settings, showing that these step sizes allow the algorithms to automatically adapt to the level of noise of the stochastic gradients. We show sufficient conditions for Delayed AdaGrad to achieve almost sure convergence of the gradients to zero, which is the first guarantee for Delayed AdaGrad in the non-convex setting. Moreover, we present a high probability analysis for Delayed AdaGrad and its momentum variant in the non-convex setting. Second, we present an analysis of SGD with exponential and cosine step sizes, which are simple-to-use, empirically successful but lack of theoretical support. We provide the very first convergence guarantees for them in the smooth and non-convex setting, with and without the Polyak-Łojasiewicz (PL) condition. We show that these two strategies also have the good property of adaptivity to noise under PL condition. Third, we study the last iterate of momentum methods. We prove the first lower bound in the general convex setting for the last iterate of SGD with constant momentum. Based on the fact that the lower bound is suboptimal, we investigate a class of (both adaptive and non-adaptive) Follow-The-Regularized-Leader-based momentum algorithms (FTRL-based SGDM) with increasing momentum and shrinking updates. We show that their last iterate has optimal convergence for unconstrained convex stochastic optimization problems without projections onto bounded domains nor knowledge of the number of iterations.

No description available.

No description available.

No description available.

No description available.

An experimental design was used to investigate the relationship between stress and eyewitness memory in adolescents (Experiment 1) and young adults (Experiment 2). Psychosocial stress was induced using a public speaking task. As eyewitness outcome measures, participants were asked to provide a physical description and to make a lineup identification of a) a research assistant with whom they had interacted under stressful circumstances (prior to public speaking), and b) a research assistant with whom they had interacted under neutral circumstances. Participants also responded to questions that tested their memory for central and peripheral details of the interactions.

No description available.

The point of this study is to elucidate and expand upon the concept of "irrationality" in the work of Max Weber and, to a lesser extent, in that of Vilfredo Pareto, whose viewpoints are antithetical, but whose goals for social theory were similar. This attempt, to compare two exemplars of classical social theory who are often considered absolutely alien to one another, is carried out through "hermeneutic" interpretation. To the end of determining the utility of hermeneutics to the study of social thought, a lengthy Appendix is provided in which various hermeneutical theories (including those of Ast, Wolf, Schleiermacher, Dilthey, Heidegger, Betti, and Gadamer) are assayed. With propositions drawn from this subordinate study, a detailed analysis is performed upon Weber's writings, which seeks to remain within the boundaries of a proper hermeneutics, to the end of determining Weber's approach to "irrationality" as part of "social action." These results are compared with a more cursory analysis of Pareto's Trattato. Finally, ramifications for contemporary social theory are briefly introduced, with special attention to the changing relationship between "rational action" and "meaning" for "social actors" in contemporary life.

Facility layout and location, land use planning and other problems that concern the optimal assignment of activities to locations are complex combinatorial optimization problems. Such problems can be represented by the Quadratic Set Packing (QSP) model, which offers distinct computational advantages over the more general Quadratic Assignment Problem formulation. Further, a lagrangian relaxation of the QSP is a simpler relaxed assignment problem that can be solved efficiently with an exact branch and bound algorithm embedded in a program called MAFLAD (Smith and Macleod, 1988). MAFLAD is based on the assumption that the study region is tesselated by a cartesian grid, and that clusters of grid cells representing alternate locations for each activity are input by the user. The first section of this thesis addresses the topological clustering problem, that is the problem of generating clusters of grid cells for input to MAFLAD. This is a bi-criterion problem in which both attribute data and spatial location impact the optimal grouping. Initially, this bi-criterion problem is decomposed and formulated as two non-linear zero-one integer programming problems that are solved successively to identify the optimal grouping. Due to the complexity of this approach, the topological clustering problem is reformulated as a QSP problem which is optimally solved with the branch and bound algorithm embedded in MAFLAD. The efficiency of this solution method is enhanced via data reduction techniques that exploit the geometry of the topological clustering problem to reduce the depth and breadth of the branch and bound tree. The second section of this thesis presents a parallel version of MAFLAD designed for an implemented on a Sequent Symmetry S-81 multiprocessor. This is an asynchronous algorithm in which multiple processors execute the branch and bound procedure on private data while sharing the best bound. This algorithm avoids problems associated with data dependency, minimizes overhead, and exploits the tightly coupled/shared memory architecture of the Sequent multiprocessor. A heuristic for ordering the sequence of assignments in the branch and bound procedure and a data partitioning algorithm to generate input for the parallel version of MAFLAD are presented, along with the results and analysis of experiments.

Drawing on the principles of Social Identity Theory and the Contact Hypothesis, this study investigated the relationship between levels of integration and racial prejudice in former Model C, desegregated schools. The sample consisted of 1119 black African, 'Coloured', Afrikaans speaking white and English-speaking white learners from desegregated high schools in Cape Town. ·A pilot study was conducted with 29 learners to establish the face validity and any practical problems, such as ambiguity, that might become evident. The independent variables, Socio-Economic Status, Level of Integration, Racial Identification, Contact at School, Contact Outside School and Contact In-And-Outside School were assessed for their effects on the attitudes of the learners. The dependent measure was the extent of racial prejudice displayed in social distance, subtle racism and ethnic attitudes toward each other. The independent and dependent measures were compiled into a questionnaire which was then administered in situ to the learners by the researcher over a period of six months. Only the responses from black African, 'Coloured', Afrikaans-speaking white and English-speaking white learners were required for the study and the data from other groups was discarded. Multiple regression analysis was used as a statistical technique to analyse the data. Statistically significant results were found for all the dependent measures. There were differences between the intergroup attitudes of black African, 'Coloured', Afrikaans speaking white and English-speaking white learners. The variation in intergroup attitudes could be significantly explained by combinations of Socio-Economic Status (Class), Level of Integration, Racial Identification, Contact at School, Contact Outside School and Contact In-And-Outside School premises. Statistically significant results for Level of Integration were not found for all the dependent measures, indicating that the extent to which schools had been desegregated was not as strong a predictor of intergroup attitudes as was expected. Overall, intergroup contact emerged as a strong predictor of social distance and ethnic attitudes for all groups. Partial support was found for Social Identity Theory as well as for the Contact Hypothesis. The findings indicated a relationship between pleasant intergroup contact, increased social contact outside school and more positive attitudes. Methodological problems associated with the research limit the generalizability of the results.