This study investigated whether contingent-anxious conditioned stimulus termination was more important than temporally yoked termination in producing conservation or enhancement of learned fear. Thirty psychology students, twenty-six females and four males, were administered item thirty-nine from the Fear Survey Schedule and an avoidance test. After in vivo treatment exposure to a harmless snake, post-test measures identical to pretests revealed that contingent-anxious subjects retained significantly more fear (p <.05) on both indexes than temporally yoked subjects. No enhancement was found and only on the subjective measure did contingent-anxious subjects show fear conservation when contrasted with no-treatment controls (p >.05). Implications for "implosive" therapies were discussed.

The motivational properties of Longstreth's (1970) definitions of incentive and frustrative cues were tested using 32 rats in a two phase straight alleyway experiment. During pretraining, incentive cue Ss were presented a visual cue prior to reinforcement; frustrative cue Ss experienced the visual cue simultaneously with reinforcement. Ss encountered the same cue in mid-alley during 40 CRF training trials. Significant inhibition developed as frustrative cue Ss passed through the cue and postcue segments. Significant incentive effects occurred midway through training only in the postcue segment. Differential resistance to extinction was not found. The results did not support all of Longstreth's assumed functions. The motivational effects were interpreted using Spence's and Amsel's instrumental learning paradigms.

The purpose of the present study was to compare their (avoidance contingency and positive-reinforcement contingency) relative effectiveness in producing a desired behavior.

A choice paradigm was used to evaluate allocation of interlocking behavior of two groups of two participants between responses having operant consequences only and responses having cultural consequences. In a discrete trial BABABAB design, each participant could select one of three options, which delivered either 3 or 5 points. In B (cultural consequence) conditions, two of the options had additional effects: the 3-point option also added 3 points to the other participant's earnings, and one of the 5-point options also subtracted 5 points from the other participant's earnings. The third option was unchanged in both conditions and delivered 5 points to the participant who selected it. Results indicated that participants in both groups initially frequently produced response combinations that earned 8 points for one or the other individual (and 0 or 3 points for the other), but allocation of responding increasingly changed to combinations that produced 6 points for each individual. This shift in performances away from maximum individual reinforcement towards maximum group reinforcement indicates cultural contingencies did not act in concert with operant contingencies, suggesting they are different mechanisms of selection.

This study examined effects of two response acquisition procedures on topography of responding using the revealed operant technique and compared results to previous experiments on this topic. Subjects emitted 100 repetitions each of 4 response patterns on a continuous schedule of reinforcement. A 30-min extinction condition followed acquisition. One group of subjects learned the first response through a series of shaping steps designed to reduce acquisition variability. Another group of subjects was instructed in the correct response topography and was told there was no penalty for attempting other sequences. The first group of subjects produced high variability during extinction despite reduced variability in acquisition. The second group of subjects responded with moderate to high variability during extinction and little variability during acquisition. Most extinction responses for the first group were variations of the last pattern reinforced. Most extinction responses for the second group were repetitions of the last pattern reinforced.

This experiment examined the effects of extinction on rate of responding and several topographical and temporal measures in adult humans. Three college students were trained to type the sequence 1•5•3 on a numeric keypad on a computer. The subjects were exposed to different fixed-ratio schedules of reinforcement (FR1, FR 5, and FR10 respectively) and extinction. Subjects displayed typical schedule performances during the maintenance phase of the experiment. During extinction the performances were disrupted, they showed a "break and run" pattern and a general decrease in responding. Also, new topographical and temporal patterns emerged. These data are consistent with those reported for non-human species and special human populations.

Master of Science / Department of Civil Engineering / Asadollah Esmaeily / Bridge and building construction in areas that sustain frequent seismic activity require the use of heavy lateral steel reinforcement within concrete columns to handle the lateral loads. Multi-spiral lateral reinforcement has been recently introduced to the construction field to offer an alternative to the traditional hoop and tie reinforcement. This report evaluates the experimental data observed in multiple experimental studies done on different concrete specimens. These specimens include multiple rectilinear reinforcement and several multi-spiral configurations in both rectangular and oblong columns. Due to multi-spiral reinforcement being a relatively new design, traditional computer programs have yet to include design analysis for this type of reinforcement in computer programs. Dr. Asad Esmaeily developed the program KSU RC 2.0 that can implement multiple analytical models to evaluate different multi-spiral configurations, as well as traditional hoop and tie confinement, that may be compared with experimental data. This report illustrates the comparative data from several different reinforced concrete column models. The data clearly indicates that multi-spiral reinforced columns exhibit higher compressive strength in the axial direction as well as higher ductility capabilities when compared to traditional rectilinear reinforcement of similar lateral steel reinforcement ratios. The use of multi-spiral reinforcement is also shown to lower costs for both the work time needed to install the structures as well as lowering the required steel ratio; all while maintaining the structural integrity of the columns.

The growing popularity of online virtual communities such as Second Life and ActiveWorlds demands the presence of intelligent agents to assist users in their daily online activities (e.g., exploring, shopping, and socializing). As these virtual environments become more crowded, multiple agents are needed to support the increasing number of users. Multi-agent environments, however, can suffer from the problem of resource competition among agents. It is therefore necessary that agents within multi-agent environments include a coordination mechanism to prevent unrealistic behaviors. Moreover, it is essential that these agents exhibit some form of intelligence, or the ability to learn, to support realism as well as to eliminate the need for developers to write separate scripts for each task the agents are required to perform. This research presents a coordinated reinforcement learning framework which can be used to develop task-oriented intelligent agents in multi-agent virtual environments. The framework contains a combination of a "next available agent" coordination model and a reinforcement learning model consisting of existing temporal difference reinforcement learning algorithms. Furthermore, the framework supports evaluations of reinforcement learning algorithms to determine which methods are best suited for task-oriented intelligent agents in dynamic, multi-agent virtual environments. To assess the effectiveness of the temporal difference reinforcement algorithms used in this study (Q-learning and Sarsa), experiments were conducted that measured an agent's ability to learn three tasks commonly performed by workers in a café environment. These tasks were basic sandwich making (BSM), complex sandwich making (CSM), and dynamic sandwich making (DSM). The BSM task consisted of four steps. The CSM and DSM tasks contained an additional fifth step. The agent learned the BSM and CSM tasks from scratch while the DSM task was learned after the agent became skillful in BSM. The measurements used to evaluate the efficiency of the Q-learning and Sarsa algorithms were the percentage of successful and optimally successful episodes performed by the agent and the average number of time steps taken by the agent to complete a successful episode. The experiments were run using both a fixed (FEP) and variable (VEP) ε-greedy probability rate. Results showed that the Sarsa reinforcement learning algorithm, on average, outperformed the Q-learning algorithm in almost all experiments except when measuring the percentage of successfully completed episodes using FEP for CSM and DSM, in which Sarsa performed almost equally as well as Q-learning. Overall, experiments utilizing VEP resulted in higher percentages of successes and optimal successes, and showed convergence to the optimal policy when measuring the average number of time steps per successful episode.

Traditionally robots have been preprogrammed to execute specific tasks. Thisapproach works well in industrial settings where robots have to execute highlyaccurate movements, such as when welding. However, preprogramming a robot isalso expensive, error prone and time consuming due to the fact that every featuresof the task has to be considered. In some cases, where a robot has to executecomplex tasks such as playing the ball-in-a-cup game, preprogramming it mighteven be impossible due to unknown features of the task. With all this in mind,this thesis examines the possibility of combining a modern learning framework,known as Learning from Demonstrations (LfD), to first teach a robot how toplay the ball-in-a-cup game by demonstrating the movement for the robot, andthen have the robot to improve this skill by itself with subsequent ReinforcementLearning (RL). The skill the robot has to learn is demonstrated with kinestheticteaching, modelled as a dynamic movement primitive, and subsequently improvedwith the RL algorithm Policy Learning by Weighted Exploration with the Returns.Experiments performed on the industrial robot KUKA LWR4+ showed that robotsare capable of successfully learning a complex skill such as playing the ball-in-a-cupgame. / Traditionellt sett har robotar blivit förprogrammerade för att utföra specifika uppgifter.Detta tillvägagångssätt fungerar bra i industriella miljöer var robotar måsteutföra mycket noggranna rörelser, som att svetsa. Förprogrammering av robotar ärdock dyrt, felbenäget och tidskrävande eftersom varje aspekt av uppgiften måstebeaktas. Dessa nackdelar kan till och med göra det omöjligt att förprogrammeraen robot att utföra komplexa uppgifter som att spela bollen-i-koppen spelet. Medallt detta i åtanke undersöker den här avhandlingen möjligheten att kombinera ettmodernt ramverktyg, kallat inläraning av demonstrationer, för att lära en robothur bollen-i-koppen-spelet ska spelas genom att demonstrera uppgiften för denoch sedan ha roboten att själv förbättra sin inlärda uppgift genom att användaförstärkande inlärning. Uppgiften som roboten måste lära sig är demonstreradmed kinestetisk undervisning, modellerad som dynamiska rörelseprimitiver, ochsenare förbättrad med den förstärkande inlärningsalgoritmen Policy Learning byWeighted Exploration with the Returns. Experiment utförda på den industriellaKUKA LWR4+ roboten visade att robotar är kapabla att framgångsrikt lära sigspela bollen-i-koppen spelet

Building automated trading systems has long been one of the most cutting-edge and exciting fields in the financial industry. In this research project, we built a trading system based on machine learning methods. We used the Recurrent Reinforcement Learning (RRL) algorithm as our fundamental algorithm, and by introducing Genetic Algorithms (GA) in the optimization procedure, we tackled the problems of picking good initial values of parameters and dynamically updating the learning speed in the original RRL algorithm. We call this optimization algorithm the Evolutionary Recurrent Reinforcement Learning algorithm (ERRL), or the GA-RRL algorithm. ERRL allows us to find many local optimal solutions easier and faster than the original RRL algorithm. Finally, we implemented the GA-RRL system on EUR/USD at a 5-minute level, and the backtest performance showed that our GA-RRL system has potentially promising profitability. In future research we plan to introduce some risk control mechanism, implement the system on different markets and assets, and perform backtest at higher frequency level.