One schedule of reinforcement that is used to decrease the rate of a target behavior is differential reinforcement of low rates (DRL). During this schedule, reinforcement is delivered for a target response if it occurs after a certain amount of time has passed since the last instance of this target response. The current study used a table-top game called PORTL and college student participants to investigate how collateral patterns develop and are disrupted during DRL schedules. After the participant developed a collateral pattern of behaviors with the objects, the researcher removed one of the objects that was part of the pattern and waited for a new pattern of behaviors to develop. Once the participant developed a new collateral pattern, the researcher removed a second object. This continued until there was only one object present. Results showed that the rate of reinforcement decreased following the removal of each object, then slowly increased as a new pattern developed.

Major depressive disorder (MDD) is a widespread psychiatric disorder that affects millions of people worldwide and is hypothesized to occur due to impairments in several neurotransmitter systems, including the monoaminergic and glutamatergic neurotransmitter systems. Antidepressant medications targeting multiple monoamine neurotransmitters have been shown to be effective for the treatment of depression. Racemic amisulpride is an atypical antipsychotic that has been used at low doses to treat dysthymia, a mild form of depression, and functions as an antagonist at DA2/3, 5-HT2B, and 5-HT7 receptors. Recent preclinical studies have suggested that the S(+) isomer may be more critical for amisulpride’s antidepressant-like effects; however, this interpretation has not been fully characterized in comparison to the R(-) isomer. The glutamatergic system also has been shown to play a critical role in alleviating depression. Several studies have demonstrated that the noncompetitive N-methyl-D-aspartate (NMDA) receptor antagonist ketamine produces rapid and sustained antidepressant-like effects in clinical trials; however, few studies have examined the degree to which ketamine’s isomers contribute to antidepressant-like effects. Fully characterizing these differences in a preclinical model of depression may offer important insight into the role of these neurotransmitter systems on depression. The present study used a 72-sec differential-reinforcement-of-low-rate (DRL) task to assess the antidepressant-like effects of amisulpride, ketamine, and their isomers in mice. The DRL 72-sec task has shown to be a reliable and sensitive screen for drugs that possess antidepressant-like activity as reflected by an increase in the number of reinforcers, a decrease in the number of responses, and a right-ward shift in the interresponse time distributions (IRTs; i.e. the elapsed time between two successive responses). For comparison, the effects of the tricyclic antidepressant imipramine and the N-methyl-D-aspartate antagonist MK-801 as positive and negative controls, respectively, were determined. Consistent with previous findings, we hypothesized that amisulpride and S(-)-amisulpride, but not R(+)-amisulpride, would produce antidepressant-like effects, and all formulations of ketamine would produce antidepressant effects. Racemic amisulpride and S(-)-amisulpride, but not R(+)-amisulpride, produced an antidepressant-like effect, evidenced by a significant increase in the number of reinforcers and a significant decrease in the number of responses. Racemic ketamine and R(-)-ketamine significantly increased the number of reinforcers and decreased the number of responses, while S(+)-ketamine significantly increased the number of reinforcers, but did not decrease the number of responses (at the doses tested). Overall, these results indicate that the racemic formulations of amisulpride and ketamine, S(-)-amisulpride, and both ketamine isomers demonstrate antidepressant-like effects as assessed in the DRL task and may be useful in a clinical context. If either of the ketamine isomers can be shown to produce fewer psychotomimetic effects in humans, then the isomers may offer a significant clinical advantage over the parent compound ketamine. Regarding amisulpride, the present results demonstrate that the S(-) isomer, but not the R(+) isomer, possess antidepressant-like activity similar to racemic amisulpride.

Extensive research has been conducted demonstrating the utility of differential reinforcement as an effective intervention for self-injurious behavior. However, the majority of this literature requires teaching an alternative response to access reinforcement. Further evaluation of treating self-injurious behavior in individuals that already possess the repertories to contact reinforcement appropriately. Prior to initiating the study, functional assessments were completed for both participant that demonstrated high-rate bursts of mands served as a reliable precursor to self-injurious behavior. In the present study, we evaluated a waiting period and differential reinforcement of low rate behavior on reducing mands while keeping self-injurious behavior at or near zero levels. Results indicated that shorter waiting periods and DRL values were effective at reducing mands and maintaining near zero levels of self-injurious behavior.

Deep Neural Networks (DNNs) are becoming increasingly important in various aspects of human life, particularly in safety-critical areas such as autonomous driving and aerospace systems. However, soft errors including bit-flips can significantly impact the performance of these systems, leading to serious consequences. To ensure the reliability of DNNs, it is essential to guarantee their performances. Many solutions have been proposed to enhance the trustworthiness of DNNs, including traditional methods like error correcting code (ECC) that can mitigate and detect soft errors but come at a high cost of redundancy. This thesis proposes a new method of correcting soft errors in DNNs using Deep Reinforcement Learning (DRL) and Transfer Learning (TL). DRL agent can learn the knowledge of identifying the layer-wise critical weights of a DNN. To accelerate the training time, TL is used to apply this knowledge to train other layers. The primary objective of this method is to ensure acceptable performance of a DNN by mitigating the impact of errors on it while maintaining low redundancy. As a case study, we tested the proposed method approach on a multilayer perception (MLP) and ResNet-18, and our results show that our method can save around 25% redundancy compared to the baseline method ECC while achieving the same level of performance. With the same redundancy, our approach can boost system performance by up to twice that of conventional methods. By implementing TL, the training time of MLP is shortened to around 81.11%, and that of ResNet-18 is shortened to around 57.75%. / DNNs blir allt viktigare i olika aspekter av mänskligt liv, särskilt inom säkerhetskritiska områden som autonom körning och flygsystem. Mjuka fel inklusive bit-flip kan dock påverka prestandan hos dessa system avsevärt, vilket leder till allvarliga konsekvenser. För att säkerställa tillförlitligheten hos DNNs är det viktigt att garantera deras prestanda. Många lösningar har föreslagits för att förbättra tillförlitligheten för DNNs, inklusive traditionella metoder som ECC som kan mildra och upptäcka mjuka fel men som har en hög kostnad för redundans. Denna avhandling föreslår en ny metod för att korrigera mjuka fel i DNN med DRL och TL. DRL-agenten kan lära sig kunskapen om att identifiera de lagermässiga kritiska vikterna för en DNN. För att påskynda träningstiden används TL för att tillämpa denna kunskap för att träna andra lager. Det primära syftet med denna metod är att säkerställa acceptabel prestanda för en DNN genom att mildra inverkan av fel på den samtidigt som låg redundans bibehålls. Som en fallstudie testade vi den föreslagna metodmetoden på en MLP och ResNet-18, och våra resultat visar att vår metod kan spara cirka 25% redundans jämfört med baslinjemetoden ECC samtidigt som vi uppnår samma prestationsnivå. Med samma redundans kan vårt tillvägagångssätt öka systemets prestanda med upp till dubbelt så högt som för konventionella metoder. Genom att implementera TL förkortas träningstiden för MLP till cirka 81.11%, och den för ResNet-18 förkortas till cirka 57.75%.

The noncompetitive NMDA receptor antagonist ketamine produces rapid and sustained antidepressant effects in patients suffering from major depressive disorder. However, abuse liability is a concern. To further evaluate the relationship between antidepressant-like and abuse-related effects of NMDA receptor antagonists, this study evaluated the effects of ketamine, MK-801, and phencyclidine in male Sprague-Dawley rats responding under two procedures that have been used to assess antidepressant-like effects [differential-reinforcement-of-low-rate (DRL) 72 s schedule of food reinforcement] and abuse-related drug effects [intracranial self-stimulation (ICSS)]. Under DRL 72 s, ketamine produced an antidepressant-like effect by increasing reinforcers, decreasing responses, and producing a rightward shift in the peak location of the interresponse time (IRT) distributions. Phencyclidine produced a modest antidepressant-like effect by increasing reinforcers and decreasing responses, but did not shift the IRT distributions. In contrast, MK-801 produced a psychostimulant-like effect by decreasing reinforcers, increasing responses, and producing a leftward shift in the peak location of the IRT distributions. The antidepressant-like effects of ketamine on the DRL 72 s procedure do not appear to be mediated by inhibiting the reuptake of serotonin via serotonin transporters or antagonism of 5-HT2 receptors. Additionally, the dissociable effects of ketamine and MK-801 in the DRL 72 s procedure may be mediated by 5-HT2 receptors. Following acute administration, ketamine produced only dose- and time-dependent depression of ICSS and failed to produce an abuse-related facilitation of ICSS at any dose or pretreatment time. Repeated dosing with ketamine produced dose-dependent tolerance to the rate-decreasing effects of ketamine but failed to unmask expression of ICSS facilitation. Termination of ketamine treatment failed to produce withdrawal-associated decreases in ICSS. In contrast, MK-801 and phencyclidine effects produced dose- and time-dependent facilitation of ICSS by MK-801. Taken together, our findings provide further evidence that expression of these antidepressant-like and abuse-related effects of ketamine, phencyclidine, and MK-801 may be related to NMDA receptor affinity.

Transplants of fetal neural tissue survive and develop in lesion cavities produced in adult rats. The present experiment tested the effect of grafting fetal hippocampal or brainstem tissue on the ability of rats with hippocampal lesions to perform on a differential reinforcement of low response rate (DRL) operant schedule. The DRL interval was 20 s. Eighty-six percent of the hippocampal grafts and 69% of the brainstem grafts developed to maturity. Inspection of sections stained using a silver technique for axis cylinders or taken from rats in which the mature transplant had been injected with Fast blue, indicated that these grafts formed connections with the host brain. Consistent with previous reports, rats with hippocampal lesions were impaired in performance of the DRL task. Rats given fetal grafts of hippocampal tissue into the hippocampal lesion site on the day of lesion production were significantly better in performance of the DRL requirement than were lesion-only rats or rats receiving grafts of fetal brainstem tissue. The results of this study confirm that grafts of fetal brain tissue can both develop in a lesion site in an adult brain and ameliorate lesion-induced behavioral deficits.

Navigation is the fundamental capability of mobile robots which allows them to move fromone point to another without any human interference. The autonomous operation of theserobots is depended on reliable, robust, and intelligent navigation system. With the recenttechnological progress, autonomous mobile robots are being deployed and used in differentareas and scenarios. Conventional navigation approaches depend on predefined accurateobstacle maps and costly high-end precise laser sensors. These maps are difficult and expensiveto acquire and degrade due changes in the environment. This limits the overall use of mobilerobots in dynamic settings. In this research, we investigate the end-to-end learning-basedapproach using vision and ranging sensors while using Deep Reinforcement Learning formobile robot navigation for indoor environments. Different state-of-the-art DRL algorithms were trained and compared in 3D-simulation in termsof sample efficiency and cumulative reward. Next, extensive experiments were carried outusing 10-dimensional sparse distance data from vision and ranging sensor. The trained modelswere evaluated in different environments of varying complexity to analyze the strength andgeneralizability of the learnt policies. Our results showed that ranging sensor approach was able to learn a robust navigation policywhich was able to generalize in unseen virtual environments without any additional trainingwith a high success rate. Whereas vision-based approach performed poorly due to insufficientinformation and hardware constraints. Moreover, all the experiment were carried out only insimulation. However, they should be directly transferable to an actual robot since abstractobservation space was used.

Over the years, the Internet of Things has grown significantly and integrated into many fields such as medicine, agriculture, smart homes, etc. This growth has resulted in a significant increase in the amount of data generated. IoT devices are constrained by various factors, including transient data properties, limited memory, energy consumption, and computation power. Edge caching has been used as a solution to alleviate the problem caused by the increase while also improving service quality. Because of IoT constraints, cutting-edge caching policies have proven inefficient due to the file’s limited lifetime. Several caching methods have been proposed over the years to address ephemeral IoT data because data freshness plays a significant role in caching policy for IoT. It is not only essential to develop innovative technologies and solutions; we must also consider the long-term impact on the environment. This paper proposes a collaborative edge caching method to optimise transmission latency, traffic cost, and carbon footprint, thereby improving sustainability issues. A deep reinforcement learning approach is used where each edge learns its best caching policy. It is compared with a state-of-the-art cache replacement policy LRU, a DRL model proposed in another paper, and a model that doesn’t utilise caching policies. The simulation result proves that our proposed DRL-based IoT data caching policy outperforms other baseline policies.

No / Introduction: The purpose was to determine if an age based, local diagnostic reference level for paediatric skeletal surveys could be established using retrospective data. Methods: All children below two years of age referred for a primary skeletal survey as a result of suspected physical abuse during 2017 or 2018 (n ¼ 45) were retrospectively included from a large Danish university hospital. The skeletal survey protocol included a total of 33 images. Dose Area Product (DAP) and acquisition parameters for all images were recorded from the Picture Archival and Communication System (PACS) and effective dose was estimated. The 75th percentile for DAP was considered as the diagnostic reference level (DRL). Results: The 75th percentile for DAP was 314 mGy*cm2 , 520 mGy*cm2 and 779 mGy*cm2 for children <1 month, 1e11 months and 12 < 24 months of age respectively. However, only the age group 1e11 months had a sufficient number of children (n ¼ 27) to establish a local DRL. Thus, for the other groups the DAP result must be interpreted with caution. Effective dose was 0.19, 0.26 and 0.18 mSv for children <1, 1e11 months and 12 < 24 months of age respectively. Conclusion: For children between 1 and 11 months of age, a local diagnostic reference level of 520 mGy*cm2 was determined. This may be used as an initial benchmark for primary skeletal surveys as a result of suspected physical abuse for comparison and future discussion. Implications for practice: While the data presented reflects the results of a single department, the suggested diagnostic reference level may be used as a benchmark for other departments when auditing skeletal survey radiation dose.

FPGAs (Field Programmable Gate Arrays) dinamicamente reconfiguráveis (DR-FPGAs) são soluções promissoras para muitos sistemas embarcados devido a potencial redução de área de silício. Metodologias de projeto e ferramentas CAD relacionadas são ainda muito limitadas para auxiliarem os projetistas a encontrarem soluções dinamicamente reconfiguráveis para diferentes aplicações. Este trabalho propõe uma metodologia de projeto que combina modelos de alto nível em SystemC, técnicas de projeto de baixo nível e a metodologia de projeto modular da XILINX. SystemC foi utilizada para representar o comportamento de alto nível não temporizado e não-RTL, bem como o baixo nível RTL-DCS (Chaveamento Dinâmico de Circuitos). Um estudo de caso da Banda Base de um Controlador Bluetooth foi desenvolvido. Duas partições temporais foram testadas em nove diferentes DR-FPGAs. A exploração espacial mostrou que 33% das soluções investigadas atenderam a restrição da especificação de 625µs de tempo do quadro do pacote Bluetooth, deixando diferentes parcelas de recursos livres que podem ser explorados para acomodar outros módulos IP de sistemas mais complexos no mesmo dispositivo. / Dynamically Reconfigurable Field Programmable Gate Arrays (DR-FPGAs) are promising solutions for many embedded systems due to the potential silicon area reduction. Design methodologies and related CAD tools are still very limited to assist designers to encounter dynamically reconfigurable solutions for different applications. This work proposes a design methodology that combines high level SystemC models and design techniques with the low level modular design proposed by Xilinx. SystemC has been used to represent the high level untimed non-RTL behavior as well as the low level RTL-DCS (Dynamic Circuit Switching). A Bluetooth Baseband unit case study was performed. Two temporal-functional partitions were evaluated on nine different target DR-FPGAs. The design space exploration showed that 33% of the investigated solutions complied with the 625µs Bluetooth packet time frame specification leaving different amounts if free resources that may be explored to accommodate other IP modules of more complex systems on the same device.