FPGAs (Field Programmable Gate Arrays) dinamicamente reconfiguráveis (DR-FPGAs) são soluções promissoras para muitos sistemas embarcados devido a potencial redução de área de silício. Metodologias de projeto e ferramentas CAD relacionadas são ainda muito limitadas para auxiliarem os projetistas a encontrarem soluções dinamicamente reconfiguráveis para diferentes aplicações. Este trabalho propõe uma metodologia de projeto que combina modelos de alto nível em SystemC, técnicas de projeto de baixo nível e a metodologia de projeto modular da XILINX. SystemC foi utilizada para representar o comportamento de alto nível não temporizado e não-RTL, bem como o baixo nível RTL-DCS (Chaveamento Dinâmico de Circuitos). Um estudo de caso da Banda Base de um Controlador Bluetooth foi desenvolvido. Duas partições temporais foram testadas em nove diferentes DR-FPGAs. A exploração espacial mostrou que 33% das soluções investigadas atenderam a restrição da especificação de 625µs de tempo do quadro do pacote Bluetooth, deixando diferentes parcelas de recursos livres que podem ser explorados para acomodar outros módulos IP de sistemas mais complexos no mesmo dispositivo. / Dynamically Reconfigurable Field Programmable Gate Arrays (DR-FPGAs) are promising solutions for many embedded systems due to the potential silicon area reduction. Design methodologies and related CAD tools are still very limited to assist designers to encounter dynamically reconfigurable solutions for different applications. This work proposes a design methodology that combines high level SystemC models and design techniques with the low level modular design proposed by Xilinx. SystemC has been used to represent the high level untimed non-RTL behavior as well as the low level RTL-DCS (Dynamic Circuit Switching). A Bluetooth Baseband unit case study was performed. Two temporal-functional partitions were evaluated on nine different target DR-FPGAs. The design space exploration showed that 33% of the investigated solutions complied with the 625µs Bluetooth packet time frame specification leaving different amounts if free resources that may be explored to accommodate other IP modules of more complex systems on the same device.

Rationale. Adolescence may be a period of susceptibility to the effects of drugs of abuse. This vulnerability may result from a convergence of psychological processes that contribute to drug addiction including impulsive action and incentive motivation during adolescence. Objectives. I examined age differences in incentive motivation, as measured by responding for a conditioned reinforcer (CR) previously paired with natural or drug rewards, and age and sex differences in impulsive action, as measured by responding on a differential reinforcement of low rates of responding (DRL) schedule or premature responding on the 2-Choice Serial Reaction Time Test (2-CSRTT), in Sprague-Dawley rats. The effects of drugs that affect these behaviours in adulthood and that act on neurochemical systems still developing during adolescence were also examined. Methods. In a first set of experiments (Chapter 3), I compared male adolescents and adults on responding for a CR previously paired with sucrose and the effect of amphetamine on this behaviour. In a second set of experiments (Chapter 4), I examined age differences in responding for a CR previously paired with passive or self-administered intravenous (IV) nicotine infusions. Subsequently, I investigated age and sex differences in responding on a DRL schedule in response to amphetamine (Chapter 5) and 2-CSRTT performance in response to amphetamine, nicotine and Ro 63-1908 (a glutamate N-Methyl-D-aspartic acid [NMDA] receptor subunit antagonist; Chapter 6). Results. Compared to adults, adolescents responded more for a CR previously paired with sucrose or passive, but not self-administered, IV nicotine infusions. Amphetamine only enhanced responding for a CR previously paired with sucrose. Adolescents responded more than adults on a DRL schedule, while adolescents made the most premature responses in the 2-CSRTT. No consistent sex differences were observed during the acquisition of either behaviour. Amphetamine increased premature responding most in adolescent males and adult females in the 2-CSRTT but not in responding on the DRL schedule. No consistent age or sex differences were observed for Ro 63-1908 or nicotine. Conclusions. Adolescents show enhanced impulsivity and incentive motivation than adults depending on the behavioural measure. Dopamine may contribute to age and sex differences in these behaviours.

<p>The capacity of deep reinforcement learning policy networks has been found to affect the performance of trained agents. It has been observed that policy networks with more parameters have better training performance and generalization ability than smaller networks. In this work, we find cases where this does not hold true. We observe unimodal variance in the zero-shot test return of varying width policies, which accompanies a drop in both train and test return. Empirically, we demonstrate mostly monotonically increasing performance or mostly optimal performance as the width of deep policy networks increase, except near the variance mode. Finally, we find a scenario where larger networks have increasing performance up to a point, then decreasing performance. We hypothesize that these observations align with the theory of double descent in supervised learning, although with specific differences.</p>

<div>The exponential growth of Internet of Things (IoT) and cyber-physical systems is resulting in complex environments comprising of various devices interacting with each other and with users. In addition, the rapid advances in Artificial Intelligence are making those devices able to autonomously modify their behaviors through the use of techniques such as reinforcement learning (RL). There is thus the need for an intelligent monitoring system on the network edge with a global view of the environment to autonomously predict optimal device actions. However, it is clear however that ensuring safety and security in such environments is critical. To this effect, we develop a constrained RL framework for IoT environments that determines optimal devices actions with respect to user-defined goals or required functionalities using deep Q learning. We use anomaly based intrusion detection on the network edge to dynamically generate security and safety policies to constrain the RL agent in the framework. We analyze the balance required between ‘safety/security’ and ‘functionality’ in IoT environments by manipulating the exploration of safe and unsafe benefit state spaces in the RL framework. We instantiate the framework for testing on application layer control in smart home environments, and network layer control including network functionalities like rate control and routing, for SDN based environments.</div>

Made available in DSpace on 2016-04-29T13:18:08Z (GMT). No. of bitstreams: 1 Livia Ferreira Godinho Aureliano.pdf: 559598 bytes, checksum: 8bc000da4999e46895b037747380a514 (MD5) Previous issue date: 2008-04-30 / Coordenação de Aperfeiçoamento de Pessoal de Nível Superior / This study investigated the effects of duration exposure to activities without programmed reinforcers on performance under a temporal schedule of reinforcement. Two other questions guided this study: (a) Are there any changes in the patterns of activities that occur without programmed reinforcers dependent on the duration of exposure to these activities? (b) What would be the effect of the duration of this exposure over responding under a DRL schedule of reinforcement when subjects are placed in a standard operant chamber without access to other activities? Subjects were 4 male food deprived rats and 2 chambers were used: a standard operant chamber (1 compartment with lever and food recipient) and a 7 compartment chamber (activity chamber) in which subjects could engage on different activities: as bar pressing, eating, drinking, wood-chewing, and running. Bar presses were reinforced with food according to a DRL schedule that varied from 5 to 10 to 21s, but 2 subjects were exposed to this schedule on the standard chamber and the others on the activity chamber. Experimental sessions lasted 2 hours and bar presses, reinforce deliveries, and compartments visited were recorded. Results indicated performances in teach environment when DRL 10s or higher was in effect. On DRL 10s response and reinforcer rates and percentage of reinforced responses were higher, and time between reinforcements was lower for subjects on the standard chamber. In DRL 21s, however, these measures were reversed, suggesting that the availability of other activities could facilitate the acquisition and maintenance of performance on higher values DRL. There was indication of an increase on the percentage of reinforced bar pressing responses as a function of time of exposure to the experimental contingency in each session for all subjects. There were no significant differences on the percentages of reinforced responses of subjects on the standard or activity chamber box as a function of successive experimental sessions. However, percentages of reinforced responses were lower for subjects in the standard chamber at the beginning of the first sessions on each DRL value. Sequences of compartment visits, that seemed to be patterns were identified and the beginning of this patterns coincided with the increased effectiveness of the bar pressing responses. Results are discussed taking into account the differences between the criteria used in studies reported on the literature. The possible roles of the activities on the performance submitted on DRL schedules are also discussed / O presente estudo pretendeu investigar os efeitos do tempo de exposição dos sujeitos às atividades sem reforço programado sobre a efetividade do desempenho submetido a um esquema temporal de reforçamento. Duas outras perguntas nortearam o trabalho: (a) ocorreria alguma mudança nos padrões das atividades sem reforço programado em função do tempo de exposição a estas atividades? (b) qual o efeito do tempo de exposição aos diferentes valores do esquema DRL sobre o desempenho dos sujeitos na caixa padrão? Foram sujeitos 4 ratos machos, privados de alimento, que trabalharam em duas caixas experimentais, uma com 7 compartimentos nos quais havia a possibilidade de engajamento em atividades (pressionar a barra, comer, beber, roer madeira, correr na roda de atividades) e uma caixa operante padrão. As respostas de pressão à barra dos 4 sujeitos foram submetidas a um esquema DRL5s, 10 e 21s : 2 sujeitos foram expostos aos esquemas na caixa padrão e os outros dois na outra caixa. Todas as sessões experimentais tiveram a duração de 2 horas. Foram registradas as pressões à barra, os reforços obtidos e o tempo de permanência nos compartimentos. Os resultados indicaram que as maiores diferenças entre os desempenhos nos dois ambientes ocorreram a partir do DRL 10s, quando as taxas de respostas, de reforços e as porcentagens de respostas reforçadas foram maiores, e o tempo entre reforços foi menor para os sujeitos na caixa padrão. Em DRL 21s, no entanto, a situação se inverteu, sugerindo que a disponibilidade de outras atividades poderia facilitar a aquisição e adaptação do desempenho em valores mais altos do DRL. A análise dos efeitos do tempo de exposição à contingência em cada sessão sobre a efetividade da resposta de pressão à barra indicou que as porcentagens de respostas reforçadas aumentaram em função do tempo da sessão para os 4 sujeitos e não houve diferenças expressivas entre as porcentagens de respostas reforçadas ao longo das sessões experimentais. No entanto, para os sujeitos na caixa padrão, os percentuais alcançados no início das primeiras sessões em cada fase foram mais baixos do que para os sujeitos em ambiente aberto. Em relação ao padrão das atividades, uma seqüência de visita a compartimentos foi identificada e o início deste padrão coincidiu com o aumento da efetividade das respostas de pressão à barra. A discussão dos resultados leva em consideração as diferenças entre os critérios utilizados nos estudos relatados, além dos possíveis papéis das atividades sobre o desempenho submetido ao esquema de DRL

The aim of this thesis is to create a tool which quantifies qualitative measures into an indicator system. The system is created on behalf of a company which is associated with investments in new technologies on the energy market. The indicator system is to take into consideration important factors in the first part of an investment cycle, in other words, the screening phase. Qualitative measures will, in each indicator, become quantified and will together create a weighted grade on a potential investment that can help the investor decide whether or not to move forward with said investment. The aim of the thesis is also to evaluate the indicator system on current possible investment options in order to analyse and discuss how the final product will work in the investor company ́s actual context. The method consists of two main parts, a gap analysis which is conducted within the investor company and its owners and a compilation of which indicators that are of greatest importance in a screening phase according to earlier research. The main result of the study is the full indicator system which consist of four indicators: technology readiness of the potential investment, contextual analysis of the potential investment, diversity within the company and the financial burn rate of the company. When applied to current potential investments the result showed that there was negligible to moderate correlation between the indicators, which was important for the system to be validated. It was also concluded that a potential investment should exceed 60 percent of the possible maximum grade in order to pass through the screening phase. The results also showed that there were four apparent gaps, namely consensus between the involved actors, utilization rate of the organizations competencies, to enter new markets and the right competency to do so. The future potential investments of the investor company should therefore aim to fill in these gaps in order to strengthen the role of the company. If doing so while using the indicator system, the risks of choosing investment options that does not fit into the investor company ́s context, will be minimized.

<p>Autonomous systems are often deployed in dynamic environments and are challenged with unexpected changes (novelties) in the environments where they receive novel data that was not seen during training. Given the uncertainty, they should be able to operate without (or with limited) human intervention and they are expected to (1) Adapt to such changes while still being effective and efficient in performing their multiple tasks. The system should be able to provide continuous availability of its critical functionalities. (2) Make informed decisions independently from any central authority. (3) Be Cognitive: learns the new context, its possible actions, and be rich in knowledge discovery through mining and pattern recognition. (4) Be Reflexive: reacts to novel unknown data as well as to security threats without terminating on-going critical missions. These characteristics combine to create the workflow of autonomous decision-making process in multi-agent environments (i.e.,) any action taken by the system must go through these characteristic models to autonomously make an ideal decision based on the situation. </p> <p><br></p> <p>In this dissertation, we propose novel learning-based models to enhance the decision-making process in autonomous multi-agent systems where agents are able to detect novelties (i.e., unexpected changes in the environment), and adapt to it in a timely manner. For this purpose, we explore two complex and highly dynamic domains </p> <p>(1) Transportation Networks (e.g., Ridesharing application): where we develop AdaPool: a novel distributed diurnal-adaptive decision-making framework for multi-agent autonomous vehicles using model-free deep reinforcement learning and change point detection. (2) Multi-agent games (e.g., Monopoly): for which we propose a hybrid approach that combines deep reinforcement learning (for frequent but complex decisions) with a fixed-policy approach (for infrequent but straightforward decisions) to facilitate decision-making and it is also adaptive to novelties. (3) Further, we present a domain agnostic approach for decision making without prior knowledge in dynamic environments using Bootstrapped DQN. Finally, to enhance security of autonomous multi-agent systems, (4) we develop a machine learning based resilience testing of address randomization moving target defense. Additionally, to further  improve the decision-making process, we present (5) a novel framework for multi-agent deep covering option discovery that is designed to accelerate exploration (which is the first step of decision-making for autonomous agents), by identifying potential collaborative agents and encouraging visiting the under-represented states in their joint observation space. </p>

Unmanned aerial vehicles (UAVs) are emerging as enablers for supporting many applications and services, such as precision agriculture, search and rescue, temporary network deployment, coverage extension, and security. UAVs are being considered for integration into emerging wireless networks as aerial users, aerial relays (ARs), or aerial base stations (ABSs). This dissertation proposes employing UAVs to contribute to physical layer techniques that enhance the security performance of advanced wireless networks and services in terms of availability, resilience, and confidentiality. The focus is on securing terrestrial cellular communications against eavesdropping with a cellular-connected UAV that is dispatched as an AR or ABS. The research develops mathematical tools and applies machine learning algorithms to jointly optimize UAV trajectory and advanced communication parameters for improving the secrecy rate of wireless links, covering various communication scenarios: static and mobile users, single and multiple users, and single and multiple eavesdroppers with and without knowledge of the location of attackers and their channel state information. The analysis is based on established air-to-ground and air-to-air channel models for single and multiple antenna systems while taking into consideration the limited on-board energy resources of cellular-connected UAVs. Simulation results show fast algorithm convergence and significant improvements in terms of channel secrecy capacity that can be achieved when UAVs assist terrestrial cellular networks as proposed here over state-of-the-art solutions. In addition, numerical results demonstrate that the proposed methods scale well with the number of users to be served and with different eavesdropping distributions. The presented solutions are wireless protocol agnostic, can complement traditional security principles, and can be extended to address other communication security and performance needs.

<p>    </p> <p>In order to advance next-generation communication systems, it is critical to enhance the state-of-the-art communication architectures, such as device-to-device (D2D), multiple- input multiple-output (MIMO), and intelligent reflecting surface (IRS), in terms of achieving high data rate, low latency, and high energy efficiency. In the first part of this dissertation, we address joint learning and optimization methodologies on cutting-edge network archi- tectures. First, we consider D2D networks equipped with MIMO systems. In particular, we address the problem of minimizing the network overhead in D2D networks, defined as the sum of time and energy required for processing tasks at devices, through the design for MIMO beamforming and communication/computation resource allocation. Second, we address IRS-assisted communication systems. Specifically, we study an adaptive IRS control scheme considering realistic IRS reflection behavior and channel environments, and propose a novel adaptive codebook-based limited feedback protocol and learning-based solutions for codebook updates. </p> <p><br></p> <p>Furthermore, in order for revolutionary innovations to emerge for future generations of communications, it is crucial to explore and address fundamental, long-standing open problems for communications, such as the design of practical codes for a variety of important channel models. In the later part of this dissertation, we study the design of practical codes for feedback-enabled communication channels, i.e., feedback codes. The existing feedback codes, which have been developed over the past six decades, have been demonstrated to be vulnerable to high forward/feedback noises, due to the non-triviality of the design of feedback codes. We propose a novel recurrent neural network (RNN) autoencoder-based architecture to mitigate the susceptibility to high channel noises by incorporating domain knowledge into the design of the deep learning architecture. Using this architecture, we suggest a new class of non-linear feedback codes that increase robustness to forward/feedback noise in additive White Gaussian noise (AWGN) channels with feedback. </p>

認知功能的提升是當今神經科學領域中的研究重點之一，但其神經機制尚有待釐清。本研究利用一種用於改善阿茲海默症臨床的非競爭型N-甲基-D-天門冬胺酸受體拮抗劑memantine，檢測其對於大白鼠在不同時距相關操作式制約行為及空間工作記憶行為之影響效果。實驗一為針對時間屬性的操作式制約行為實驗，運用大白鼠的區辯性增強低頻反應作業（DRL 10秒行為）與固定時距作業（FI 30秒行為）之行為作業，並操弄連續訓練與間歇訓練的兩種不同模式，測試memantine對前述四組受試的操作式制約行為在表現、消除與自發恢復等三階段之劑量反應。實驗二利用配對性延遲T迷津作業區分出不等基準線(表現好與表現差)之受試，再加以藥理實驗，測試memantine對於前述兩組受試之劑量反應。實驗一結果顯示，受試在兩種不同訓練模式下經十五次習得訓練後，在兩種操作式壓桿行為的壓桿反應相關指標中都有明顯的差異，這證實不同的行為訓練模式會導致學習後的表現有差異之別。memantine藥理實驗結果顯示，此藥對於上述四組受試的操作式行為之三階段的影響效果，會因為不同訓練模式與不同作業而異。實驗二結果顯示，memantine提高空間工作記憶的正確率在表現不好的組別有很顯著的藥效，這證實memantine對於空間式工作記憶行為的影響，也會因學習基準線的不同水平而異。在行為實驗後所進行的蛋白質表現量檢測中，memantine（5 mg/kg）只對五個測試腦區中的背側紋狀體中ERK1磷酸化程度有明顯上升的影響，而其對ERK2及CREB的磷酸化在所有腦組織中皆沒有顯著的影響。綜合以上結果，memantine影響時間與空間屬性的相關行為之藥理效果，會依行為的不同習得歷程(或行為背景經驗)及基準線表現程度而異，而此項行為藥理效果，可能與紋狀體中ERK1的磷酸化有關。 / The neural basis of cognitive enhancement is one of the intriguing topics in neuroscience research; however, the underlying neural mechanisms remain to be elucidated. This study examined the effects of memantine, a non-competitive N-methyl-D-aspartate (NMDA) receptor antagonist which is used to treat Alzheimer’s disease in clinic, on operant behaviors and spatial working memory. In Experiment 1, using the differential reinforcement for low-rate-response 10 sec (DRL 10s) and the fixed-interval 30 sec (FI 30s) operant tasks, and with the manipulation of two different training regimens (continuous vs. intermittent) in the acquisition phase, the effects of memantine were evaluated in three stages of behavioral tests including the performance (right after the end of 15-day acquisition), the extinction, and the spontaneous recovery (after the extinction). In Experiment 2, memantine were tested in the subjects with different level of baseline performance (good vs. bad) on the distinctive patterns of operant responding in four different groups which received DRL 10s and FI 30s with different training regimens; indicating that behavioral task and training background are critical to the operant performance of temporal operant behaviors. Such behavioral outcomes led the dissociable effects of memantine appeared in between the four groups as tested in all three different stages. The results of Experiment 2 showed a profound improvement of the correct responses rate on spatial working memory in the low-baseline group as compared to the higher-baseline group. With a pretreatment of memantine (5 mg/kg), brain tissues in five selected areas were collected for western blot assays of ERK 1, ERK 2, and CREB. The results only revealed a significant increase of ERK 1 phosphorylation in the dorsal striatum. Together, the effects of memantine to improve cognition-associated processes in the temporal operant behaviors and the baseline of performance, and the present observation of cognition-enhancing effects of memantine may be resulted by the ERK 1 phosphorylation in the dorsal striatum.