<p dir="ltr">Learning, planning, and representing knowledge at multiple levels of temporal abstractions provide an agent with the ability to predict consequences of different courses of actions, which is essential for improving the performance of sequential decision making. However, discovering effective temporal abstractions, which the agent can use as skills, and adopting the constructed temporal abstractions for efficient policy learning can be challenging. Despite significant advancements in single-agent settings, temporal abstractions in multi-agent systems remains underexplored. This thesis addresses this research gap by introducing novel algorithms for discovering and employing temporal abstractions in both cooperative and competitive multi-agent environments. We first develop an unsupervised spectral-analysis-based discovery algorithm, aiming at finding temporal abstractions that can enhance the joint exploration of agents in complex, unknown environments for goal-achieving tasks. Subsequently, we propose a variational method that is applicable for a broader range of collaborative multi-agent tasks. This method unifies dynamic grouping and automatic multi-agent temporal abstraction discovery, and can be seamlessly integrated into the commonly-used multi-agent reinforcement learning algorithms. Further, for competitive multi-agent zero-sum games, we develop an algorithm based on Counterfactual Regret Minimization, which enables agents to form and utilize strategic abstractions akin to routine moves in chess during strategy learning, supported by solid theoretical and empirical analyses. Collectively, these contributions not only advance the understanding of multi-agent temporal abstractions but also present practical algorithms for intricate multi-agent challenges, including control, planning, and decision-making in complex scenarios.</p>

Master of Science / Department of Agricultural Economics / Ted C. Schroeder / Very rapid growth in the ethanol industry has led to a void of information regarding price dynamics and utilization of distiller's grains. Understanding market conditions is essential for livestock producers to make informed decisions in the procurement of feedstuffs, especially as grain price levels have recently increased substantially. In this study, distiller's grain price discovery dynamics are evaluated to develop an understanding of spatial price relationships. The knowledge of price relationships reveals that users of distiller's grain should shop around when procuring the feedstuff. Additionally, because animal performance may be altered with increased inclusion of distiller's grains, regional competitive advantages could shift fed cattle production to geographic regions characterized by high of ethanol production plants. Therefore, the cost of current grain processing methods are evaluated to enhance the awareness of regional competitiveness and long term sustainability. The combination of these two objectives allows producers to better realize the implications of the ethanol industry on their ability to maintain their operations into the near future.

Doctor of Philosophy / Department of Statistics / Suzanne Dubnicka / DNA barcodes are short strands of nucleotide bases taken from the cytochrome c oxidase subunit 1 (COI) of the mitochondrial DNA (mtDNA). A single barcode may have the form C C G G C A T A G T A G G C A C T G . . . and typically ranges in length from 255 to around 700 nucleotide bases. Unlike nuclear DNA (nDNA), mtDNA remains largely unchanged as it is passed from mother to offspring. It has been proposed that these barcodes may be used as a method of differentiating between biological species (Hebert, Ratnasingham, and deWaard 2003). While this proposal is sharply debated among some taxonomists (Will and Rubinoff 2004), it has gained momentum and attention from biologists. One issue at the heart of the controversy is the use of genetic distance measures as a tool for species differentiation. Current methods of species classification utilize these distance measures that are heavily dependent on both evolutionary model assumptions as well as a clearly defined "gap" between intra- and interspecies variation (Meyer and Paulay 2005). We point out the limitations of such distance measures and propose a character-based method of species classification which utilizes an application of Bayes' rule to overcome these deficiencies. The proposed method is shown to provide accurate species-level classification. The proposed methods also provide answers to important questions not addressable with current methods.

The prospect of replacing existing fixed networks with cheap, flexible and evenmobile low-power wireless network has been a strong research driver in recent years.However, many challenges still exist: reliability is hampered by unstable and burstycommunication links; the wireless medium is getting congested by an increasingnumber of wireless devices; and life-times are limited due to difficulties in developingefficient duty-cycling mechanisms. These challenges inhibit the industry to fullyembrace and exploit the capabilities and business opportunities that low-powerwireless devices offer. In this thesis, we propose, design, implement, and evaluateprotocols and systems to increase flexibility and improve efficiency of low-powerwireless communications. First, we present MobiSense, a system architecture for energy-efficient communicationsin micro-mobility sensing scenarios. MobiSense is a hybrid architecturecombining a fixed infrastructure network and mobile sensor nodes. Simulations andexperimental results show that the system provides high throughput and reliabilitywith low-latency handoffs. Secondly, we investigate if and how multi-channel communication can mitigate theimpact of link dynamics on low-power wireless protocols. Our study is motivated bya curiosity to reconcile two opposing views: that link dynamics is best compensatedby either (i) adaptive routing, or (ii) multi-channel communication. We perform acomprehensive measurement campaign and evaluate performance both in the singlelink and over a multi-hop network. We study packet reception ratios, maximumburst losses, temporal correlation of losses and loss correlations across channels.The evaluation shows that multi-channel communication significantly reduces linkburstiness and packet losses. In multi-hop networks, multi-channel communicationsand adaptive routing achieves similar end-to-end reliability in dense topologies,while multi-channel communication outperforms adaptive routing in sparse networkswhere re-routing options are limited. Third, we address the problem of distributed information exchange in proximitybasednetworks. First, we consider randomized information exchange and assess thepotential of multi-channel epidemic discovery. We propose an epidemic neightbordiscoverymechanism that reduces discovery times considerably compared to singlechannelprotocols in large and dense networks. Then, the idea is extended todeterministic information exchange. We propose, design and evaluate an epidemicinformation dissemination mechanism with strong performance both in theory andpractice. Finally, we apply some of the concepts from epidemic discovery to the designof an asynchronous, sender-initiated multi-channel medium access protocol. Theprotocol combines a novel mechanism for rapid schedule learning that avoids perpacketchannel negotiations with the use of burst data transfer to provide efficientsupport of ’multiple contending unicast and parallel data flows. / De senaste åren har forskning inom trådlös kommunikation drivits av önskemåletom att kunna ersätta nuvarande trådbundna kommunikationslänkar med trådlösa lågenergialternativ.Dock kvarstår många utmaningar, såsom instabila och sporadiskalänkar, överbelastning på grund av en ökning i antal trådlösa enheter, hur maneffektivt kan växla duty-cycling mekanismen för att förlänga nätverkens livstid,med flera. Dessa utmaningar begränsar industrin från att ta till sig och utnyttjade fördelar som trådlösa lågenergialternativ kan medföra. I den här avhandlingenföreslår, designar, implementerar och utvärderar vi protokoll och system som kanförbättra de nuvarande trådlösa lågenergialternativen. Först presenterar vi MobiSense, en systemarkitektur för energibesparande kommunikationi mikro-mobila sensorscenarier. MobiSense är en hybridarkitektur somkombinerar ett fast infrastrukturnätverk med rörliga sensornoder. Simulerings- ochexperimentella resultat visar att systemet uppnår en högre överföringskapacitet ochtillförlitlighet samtidigt som överlämnandet mellan basstationer har låg latens. I den andra delen behandlar vi hur effekterna från länkdynamiken hos protokollför lågenergikommunikation kan minskas, och försöker förena idéerna hos två motståendesynsätt: (i) flerkanalskommunikation och (ii) adaptiv routing. Vi analyserarenkanals- och flerkanalskommunikation över en-stegslänkar i termer av andelenmottagna paket kontra andelen förlorade, den maximala sporadiska förlusten avpaket, tidskorrelation för förluster och förlustkorrelation mellan olika kanaler. Resultatenindikerar att flerkanalskommunikation med kanalhoppning kraftigt minskardet sporadiska uppträdandet hos länkarna och korrelationen mellan paketförluster.För flerstegsnätverk uppvisar flerkanalskommunikation och adaptiv routingliknande tillförlitlighet i täta topologier, medan flerkanalskommunikation har bättreprestanda än adaptiv routing i glesa nätverk med sporadiska länkar. I den tredje delen studeras distribuerat informationsutbyte i närhetsbaseradenätverk. Först betraktas det slumpmässiga fallet och vi fastställer potentialen hosflerkanalig indirekt utforskning av nätverket. Vi analyserar ett trestegs protokoll,som möjliggör en snabbare utforskning av nätverket. Sedan föreslår vi en ny algoritmför att upptäcka grannarna i ett flerkanalsnätverk, som kraftigt minskarutforskningstiden i jämförelse med ett enkanalsprotokoll. Vi utökar även problemettill det deterministiska fallet och föreslår en mekanism för informationsspridningsom påskyndar utforskningstiderna för deterministiska protokoll. Utvidgningen hartvå huvudförbättringar som leder till kraftigt ökad prestanda samtidigt som degaranterar att utforskningsprocessen är deterministisk. Till sist applicerar vi koncepten rörande indirekt utforskning för att designa,implementera och evaluera ett asynkront sändare-initierat flerkanals MAC protokollför trådlös lågenergikommunikation. Protokollet kombinerar en ny mekanism försnabbt lärande av tidsschemat, vilket undviker kanalförhandling för varje paket,med sporadisk dataöverföring. Detta möjliggör ett effektivt tillhandahållande avflera konkurrerande och parallella dataflöden. / <p>QC 20151204</p>

Broadcasting is used in on-demand routing protocols to discover routes in Mobile Ad-hoc Networks (MANETs). On-demand routing protocols, such as Ad-hoc On-demand Distance Vector (AODV) commonly employ pure flooding based broadcasting to discover new routes. In pure flooding, a route request (RREQ) packet is broadcast by the source node and each receiving node rebroadcasts it. This continues until the RREQ packet arrives at the destination node. Pure flooding generates excessive redundant routing traffic that may lead to the broadcast storm problem (BSP) and deteriorate the performance of MANETs significantly. A number of probabilistic broadcasting schemes have been proposed in the literature to address BSP. However, these schemes do not consider thermal noise and interference which exist in real life MANETs, and therefore, do not perform well in real life MANETs. Real life MANETs are noisy and the communication is not error free. This research argues that a broadcast scheme that considers the effects of thermal noise, co-channel interference, and node density in the neighbourhood simultaneously can reduce the broadcast storm problem and enhance the MANET performance. To achieve this, three investigations have been carried out: First, the effect of carrier sensing ranges on on-demand routing protocol such as AODV and their impact on interference; second, effects of thermal noise on on-demand routing protocols and third, evaluation of pure flooding and probabilistic broadcasting schemes under noisy and noiseless conditions. The findings of these investigations are exploited to propose a Channel Adaptive Probabilistic Broadcast (CAPB) scheme to disseminate RREQ packets efficiently. The proposed CAPB scheme determines the probability of rebroadcasting RREQ packets on the fly according to the current Signal to Interference plus Noise Ratio (SINR) and node density in the neighbourhood. The proposed scheme and two related state of the art (SoA) schemes from the literature are implemented in the standard AODV to replace the pure flooding based broadcast scheme. Ns-2 simulation results show that the proposed CAPB scheme outperforms the other schemes in terms of routing overhead, average end-to-end delay, throughput and energy consumption.

No description available.

No description available.

Inter-molecular interaction is at the heart of biological function. Proteins can interact with ligands, peptides, small molecules, and other proteins to serve their structural or functional purpose. With advances in combinatorial chemistry and the development of high throughput binding assays, the available inter-molecular interaction data is increasing exponentially. As the space of testable compounds increases, the complexity and cost of finding a suitable inhibitor for a protein interaction increases. Computational drug discovery plays an important role in minimizing the time and cost needed to study the space of testable compounds. This work focuses on the usage of various computational methods in identifying protein interaction inhibitors and demonstrates the ability of computational drug discovery to contribute to the ever growing field of molecular interaction. A program to predict the location of binding surfaces on proteins, STP (Mehio et al., Bioinformatics, 2010, in press), has been created based on calculating the propensity of triplet-patterns of surface protein atoms that occur in binding sites. The use of STP in predicting ligand binding sites, allosteric binding sites, enzyme classification numbers, and binding details in multi-unit complexes is demonstrated. STP has been integrated into the in-house high throughput drug discovery pipeline, allowing the identification of inhibitors for proteins whose binding sites are unknown. Another computational paradigm is introduced, creating a virtual library of -turn peptidomimetics, designed to mimic the interaction of the Baff-Receptor (Baff-R) with the B-Lymphocyte Stimulator (Blys). LIDAEUS (Taylor, et al., Br J Pharmacol, 2008; 153, p. S55-S67) is used to identify chemical groups with favorable binding to Blys. Natural and non-natural sidechains are then used to create a library of synthesizable cyclic hexapeptides that would mimic the Blys:Baff-R interaction. Finally, this work demonstrates the usage and synergy of various in-house computational resources in drug discovery. The ProPep database is a repository used to study trends, motifs, residue pairing frequencies, and aminoacid enrichment propensities in protein-peptide interaction. The LHRLL protein-peptide interaction motif is identified and used with UFSRAT (S. Shave, PhD Thesis, University of Edinburgh, 2010) to conduct ligand-based virtual screening and generate a list of possible antagonists from the EDULISS (K. Hsin, PhD Thesis, University of Edinburgh, 2010) compound repository. A high throughput version of AutoDock (Morris, et al., J Comput Chem, 1998; 19, p. 1639-62) was adapted and used for precision virtual screening of these molecules, resulting in a list of compounds that are likely to inhibit the binding of this motif to several Nuclear Receptors.

published_or_final_version / Education / Master / Master of Arts in Education

Methamphetamine abuse is a serious public health concern affecting millions of people worldwide, and there are currently no viable pharmacotherapies to treat methamphetamine abuse. Methamphetamine increases extracellular dopamine (DA) concentrations through an interaction with the DA transporter (DAT) and the vesicular monoamine transporter-2 (VMAT2), leading to reward and abuse. While numerous studies have focused on DAT as a target for the discovery of pharmacotherapies to treat psychostimulant abuse, these efforts have been met with limited success. Taking into account the fact that methamphetamine interacts with VMAT2 to increase DA extracellular concentrations; the focus of the current work was to develop novel compounds that interact with VMAT2 to inhibit the effects of methamphetamine. Lobeline, the principal alkaloid found in Lobelia inflata, inhibits VMAT2 binding and function. Inhibition of VMAT2 was hypothesized to be responsible for the observed lobeline-induced inhibition of methamphetamine-evoked DA release in striatal slices and decrease in methamphetamine self-administration in rats. Lobeline has recently completed Phase Ib clinical trials demonstrating safety in methamphetamine abusers. Lobeline is also a potent inhibitor of nicotinic acetylcholine receptors (nAChRs), limiting selectivity for VMAT2. Chemical defunctionalization of the lobeline molecule afforded analogs, meso-transdiene (MTD) and lobelane, which exhibited decreased affinity for nAChRs. MTD, an unsaturated analog of lobeline, exhibited similar affinity for VMAT2 and increased affinity for DAT compared to lobeline. Conformationally-restricted MTD analogs exhibited decreased affinity for DAT compared to MTD, while retaining affinity at VMAT2. One analog, UKMH-106 exhibited high affinity and selectivity for VMAT2 and inhibited METH-evoked DA release from striatal slices. Unfortunately, the MTD analogs exhibited poor water solubility which limited further investigation of these promising analogs. Importantly lobelane, a saturated analog of lobeline, exhibited increased affinity and selectivity for VMAT2 compared to lobeline. To improve water solubility, a N-1,2-dihydroxypropyl (diol) moiety was incorporated into the lobelane molecule. GZ-793A, an N-1,2-diol analog, potently and competitively inhibited VMAT2 function, exhibiting over 50-fold selectivity for VMAT2 over DAT, serotonin transporters and nAChRs. GZ-793A released DA from preloaded synaptic vesicles, fitting a two-site model with the high-affinity site inhibited by tetrabenazine and reserpine (classical VMAT2 inhibitors), suggesting a VMAT2-mediated mechanism of release. Further, low concentrations of GZ-793A that selectively interact with high-affinity sites on VMAT2 to evoke DA release, inhibit methamphetamine-evoked DA release from synaptic vesicles. Results showed that increasing concentrations of GZ-793A produced a rightward shift in the METH concentration response; however, the Schild regression revealed a slope different from unity, consistent with surmountable allosteric inhibition. In addition, GZ-793A specifically inhibited methamphetamine-evoked DA release in striatal slices and methamphetamine self-administration in rats. To examine the possibility that GZ-793A produced DA depletion, the effect of a behaviorally active dose of GZ-793A on DA content in striatal tissue and striatal vesicles was determined. GZ-793A administration did not alter DA content in striatal tissue or vesicles and pretreatment with GZ-793A prior to methamphetamine administration did not exacerbate the DA depleting effects of methamphetamine. Importantly, GZ-793A was shown to protect against methamphetamine-induced striatal DA depletions. Thus, GZ-793A represents an exciting new lead in the development of pharmacotherapies to treat methamphetamine abuse.