Planning in partially observable domains is a notoriously difficult problem. However, in many real-world scenarios, planning can be simplified by decomposing the task into a hierarchy of smaller planning problems which, can then be solved independently of one another. Several approaches, mainly dealing with fully observable domains, have been proposed to optimize a plan that decomposes according to a hierarchy specified a priori. Some researchers have also proposed to discover hierarchies in fully observable domains. In this thesis, we investigate the problem of automatically discovering planning hierarchies in partially observable domains. The main advantage of discovering hierarchies is that, for a plan of a fixed size, hierarchical plans can be more expressive than non-hierarchical ones. Our solution frames the discovery and optimization of a hierarchical policy as a non-convex optimization problem. By encoding the hierarchical structure as variables of the optimization problem, we can automatically discover a hierarchy. Successfully solving the optimization problem therefore yields an optimal hierarchy and an optimal policy. We describe several techniques to solve the optimization problem. Namely, we provide results using general non-linear solvers, mixed-integer linear and non-linear solvers or a form of bounded hierarchical policy iteration. Our method is flexible enough to allow any parts of the hierarchy to be specified based on prior knowledge while letting the optimization discover the unknown parts. It can also discover hierarchical policies, including recursive policies, that are more compact (potentially infinitely fewer parameters).

Large Eddy Simulation (LES) of bluff body stabilized premixed and partially premixed combustion close to the flammability limit is carried out in this thesis. The LES algorithm has no ad-hoc adjustable model parameters and is able to respond automatically to variations in the inflow conditions. Algorithm validation is achieved by comparison with reactive and non-reactive experimental data. In the reactive flow, two scalar closure models, Eddy Break-Up (EBULES) and Linear Eddy Mixing (LEMLES), are used and compared. Over important regions, the flame lies in the Broken Reaction Zone regime. Here, the EBU model assumptions fail. The flame thickness predicted by LEMLES is smaller and the flame is faster to respond to turbulent fluctuations, resulting in a more significant wrinkling of the flame surface. As a result, LEMLES captures better the subtle effects of the flame-turbulence interaction. Three premixed (equivalence ratio = 0.6, 0.65, and 0.75) cases are simulated. For the leaner case, the flame temperature is lower, the heat release is reduced and vorticity is stronger. As a result, the flame in this case is found to be unstable. In the rich case, the flame temperature is higher, and the spreading rate of the wake is increased due to the higher amount of heat release Partially premixed combustion is simulated for cases where the transverse profile of the inflow equivalence ratio is variable. The simulations show that for mixtures leaner in the core the vortical pattern tends towards anti-symmetry and the heat release decreases, resulting also in instability of the flame. For mixtures richer in the core, the flame displays sinusoidal flapping resulting in larger wake spreading. More accurate predictions of flame stability will require the use of detailed chemistry, raising the computational cost of the simulation. To address this issue, a novel algorithm for training Artificial Neural Networks (ANN) for prediction of the chemical source terms has been implemented and tested. Compared to earlier methods, the main advantages of the ANN method are in CPU time and disk space and memory reduction.

The Central and Eastern United States (CEUS) is a region that is characterized by low frequency-high consequence seismic events such as the New Madrid sequence of 18111812. The infrequent nature of earthquakes in the region has led to a perception that the seismic risk in the area is low, and the current building stock reflects this perception. The majority of steel-framed buildings in the CEUS were designed without regard to seismic loads. Such frames possess limited seismic resistance, and may pose an unacceptable risk if a large earthquake were to occur in the region. A key ingredient of building performance and seismic risk assessment is the fragility, a term that describes the probability of failure to meet a performance objective as a function of demand on the system. The effects of uncertainties on building seismic performance can be displayed by a seismic fragility relationship. This fragility can be used in a conditional scenario-based seismic risk assessment or can be integrated with seismic hazard to obtain an estimate of annual or lifetime risk. The seismic fragility analyses in this study focus on steel frames that are typical of building construction in regions of infrequent seismicity; such frames have received little attention to date in building seismic risk assessment. Current steel building stock in Shelby Co., TN has been represented by five code-compliant model frames with different lateral force-resisting systems, i.e., braced-frames, partially-restrained moment frames and a rigid moment frame. The performance of model frames under certain hazard levels was assessed using fragility curves. Different rehabilitation methods were discussed and applied. Results indicate that PR frames behave better than expected and rehabilitated frames perform quite well even under severe earthquakes.

The need to analyze and closely study the gene related mechanisms motivated the research on the modeling and control of gene regulatory networks (GRN). Dierent approaches exist to model GRNs / they are mostly simulated as mathematical models that represent relationships between genes. Though it turns into a more challenging problem, we argue that partial observability would be a more natural and realistic method for handling the control of GRNs. Partial observability is a fundamental aspect of the problem / it is mostly ignored and substituted by the assumption that states of GRN are known precisely, prescribed as full observability. On the other hand, current works addressing partially observability focus on formulating algorithms for the nite horizon GRN control problem. So, in this work we explore the feasibility of realizing the problem in a partially observable setting, mainly with Partially Observable Markov Decision Processes (POMDP). We proposed a POMDP formulation for the innite horizon version of the problem. Knowing the fact that POMDP problems suer from the curse of dimensionality, we also proposed a POMDP solution method that automatically decomposes the problem by isolating dierent unrelated parts of the problem, and then solves the reduced subproblems. We also proposed a method to enrich gene expression data sets given as input to POMDP control task, because in available data sets there are thousands of genes but only tens or rarely hundreds of samples. The method is based on the idea of generating more than one model using the available data sets, and then sampling data from each of the models and nally ltering the generated samples with the help of metrics that measure compatibility, diversity and coverage of the newly generated samples.

Despite very extensive studies related to molecular processes underlying memory formation, still little known about the potential differences in the brain biochemistry between &ldquo / good&rdquo / and &ldquo / poor&rdquo / learners belonging to a random population of young animals. In the present study, an attempt was taken to correlate the individual variation in short- and long-term spatial memory in three different lines of young, healthy rats: inbred Wistar (W), outcrossed Wistar/Spraque Dawley (W/S) and pigmented Long-Evans rats, with hippocampal levels of selected enzymes known as &ldquo / memory molecules&rdquo / including neuronal (n), endothelial (e) and inducible (i) NOS, CaMKII&alpha / , PKA and ChAT. Additionally, in order to indirectly estimate the activity of CaMKII&alpha / and PKA, hippocampal levels of their phosphorylated forms (pCaMKII&alpha / and pPKA) were assessed. Rats were classified as &ldquo / good&rdquo / and &ldquo / poor&rdquo / learners on the basis of their performance in a partially baited 12-arm radial maze. The hippocampal protein levels were measured using Western Blot technique. In addition to individual variation in animals&rsquo / learning capacity, strain-depended differences have also been observed. Deficient performance recorded in inbred W rats compared to outcrossed W/S rats, and &ldquo / poor&rdquo / learners from both rat groups had predominantly related to the higher frequency of reference memory errors. The results of biochemical assays showed strain-depended differences in the NOS expression. The overall NOS levels were significantly higher in outcrossed W/S rats compared to inbred W rats. In both rat lines, the rate of learning positively correlated with hippocampal levels of nNOS and negatively correlated with iNOS levels. Hippocampal eNOS levels correlated negatively with animals&rsquo / performance but only in the W rats. These results suggested that all 3 NOS isoforms are implemented in the learning process playing, however, different roles in neural signaling. Experiments carried out on Long-Evans rats did not reveal a significant difference in the basal hippocampal levels of the CaMKII&alpha / , however, the level of the pCaMKII&alpha / , was significantly higher in &ldquo / good&rdquo / learners. Also, hippocampal levels of both PKA and pPKA, as well as that of ChAT were significantly higher in &ldquo / good&rdquo / as compared to &ldquo / poor&rdquo / learners. Taken together, the latter findings indicate that low hippocampal expression of PKA and ChAT as well as low CaMKII&alpha / or PKA activation may cause learning deficits in random population of young rats, and thus, these enzymes can be considered target molecules when looking for cognitive enhancers to treat memory deficits in young subjects.

Three different topics of turbulence research that cover modeling, theory and model computation categories are selected and studied in depth. In the first topic, "velocity gradient dynamics in turbulence" (modeling), the Lagrangian linear diffusion model that accounts for the viscous-effect is proposed to make the existing restricted-Euler velocity gradient dynamics model quantitatively useful. Results show good agreement with DNS data. In the second topic, "pressure-strain correlation in homogeneous anisotropic turbulence subject to rapid strain-dominated distortion" (theory), extensive rapid distortion calculation is performed for various anisotropic initial turbulence conditions in strain-dominated mean flows. The behavior of the rapid pressure-strain correlation is investigated and constraining criteria for the rapid pressure-strain correlation models are developed. In the last topic, "unsteady computation of turbulent flow past a square cylinder using partially-averaged Navier-Stokes method" (model computation), the basic philosophy of the PANS method is reviewed and a practical problem of flow past a square cylinder is computed for various levels of physical resolution. It is revealed that the PANS method can capture many important unsteady flow features at an affordable computational effort.

In an experiment to understand colon carcinogenesis, all animals were exposed to a carcinogen while half the animals were also exposed to radiation. Spatially, we measured the existence of aberrant crypt foci (ACF), namely morphologically changed colonic crypts that are known to be precursors of colon cancer development. The biological question of interest is whether the locations of these ACFs are spatially correlated: if so, this indicates that damage to the colon due to carcinogens and radiation is localized. Statistically, the data take the form of binary outcomes (corresponding to the existence of an ACF) on a regular grid. We develop score??type methods based upon the Matern and conditionally autoregression (CAR) correlation models to test for the spatial correlation in such data, while allowing for nonstationarity. Because of a technical peculiarity of the score??type test, we also develop robust versions of the method. The methods are compared to a generalization of Moran??s test for continuous outcomes, and are shown via simulation to have the potential for increased power. When applied to our data, the methods indicate the existence of spatial correlation, and hence indicate localization of damage. Assuming that there are correlations in the locations of the ACF, the questions are how great are these correlations, and whether the correlation structures di?er when an animal is exposed to radiation. To understand the extent of the correlation, we cast the problem as a spatial binary regression, where binary responses arise from an underlying Gaussian latent process. We model these marginal probabilities of ACF semiparametrically, using ?xed-knot penalized regression splines and single-index models. We ?t the models using pairwise pseudolikelihood methods. Assuming that the underlying latent process is strongly mixing, known to be the case for many Gaussian processes, we prove asymptotic normality of the methods. The penalized regression splines have penalty parameters that must converge to zero asymptotically: we derive rates for these parameters that do and do not lead to an asymptotic bias, and we derive the optimal rate of convergence for them. Finally, we apply the methods to the data from our experiment.

This dissertation has focused on i) optimizing charge transfer rates at indium-tinoxide (ITO) electrodes, and ii) characterization of the supramolecular structure and properties of ultra thin surface modifier films on modified electrodes for various device applications. Commercial ITO surfaces were modified using conducting polymer thin film architectures with and without various chemical activation procedures. Ferrocene derivatives were used as redox probes, which showed dramatic changes in electron transfer rate as the SA-PANI/PAA layers were added to the ITO surface. Highest rates of electron transfer were observed for DMFc, whose oxidation potential coincides with the potential region where these SA-PANI/PAA films reach their optimal electroactivity. Apparent heterogeneous electron transfer rate constants, kS, measured voltammetrically, were ca.10 x higher for SA-PANI/PAA films on ITO, versus clean ITO substrates. These films also showed linear potentiometric responses with retention of the ITO transparency with the capability to create smoothest films using an aqueous deposition protocol, which proved important in other applications. ITO electrodes were also modified via chemisorption of carboxy functionalized EDOTCA and electropolymerization of PEDOTCA/PEDOT copolymers, when properly optimized for thickness and structure, enhance voltammetrically determined electron transfer rates (kS) to solution probe molecules, such as dimethylferrocene (DMFc). Values of kS ≥ 0.4 cm•sec⁻¹, were determined, approaching rates seen on clean gold surfaces. ITO activation combined with formation of these co-polymer films has the effect of enhancing the electroactive fraction of electrode surface, versus a non-activated, unmodified ITO electrode, which acts as a “blocked” electrode. The electroactivity and spectroelectrochemistry of these films helped to resolve the electron transfer rate mechanism and enabled the construction of models in combination with AFM, XPS, UPS and RAIRS studies. The surface topography, structure, composition, work function and contact angle, also revealed other desirable properties for molecular electronic devices. The carboxylic functionality of the EDOTCA molecule adds more desirable properties compared to normal PEDOT films, such as favoring the deposition of smooth films, increasing the optical contrast, participating in hydrogen-bonding, chemisorption to oxide surface, self-doping and providing a linker for incorporation of different functional groups, new molecules, or nanoparticles. Periodic sub-micron electrode arrays can be created using micro-contact printing and electropolymerization. The sinusoidal modulation of the refractive index of such confined conducting polymer nanostructures or nanoparticle stripes allows efficient visible light diffraction. The modulation of the diffraction efficiency at PANI and PEDOT gratings in the presence of an analytical stimulus such as pH or potential demonstrate the sensing capability at these surfaces. The template stripped gold surfaces that are being developed in our lab demonstrate several advantages over commercially available evaporated gold films especially for nanoscale surface modification.

No description available.

Stochastic control problems that arise in sequential decision making applications typically assume that information used for decision-making is obtained according to a predetermined sampling schedule. In many real applications however, there is a high sampling cost associated with collecting such data. It is therefore of equal importance to determine when information should be collected as it is to decide how this information should be utilized for optimal decision-making. This type of joint optimization has been a long-standing problem in the operations research literature, and very few results regarding the structure of the optimal sampling and control policy have been published. In this thesis, the joint optimization of sampling and control is studied in the context of maintenance optimization. New theoretical results characterizing the structure of the optimal policy are established, which have practical interpretation and give new insight into the value of condition-based maintenance programs in life-cycle asset management. Applications in other areas such as healthcare decision-making and statistical process control are discussed. Statistical parameter estimation results are also developed with illustrative real-world numerical examples.