The need to analyze and closely study the gene related mechanisms motivated the research on the modeling and control of gene regulatory networks (GRN). Dierent approaches exist to model GRNs / they are mostly simulated as mathematical models that represent relationships between genes. Though it turns into a more challenging problem, we argue that partial observability would be a more natural and realistic method for handling the control of GRNs. Partial observability is a fundamental aspect of the problem / it is mostly ignored and substituted by the assumption that states of GRN are known precisely, prescribed as full observability. On the other hand, current works addressing partially observability focus on formulating algorithms for the nite horizon GRN control problem. So, in this work we explore the feasibility of realizing the problem in a partially observable setting, mainly with Partially Observable Markov Decision Processes (POMDP). We proposed a POMDP formulation for the innite horizon version of the problem. Knowing the fact that POMDP problems suer from the curse of dimensionality, we also proposed a POMDP solution method that automatically decomposes the problem by isolating dierent unrelated parts of the problem, and then solves the reduced subproblems. We also proposed a method to enrich gene expression data sets given as input to POMDP control task, because in available data sets there are thousands of genes but only tens or rarely hundreds of samples. The method is based on the idea of generating more than one model using the available data sets, and then sampling data from each of the models and nally ltering the generated samples with the help of metrics that measure compatibility, diversity and coverage of the newly generated samples.

Stochastic control problems that arise in sequential decision making applications typically assume that information used for decision-making is obtained according to a predetermined sampling schedule. In many real applications however, there is a high sampling cost associated with collecting such data. It is therefore of equal importance to determine when information should be collected as it is to decide how this information should be utilized for optimal decision-making. This type of joint optimization has been a long-standing problem in the operations research literature, and very few results regarding the structure of the optimal sampling and control policy have been published. In this thesis, the joint optimization of sampling and control is studied in the context of maintenance optimization. New theoretical results characterizing the structure of the optimal policy are established, which have practical interpretation and give new insight into the value of condition-based maintenance programs in life-cycle asset management. Applications in other areas such as healthcare decision-making and statistical process control are discussed. Statistical parameter estimation results are also developed with illustrative real-world numerical examples.

Stochastic control problems that arise in sequential decision making applications typically assume that information used for decision-making is obtained according to a predetermined sampling schedule. In many real applications however, there is a high sampling cost associated with collecting such data. It is therefore of equal importance to determine when information should be collected as it is to decide how this information should be utilized for optimal decision-making. This type of joint optimization has been a long-standing problem in the operations research literature, and very few results regarding the structure of the optimal sampling and control policy have been published. In this thesis, the joint optimization of sampling and control is studied in the context of maintenance optimization. New theoretical results characterizing the structure of the optimal policy are established, which have practical interpretation and give new insight into the value of condition-based maintenance programs in life-cycle asset management. Applications in other areas such as healthcare decision-making and statistical process control are discussed. Statistical parameter estimation results are also developed with illustrative real-world numerical examples.

Orientador: João Frnacisco Galera Monico / Banca: Paulo de Oliveira Camargo / Banca: Leonardo Castro de Oliveira / Nesta dissertação investiga-se o Posicionamento por Ponto Preciso (PPP), nos modos estático e cinemático, utilizando o Filtro de Kalman Estendido. Foram corrigidos os erros devido aos efeitos da refração troposférica, relatividade, movimento de rotação da Terra, variações das marés terrestres, correção do centro de fase da antena do satélite e fase Wind- up. Os efeitos devido à ionosfera foram minimizados através da combinação linear Ion-Free. A perda de ciclo foi detectada através da combinação linear widelane, envolvendo as duas observáveis: fase da onda portadora e pseudodistância. Quando a perda de ciclo é detectada, uma nova ambigüidade é inicializada no lugar da antiga. As implementações foram realizadas no software FILTER_FCT, em desenvolvimento na FCT/UNESP, o qual processava apenas a pseudodistância. Visando mostrar a acurácia que pode ser obtida com o PPP, foram realizados três experimentos: PPP no modo estático, PPP estático simulando o modo cinemático e PPP cinemático. No primeiro e no segundo experimentos foram utilizados dados das estações GPS: PPTE, VICO, RIOD e FORT, pertencentes à Rede Brasileira de Monitoramento Contínuo (RBMC), e da estação FRDN, localizada em Fredericton, no Canadá. No terceiro experimento foram utilizados dados do receptor GPS a bordo do satélite CHAMP. No PPP estático, utilizou-se como estratégia de processamento a modificação da variância das coordenadas e do relógio do receptor na matriz de variância-covariância do Filtro de Kalman Estendido. Verificou-se uma discrepância em relação às coordenadas consideradas verdadeiras na ordem de decímetros, com exceção das estações VICO e RIOD, que foram da ordem de centímetros. / In this dissertation the Precise Point Positioning (PPP), in static and kinematic modes, using Extended Kalman Filter is investigated. The errors due to troposphere refraction, relativity, movement of Earth's rotation, tide loading, satellite antenna phase center offset and phase wind-up were corrected. The effects due to ionosphere were minimized through the Ionospheric-Free linear combination. The widelane combination, involving phase and pseduorange, was used to detect cycle slips. When a cycle slip is detected, a new ambiguity is initialized in the place of the old one. The implementations were accomplished in the FILTER_FCT software. To show the accuracy in the PPP, three experiments were accomplished: PPP in the static mode, PPP in the static mode simulating the kinematic mode and PPP in kinematic mode. In the first and second experiments, data from PPTE, VICO, RIOD and FORT stations belonging to Brazilian Network for Continuous Monitoring (RBMC), and FRDN station located in Fredericton, Canada, were used. In the third experiment data from a GPS receiver on board of the CHAMP satellite were used. In static PPP, the modification of the coordinates variance and the receiver clock was used as processing strategy. The discrepanc ies obtained in the static positioning were on the order of decimeters, except for VICO and RIOD stations where they were on the order of centimeters. When the clock receiver variance was increased from (3.300)ø mø to (10.000)ø mø , the values of the discrepancies also increased, on the order of centimeters. Then, there is influence of the receiver clock variance in the estimated coordinates. The residual part of the tropospheric effects was estimated with the introduction of a scale factor in the processing. / Mestre

Made available in DSpace on 2014-06-11T19:22:25Z (GMT). No. of bitstreams: 0 Previous issue date: 2006-10-31Bitstream added on 2014-06-13T19:48:45Z : No. of bitstreams: 1 faustino_rc_me_prud.pdf: 956028 bytes, checksum: 4518bd21cfc75ed14c3e4e793a14ec55 (MD5) / Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES) / Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP) / Nesta dissertação investiga-se o Posicionamento por Ponto Preciso (PPP), nos modos estático e cinemático, utilizando o Filtro de Kalman Estendido. Foram corrigidos os erros devido aos efeitos da refração troposférica, relatividade, movimento de rotação da Terra, variações das marés terrestres, correção do centro de fase da antena do satélite e fase Wind- up. Os efeitos devido à ionosfera foram minimizados através da combinação linear Ion-Free. A perda de ciclo foi detectada através da combinação linear widelane, envolvendo as duas observáveis: fase da onda portadora e pseudodistância. Quando a perda de ciclo é detectada, uma nova ambigüidade é inicializada no lugar da antiga. As implementações foram realizadas no software FILTER_FCT, em desenvolvimento na FCT/UNESP, o qual processava apenas a pseudodistância. Visando mostrar a acurácia que pode ser obtida com o PPP, foram realizados três experimentos: PPP no modo estático, PPP estático simulando o modo cinemático e PPP cinemático. No primeiro e no segundo experimentos foram utilizados dados das estações GPS: PPTE, VICO, RIOD e FORT, pertencentes à Rede Brasileira de Monitoramento Contínuo (RBMC), e da estação FRDN, localizada em Fredericton, no Canadá. No terceiro experimento foram utilizados dados do receptor GPS a bordo do satélite CHAMP. No PPP estático, utilizou-se como estratégia de processamento a modificação da variância das coordenadas e do relógio do receptor na matriz de variância-covariância do Filtro de Kalman Estendido. Verificou-se uma discrepância em relação às coordenadas consideradas verdadeiras na ordem de decímetros, com exceção das estações VICO e RIOD, que foram da ordem de centímetros. / In this dissertation the Precise Point Positioning (PPP), in static and kinematic modes, using Extended Kalman Filter is investigated. The errors due to troposphere refraction, relativity, movement of Earth's rotation, tide loading, satellite antenna phase center offset and phase wind-up were corrected. The effects due to ionosphere were minimized through the Ionospheric-Free linear combination. The widelane combination, involving phase and pseduorange, was used to detect cycle slips. When a cycle slip is detected, a new ambiguity is initialized in the place of the old one. The implementations were accomplished in the FILTER_FCT software. To show the accuracy in the PPP, three experiments were accomplished: PPP in the static mode, PPP in the static mode simulating the kinematic mode and PPP in kinematic mode. In the first and second experiments, data from PPTE, VICO, RIOD and FORT stations belonging to Brazilian Network for Continuous Monitoring (RBMC), and FRDN station located in Fredericton, Canada, were used. In the third experiment data from a GPS receiver on board of the CHAMP satellite were used. In static PPP, the modification of the coordinates variance and the receiver clock was used as processing strategy. The discrepanc ies obtained in the static positioning were on the order of decimeters, except for VICO and RIOD stations where they were on the order of centimeters. When the clock receiver variance was increased from (3.300)ø mø to (10.000)ø mø , the values of the discrepancies also increased, on the order of centimeters. Then, there is influence of the receiver clock variance in the estimated coordinates. The residual part of the tropospheric effects was estimated with the introduction of a scale factor in the processing.

The first-time measurement of the angular dependence of the beam-helicity asymmetry for $\gamma p \rightarrow pK^{+}K^{-}$ is shown and compared to $\gamma p \rightarrow p\pi^{+}\pi^{-}$. The data obtained were from the CLAS g12 experiment at Jefferson Lab. The experiment utilized a beam of circularly polarized photons with energies between 1.1 and 5.4 GeV incident on an unpolarized liquid hydrogen target, which produced an unprecedented number of strange hadrons in photoproduction. The production mechanism for strange hadrons is not well understood. The beam-helicity asymmetry is a polarization observable that provides information on interfering production mechanisms in the reaction. It is shown that the asymmetry is sensitive to several kinematic variables that are key in modeling the reaction dynamics. Furthermore, the comparison of the beam-helicity asymmetry between the kaon and pion channels serves as a platform for the investigation of flavor dependence. A partial wave analysis on the $pK^{-}$ system is also performed in a search for missing hyperon excitations.

Across a range of sectors, including transportation and healthcare, the use of automation to assist humans with increasingly complex tasks is also demanding that such systems are more interactive with human users. Given the role of cognitive factors in human decision-making during their interactions with automation, models enabling human cognitive state estimation and prediction could be used by autonomous systems to appropriately adapt their behavior. However, accomplishing this requires mathematical models of human cognitive state evolution that are suitable for algorithm design. In this thesis, a computational model of coupled human trust and self-confidence dynamics is proposed. The dynamics are modeled as a partially observable Markov decision process that leverages behavioral and self-report data as observations for estimation of the cognitive states. The use of an asymmetrical structure in the emission probability functions enables labeling and interpretation of the coupled cognitive states. The model is trained and validated using data collected from 340 participants. Analysis of the transition probabilities shows that the model captures nuanced effects, in terms of participants' decisions to rely on an autonomous system, that result as a function of the combination of their trust in the automation and self-confidence. Implications for the design of human-aware autonomous systems are discussed, particularly in the context of human trust and self-confidence calibration.

Les résultats présentés dans cette thèse s’articulent autour de la synthèse d’observateurs de type grand gain pour des classes de systèmes non linéaires multi-entrées, multi-sorties non uniformément observables. Dans un premier temps, la classe de systèmes considérées est telle que la dynamique des variables d’état est décrite par la somme de deux termes. Le premier correspond à une partie affine en l’état décrite par le produit d’une matrice, dont les entrées (fonctions non linéaires de l’état) ont une structure triangulaire, par le vecteur d’état. Le deuxième terme est composé par les non linéarités du système qui ont aussi une structure triangulaire. Le gain de l’observateur proposé est issu de la résolution d’une équation différentielle ordinaire de type Lyapunov.La convergence exponentielle de l’erreur d’observation sous-jacente est établie sous une une certaine condition d’excitation persistante dépendant de l’entrée du système et de l’état de l’observateur.Dans un deuxième temps, la synthèse de cet observateur est étendue à une classe plus large de systèmes non linéaires où des états peuvent intervenir de manière non triangulaire.La notion d’indices caractéristiques associés à ces états est alors introduite et elle a permis de définir une structure triangulaire étendue pour la quelle la synthèse de l’observateur a aussi été effectuée.Enfin, il a été établi que les observateurs proposés peuvent être utiliséscomme observateurs adaptatifs pour l’estimation simultanée de l’état et de certains paramètres et une forme adaptative de ces observateurs a été générée.Les performances des différents observateurs proposés ont été illustrées à travers des exemples en simulation / The results given in this thesis deal with the design of high gain observers for some classes on Multi Input Multi Output non uniformly observable nonlinear systems. In a first step, the class of considered systems is such that the dynamics of the state variables is the sum of two terms. The first term is affine in the state and is composed by the product of a matrix, whose entries are nonlinear functions of the state with a triangular structure, by the state vector. The second term describes the system nonlinearities which also assume a triangular structure. The gain of the proposed observer is issued from the resolution of a Lyapunov ordinary differential equation. The exponential convergence of the underlying observation error is established under a persistent excitation condition involving the system inputs and the state of the observer. In a second step, the observer design has been extended to a larger class of nonlinear systems where some state variables may intervene in a non triangular way. The notion of the characteristic indices associated to these state variables is then introduced and it allowed to define an extended triangular structure for which a high gain observer has been designed. Finally, it has been established that the proposed observers can be used as adaptive ones to jointly estimate the system state together with some unknown parameters and an adaptive form of these observers has been derived. The performance and main properties of the proposed observers have been illustrated in simulation by considering many examples throughout this thesis.

Unmanned aerial vehicles (UAVs) have emerged as a promising technology in search and rescue operations (SAR). UAVs have the ability to provide more timely localization, thus decreasing the crucial duration of SAR operations. Previous work have demonstrated proof-of-concept in regard to localizing missing people by utilizing received signal strength (RSS) and UAVs. The localization system is based on the assumption that the missing person wears an enabled smartphone whose Wi-Fi signal can be intercepted. This thesis proposes a two-staged path planner for UAVs, utilizing RSS-signals and an initial belief regarding the missing person's location. The objective of the first stage is to locate an RSS-signal. By dividing the search area into grids, a hierarchical solution based on several Markov decision processes (MDPs) can be formulated which takes different areas probabilities into consideration. The objective of the second stage is to isolate the RSS-signal and provide a location estimate. The environment is deemed to be partially observable, and the problem is formulated as a partially observable Markov decision process (POMDP). Two different filters, a point mass filter (PMF) and a particle filter (PF), are evaluated in regard to their ability to correctly estimate the state of the environment. The state of the environment then acts as input to a deep Q-network (DQN) which selects appropriate actions for the UAV. Thus, the DQN becomes a path planner for the UAV and the trajectory it generates is compared to trajectories generated by, among others, a greedy-policy.  Results for Stage 1 demonstrate that the path generated by the MDPs prioritizes areas with higher probability, and intuitively seems very reasonable. The results also illustrate potential drawbacks with a hierarchical solution, which potentially can be addressed by considering more factors into the problem. Simulation results for Stage 2 show that both a PMF and a PF can successfully be used to estimate the state of the environment and provide an accurate localization estimate. The PMF generated slightly more accurate estimations compared to the PF. The DQN is successful in isolating the missing person's probable location, by relatively few actions. However, it only performs marginally better than the greedy policy, indicating that it may be a complicated solution to a simpler problem.

Intelligent tutoring systems (ITSs) are computer programs that model individual learners and adapt instruction to help each learner differently. One way ITSs differ from human tutors is that few ITSs give learners a way to ask questions. When learners can ask for help, their questions have the potential to improve learning directly and also act as a new source of model data to help the ITS personalize instruction. Inquiry modeling gives ITSs the ability to answer learner questions and refine their learner models with an inexpensive new input channel. In order to support inquiry modeling, an advanced planning formalism is applied to ITS learner modeling. Partially observable Markov decision processes (POMDPs) differ from more widely used ITS architectures because they can plan complex action sequences in uncertain situations with machine learning. Tractability issues have previously precluded POMDP use in ITS models. This dissertation introduces two improvements, priority queues and observation chains, to make POMDPs scale well and encompass the large problem sizes that real-world ITSs must confront. A new ITS was created to support trainees practicing a military task in a virtual environment. The development of the Inquiry Modeling POMDP Adaptive Trainer (IMP) began with multiple formative studies on human and simulated learners that explored inquiry modeling and POMDPs in intelligent tutoring. The studies suggest the new POMDP representations will be effective in ITS domains having certain common characteristics. iv Finally, a summative study evaluated IMP’s ability to train volunteers in specific practice scenarios. IMP users achieved post-training scores averaging up to 4.5 times higher than users who practiced without support and up to twice as high as trainees who used an ablated version of IMP with no inquiry modeling. IMP’s implementation and evaluation helped explore questions about how inquiry modeling and POMDP ITSs work, while empirically demonstrating their efficacy