Tato práce se zabývá modely s promìnlivými koe cienty se za- mìøením na statistickou inferenci. Hlavní my¹lenkou tìchto modelù je pou¾ití regresních koe cientù, mìnících se v závislosti na nìjakém modi kátoru vlivu, namísto konstantních koe cientù klasické lineární regrese. Nejprve si de nujeme tyto modely a jejich odhadové procedury, kterých bylo doposud publikováno nì- kolik variant. K odhadu se pou¾ívá lokální regrese nebo rùzné druhy splajnù { vyhlazovací, polynomiální èi penalizované. Od metody odhadu se následnì od- víjí i daná statistická inference, ke které uvedeme odvozené vychýlení, rozptyl, asymptotickou normalitu, kon denèní pásma a testování hypotéz. Hlavním cílem na¹í práce je kompaktnì shrnout vybrané metody a jejich inferenci. Na závìr je navr¾ena proceduru pro výbìr promìnných.

This thesis is divided into two parts. In the first part, we show how problems of statistical inference and combinatorial optimization may be approached within a unified framework that employs tools from fields as diverse as machine learning, statistical physics and information theory, allowing us to i) design algorithms to solve the problems, ii) analyze the performance of these algorithms both empirically and analytically, and iii) to compare the results obtained with the optimal achievable ones. In the second part, we use this framework to study two specific problems, one of inference (compressed sensing) and the other of optimization (information hiding). In both cases, we review current approaches, identify their flaws, and propose new schemes to address these flaws, building on the use of message-passing algorithms, variational inference techniques, and spin glass models from statistical physics. / Esta tese está dividida em duas partes. Na primeira delas, mostramos como problemas de inferência estatística e de otimização combinatória podem ser abordados sob um framework unificado que usa ferramentas de áreas tão diversas quanto o aprendizado de máquina, a física estatística e a teoria de informação, permitindo que i) projetemos algoritmos para resolver os problemas, ii) analisemos a performance destes algoritmos tanto empiricamente como analiticamente, e iii) comparemos os resultados obtidos com os limites teóricos. Na segunda parte, este framework é usado no estudo de dois problemas específicos, um de inferência (compressed sensing) e outro de otimização (ocultação de dados). Em ambos os casos, revisamos abordagens recentes, identificamos suas falhas, e propomos novos esquemas que visam corrigir estas falhas, baseando-nos sobretudo em algoritmos de troca de mensagens, técnicas de inferência variacional, e modelos de vidro de spin da física estatística.

Este trabalho apresenta três principais contribuições: i. a proposta de uma nova lógica de descrição probabilística; ii. um novo algoritmo de inferência em primeira ordem a ser utilizado em terminologias representadas nessa lógica; e iii. aplicações práticas em problemas reais. A lógica aqui proposta, crALC (credal ALC), adiciona inclusões probabilísticas na popular lógica ALC combinando as terminologias com condições de aciclicidade, de Markov, e adotando uma semântica baseada em interpretações. Como os métodos de inferência exata tradicionalmente apresentam problemas de escalabilidade devido à presença de quantificadores (restrições universal e existencial), apresentamos um algoritmo de loopy propagation em primeira-ordem que se comporta bem para terminologias com domínios não triviais. Uma série de testes foi feita com o algoritmo proposto em comparação com algoritmos tradicionais da literatura; os resultados apresentados mostram uma clara vantagem em relação aos outros algoritmos. São apresentadas ainda duas aplicações da lógica e do algoritmo para resolver problemas reais da área de robótica móvel. Embora os problemas tratados sejam relativamente simples, eles constituem a base de muitos outros problemas da área, sendo um passo importante na representação de conhecimento de agentes/robôs autônomos e no raciocínio sobre esse conhecimento. / This work presents two major contributions: i. a new probabilistic description logic; ii. a new algorithm for inference in terminologies expressed in this logic; iii. practical applications in real tasks. The proposed logic, referred to as crALC (credal ALC), adds probabilistic inclusions to the popular logic ALC, combining the usual acyclicity and Markov conditions, and adopting interpretation-based semantics. As exact inference does not seem scalable due to the presence of quantifiers (existential and universal), we present a first-order loopy propagation algorithm that behaves appropriately for non-trivial domain sizes. A series of tests were done comparing the performance of the proposed algorithm against traditional ones; the presented results are favorable to the first-order algorithm. Two applications in the field of mobile robotics are presented, using the new probabilistic logic and the inference algorithm. Though the problems can be considered simple, they constitute the basis for many other tasks in mobile robotics, being a important step in knowledge representation and in reasoning about it.

Este trabalho tem o objetivo de divulgar a metodologia de ponderação de modelos ou Bayesian Model Averaging (BMA) entre os pesquisadores da área agronômica e discutir suas vantagens e limitações. Com o BMA é possível combinar resultados de diferentes modelos acerca de determinada quantidade de interesse, com isso, o BMA apresenta-se como sendo uma metodologia alternativa de análise de dados frente os usuais métodos de seleção de modelos tais como o Coeficiente de Determinação Múltipla (R2 ), Coeficiente de Determinação Múltipla Ajustado (R2), Estatística de Mallows ( Cp) e Soma de Quadrados de Predição (PRESS). Vários trabalhos foram, recentemente, realizados com o objetivo de comparar o desempenho do BMA em relação aos métodos de seleção de modelos, porém, há ainda muitas situações para serem exploradas até que se possa chegar a uma conclusão geral acerca desta metodologia. Neste trabalho, o BMA foi aplicado a um conjunto de dados proveniente de um experimento agronômico. A seguir, o desempenho preditivo do BMA foi comparado com o desempenho dos métodos de seleção acima citados por meio de um estudo de simulação variando o grau de multicolinearidade e o tamanho amostral. Em cada uma dessas situações, foram utilizadas 1000 amostras geradas a partir de medidas descritivas de conjuntos de dados reais da área agronômica. O desempenho preditivo das metodologias em comparação foi medido pelo Logaritmo do Escore Preditivo (LEP). Os resultados empíricos obtidos indicaram que o BMA apresenta desempenho semelhante aos métodos usuais de seleção de modelos nas situações de multicolinearidade exploradas neste trabalho. / The objective of this work was divulge to Bayesian Model Averaging (BMA) between the researchers of the agronomy area and discuss its advantages and limitations. With the BMA is possible combine results of difeerent models about determined quantity of interest, with that, the BMA presents as being a metodology alternative of data analysis front the usual models selection approaches, for example the Coefficient of Multiple Determination (R2), Coefficient of Multiple Determination Adjusted (R2), Mallows (Cp Statistics) and Prediction Error Sum Squares (PRESS). Several works recently were carried out with the objective of compare the performance of the BMA regarding the approaches of models selection, however, there is still many situations for will be exploited to that can arrive to a general conclusion about this metodology. In this work, the BMA was applied to data originating from an agronomy experiment. It follow, the predictive performance of the BMA was compared with the performance of the approaches of selection above cited by means of a study of simulation varying the degree of multicollinearity, measured by the number of condition of the matrix standardized X'X and the number of observations in the sample. In each one of those situations, were utilized 1000 samples generated from the descriptive information of agronomy data. The predictive performance of the metodologies in comparison was measured by the Logarithm of the Score Predictive (LEP). The empirical results obtained indicated that the BMA presents similar performance to the usual approaches of selection of models in the situations of multicollinearity exploited.

O uso de fronteira de produ¸c˜ ao estoc´ astica com m´ ultiplos produtos tem despertado um interesse especial em ´areas da economia que defrontam-se com o problema de quantificar a eficiˆencia t´ecnica de firmas. Na estat´ýstica cl´ assica, quando se defronta com firmas que possuem v´arios produtos, as fun¸c˜ oes custo ou demanda s˜ ao mais utilizadas para calcular essa eficiˆencia, mas isso requer uma quantidade maior de informa¸c˜ oes sobre os dados, al´em das quantidades de insumos e produtos, tamb´em s˜ ao necess´ arios seus pre¸cos e custos. Quando existem apenas informa¸c˜ oes sobre os insumos (x) e os produtos (y) h´a a necessidade de se trabalhar com a fun¸c˜ ao de produ¸c˜ ao e a inexistˆencia de estat´ýsticas suficientes para alguns parˆ ametros tornam a an´alise d´ýficil. A abordagem Bayesiana pode se tornar uma ferramenta muito ´ util para esse caso, pois ´e poss´ývel obter uma amostra da distribui¸ c˜ ao de probabilidade dos parˆ ametros do modelo, possibilitando a obten¸c˜ ao de resumos de interesse. Para obter as amostras dessas distribui¸ c˜ oes m´etodos Monte Carlo com cadeias de Markov, tais como, amostrador de Gibbs, Metropolis-Hastings e “Slice sampling” s˜ ao utilizados. / The use of stochastic production frontier with multiple-outputs has been waking up a special interest in areas of the economy that are confronted with the problem of quantifying the technical efficiency of firms. In the classic statistics, when it is confronted with firms that possess several outputs, cost or profit functions are more used to calculate that efficiency, but that requests an amount larger of information about data set, besides the amounts of inputs and outputs, are also necessary your prices and costs. When just exist information on inputs (x) and outputs (y) there is need to work with the production function and the lack of enough statistics for some parameters turn the difficult analysis. Bayesian approach can become a useful tool for that case, because is possible to obtain a sample of the distribution of probability of the parameters of the model, making possible the obtaining of summaries of interest. To obtain samples of those distributions methods Markov chains Monte Carlo, that is, Gibbs sampling, Metropolis-Hastings and Slice sampling are used.

Abstract In this thesis, iterative detection and estimation algorithms for digital communications systems in the presence of parametric uncertainty are explored and further developed. In particular, variational methods, which have been extensively applied in other research fields such as artificial intelligence and machine learning, are introduced and systematically used in deriving approximations to the optimal receivers in various channel conditions. The key idea behind the variational methods is to transform the problem of interest into an optimization problem via an introduction of extra degrees of freedom known as variational parameters. This is done so that, for fixed values of the free parameters, the transformed problem has a simple solution, solving approximately the original problem. The thesis contributes to the state of the art of advanced receiver design in a number of ways. These include the development of new theoretical and conceptual viewpoints of iterative turbo-processing receivers as well as a new set of practical joint estimation and detection algorithms. Central to the theoretical studies is to show that many of the known low-complexity turbo receivers, such as linear minimum mean square error (MMSE) soft-input soft-output (SISO) equalizers and demodulators that are based on the Bayesian expectation-maximization (BEM) algorithm, can be formulated as solutions to the variational optimization problem. This new approach not only provides new insights into the current designs and structural properties of the relevant receivers, but also suggests some improvements on them. In addition, SISO detection in multipath fading channels is considered with the aim of obtaining a new class of low-complexity adaptive SISOs. As a result, a novel, unified method is proposed and applied in order to derive recursive versions of the classical Baum-Welch algorithm and its Bayesian counterpart, referred to as the BEM algorithm. These formulations are shown to yield computationally attractive soft decision-directed (SDD) channel estimators for both deterministic and Rayleigh fading intersymbol interference (ISI) channels. Next, by modeling the multipath fading channel as a complex bandpass autoregressive (AR) process, it is shown that the statistical parameters of radio channels, such as frequency offset, Doppler spread, and power-delay profile, can be conveniently extracted from the estimated AR parameters which, in turn, may be conveniently derived via an EM algorithm. Such a joint estimator for all relevant radio channel parameters has a number of virtues, particularly its capability to perform equally well in a variety of channel conditions. Lastly, adaptive iterative detection in the presence of phase uncertainty is investigated. As a result, novel iterative joint Bayesian estimation and symbol a posteriori probability (APP) computation algorithms, based on the variational Bayesian method, are proposed for both constant-phase channel models and dynamic phase models, and their performance is evaluated via computer simulations.

Tato práce se zabývá modely s promìnlivými koe cienty se za- mìøením na statistickou inferenci. Hlavní my¹lenkou tìchto modelù je pou¾ití regresních koe cientù, mìnících se v závislosti na nìjakém modi kátoru vlivu, namísto konstantních koe cientù klasické lineární regrese. Nejprve si de nujeme tyto modely a jejich odhadové procedury, kterých bylo doposud publikováno nì- kolik variant. K odhadu se pou¾ívá lokální regrese nebo rùzné druhy splajnù { vyhlazovací, polynomiální èi penalizované. Od metody odhadu se následnì od- víjí i daná statistická inference, ke které uvedeme odvozené vychýlení, rozptyl, asymptotickou normalitu, kon denèní pásma a testování hypotéz. Hlavním cílem na¹í práce je kompaktnì shrnout vybrané metody a jejich inferenci. Na závìr je navr¾ena proceduru pro výbìr promìnných.

Different from the standard treatment discovery framework which is used for finding single treatments for a homogenous group of patients, personalized medicine involves finding therapies that are tailored to each individual in a heterogeneous group. In this paper, we propose a new semiparametric additive single-index model for estimating individualized treatment strategy. The model assumes a flexible and nonparametric link function for the interaction between treatment and predictive covariates. We estimate the rule via monotone B-splines and establish the asymptotic properties of the estimators. Both simulations and an real data application demonstrate that the proposed method has a competitive performance.

Humans readily infer the meanings of novel symbols in communicative contexts of varying complexity, and several researchers in the field of language evolution have explicitly acknowledged that inference plays a key role in accounting for the evolution of symbolic communication. However, in this field at least, there has been very little investigation into the nature of inference in this regard. That is, evolutionary linguists have yet to address the following questions if we are to have a fuller picture of how humans came to communicate symbolically: 1. What kinds of inference are there? Specifically, i Diachronically, what forms of inference are comparatively simpler in evolutionary terms, and thus shared with a wider range of species? What forms of inference are more complex, and limited to humans or to us and our closest relatives? ii Synchronically, if humans are capable of several kinds of complex inference, how do we know which particular kind of inference is being applied in solving a given problem? 2. How do symbol-learning problems vary? Specifically, i What makes a particular symbol-learning problem more or less complex in terms of the kind of inference needed to solve it? ii How would the communicative context of our pre-linguistic ancestors have been different from that of a human child learning words from its linguistic parent? This dissertation takes a step towards answering these questions by investigating a little-known form of inference called `abduction' (or insightful hypothesis generation), which has thus far been wholly overshadowed in language evolution by a much better understood form called `induction' (or probabilistic hypothesis evaluation). I will argue that abduction and induction are both comparatively complex in the diachronic terms expressed above in 1.i, and while induction is useful in accounting for how modern children learn words from linguistic adults, abduction is more important in situations like those that would have faced our pre-lingistic ancestors as they first began to use symbols. That is, I will argue on both theoretical and empirical grounds that abductive inference was an evolutionary milestone as our ancestors crossed what Deacon (1997) calls the symbolic threshold.

The extension of the CJS model to include individual, continuous, time-varying covariates relies on the estimation of covariate values on occasions on which individuals were not captured. Fitting this model in a Bayesian framework typically involves the implementation of a Markov chain Monte Carlo (MCMC) algorithm, such as a Gibbs sampler, to sample from the posterior distribution. For large data sets with many missing covariate values that must be estimated, this creates a computational issue, as each iteration of the MCMC algorithm requires sampling from the full conditional distributions of each missing covariate value. This dissertation examines two solutions to address this problem. First, I explore variational Bayesian algorithms, which derive inference from an approximation to the posterior distribution that can be fit quickly in many complex problems. Second, I consider an alternative approximation to the posterior distribution derived by truncating the individual capture histories in order to reduce the number of missing covariates that must be updated during the MCMC sampling algorithm. In both cases, the increased computational efficiency comes at the cost of producing approximate inferences. The variational Bayesian algorithms generally do not estimate the posterior variance very accurately and do not directly address the issues with estimating many missing covariate values. Meanwhile, the truncated CJS model provides a more significant improvement in computational efficiency while inflating the posterior variance as a result of discarding some of the data. Both approaches are evaluated via simulation studies and a large mark-recapture data set consisting of cliff swallow weights and capture histories.