In the first half of this thesis, we extend the results of Halfin and Whitt to generally distributed service times. This is accomplished by first writing the system equations for the G/GI/N queue in a manner similar to the system equations for G/GI/Infinity queue. We next identify a key relationship between these two queues. This relationship allows us to leverage several existing results for the G/GI/Infinity queue in order to prove our main result. Our main result in the first part of this thesis is to show that the diffusion scaled queue length process for the G/GI/N queue in the Halfin-Whitt regime converges to a limiting stochastic process which is driven by a Gaussian process and satisfies a stochastic convolution equation. We also show that a similar result holds true for the fluid scaled queue length process under general initial conditions. Customer abandonment is also a common feature of many call centers. Some researchers have even gone so far as to suggest that the level of customer abandonment is the single most important metric with regards to a call center's performance. In the second half of this thesis, we improve upon a result of Ward and Glynn's concerning the GI/GI/1+GI queue in heavy traffic. Whereas Ward and Glynn obtain a diffusion limit result for the GI/GI/1+GI queue in heavy traffic which incorporates only the density the abandonment distribution at the origin, our result incorporate the entire abandonment distribution. This is accomplished by a scaling the hazard rate function of the abandonment distribution as the system moves into heavy traffic. Our main results are to obtain diffusion limits for the properly scaled workload and queue length processes in the GI/GI/1+GI queue. The limiting diffusions we obtain are reflected at the origin with a negative drift which is dependent upon the hazard rate of the abandonment distribution. Because these diffusions have an analytically tractable steady-state distribution, they can be used to provide a closed-form approximation for the steady-state distribution of the queue length and workload processes in a GI/GI/1+GI queue. We demonstrate the accuracy of these approximations through simulation.

The genealogy of a random sample of a population of organisms can be represented as a rooted binary tree. Population dynamics determine a distribution over sample genealogies. For large populations of constant size and in the absence of selection effects, the coalescent process of Kingman determines a suitable distribution. Neuhauser and Krone gave a stochastic model generalising the Kingman coalescent in a natural way to include the effects of selection. The model of Neuhauser and Krone determines a distribution over a class of graphs of randomly variable vertex number, known as ancestral selection graphs. Because vertices have associated scalar ages, realisations of the ancestral selection graph process have randomly variable dimensions. A Markov chain Monte Carlo method is used to simulate the posterior distribution for population parameters of interest. The state of the Markov chain Monte Carlo is a random graph, with random dimension and equilibrium distribution equal to the posterior distribution. The aim of the project is to determine if the data is informative of the selection parameter by fitting the model to synthetic data. / Foundation for Research Science and Technology Top Achiever Doctoral Scolarship

The genealogy of a random sample of a population of organisms can be represented as a rooted binary tree. Population dynamics determine a distribution over sample genealogies. For large populations of constant size and in the absence of selection effects, the coalescent process of Kingman determines a suitable distribution. Neuhauser and Krone gave a stochastic model generalising the Kingman coalescent in a natural way to include the effects of selection. The model of Neuhauser and Krone determines a distribution over a class of graphs of randomly variable vertex number, known as ancestral selection graphs. Because vertices have associated scalar ages, realisations of the ancestral selection graph process have randomly variable dimensions. A Markov chain Monte Carlo method is used to simulate the posterior distribution for population parameters of interest. The state of the Markov chain Monte Carlo is a random graph, with random dimension and equilibrium distribution equal to the posterior distribution. The aim of the project is to determine if the data is informative of the selection parameter by fitting the model to synthetic data. / Foundation for Research Science and Technology Top Achiever Doctoral Scolarship

The genealogy of a random sample of a population of organisms can be represented as a rooted binary tree. Population dynamics determine a distribution over sample genealogies. For large populations of constant size and in the absence of selection effects, the coalescent process of Kingman determines a suitable distribution. Neuhauser and Krone gave a stochastic model generalising the Kingman coalescent in a natural way to include the effects of selection. The model of Neuhauser and Krone determines a distribution over a class of graphs of randomly variable vertex number, known as ancestral selection graphs. Because vertices have associated scalar ages, realisations of the ancestral selection graph process have randomly variable dimensions. A Markov chain Monte Carlo method is used to simulate the posterior distribution for population parameters of interest. The state of the Markov chain Monte Carlo is a random graph, with random dimension and equilibrium distribution equal to the posterior distribution. The aim of the project is to determine if the data is informative of the selection parameter by fitting the model to synthetic data. / Foundation for Research Science and Technology Top Achiever Doctoral Scolarship

The genealogy of a random sample of a population of organisms can be represented as a rooted binary tree. Population dynamics determine a distribution over sample genealogies. For large populations of constant size and in the absence of selection effects, the coalescent process of Kingman determines a suitable distribution. Neuhauser and Krone gave a stochastic model generalising the Kingman coalescent in a natural way to include the effects of selection. The model of Neuhauser and Krone determines a distribution over a class of graphs of randomly variable vertex number, known as ancestral selection graphs. Because vertices have associated scalar ages, realisations of the ancestral selection graph process have randomly variable dimensions. A Markov chain Monte Carlo method is used to simulate the posterior distribution for population parameters of interest. The state of the Markov chain Monte Carlo is a random graph, with random dimension and equilibrium distribution equal to the posterior distribution. The aim of the project is to determine if the data is informative of the selection parameter by fitting the model to synthetic data. / Foundation for Research Science and Technology Top Achiever Doctoral Scolarship

O objetivo central desta pesquisa foi estimar o benefício econômico do Parque Nacional Cerro Corra - PNCC localizado no Departamento de Amambay no Paraguai mensurando o valor obtido pelas pessoas que o visitam. Os métodos aplicados foram o Método de Custo de Viagem TCM assumindo uma distribuição de Poisson e o Método de Valoração Contingente CVM utilizando o modelo referendo com distribuição de probabilidade logística. Realizando uma amostra aleatória os dados foram coletados durante os meses de setembro a dezembro de 2007 e janeiro de 2008, efetuando-se 380 entrevistas com aproveitamento de 347 delas. Através do TCM o valor médio do excedente do consumidor por visitante foi de R$ 9,54, enquanto que, por meio do CVM, obteve-se um valor médio da DAP e/ ou benefício por visitante de R$ 8,95, notando-se uma diferença de 6,18% a menos para o CVM. Evidencia-se que por ambos os métodos o PNCC possui em geral um benefício econômico para seus visitantes. / The central objective of this research was to estimate the economic benefit of the National Park Cerro Corá PNCC located in the Department of Amambay Paraguay measuring the benefit received by its visitors. The methods applied were the Travel Cost Method TCM assuming a Poisson distribution and Contingent Valuation Method CVM using the referendum model with logistic probability distribution. Data were collected between September of 2007 and January 2008. The total number of questionnaires applied was 380 with 347 of them used for econometric purposes. A random sample was used to collect the data. Through TCM the average consumer\'s surplus for the visitor was R$ 9.54, while using the CVM the estimated mean value of WTP (or benefit by visitors) was $ 8.95. The difference between the two methods is 6.18%. There is evidence from these estimates hat PNCC have an overall positive economic benefit to its visitors.

Stochastic control problems that arise in sequential decision making applications typically assume that information used for decision-making is obtained according to a predetermined sampling schedule. In many real applications however, there is a high sampling cost associated with collecting such data. It is therefore of equal importance to determine when information should be collected as it is to decide how this information should be utilized for optimal decision-making. This type of joint optimization has been a long-standing problem in the operations research literature, and very few results regarding the structure of the optimal sampling and control policy have been published. In this thesis, the joint optimization of sampling and control is studied in the context of maintenance optimization. New theoretical results characterizing the structure of the optimal policy are established, which have practical interpretation and give new insight into the value of condition-based maintenance programs in life-cycle asset management. Applications in other areas such as healthcare decision-making and statistical process control are discussed. Statistical parameter estimation results are also developed with illustrative real-world numerical examples.

Stochastic control problems that arise in sequential decision making applications typically assume that information used for decision-making is obtained according to a predetermined sampling schedule. In many real applications however, there is a high sampling cost associated with collecting such data. It is therefore of equal importance to determine when information should be collected as it is to decide how this information should be utilized for optimal decision-making. This type of joint optimization has been a long-standing problem in the operations research literature, and very few results regarding the structure of the optimal sampling and control policy have been published. In this thesis, the joint optimization of sampling and control is studied in the context of maintenance optimization. New theoretical results characterizing the structure of the optimal policy are established, which have practical interpretation and give new insight into the value of condition-based maintenance programs in life-cycle asset management. Applications in other areas such as healthcare decision-making and statistical process control are discussed. Statistical parameter estimation results are also developed with illustrative real-world numerical examples.

O objetivo central desta pesquisa foi estimar o benefício econômico do Parque Nacional Cerro Corra - PNCC localizado no Departamento de Amambay no Paraguai mensurando o valor obtido pelas pessoas que o visitam. Os métodos aplicados foram o Método de Custo de Viagem TCM assumindo uma distribuição de Poisson e o Método de Valoração Contingente CVM utilizando o modelo referendo com distribuição de probabilidade logística. Realizando uma amostra aleatória os dados foram coletados durante os meses de setembro a dezembro de 2007 e janeiro de 2008, efetuando-se 380 entrevistas com aproveitamento de 347 delas. Através do TCM o valor médio do excedente do consumidor por visitante foi de R$ 9,54, enquanto que, por meio do CVM, obteve-se um valor médio da DAP e/ ou benefício por visitante de R$ 8,95, notando-se uma diferença de 6,18% a menos para o CVM. Evidencia-se que por ambos os métodos o PNCC possui em geral um benefício econômico para seus visitantes. / The central objective of this research was to estimate the economic benefit of the National Park Cerro Corá PNCC located in the Department of Amambay Paraguay measuring the benefit received by its visitors. The methods applied were the Travel Cost Method TCM assuming a Poisson distribution and Contingent Valuation Method CVM using the referendum model with logistic probability distribution. Data were collected between September of 2007 and January 2008. The total number of questionnaires applied was 380 with 347 of them used for econometric purposes. A random sample was used to collect the data. Through TCM the average consumer\'s surplus for the visitor was R$ 9.54, while using the CVM the estimated mean value of WTP (or benefit by visitors) was $ 8.95. The difference between the two methods is 6.18%. There is evidence from these estimates hat PNCC have an overall positive economic benefit to its visitors.

<p dir="ltr">This thesis presents probably approximately correct (PAC) bounds on estimates of the transition kernels of Controlled Markov chains (CMC’s). CMC’s are a natural choice for modelling various industrial and medical processes, and are also relevant to reinforcement learning (RL). Learning the transition dynamics of CMC’s in a sample efficient manner is an important question that is open. This thesis aims to close this gap in knowledge in literature.</p><p dir="ltr">In Chapter 2, we lay the groundwork for later chapters by introducing the relevant concepts and defining the requisite terms. The two subsequent chapters focus on non-parametric estimation. </p><p dir="ltr">In Chapter 3, we restrict ourselves to a finitely supported CMC with d states and k controls and produce a general theory for minimax sample complexity of estimating the transition matrices.</p><p dir="ltr">In Chapter 4 we demonstrate the applicability of this theory by using it to recover the sample complexities of various controlled Markov chains, as well as RL problems.</p><p dir="ltr">In Chapter 5 we move to a continuous state and action spaces with compact supports. We produce a robust, non-parametric test to find the best histogram based estimator of the transition density; effectively reducing the problem into one of model selection based on empricial processes.</p><p dir="ltr">Finally, in Chapter 6 we move to a parametric and Bayesian regime, and restrict ourselves to Markov chains. Under this setting we provide a PAC-Bayes bound for estimating model parameters under tempered posteriors.</p>