No description available.

In this thesis, we consider stochastic systems which arise in different real-world application contexts. The first problem we consider is based on product adoption and pricing. A monopolist selling a product has to appropriately price the product over time in order to maximize the aggregated profit. The demand for a product is uncertain and is influenced by a number of factors, some of which are price, advertising, and product technology. We study the influence of price on the demand of a product and also how demand affects future prices. Our approach involves mathematically modelling the variation in demand as a function of price and current sales. We present a simulation-based algorithm for computing the optimal price path of a product for a given period of time. The algorithm we propose uses a smoothed-functional based performance gradient descent method to find a price sequence which maximizes the total profit over a planning horizon. The second system we consider is in the domain of sensor networks. A sensor network is a collection of autonomous nodes, each of which senses the environment. Sensor nodes use energy for sensing and communication related tasks. We consider the problem of finding optimal energy sharing policies that maximize the network performance of a system comprising of multiple sensor nodes and a single energy harvesting(EH) source. Nodes periodically sense a random field and generate data, which is stored in their respective data queues. The EH source harnesses energy from ambient energy sources and the generated energy is stored in a buffer. The nodes require energy for transmission of data and and they receive the energy for this purpose from the EH source. There is a need for efficiently sharing the stored energy in the EH source among the nodes in the system, in order to minimize average delay of data transmission over the long run. We formulate this problem in the framework of average cost infinite-horizon Markov Decision Processes[3],[7]and provide algorithms for the same.