Up to now various recommendation systems have been proposed for web based applications such as e-commerce and information retrieval where a large amount of product or information is available. Basically, the task of the recommendation systems in those applications, for example the e-commerce, is to find and recommend the most relevant items to users/customers. In this domain, the most prominent approaches are collaborative filtering and content-based filtering. Sometimes these approaches are called as user profiling as well. In this work, a context-aware recommendation system is proposed for mobile environment, which also can be considered as an extension of those recommendation systems proposed for web-based information retrieval and e-commerce applications. In the web-based information retrieval and e-commerce applications, for example in an online book store (e-commerce), the users&amp / #8217 / actions are independent of their instant context (location, time&amp / #8230 / etc). But as for mobile environment, the users&amp / #8217 / actions are strictly dependent on their instant context. These dependencies give raise to need of filtering items/actions with respect to the users&amp / #8217 / instant context. In this thesis, an approach coupling approaches from two different domains, one is the mobile environment and other is the web, is proposed. Hence, it will be possible to separate whole approach into two phases: context-aware prediction and user profiling. In the first phase, combination of two methods called fuzzy c-means clustering and learning automata will be used to predict the mobile user&amp / #8217 / s motions in context space beforehand. This provides elimination of a large amount of items placed in the context space. In the second phase, hierarchical fuzzy clustering for users profiling will be used to determine the best recommendation among the remaining items.

The growing demand for ubiquitous broadband network connectivity and continuously falling prices in hardware operating on the unlicensed bands have put Wi-Fi technology in a position to lead the way in rapid innovation towards high performance wireless for the future. The success story of Wi-Fi contributed to the development of widespread variety of options for unlicensed access (e.g., Bluetooth, Zigbee) and has even sparked regulatory bodies in several countries to permit access to unlicensed devices in portions of the spectrum initially licensed to TV services. In this thesis we present novel spectrum management algorithms for networks employing 802.11 and TV white spaces broadly aimed at efficient use of spectrum under consideration, lower contention (interference) and high performance. One of the target scenarios of this thesis is neighbourhood or citywide wireless access. For this, we propose the use of IEEE 802.11-based multi-radio wireless mesh network using omnidirectional antennae. We develop a novel scalable protocol termed LCAP for efficient and adaptive distributed multi-radio channel allocation. In LCAP, nodes autonomously learn their channel allocation based on neighbourhood and channel usage information. This information is obtained via a novel neighbour discovery protocol, which is effective even when nodes do not share a common channel. Extensive simulation-based evaluation of LCAP relative to the state-of-the-art Asynchronous Distributed Colouring (ADC) protocol demonstrates that LCAP is able to achieve its stated objectives. These objectives include efficient channel utilisation across diverse traffic patterns, protocol scalability and adaptivity to factors such as external interference. Motivated by the non-stationary nature of the network scenario and the resulting difficulty of establishing convergence of LCAP, we consider a deterministic alternative. This approach employs a novel distributed priority-based mechanism where nodes decide on their channel allocations based on only local information. Key enabler of this approach is our neighbour discovery mechanism. We show via simulations that this mechanism exhibits similar performance to LCAP. Another application scenario considered in this thesis is broadband access to rural areas. For such scenarios, we consider the use of long-distance 802.11 mesh networks and present a novel mechanism to address the channel allocation problem in a traffic-aware manner. The proposed approach employs a multi-radio architecture using directional antennae. Under this architecture, we exploit the capability of the 802.11 hardware to use different channel widths and assign widths to links based on their relative traffic volume such that side-lobe interference is mitigated. We show that this problem is NP-complete and propose a polynomial time, greedy channel allocation algorithm that guarantees valid channel allocations for each node. Evaluation of the proposed algorithm via simulations of real network topologies shows that it consistently outperforms fixed width allocation due to its ability to adapt to spatio-temporal variations in traffic demands. Finally, we consider the use of TV-white-spaces to increase throughput for in-home wireless networking and relieve the already congested unlicensed bands. To the best of our knowledge, our work is the first to develop a scalable micro auctioning mechanism for sharing of TV white space spectrum through a geolocation database. The goal of our approach is to minimise contention among secondary users, while not interfering with primary users of TV white space spectrum (TV receivers and microphone users). It enables interference-free and dynamic sharing of TVWS among home networks with heterogeneous spectrum demands, while resulting in revenue generation for database and broadband providers. Using white space availability maps from the UK, we validate our approach in real rural, urban and dense-urban residential scenarios. Our results show that our mechanism is able to achieve its stated objectives of attractiveness to both the database provider and spectrum requesters, scalability and efficiency for dynamic spectrum distribution in an interference-free manner.

Indiana University-Purdue University Indianapolis (IUPUI) / Multi-agent systems consist of multiple agents that interact and coordinate with each other to work towards to certain goal. Multi-agent systems naturally arise in a variety of domains such as robotics, telecommunications, and economics. The dynamic and complex nature of these systems entails the agents to learn the optimal solutions on their own instead of following a pre-programmed strategy. Reinforcement learning provides a framework in which agents learn optimal behavior based on the response obtained from the environment. In this thesis, we propose various novel de- centralized, learning automaton based algorithms which can be employed by a group of interacting learning automata. We propose a completely decentralized version of the estimator algorithm. As compared to the completely centralized versions proposed before, this completely decentralized version proves to be a great improvement in terms of space complexity and convergence speed. The decentralized learning algorithm was applied; for the first time; to the domains of distributed object tracking and distributed watershed management. The results obtained by these experiments show the usefulness of the decentralized estimator algorithms to solve complex optimization problems. Taking inspiration from the completely decentralized learning algorithm, we propose the novel concept of partial decentralization. The partial decentralization bridges the gap between the completely decentralized and completely centralized algorithms and thus forms a comprehensive and continuous spectrum of multi-agent algorithms for the learning automata. To demonstrate the applicability of the partial decentralization, we employ a partially decentralized team of learning automata to control multi-agent Markov chains. More flexibility, expressiveness and flavor can be added to the partially decentralized framework by allowing different decentralized modules to engage in different types of games. We propose the novel framework of heterogeneous games of learning automata which allows the learning automata to engage in disparate games under the same formalism. We propose an algorithm to control the dynamic zero-sum games using heterogeneous games of learning automata.

Indiana University-Purdue University Indianapolis (IUPUI) / Learning in Partially Observable Markov Decision process (POMDP) is motivated by the essential need to address a number of realistic problems. A number of methods exist for learning in POMDPs, but learning with limited amount of information about the model of POMDP remains a highly anticipated feature. Learning with minimal information is desirable in complex systems as methods requiring complete information among decision makers are impractical in complex systems due to increase of problem dimensionality. In this thesis we address the problem of decentralized control of POMDPs with unknown transition probabilities and reward. We suggest learning in POMDP using a tree based approach. States of the POMDP are guessed using this tree. Each node in the tree has an automaton in it and acts as a decentralized decision maker for the POMDP. The start state of POMDP is known as the landmark state. Each automaton in the tree uses a simple learning scheme to update its action choice and requires minimal information. The principal result derived is that, without proper knowledge of transition probabilities and rewards, the automata tree of decision makers will converge to a set of actions that maximizes the long term expected reward per unit time obtained by the system. The analysis is based on learning in sequential stochastic games and properties of ergodic Markov chains. Simulation results are presented to compare the long term rewards of the system under different decision control algorithms.