The work in this thesis concerns the use of reinforcement learning solutions to re-source allocation problems in channelised cellular networks. The methodology of re-inforcement learning techniques was chosen for application to these problems due to its capability of finding efficient policies in a fully on-line, adaptable manner, without requiring specific environment models. All of the presented agent architectures are assumed to simultaneously learn and perform network control functions in a totally on-line and unsupervised manner, and agents are developed with a view to real-world implementability by focussing on techniques that have low resource requirements and make use of only local system information.

An adaptive resource reservation and handoff priority scheme, which jointly considers the characteristics from the physical, link and network layers, is proposed for a packet switching Multicode (MC)-CDMA cellular network supporting multimedia applications. A call admission region is derived for call admission control (CAC) and handoff management with the satisfaction of quality of service (QoS) requirements for all kinds of multimedia traffic, where the QoS parameters include the wireless transmission bit error rate (BER), the packet loss rate (PLR) and delay requirement. The BER requirement is guaranteed by properly arranging simultaneous packet transmissions, whereas the PLR and delay requirements are guaranteed by the proposed packet scheduling and partial packet integration scheme. To give service priority to handoff calls, a threshold-based adaptive resource reservation scheme is proposed on the basis of a practical user mobility model and a proper handoff request prediction scheme. The resource reservation scheme gives handoff calls a higher admission priority over new calls, and is designed to adjust the reservation-request time threshold adaptively according to the varying traffic load. The individual reservation requests form a common reservation pool, and handoff calls are served on a first-come-first-serve basis. By exploiting the transmission rate adaptability of video calls to the available radio resources, the resources freed from rate-adaptive high-quality video calls by service degradation can be further used to prioritize handoff calls. With the proposed resource reservation and handoff priority scheme, the dynamic properties of the system can be closely captured and a better grade of service (GoS) in terms of new call blocking and handoff call dropping probabilities(rates) can be achieved compared to other schemes in literature. Numerical results are presented to show the improvement of the GoS performance and the efficient utilization of the radio resources.

This thesis studies the impact of mobility and wireless channel characteristics, i. e. , variability and high bit-error-rate, on the performance of integrated voice and data wireless systems from network, transport protocol and application perspectives. <br /><br /> From the network perspective, we study the impact of user mobility on radio resource allocation. The goal is to design resource allocation mechanisms that provide seamless mobility for voice calls while being fair to data calls. In particular, we develop a distributed admission control for a general integrated voice and data wireless system. We model the number of active calls in a cell of the network as a Gaussian process with time-dependent mean and variance. The Gaussian model is updated periodically using the information obtained from neighboring cells about their load conditions. We show that the proposed scheme guarantees a prespecified dropping probability for voice calls while being fair to data calls. Furthermore, the scheme is stable, insensitive to user mobility process and robust to load variations. <br /><br /> From the transport protocol perspective, we study the impact of wireless channel variations and rate scheduling on the performance of elastic data traffic carried by TCP. We explore cross-layer optimization of the rate adaptation feature of cellular networks to optimize TCP throughput. We propose a TCP-aware scheduler that switches between two rates as a function of TCP sending rate. We develop a fluid model of the steady-state TCP behavior for such a system and derive analytical expressions for TCP throughput that explicitly account for rate variability as well as the dependency between the scheduler and TCP. The model is used to choose RF layer parameters that, in conjunction with the TCP-aware scheduler, improve long-term TCP throughput in wireless networks. A distinctive feature of our model is its ability to capture variability of round-trip-time, channel rate and packet error probability inherent to wireless communications. <br /><br /> From the application perspective, we study the performance of wireless messaging systems. Two popular wireless applications, the short messaging service and multimedia messaging service are considered. We develop a mathematical model to evaluate the performance of these systems taking into consideration the fact that each message tolerates only a limited amount of waiting time in the system. Using the model, closed-form expressions for critical performance parameters such as message loss, message delay and expiry probability are derived. Furthermore, a simple algorithm is presented to find the optimal temporary storage size that minimizes message delay for a given set of system parameters.

This thesis contributes toward the design of a quality-of-service (QoS) aware network layer for wireless ad hoc networks. With the lack of an infrastructure in ad hoc networks, the role of the network layer is not only to perform multihop routing between a source node and a destination node, but also to establish an end-to-end connection between communicating peers that satisfies the service level requirements of multimedia applications running on those peers. Wireless ad hoc networks represent autonomous distributed systems that are infrastructure-less, fully distributed, and multi-hop in nature. Over the last few years, wireless ad hoc networks have attracted significant attention from researchers. This has been fueled by recent technological advances in the development of multifunction and low-cost wireless communication gadgets. Wireless ad hoc networks have diverse applications spanning several domains, including military, commercial, medical, and home networks. Projections indicate that these self-organizing wireless ad hoc networks will eventually become the dominant form of the architecture of telecommunications networks in the near future. Recently, due to increasing popularity of multimedia applications, QoS support in wireless ad hoc networks has become an important yet challenging objective. The challenge lies in the need to support the heterogeneous QoS requirements (e.g., data rate, packet loss probability, and delay constraints) for multimedia applications and, at the same time, to achieve efficient radio resource utilization, taking into account user mobility and dynamics of multimedia traffic. In terms of research contributions, we first present a position-based QoS routing framework for wireless ad-hoc networks. The scheme provides QoS guarantee in terms of packet loss ratio and average end-to-end delay (or throughput) to ad hoc networks loaded with constant rate traffic. Via cross-layer design, we apply call admission control and temporary bandwidth reservation on discovered routes, taking into consideration the physical layer multi-rate capability and the medium access control (MAC) interactions such as simultaneous transmission and self interference from route members. Next, we address the network-layer resource allocation where a single-hop ad hoc network is loaded with random traffic. As a starting point, we study the behavior of the service process of the widely deployed IEEE 802.11 DCF MAC when the network is under different traffic load conditions. Our study investigates the near-memoryless behavior of the service time for IEEE 802.11 saturated single-hop ad hoc networks. We show that the number of packets successfully transmitted by any node over a time interval follows a general distribution, which is close to a Poisson distribution with an upper bounded distribution distance. We also show that the service time distribution can be approximated by the geometric distribution and illustrate that a simplified queuing system can be used efficiently as a resource allocation tool for single hop IEEE 802.11 ad hoc networks near saturation. After that, we shift our focus to providing probabilistic packet delay guarantee to multimedia users in non-saturated IEEE 802.11 single hop ad hoc networks. We propose a novel stochastic link-layer channel model to characterize the variations of the IEEE 802.11 channel service process. We use the model to calculate the effective capacity of the IEEE 802.11 channel. The channel effective capacity concept is the dual of the effective bandwidth theory. Our approach offers a tool for distributed statistical resource allocation in single hop ad hoc networks, which combines both efficient resource utilization and QoS provisioning to a certain probabilistic limit. Finally, we propose a statistical QoS routing scheme for multihop IEEE 802.11 ad hoc networks. Unlike most of QoS routing schemes in literature, the proposed scheme provides stochastic end-to-end delay guarantee, instead of average delay guarantee, to delay-sensitive bursty traffic sources. Via a cross-layer design approach, the scheme selects the routes based on a geographical on-demand ad hoc routing protocol and checks the availability of network resources by using traffic source and link-layer channel models, incorporating the IEEE 802.11 characteristics and interaction. Our scheme extends the well developed effective bandwidth theory and its dual effective capacity concept to multihop IEEE 802.11 ad hoc networks in order to achieve an efficient utilization of the shared radio channel while satisfying the end-to-end delay bound.

An adaptive resource reservation and handoff priority scheme, which jointly considers the characteristics from the physical, link and network layers, is proposed for a packet switching Multicode (MC)-CDMA cellular network supporting multimedia applications. A call admission region is derived for call admission control (CAC) and handoff management with the satisfaction of quality of service (QoS) requirements for all kinds of multimedia traffic, where the QoS parameters include the wireless transmission bit error rate (BER), the packet loss rate (PLR) and delay requirement. The BER requirement is guaranteed by properly arranging simultaneous packet transmissions, whereas the PLR and delay requirements are guaranteed by the proposed packet scheduling and partial packet integration scheme. To give service priority to handoff calls, a threshold-based adaptive resource reservation scheme is proposed on the basis of a practical user mobility model and a proper handoff request prediction scheme. The resource reservation scheme gives handoff calls a higher admission priority over new calls, and is designed to adjust the reservation-request time threshold adaptively according to the varying traffic load. The individual reservation requests form a common reservation pool, and handoff calls are served on a first-come-first-serve basis. By exploiting the transmission rate adaptability of video calls to the available radio resources, the resources freed from rate-adaptive high-quality video calls by service degradation can be further used to prioritize handoff calls. With the proposed resource reservation and handoff priority scheme, the dynamic properties of the system can be closely captured and a better grade of service (GoS) in terms of new call blocking and handoff call dropping probabilities(rates) can be achieved compared to other schemes in literature. Numerical results are presented to show the improvement of the GoS performance and the efficient utilization of the radio resources.

This thesis studies the impact of mobility and wireless channel characteristics, i. e. , variability and high bit-error-rate, on the performance of integrated voice and data wireless systems from network, transport protocol and application perspectives. <br /><br /> From the network perspective, we study the impact of user mobility on radio resource allocation. The goal is to design resource allocation mechanisms that provide seamless mobility for voice calls while being fair to data calls. In particular, we develop a distributed admission control for a general integrated voice and data wireless system. We model the number of active calls in a cell of the network as a Gaussian process with time-dependent mean and variance. The Gaussian model is updated periodically using the information obtained from neighboring cells about their load conditions. We show that the proposed scheme guarantees a prespecified dropping probability for voice calls while being fair to data calls. Furthermore, the scheme is stable, insensitive to user mobility process and robust to load variations. <br /><br /> From the transport protocol perspective, we study the impact of wireless channel variations and rate scheduling on the performance of elastic data traffic carried by TCP. We explore cross-layer optimization of the rate adaptation feature of cellular networks to optimize TCP throughput. We propose a TCP-aware scheduler that switches between two rates as a function of TCP sending rate. We develop a fluid model of the steady-state TCP behavior for such a system and derive analytical expressions for TCP throughput that explicitly account for rate variability as well as the dependency between the scheduler and TCP. The model is used to choose RF layer parameters that, in conjunction with the TCP-aware scheduler, improve long-term TCP throughput in wireless networks. A distinctive feature of our model is its ability to capture variability of round-trip-time, channel rate and packet error probability inherent to wireless communications. <br /><br /> From the application perspective, we study the performance of wireless messaging systems. Two popular wireless applications, the short messaging service and multimedia messaging service are considered. We develop a mathematical model to evaluate the performance of these systems taking into consideration the fact that each message tolerates only a limited amount of waiting time in the system. Using the model, closed-form expressions for critical performance parameters such as message loss, message delay and expiry probability are derived. Furthermore, a simple algorithm is presented to find the optimal temporary storage size that minimizes message delay for a given set of system parameters.

This thesis contributes toward the design of a quality-of-service (QoS) aware network layer for wireless ad hoc networks. With the lack of an infrastructure in ad hoc networks, the role of the network layer is not only to perform multihop routing between a source node and a destination node, but also to establish an end-to-end connection between communicating peers that satisfies the service level requirements of multimedia applications running on those peers. Wireless ad hoc networks represent autonomous distributed systems that are infrastructure-less, fully distributed, and multi-hop in nature. Over the last few years, wireless ad hoc networks have attracted significant attention from researchers. This has been fueled by recent technological advances in the development of multifunction and low-cost wireless communication gadgets. Wireless ad hoc networks have diverse applications spanning several domains, including military, commercial, medical, and home networks. Projections indicate that these self-organizing wireless ad hoc networks will eventually become the dominant form of the architecture of telecommunications networks in the near future. Recently, due to increasing popularity of multimedia applications, QoS support in wireless ad hoc networks has become an important yet challenging objective. The challenge lies in the need to support the heterogeneous QoS requirements (e.g., data rate, packet loss probability, and delay constraints) for multimedia applications and, at the same time, to achieve efficient radio resource utilization, taking into account user mobility and dynamics of multimedia traffic. In terms of research contributions, we first present a position-based QoS routing framework for wireless ad-hoc networks. The scheme provides QoS guarantee in terms of packet loss ratio and average end-to-end delay (or throughput) to ad hoc networks loaded with constant rate traffic. Via cross-layer design, we apply call admission control and temporary bandwidth reservation on discovered routes, taking into consideration the physical layer multi-rate capability and the medium access control (MAC) interactions such as simultaneous transmission and self interference from route members. Next, we address the network-layer resource allocation where a single-hop ad hoc network is loaded with random traffic. As a starting point, we study the behavior of the service process of the widely deployed IEEE 802.11 DCF MAC when the network is under different traffic load conditions. Our study investigates the near-memoryless behavior of the service time for IEEE 802.11 saturated single-hop ad hoc networks. We show that the number of packets successfully transmitted by any node over a time interval follows a general distribution, which is close to a Poisson distribution with an upper bounded distribution distance. We also show that the service time distribution can be approximated by the geometric distribution and illustrate that a simplified queuing system can be used efficiently as a resource allocation tool for single hop IEEE 802.11 ad hoc networks near saturation. After that, we shift our focus to providing probabilistic packet delay guarantee to multimedia users in non-saturated IEEE 802.11 single hop ad hoc networks. We propose a novel stochastic link-layer channel model to characterize the variations of the IEEE 802.11 channel service process. We use the model to calculate the effective capacity of the IEEE 802.11 channel. The channel effective capacity concept is the dual of the effective bandwidth theory. Our approach offers a tool for distributed statistical resource allocation in single hop ad hoc networks, which combines both efficient resource utilization and QoS provisioning to a certain probabilistic limit. Finally, we propose a statistical QoS routing scheme for multihop IEEE 802.11 ad hoc networks. Unlike most of QoS routing schemes in literature, the proposed scheme provides stochastic end-to-end delay guarantee, instead of average delay guarantee, to delay-sensitive bursty traffic sources. Via a cross-layer design approach, the scheme selects the routes based on a geographical on-demand ad hoc routing protocol and checks the availability of network resources by using traffic source and link-layer channel models, incorporating the IEEE 802.11 characteristics and interaction. Our scheme extends the well developed effective bandwidth theory and its dual effective capacity concept to multihop IEEE 802.11 ad hoc networks in order to achieve an efficient utilization of the shared radio channel while satisfying the end-to-end delay bound.

In this thesis, our goal is to achieve customer-specified performance objectives for workloads in a database management system (DBMS). Competing workloads in current DBMSs have detrimental effects on performance. Differentiated levels of service become important to ensure that critical work takes priority. We design a feedback-based admission differentiation framework, which consists of three components: workload classifier, workload monitor and adaptive admission controller. The adaptive admission controller uses the workload management capabilities of IBM DB2’s Workload Manager (WLM) to achieve the performance objectives of the most important workload by applying admission control on the rest of the work, which is less important and may or may not have performance objectives. The controller uses a feedback-based technique to automatically adjust the admission control on the less important work to achieve performance objectives for the important workload. The adaptive admission controller is implemented on an instance of DB2 to the test the effectiveness of the controller.

Due to the World Wide Web (WWW) has expanded speedily, the interaction between the user and the Web site is becoming persistently. Most proposed load control schemes developed based on the traditional Web site providing static files are not suitable for the Web site today. The new generation of WWW provides more varied and secure services. The greater part of applications are belonging to the session-based service. In order to provide better quality of services for the users, Web cluster architecture becomes the popular solution for most Web sites. This architecture combines serveral servers to work together and deals satisfied with the exponential increasing in service on WWW. In this paper, we proposed a session-based admission control and load balancing mechanism for Web cluster systems. The admission control scheme is used to prevent Web system from becoming overload by determining whether an establishment of TCP connection is admitted or not. The load balancing scheme assigns new sessions to the suitable back-end server and achieves good load balancing among back-end servers, which also increases the throughput of system. The simulation results demonstrate that our mechanism utilizes system resource efficiently and takes system utilization and percentage of completed session into account simultaneously. Therefore, our mechanism can ensure the sessions would not be aborted midway. To compare with others, our mechanism can attain higher throughput and maintain the request average delay time. Furthermore, whether the back-end servers are identical in capacity or not, their load always get good balance.

Because there has been a rapid development in wireless networks, it is important to provide quality-of-service (QoS) guarantees as they are expected to support multimedia applications. In this paper we propose a new bandwidth reservation scheme based on the characteristic of the cell equipped with sector antenna and 2-tier cell structure. According to this information, the proposed scheme can predict the next location of each connection and precisely reserve bandwidth in appropriate neighboring cells, not all of its neighboring cells. In addition, the proposed scheme incorporates bandwidth borrowing mechanism into call admission control strategy. The combination of bandwidth reservation and bandwidth borrowing provides network users with QoS in terms of guaranteed bandwidth, call blocking and call dropping probabilities.