Energy consumption has recently become a key issue from both environmental and economic considerations. A typical mobile phone network in the UK may consume approximately 40-50 MW, contributing a significant proportion of the total energy consumed by the information technology industry. With the worldwide growth in the number of mobile subscribers, the associated carbon emissions and growing energy costs are becoming a significant operational expense, leading to the need for energy reduction. The Mobile VCE Green Radio Project has been launched, which targets to achieve 100x energy reduction of the current wireless networks by 2020. In this thesis, energy-efficient resource allocation strategies have been investigated taking the LTE system as an example. Firstly, theoretical analysis of energy-efficient design in cellular environments is provided according to the Shannon Theory. Based on a two-link scenario the performance of simultaneous transmission and orthogonal transmission for network power minimization under the specified rate constraints is investigated. It is found that simultaneous transmission consumes less power than orthogonal transmission close to the base station, but much more power in the cell-edge area. Also, simulation results suggest that the energy-efficient switching margins between these two schemes are dominated by the sum total of their required data rates. New definitions of power-utility and fairness metrics are further proposed, following by the design of weighted resource allocation approaches based on efficiency-fairness trade-offs. Apart from energy-efficient multiple access between different links, the energy used by individual base stations can also be reduced. For example, deploying sleep modes is an effective approach to reduce radio base station operational energy consumption. By periodically switching off the base station transmission, or using fewer transmit antennas, the energy consumption of base station hardware may decrease. By delivering less control signalling overhead, the radio frequency energy consumption can also be reduced. Simulation results suggest that up to 90% energy reduction can be obtained in low traffic conditions by employing time-domain optimization in each radio frame. The optimum on/off duty cycle is derived, enabling the energy consumption of the base station to scale with traffic loads. In the spatial-domain, an antenna selection criterion is proposed, indicating the most energy-efficient antenna configuration with the knowledge of users’ locations and quality of service requirements. Without time-domain sleep modes, using fewer transmit antennas could outperform full antenna transmission. However, with time-domain sleep modes, using all available antennas is generally the most energy-efficient choice.

Wireless access networks today consume 0.5 percent of the global energy. Rapidly growing demand for capacity will further increase the energy consumption. Thus, improving energy efficiency has a great importance not only for environmental awareness but also to lower the operational cost of network operators. However, current networks which are optimized based on non-energy related objectives introduce challenges towards green wireless access networks. In this thesis we investigate the solutions at the deployment level and handle energy efficiency assessment issues in wireless access networks. The precise characterization of the power consumption of the whole network has a crucial importance in order to obtain consistent conclusions from any proposed solution at the network level. For this purpose, we propose a novel power consumption model  considering  the impact of backhaul for two established technologies, i.e., fiber and microwave, which is often ignored in the literature. We show that there is a tradeoff between the power saved by using low power base stations and the excess power that has to be spent for backhauling their traffic which therefore needs to carefully be included into energy efficiency analysis. Furthermore, among the solutions that are analyzed, fiber-based backhaul solution is identified to outperform microwave regardless of the considered topology. The proposed model is then used to gain a general insight regarding the important design parameters and their possible impact on energy- and cost oriented network design. To this end, we present a  high-level framework to see the main tradeoffs between energy, infrastructure cost, spectrum and show that future high-capacity systems are increasingly limited by infrastructure and energy costs where spectrum has a strong positive impact on both. We then investigate different network deployment strategies to improve the energy efficiency where we focus on the impact of various base station types, cell size, power consumption parameters and the capacity demand. We propose a refined power consumption model where the parameters are determined in accordance with cell size. We show that network densification can only be justified when capacity expansion is anticipated and over-provisioning of the network is not plausible for greener network. The improvement through heterogeneous networks is indicated to be highly related to the traffic demand where up to 30% improvement is feasible for high area throughput targets. Furthermore, we consider the problem of energy efficiency assessment at the network level in order to allow operators to know their current status and quantify the potential energy savings of different solutions to establish future strategies. We propose elaborate metric forms that can characterize the efficiency and a methodology that indicate how to perform a reliable and accurate measurement considering the complexity of wireless networks. We show the weakness of the current metrics reporting the "effectiveness" and how these might indicate disputable improvement directions unless they are properly revised. This illustrates the need for a standardized network level energy efficiency evaluation methodology towards green wireless access. / <p>QC 20121109</p> / Energy-efficient wireless networking (eWIN)

This thesis proposes four novel techniques to solve the problem of growing energy consumption requirements in cellular communication networks. The first and second part of this work propose a novel energy efficient scheduling mechanism and two new bandwidth management techniques, while the third part provides an algorithm to actively manage the power state of base stations (BSs) so that energy consumption is minimized throughout the day while users suffer a minimal loss in achieved data rate performance within the system. The proposed energy efficient score based scheduler (EESBS) is based on the already existing principle of score based resource allocation. Resource blocks (RBs) are given scores based on their energy efficiency for every user and then their allocation is decided based on a comparison between the scores of the different users on each RB. Two additional techniques are introduced that allow the scheduler to manage the user’s bandwidth footprint or in other words the number of RBs allocated. The first one, bandwidth expansion mode (BEM), allows users to expand their bandwidth footprint while retaining their overall transmission data rate. This allows the system to save energy due to the fact that data rate scales linearly with bandwidth and only logarithmically with transmission power. The second technique, time compression mode (TCoM), is targeted at users whose energy consumption is dominated by signalling overhead transmissions. If the assumption is made that the overhead is proportional to the number of RBs allocated, then users who find themselves having low data rate demands can release some of their allocated RBs by using a higher order modulation on the remaining ones and thus reduce their overall energy expenditure. Moreover, a system that combines all of the aforementioned scheduling techniques is also discussed. Both theoretical and simulation results on the performance of the described systems are provided. The energy efficient hardware state control (EESC) algorithm works by first collecting statistical information about the loading of each BS during the day that is due to the particular mobility patterns of users. It then uses that information to allow the BSs to turn off for parts of the day when the expected load is low and they can offload their current users to nearby cell sites. Simplified theoretical, along with complete system computer simulation, results are included. All the algorithms presented are very straightforward to implement and are not computationally intensive. They provide significant energy consumption reductions at none to minimal cost in terms of experienced user data rate.

Dans cette thèse nous nous intéressons au développement d'une méthode de prise de décision pour un équipement de réception de Radio Intelligente qui s’adapte dynamiquement à son environnement. L'approche que nous adoptons est basée sur la modélisation statistique de l'environnement radio. En caractérisant statistiquement les observations fournies par les capteurs de l'environnement, nous mettons en place des règles de décisions statistiques qui prennent en considération les erreurs d'observation des métriques radio, ce qui contribue à minimiser les taux des décisions erronées. Nous visons aussi à travers cette thèse à utiliser les capacités intelligentes de prise de décision pour contribuer à la réduction de la complexité de calcul au niveau de l'équipement de réception. En effet, nous identifions des scénarios de prise de décision de reconfiguration qui limitent la présence de certains composants ou fonctions de la chaîne de réception. En particulier, nous traitons, deux scénarios de décision qui adaptent respectivement la présence des fonctions d’égalisation et du beamforming en réception. La limitation de ces deux opérations contribue à la réduction de la complexité de calcul au niveau de la chaîne de réception sans dégrader ses performances. Enfin, nous intégrons notre méthode de décision par modélisation statistique ainsi que les deux scénarios de décision traités dans une architecture de gestion d'une radio intelligente, afin de mettre en valeur le contrôle de l'intelligence et de la reconfiguration dans un équipement radio. / In this thesis we focus on the development of a decision making method for the cognitive radio receiver that dynamically adapts to its environment. The approach that we use is based on the statistical modeling of the radio environment. By statistically characterizing the observations provided by the radio sensor, we set up statistical decision rules that take into account the observations’ errors. This helps to minimize the rate of bad decisions. Also, we aim to use the intelligent capacities to reduce the computational complexity in the receiver chain. Indeed, we identify decision scenarios that limit some operators. In particular, we address two decision scenarios that adapt the presence of the equalization and of the beamforming to the environment. The limitation of these two operations helps to reduce the computational complexity in reception. Finally, we integrate our decision method and the two decision scenarios in a management architecture of reconfiguration and intelligence.

Pour supporter l’évolution constante des standards de communication numérique, du GSM vers la 5G, les équipements de communication doivent continuellement s’adapter. Face à l’utilisation croissante de l’internet, on assiste à une explosion du trafic de données, ce qui augmente la consommation d'énergie des appareils de communication sans fil et conduit donc à un impact significatif sur les émissions mondiales de CO2. De plus en plus de recherches se sont concentrées sur l'efficacité énergétique de la communication sans fil. La radio Intelligente, ou Cognitive Radio (CR), est considérée comme une technologie pertinente pour les communications radio vertes en raison de sa capacité à adapter son comportement à son environnement. Sur la base de métriques fournissant suffisamment d'informations sur l'état de fonctionnement du système, une décision optimale peut être effectuée en vue d'une action de reconfiguration, dans le but de réduire au minimum la dissipation d'énergie tout en ne compromettant pas les performances. Par conséquent, tout équipement intelligent doit disposer d’une architecture de gestion de la reconfiguration. Nous avons retenu l’architecture HDCRAM (Hierarchical and Distributed Cognitive Radio Architecture Management), développée dans notre équipe, et nous l’avons déployée sur des plates-formes hétérogènes. L'un des objectifs est d'améliorer l'efficacité énergétique par la mise en œuvre de l’architecture HDCRAM. Nous l’avons appliquée à un système OFDM simplifié pour illustrer comment HDCRAM permet de gérer efficacement le système et son adaptation à un environnement évolutif. / As the digital communication systems evolve from GSM and now toward 5G, the supported standards are also growing. The desired communication equipments are required to support different standards in a single device at the same time. And more and more wireless Internet services have been being provided resulting in the explosive growth in data traffic, which increase the energy consumption of the communication devices thus leads to significant impact on global CO2 emission. More and more researches have focused on the energy efficiency of wireless communication. Cognitive Radio (CR) has been considered as an enabling technology for green radio communications due to its ability to adapt its behavior to the changing environment. In order to efficiently manage the sensing information and the reconfiguration of a cognitive equipment, it is essential, first of all, to gather the necessary metrics so as to provide enough information about the operating condition thus helping decision making. Then, on the basis of the metrics obtained, an optimal decision can be made and is followed by a reconfiguration action, whose aim is to minimize the power dissipation while not compromising on performance. Therefore, a management architecture is necessary to be added into the cognitive equipment acting as a glue to realize the CR capabilities. We introduce a management architecture, namely Hierarchical and Distributed Cognitive Radio Architecture Management (HDCRAM), which has been proposed for CR management by our team. This work focuses on the implementation of HDCRAM on heterogeneous platforms. One of the objectives is to improve the energy efficiency by the management of HDCRAM. And an example of a simplified OFDM system is used to explain how HDCRAM works to efficiently manage the system to adapt to the changing environment.

Pour supporter l’évolution constante des standards de communication numérique, du GSM vers la 5G, les équipements de communication doivent continuellement s’adapter. Face à l’utilisation croissante de l’internet, on assiste à une explosion du trafic de données, ce qui augmente la consommation d'énergie des appareils de communication sans fil et conduit donc à un impact significatif sur les émissions mondiales de CO2. De plus en plus de recherches se sont concentrées sur l'efficacité énergétique de la communication sans fil. La radio Intelligente, ou Cognitive Radio (CR), est considérée comme une technologie pertinente pour les communications radio vertes en raison de sa capacité à adapter son comportement à son environnement. Sur la base de métriques fournissant suffisamment d'informations sur l'état de fonctionnement du système, une décision optimale peut être effectuée en vue d'une action de reconfiguration, dans le but de réduire au minimum la dissipation d'énergie tout en ne compromettant pas les performances. Par conséquent, tout équipement intelligent doit disposer d’une architecture de gestion de la reconfiguration. Nous avons retenu l’architecture HDCRAM (Hierarchical and Distributed Cognitive Radio Architecture Management), développée dans notre équipe, et nous l’avons déployée sur des plates-formes hétérogènes. L'un des objectifs est d'améliorer l'efficacité énergétique par la mise en œuvre de l’architecture HDCRAM. Nous l’avons appliquée à un système OFDM simplifié pour illustrer comment HDCRAM permet de gérer efficacement le système et son adaptation à un environnement évolutif. / As the digital communication systems evolve from GSM and now toward 5G, the supported standards are also growing. The desired communication equipments are required to support different standards in a single device at the same time. And more and more wireless Internet services have been being provided resulting in the explosive growth in data traffic, which increase the energy consumption of the communication devices thus leads to significant impact on global CO2 emission. More and more researches have focused on the energy efficiency of wireless communication. Cognitive Radio (CR) has been considered as an enabling technology for green radio communications due to its ability to adapt its behavior to the changing environment. In order to efficiently manage the sensing information and the reconfiguration of a cognitive equipment, it is essential, first of all, to gather the necessary metrics so as to provide enough information about the operating condition thus helping decision making. Then, on the basis of the metrics obtained, an optimal decision can be made and is followed by a reconfiguration action, whose aim is to minimize the power dissipation while not compromising on performance. Therefore, a management architecture is necessary to be added into the cognitive equipment acting as a glue to realize the CR capabilities. We introduce a management architecture, namely Hierarchical and Distributed Cognitive Radio Architecture Management (HDCRAM), which has been proposed for CR management by our team. This work focuses on the implementation of HDCRAM on heterogeneous platforms. One of the objectives is to improve the energy efficiency by the management of HDCRAM. And an example of a simplified OFDM system is used to explain how HDCRAM works to efficiently manage the system to adapt to the changing environment.

Cette thèse étudie les techniques de gestion intelligente du spectre et de topologie des réseaux via une approche radio intelligente dans le but d’améliorer leur capacité, leur qualité de service (QoS – Quality of Service) et leur consommation énergétique. Les techniques d’apprentissage par renforcement y sont utilisées dans le but d’améliorer les performances d’un système radio intelligent. Dans ce manuscrit, nous traitons du problème d’accès opportuniste au spectre dans le cas de réseaux intelligents sans infrastructure. Nous nous plaçons dans le cas où aucune information n’est échangée entre les utilisateurs secondaires (pour éviter les surcoûts en transmissions). Ce problème particulier est modélisé par une approche dite de bandits manchots « restless » markoviens multi-utilisateurs (multi-user restless Markov MAB -multi¬armed bandit). La contribution principale de cette thèse propose une stratégie d’apprentissage multi-joueurs qui prend en compte non seulement le critère de disponibilité des canaux (comme déjà étudié dans la littérature et une thèse précédente au laboratoire), mais aussi une métrique de qualité, comme par exemple le niveau d’interférence mesuré (sensing) dans un canal (perturbations issues des canaux adjacents ou de signaux distants). Nous prouvons que notre stratégie, RQoS-UCB distribuée (distributed restless QoS-UCB – Upper Confidence Bound), est quasi optimale car on obtient des performances au moins d’ordre logarithmique sur son regret. En outre, nous montrons par des simulations que les performances du système intelligent proposé sont améliorées significativement par l’utilisation de la solution d’apprentissage proposée permettant à l’utilisateur secondaire d’identifier plus efficacement les ressources fréquentielles les plus disponibles et de meilleure qualité. Cette thèse propose également un nouveau modèle d’apprentissage par renforcement combiné à un transfert de connaissance afin d’améliorer l’efficacité énergétique (EE) des réseaux cellulaires hétérogènes. Nous formulons et résolvons un problème de maximisation de l’EE pour le cas de stations de base (BS – Base Stations) dynamiquement éteintes et allumées (ON-OFF). Ce problème d’optimisation combinatoire peut aussi être modélisé par des bandits manchots « restless » markoviens. Par ailleurs, une gestion dynamique de la topologie des réseaux hétérogènes, utilisant l’algorithme RQoS-UCB, est proposée pour contrôler intelligemment le mode de fonctionnement ON-OFF des BS, dans un contexte de trafic et d’étude de capacité multi-cellulaires. Enfin une méthode incluant le transfert de connaissance « transfer RQoS-UCB » est proposée et validée par des simulations, pour pallier les pertes de récompense initiales et accélérer le processus d’apprentissage, grâce à la connaissance acquise à d’autres périodes temporelles correspondantes à la période courante (même heure de la journée la veille, ou même jour de la semaine par exemple). La solution proposée de gestion dynamique du mode ON-OFF des BS permet de diminuer le nombre de BS actives tout en garantissant une QoS adéquate en atténuant les fluctuations de la QoS lors des variations du trafic et en améliorant les conditions au démarrage de l’apprentissage. Ainsi, l’efficacité énergétique est grandement améliorée. Enfin des démonstrateurs en conditions radio réelles ont été développés pour valider les solutions d’apprentissage étudiées. Les algorithmes ont également été confrontés à des bases de données de mesures effectuées par un partenaire dans la gamme de fréquence HF, pour des liaisons transhorizon. Les résultats confirment la pertinence des solutions d’apprentissage proposées, aussi bien en termes d’optimisation de l’utilisation du spectre fréquentiel, qu’en termes d’efficacité énergétique. / Future cellular network technologies are targeted at delivering self-organizable and ultra-high capacity networks, while reducing their energy consumption. This thesis studies intelligent spectrum and topology management through cognitive radio techniques to improve the capacity density and Quality of Service (QoS) as well as to reduce the cooperation overhead and energy consumption. This thesis investigates how reinforcement learning can be used to improve the performance of a cognitive radio system. In this dissertation, we deal with the problem of opportunistic spectrum access in infrastructureless cognitive networks. We assume that there is no information exchange between users, and they have no knowledge of channel statistics and other user's actions. This particular problem is designed as multi-user restless Markov multi-armed bandit framework, in which multiple users collect a priori unknown reward by selecting a channel. The main contribution of the dissertation is to propose a learning policy for distributed users, that takes into account not only the availability criterion of a band but also a quality metric linked to the interference power from the neighboring cells experienced on the sensed band. We also prove that the policy, named distributed restless QoS-UCB (RQoS-UCB), achieves at most logarithmic order regret. Moreover, numerical studies show that the performance of the cognitive radio system can be significantly enhanced by utilizing proposed learning policies since the cognitive devices are able to identify the appropriate resources more efficiently. This dissertation also introduces a reinforcement learning and transfer learning frameworks to improve the energy efficiency (EE) of the heterogeneous cellular network. Specifically, we formulate and solve an energy efficiency maximization problem pertaining to dynamic base stations (BS) switching operation, which is identified as a combinatorial learning problem, with restless Markov multi-armed bandit framework. Furthermore, a dynamic topology management using the previously defined algorithm, RQoS-UCB, is introduced to intelligently control the working modes of BSs, based on traffic load and capacity in multiple cells. Moreover, to cope with initial reward loss and to speed up the learning process, a transfer RQoS-UCB policy, which benefits from the transferred knowledge observed in historical periods, is proposed and provably converges. Then, proposed dynamic BS switching operation is demonstrated to reduce the number of activated BSs while maintaining an adequate QoS. Extensive numerical simulations demonstrate that the transfer learning significantly reduces the QoS fluctuation during traffic variation, and it also contributes to a performance jump-start and presents significant EE improvement under various practical traffic load profiles. Finally, a proof-of-concept is developed to verify the performance of proposed learning policies on a real radio environment and real measurement database of HF band. Results show that proposed multi-armed bandit learning policies using dual criterion (e.g. availability and quality) optimization for opportunistic spectrum access is not only superior in terms of spectrum utilization but also energy efficient.

Cette thèse étudie les techniques de gestion intelligente du spectre et de topologie des réseaux via une approche radio intelligente dans le but d’améliorer leur capacité, leur qualité de service (QoS – Quality of Service) et leur consommation énergétique. Les techniques d’apprentissage par renforcement y sont utilisées dans le but d’améliorer les performances d’un système radio intelligent. Dans ce manuscrit, nous traitons du problème d’accès opportuniste au spectre dans le cas de réseaux intelligents sans infrastructure. Nous nous plaçons dans le cas où aucune information n’est échangée entre les utilisateurs secondaires (pour éviter les surcoûts en transmissions). Ce problème particulier est modélisé par une approche dite de bandits manchots « restless » markoviens multi-utilisateurs (multi-user restless Markov MAB -multi¬armed bandit). La contribution principale de cette thèse propose une stratégie d’apprentissage multi-joueurs qui prend en compte non seulement le critère de disponibilité des canaux (comme déjà étudié dans la littérature et une thèse précédente au laboratoire), mais aussi une métrique de qualité, comme par exemple le niveau d’interférence mesuré (sensing) dans un canal (perturbations issues des canaux adjacents ou de signaux distants). Nous prouvons que notre stratégie, RQoS-UCB distribuée (distributed restless QoS-UCB – Upper Confidence Bound), est quasi optimale car on obtient des performances au moins d’ordre logarithmique sur son regret. En outre, nous montrons par des simulations que les performances du système intelligent proposé sont améliorées significativement par l’utilisation de la solution d’apprentissage proposée permettant à l’utilisateur secondaire d’identifier plus efficacement les ressources fréquentielles les plus disponibles et de meilleure qualité. Cette thèse propose également un nouveau modèle d’apprentissage par renforcement combiné à un transfert de connaissance afin d’améliorer l’efficacité énergétique (EE) des réseaux cellulaires hétérogènes. Nous formulons et résolvons un problème de maximisation de l’EE pour le cas de stations de base (BS – Base Stations) dynamiquement éteintes et allumées (ON-OFF). Ce problème d’optimisation combinatoire peut aussi être modélisé par des bandits manchots « restless » markoviens. Par ailleurs, une gestion dynamique de la topologie des réseaux hétérogènes, utilisant l’algorithme RQoS-UCB, est proposée pour contrôler intelligemment le mode de fonctionnement ON-OFF des BS, dans un contexte de trafic et d’étude de capacité multi-cellulaires. Enfin une méthode incluant le transfert de connaissance « transfer RQoS-UCB » est proposée et validée par des simulations, pour pallier les pertes de récompense initiales et accélérer le processus d’apprentissage, grâce à la connaissance acquise à d’autres périodes temporelles correspondantes à la période courante (même heure de la journée la veille, ou même jour de la semaine par exemple). La solution proposée de gestion dynamique du mode ON-OFF des BS permet de diminuer le nombre de BS actives tout en garantissant une QoS adéquate en atténuant les fluctuations de la QoS lors des variations du trafic et en améliorant les conditions au démarrage de l’apprentissage. Ainsi, l’efficacité énergétique est grandement améliorée. Enfin des démonstrateurs en conditions radio réelles ont été développés pour valider les solutions d’apprentissage étudiées. Les algorithmes ont également été confrontés à des bases de données de mesures effectuées par un partenaire dans la gamme de fréquence HF, pour des liaisons transhorizon. Les résultats confirment la pertinence des solutions d’apprentissage proposées, aussi bien en termes d’optimisation de l’utilisation du spectre fréquentiel, qu’en termes d’efficacité énergétique. / Future cellular network technologies are targeted at delivering self-organizable and ultra-high capacity networks, while reducing their energy consumption. This thesis studies intelligent spectrum and topology management through cognitive radio techniques to improve the capacity density and Quality of Service (QoS) as well as to reduce the cooperation overhead and energy consumption. This thesis investigates how reinforcement learning can be used to improve the performance of a cognitive radio system. In this dissertation, we deal with the problem of opportunistic spectrum access in infrastructureless cognitive networks. We assume that there is no information exchange between users, and they have no knowledge of channel statistics and other user's actions. This particular problem is designed as multi-user restless Markov multi-armed bandit framework, in which multiple users collect a priori unknown reward by selecting a channel. The main contribution of the dissertation is to propose a learning policy for distributed users, that takes into account not only the availability criterion of a band but also a quality metric linked to the interference power from the neighboring cells experienced on the sensed band. We also prove that the policy, named distributed restless QoS-UCB (RQoS-UCB), achieves at most logarithmic order regret. Moreover, numerical studies show that the performance of the cognitive radio system can be significantly enhanced by utilizing proposed learning policies since the cognitive devices are able to identify the appropriate resources more efficiently. This dissertation also introduces a reinforcement learning and transfer learning frameworks to improve the energy efficiency (EE) of the heterogeneous cellular network. Specifically, we formulate and solve an energy efficiency maximization problem pertaining to dynamic base stations (BS) switching operation, which is identified as a combinatorial learning problem, with restless Markov multi-armed bandit framework. Furthermore, a dynamic topology management using the previously defined algorithm, RQoS-UCB, is introduced to intelligently control the working modes of BSs, based on traffic load and capacity in multiple cells. Moreover, to cope with initial reward loss and to speed up the learning process, a transfer RQoS-UCB policy, which benefits from the transferred knowledge observed in historical periods, is proposed and provably converges. Then, proposed dynamic BS switching operation is demonstrated to reduce the number of activated BSs while maintaining an adequate QoS. Extensive numerical simulations demonstrate that the transfer learning significantly reduces the QoS fluctuation during traffic variation, and it also contributes to a performance jump-start and presents significant EE improvement under various practical traffic load profiles. Finally, a proof-of-concept is developed to verify the performance of proposed learning policies on a real radio environment and real measurement database of HF band. Results show that proposed multi-armed bandit learning policies using dual criterion (e.g. availability and quality) optimization for opportunistic spectrum access is not only superior in terms of spectrum utilization but also energy efficient.