In the cooperative communications, the users relay each other¡¦s signal and thus forming multiple transmission paths to the destination and therefore the system can achieve spatial diversity gain. Decode-and-forward and amplify-and-forward are the most popular relaying strategies in the literature due to their simplicity. However, in practice, cooperative users acting as relays may not always normally operated or trustworthy. When the relay misbehavior is present in the cooperative networks, the communication performance may degrade dramatically and the users may be even better off without cooperation. Therefore, it is necessary for the destination to determine the misbehaving relays and to take appropriate actions to ensure that cooperative advantages are preserved. In this thesis, we focus on developing a misbehaving relay detection method to detect whether or not the system is in the presence of some misbehaving relays. After performing misbehaving relay detection, the destination removes the signals from the un- reliable paths and then uses maximal ratio combing to achieve spatial diversity. The simulation results conducted by the thesis show that the proposed method is more robust as compared with those without employing misbehaving relay detection when the system is in the presence of some misbehaving relays.

In the cooperative communications, the users relay each other's signal and thus form multiple transmission paths to the destination and therefore the system can achieve spatial diversity gain. Most studies in the literature assumed that cooperative users acting as the relays are normally operated and trustworthy. However, this may not always be true in practice. When the relay misbehaviors are present in the cooperative communications, the communication performance may degrade dramatically and the users may be even better off without cooperation. Therefore, it is necessary for the destination to determine the misbehaving relays and to take appropriate actions to ensure that cooperative advantages are preserved. This thesis considers both models in which the cooperative communications are with direct path (WDP) and without direct path (WODP). Utilizing the proposed Kolmogorov-Smirnov test mechanism, the destination identifies the misbehaving relays within the cooperative communications and then excludes their transmitting messages when performing the diversity combining to infer the symbols of interest sent by the source. In addition, this thesis provides the bit error rate (BER) analysis of the cooperative communications employing the proposed misbehaving relay detectors. The simulation results demonstrate that the proposed methods have robust performance when the relay misbehaviors are present in the cooperative communications.