In this paper, the channel estimation problem for a two-way relay network (TWRN) based on two different wireless channel assumptions is considered. Previous works have proposed a training-based channel estimation method to obtain the channel state information (CSI). But in practice the channel change from one data block to another, which may cause the performance degradation due to the outdated CSI. To enhance the performance, the system has to insert more training signal. In order to improve the bandwidth efficiency, we propose a joint channel estimation and data detection method based on expectation-maximization (EM) algorithm. From the simulation results, the proposed method can combat the effect of fading channel and still the MSE results are very close to Cramer-Rao Lower Bound (CRLB) at the high signal-to-noise ratio (SNR) region. Additionally, as compare with the previous work, the proposed scheme also has a better detection performance for both time-varying and time-invariant channels.

This thesis proposes resource allocation algorithms for multi-input multi-output (MIMO) relay and Scalable H.264/AVC video transmission over cooperative communication networks. For MIMO relay, we explore the reception diversity with maximal ratio combining (MRC) and transmission diversity with space-time block codes (STBC) respectively. Then, a condition is proposed to maximize the overall output signal-to-noise ratio (SNR). In this condition, the ineffective relays will be excluded in sequence from the cooperation. Simulation results indicate that the effect of bit error rate (BER) through the relay selection is similar to the scheme which applies all relays, but the amounts of used relay decreased. For Scalable H.264/AVC video, by introducing frame significance analysis, the video quality dependency between coding frame and its references is investigated for temporal layers and quality layers. The proposed algorithm allocates the relay and sub-band to each layer based on channel conditions and the priority of classified video packets. Experimental results indicate that the proposed algorithm is superior to the temporal-based allocation and quality-based allocation cooperative schemes.