Relay-assisted cooperative communications are promising solutions for error-performance improvement and cell coverage extension. In this thesis, we propose several efficient cooperative relay communication schemes. First, an efficient space-time coded cooperative relay communications scheme that employs linear precoding and transmission-pattern selection is proposed. This is built upon an existing block linear precoding technique for conventional multiple-input multiple-output systems in order to improve the diversity performance of a multihop relay network. Second, we consider several multiuser cooperative relay communication schemes employing layered modulations, such as hierarchical modulation and superposition coding. Conventional cooperative relay communication is effective in mitigating fading effects. However, additional resources, such as time slots or frequency bands are required for the relay, which reduce the overall throughput. Reduction of throughput will become more severe as the number of users increases. In order to overcome this limitation, multiuser cooperative relaying schemes that employ hierarchical modulation and superposition coding are proposed. These schemes exploit the superimposed message for users in the network and allow the system to transmit two or more independent data streams simultaneously. The proposed schemes do not require additional resources than the conventional schemes, while improving the error performance by flexibly controlling the power division coefficient of superposition coding or the distance parameter of hierarchical modulation. / Graduation date: 2012 / Access restricted to the OSU Community at author's request from Sept. 28, 2011 - Sept. 28, 2012