Chei Kwok-hung. / Thesis (M.Phil.)--Chinese University of Hong Kong, 2000. / Includes bibliographical references (leaves 66-73). / Abstracts in English and Chinese. / Chapter Chapter 1 --- Introduction --- p.1 / Chapter 1.1 --- Typical Digital Communication Systems --- p.2 / Chapter 1.1.1 --- Source coding --- p.3 / Chapter 1.1.2 --- Channel coding --- p.5 / Chapter 1.2 --- Joint Source-Channel Coding System --- p.5 / Chapter 1.3 --- Thesis Organization --- p.7 / Chapter Chapter 2 --- Trellis Coding --- p.9 / Chapter 2.1 --- Convolutional Codes --- p.9 / Chapter 2.2 --- Trellis-Coded Modulation --- p.12 / Chapter 2.2.1 --- Set Partitioning --- p.13 / Chapter 2.3 --- Trellis-Coded Quantization --- p.14 / Chapter 2.4 --- Joint TCQ/TCM System --- p.17 / Chapter 2.4.1 --- The Combined Receiver --- p.17 / Chapter 2.4.2 --- Viterbi Decoding --- p.19 / Chapter 2.4.3 --- Sequence MAP Decoding --- p.20 / Chapter 2.4.4 --- Sliding Window Decoding --- p.21 / Chapter 2.4.5 --- Block-Based Decoding --- p.23 / Chapter Chapter 3 --- Soft Decoding Joint TCQ/TCM over AWGN Channel --- p.25 / Chapter 3.1 --- System Model --- p.26 / Chapter 3.2 --- TCQ with Optimal Soft-Decoder --- p.27 / Chapter 3.3 --- Gaussian Memoryless Source --- p.30 / Chapter 3.3.1 --- Theorem Limit --- p.31 / Chapter 3.3.2 --- Performance on PAM Constellations --- p.32 / Chapter 3.3.3 --- Performance on PSK Constellations --- p.36 / Chapter 3.4 --- Uniform Memoryless Source --- p.38 / Chapter 3.4.1 --- Theorem Limit --- p.38 / Chapter 3.4.2 --- Performance on PAM Constellations --- p.39 / Chapter 3.4.3 --- Performance on PSK Constellations --- p.40 / Chapter Chapter 4 --- Soft Decoding Joint TCQ/TCM System over Rayleigh Fading Channel --- p.42 / Chapter 4.1 --- Wireless Channel --- p.43 / Chapter 4.2 --- Rayleigh Fading Channel --- p.44 / Chapter 4.3 --- Idea Interleaving --- p.45 / Chapter 4.4 --- Receiver Structure --- p.46 / Chapter 4.5 --- Numerical Results --- p.47 / Chapter 4.5.1 --- Performance on 4-PAM Constellations --- p.48 / Chapter 4.5.2 --- Performance on 8-PAM Constellations --- p.50 / Chapter 4.5.3 --- Performance on 16-PAM Constellations --- p.52 / Chapter Chapter 5 --- Joint TCVQ/TCM System --- p.54 / Chapter 5.1 --- Trellis-Coded Vector Quantization --- p.55 / Chapter 5.1.1 --- Set Partitioning in TCVQ --- p.56 / Chapter 5.2 --- Joint TCVQ/TCM --- p.59 / Chapter 5.2.1 --- Set Partitioning and Index Assignments --- p.60 / Chapter 5.2.2 --- Gaussian-Markov Sources --- p.61 / Chapter 5.3 --- Simulation Results and Discussion --- p.62 / Chapter Chapter 6 --- Conclusion and Future Work --- p.64 / Chapter 6.1 --- Conclusion --- p.64 / Chapter 6.2 --- Future Works --- p.65 / Bibliography --- p.66 / Appendix-Publications --- p.73

Kwong Cheuk Fai. / Thesis (M.Phil.)--Chinese University of Hong Kong, 2001. / Includes bibliographical references (leaves 72-74). / Abstracts in English and Chinese. / Acknowledgements --- p.i / Abstract --- p.ii / Table of Contents --- p.iv / Chapter 1 --- Introduction --- p.1 / Chapter 1.1 --- Quantization --- p.2 / Chapter 1.2 --- Trellis-Coded Quantization --- p.3 / Chapter 1.3 --- Thesis Organization --- p.4 / Chapter 2 --- Trellis-Coded Modulation --- p.6 / Chapter 2.1 --- Convolutional Codes --- p.7 / Chapter 2.1.1 --- Generator Polynomials and Generator Matrix --- p.9 / Chapter 2.1.2 --- Circuit Diagram --- p.10 / Chapter 2.1.3 --- State Transition Diagram --- p.11 / Chapter 2.1.4 --- Trellis Diagram --- p.12 / Chapter 2.2 --- Trellis-Coded Modulation --- p.13 / Chapter 2.2.1 --- Uncoded Transmission verses TCM --- p.14 / Chapter 2.2.2 --- Trellis Representation --- p.17 / Chapter 2.2.3 --- Ungerboeck Codes --- p.18 / Chapter 2.2.4 --- Set Partitioning --- p.19 / Chapter 2.2.5 --- Decoding for TCM --- p.22 / Chapter 3 --- Trellis-Coded Quantization --- p.26 / Chapter 3.1 --- Scalar Trellis-Coded Quantization --- p.26 / Chapter 3.2 --- Trellis-Coded Vector Quantization --- p.31 / Chapter 3.2.1 --- Set Partitioning in TCVQ --- p.33 / Chapter 3.2.2 --- Codebook Optimization --- p.34 / Chapter 3.2.3 --- Numerical Data and Discussions --- p.35 / Chapter 4 --- Trellis-Coded Quantization with Unequal Distortion --- p.38 / Chapter 4.1 --- Design Procedures --- p.40 / Chapter 4.2 --- Fine and Coarse Codebooks --- p.41 / Chapter 4.3 --- Set Partitioning --- p.44 / Chapter 4.4 --- Codebook Optimization --- p.45 / Chapter 4.5 --- Decoding for Unequal Distortion TCVQ --- p.46 / Chapter 5 --- Unequal Distortion TCVQ on Memoryless Gaussian Source --- p.47 / Chapter 5.1 --- Memoryless Gaussian Source --- p.49 / Chapter 5.2 --- Set Partitioning of Codewords of Memoryless Gaussian Source --- p.49 / Chapter 5.3 --- Numerical Results and Discussions --- p.51 / Chapter 6 --- Unequal Distortion TCVQ on Markov Gaussian Source --- p.57 / Chapter 6.1 --- Markov Gaussian Source --- p.57 / Chapter 6.2 --- Set Partitioning of Codewords of Markov Gaussian Source --- p.58 / Chapter 6.3 --- Numerical Results and Discussions --- p.59 / Chapter 7 --- Conclusions --- p.70 / Bibliography --- p.72

Ho Man-Shing. / Thesis (M.Phil.)--Chinese University of Hong Kong, 2001. / Includes bibliographical references (leaves 65-66). / Abstracts in English and Chinese. / Acknowledgement --- p.i / Abstract --- p.ii / List of Abbreviation --- p.iii / Chapter 1. --- Introduction --- p.1 / Chapter 1.1 --- Digital Communication System --- p.3 / Chapter 1.2 --- Thesis Organization --- p.4 / Chapter 2. --- Error-Correcting Codes --- p.6 / Chapter 2.1 --- Convolutional Codes --- p.7 / Chapter 2.1.1 --- Generator Polynomials --- p.8 / Chapter 2.1.2 --- Generator Matrix --- p.9 / Chapter 2.1.3 --- Circuit Diagram --- p.10 / Chapter 2.1.4 --- State-transition Diagram --- p.11 / Chapter 2.1.4 --- Trellis Diagram --- p.12 / Chapter 2.1.5 --- Distance property --- p.13 / Chapter 2.2 --- Rate-Compatible Punctured Convolutional Codes --- p.14 / Chapter 2.3 --- Trellis-Coded Modulation --- p.17 / Chapter 2.3.1 --- General Model of TCM --- p.18 / Chapter 2.3.2 --- Trellis Representation --- p.20 / Chapter 2.3.3 --- Set Partitioning --- p.21 / Chapter 2.3.4 --- Code Modulation --- p.23 / Chapter 2.4 --- Decoding Algorithm --- p.25 / Chapter 2.4.1 --- Viterbi Algorithm --- p.27 / Chapter 2.4.2 --- List Viterbi Algorithm --- p.30 / Chapter 3. --- Unequal-Error-Protection for Embedded Image Coder --- p.33 / Chapter 3.1 --- SPIHT Coder --- p.35 / Chapter 3.1.1 --- Progressive Image Transmission --- p.36 / Chapter 3.1.2 --- Set Partitioning Sorting Algorithm --- p.37 / Chapter 3.1.3 --- Spatial Orientation Trees --- p.38 / Chapter 3.2 --- System Description --- p.39 / Chapter 3.3 --- Code Allocation --- p.41 / Chapter 3.4 --- System Complexity --- p.42 / Chapter 3.5 --- Simulation Result --- p.43 / Chapter 4. --- Unequal-Error-Protection Provided by Trellis-Coded Modulation --- p.51 / Chapter 4.1 --- System Description --- p.52 / Chapter 4.2 --- Unequal Constellation --- p.53 / Chapter 4.3 --- Free Distance --- p.55 / Chapter 4.4 --- Simulation Results --- p.59 / Chapter 5. --- Conclusion --- p.63 / Bibliography --- p.65

The two main goals of this research are to study the implementation aspects of space-time turbo trellis codes (ST Turbo TC) and to develop efficient simulation methods for space-time and turbo coded systems using the importance sampling (IS) technique. The design of ST Turbo TC for improving the bandwidth efficiency and the reliability of wireless communication networks, which is based on the turbo structure, has been proposed in the literature. To achieve memory savings and reduce the decoding delay, this thesis proposes a simplified ST Turbo TC decoder using a sliding window (SW) technique. Different window sizes are employed and investigated. Through computer simulation, the optimum window sizes are determined for various system configurations. The effect of finite word length representation on the performance of ST Turbo TC is then studied. Simulation results show that ST Turbo TC is feasible for finite word length representation without significant degradation in the frame error rate performance. The optimum word length configurations are defined for all quantities external and internal to the ST Turbo TC decoder. For complex communication systems such as space-time codes and turbo codes, computer simulation is in fact the useful approach to obtain the estimated performance. To overcome the lengthy run-time requirements of the conventional Monte-Carlo (MC) method, this thesis introduces importance sampling simulation methods that accurately estimate the performances of turbo codes and space-time codes including orthogonal space-time block codes (OSTBC) and concatenated OSTBC. It is demonstrated that the proposed methods require much smaller sample sizes to achieve the same accuracy required by a conventional MC estimator. / Doctor of Philosophy (PhD)

<p>The thesis is largely built on a collection of published and submitted papers where the main focus is to analyze and optimize single-carrier adaptive coded modulation systems with and without antenna diversity. Multidimensional trellis codes are used as component codes. The majority of the analysis is done with both estimation and prediction errors being incorporated. Both channel estimation and prediction are performed using a pilot-symbol-assisted modulation scheme. Thus, known pilot symbols (overhead information) must be transmitted; which consumes power and also degrades system spectral efficiency. Both power consumption and pilot insertion frequency are optimized such that they are kept at necessary values to maximize system throughput without sacrificing the error rate performance. The results show that efficient and reliable system performance can be achieved over a wide range of the considered average channel quality. Going from a single-input single-output system to both spatially uncorrelated and correlated single-input multiple-ouput (SIMO) systems, and further to an uncorrelated multiple-input multiple-output (MIMO) diversity system, is the evolution of the thesis. In the SIMO case, maximum ratio combining is used to combine the incoming signals, whereas the signals are space-time combined in the MIMO diversity system. The multiple-input single-output system comes out as a special case of a MIMO system. Besides the spatially uncorrelated antenna array, the effect of spatial correlation is also considered in the SIMO case. In this case, only prediction error is considered and channel estimation is assumed to be perfect. At first, the impact of spatial correlation in a predicted system originally designed to operate on uncorrelated channels is quanitifed. Then, a maximum a posteriori (MAP)-optimal “space-time predictor” is derived to take spatial correlation into account. As expected, the results show that the throughput is still lower than the uncorrelated system, but the degradation is decreased when the MAP-optimal space-time predictor is used. Thus, by exploiting the correlation properly, the degradation can be reduced. By numerical examples, we demonstrate the potential effect of limiting the predictor complexity, of fixing the pilot spacing, as well as of assuming perfect estimation. The two first simplifications imply lower system complexity and feedback rate, whereas the last assumption is usually made to ease the mathematical analysis. The numerical examples indicate that all the simplifications can be done without serious impact on the predicted system performance.</p>

The thesis is largely built on a collection of published and submitted papers where the main focus is to analyze and optimize single-carrier adaptive coded modulation systems with and without antenna diversity. Multidimensional trellis codes are used as component codes. The majority of the analysis is done with both estimation and prediction errors being incorporated. Both channel estimation and prediction are performed using a pilot-symbol-assisted modulation scheme. Thus, known pilot symbols (overhead information) must be transmitted; which consumes power and also degrades system spectral efficiency. Both power consumption and pilot insertion frequency are optimized such that they are kept at necessary values to maximize system throughput without sacrificing the error rate performance. The results show that efficient and reliable system performance can be achieved over a wide range of the considered average channel quality. Going from a single-input single-output system to both spatially uncorrelated and correlated single-input multiple-ouput (SIMO) systems, and further to an uncorrelated multiple-input multiple-output (MIMO) diversity system, is the evolution of the thesis. In the SIMO case, maximum ratio combining is used to combine the incoming signals, whereas the signals are space-time combined in the MIMO diversity system. The multiple-input single-output system comes out as a special case of a MIMO system. Besides the spatially uncorrelated antenna array, the effect of spatial correlation is also considered in the SIMO case. In this case, only prediction error is considered and channel estimation is assumed to be perfect. At first, the impact of spatial correlation in a predicted system originally designed to operate on uncorrelated channels is quanitifed. Then, a maximum a posteriori (MAP)-optimal “space-time predictor” is derived to take spatial correlation into account. As expected, the results show that the throughput is still lower than the uncorrelated system, but the degradation is decreased when the MAP-optimal space-time predictor is used. Thus, by exploiting the correlation properly, the degradation can be reduced. By numerical examples, we demonstrate the potential effect of limiting the predictor complexity, of fixing the pilot spacing, as well as of assuming perfect estimation. The two first simplifications imply lower system complexity and feedback rate, whereas the last assumption is usually made to ease the mathematical analysis. The numerical examples indicate that all the simplifications can be done without serious impact on the predicted system performance.

<p>This dissertation studies a state-of-the-art bandwidth-efficient coded modulation technique, known as bit interleaved coded modulation with iterative decoding (BICM-ID), together with various diversity techniques to dramatically improve the performance of digital communication systems over wireless channels.</p> <p>For BICM-ID over a single-antenna frequency non-selective fading channel, the problem of mapping over multiple symbols, i.e., multi-dimensional (multi-D) mapping, with 8-PSK constellation is investigated. An explicit algorithm to construct a good multi-D mapping of 8-PSK to improve the asymptotic performance of BICM-ID systems is introduced. By comparing the performance of the proposed mapping with an unachievable lower bound, it is conjectured that the proposed mapping is the global optimal mapping. The superiority of the proposed mapping over the best conventional (1-dimensional complex) mapping and the multi-D mapping found previously by computer search is thoroughly demonstrated.</p> <p>In addition to the mapping issue in single-antenna BICM-ID systems, the use of signal space diversity (SSD), also known as linear constellation precoding (LCP), is considered in BICM-ID over frequency non-selective fading channels. The performance analysis of BICM-ID and complex N-dimensional signal space diversity is carried out to study its performance limitation, the choice of the rotation matrix and the design of a low-complexity receiver. Based on the design criterion obtained from a tight error bound, the optimality of the rotation matrix is established. It is shown that using the class of optimal rotation matrices, the performance of BICM-ID systems over a frequency non-selective Rayleigh fading channel approaches that of the BICM-ID systems over an additive white Gaussian noise (AWGN) channel when the dimension of the signal constellation increases. Furthermore, by exploiting the sigma mapping for any M-ary quadrature amplitude modulation (QAM) constellation, a very simple sub-optimal, yet effective iterative receiver structure suitable for signal constellations with large dimensions is proposed. Simulation results in various cases and conditions indicate that the proposed receiver can achieve the analytical performance bounds with low complexity.</p> <p>The application of BICM-ID with SSD is then extended to the case of cascaded Rayleigh fading, which is more suitable to model mobile-to-mobile communication channels. By deriving the error bound on the asymptotic performance, it is first illustrated that for a small modulation constellation, a cascaded Rayleigh fading causes a much more severe performance degradation than a conventional Rayleigh fading. However, BICM-ID employing SSD with a sufficiently large constellation can close the performance gap between the Rayleigh and cascaded Rayleigh fading channels, and their performance can closely approach that over an AWGN channel.</p> <p>In the next step, the use of SSD in BICM-ID over frequency selective Rayleigh fading channels employing a multi-carrier modulation technique known as orthogonal frequency division multiplexing (OFDM) is studied. Under the assumption of correlated fading over subcarriers, a tight bound on the asymptotic error performance for the general case of applying SSD over all N subcarriers is derived and used to establish the best achievable asymptotic performance by SSD. It is then shown that precoding over subgroups of at least L subcarriers per group, where L is the number of channel taps, is sufficient to obtain this best asymptotic error performance, while significantly reducing the receiver complexity. The optimal joint subcarrier grouping and rotation matrix design is subsequently determined by solving the Vandermonde linear system. Illustrative examples show a good agreement between various analytical and simulation results.</p> <p>Further, by combining the ideas of multi-D mapping and subcarrier grouping, a novel power and bandwidth-efficient bit-interleaved coded modulation with OFDM and iterative decoding (BI-COFDM-ID) in which multi-D mapping is performed over a group of subcarriers for broadband transmission in a frequency selective fading environment is proposed. A tight bound on the asymptotic error performance is developed, which shows that subcarrier mapping and grouping have independent impacts on the overall error performance, and hence they can be independently optimized. Specifically, it is demonstrated that the optimal subcarrier mapping is similar to the optimal multi-D mapping for BICM-ID in frequency non-selective Rayleigh fading environment, whereas the optimal subcarrier grouping is the same with that of OFDM with SSD. Furthermore, analytical and simulation results show that the proposed system with the combined optimal subcarrier mapping and grouping can achieve the full channel diversity without using SSD and provide significant coding gains as compared to the previously studied BI-COFDM-ID with the same power, bandwidth and receiver complexity.</p> <p>Finally, the investigation is extended to the application of BICM-ID over a multiple-input multiple-output (MIMO) system equipped with multiple antennas at both the transmitter and the receiver to exploit both time and spatial diversities, where neither the transmitter nor the receiver knows the channel fading coefficients. The concentration is on the class of unitary constellation, due to its advantages in terms of both information-theoretic capacity and error probability. The tight error bound with respect to the asymptotic performance is also derived for any given unitary constellation and mapping rule. Design criteria regarding the choice of unitary constellation and mapping are then established. Furthermore, by using the unitary constellation obtained from orthogonal design with quadrature phase-shift keying (QPSK or 4-PSK) and 8-PSK, two different mapping rules are proposed. The first mapping rule gives the most suitable mapping for systems that do not implement iterative processing, which is similar to a Gray mapping in coherent channels. The second mapping rule yields the best mapping for systems with iterative decoding. Analytical and simulation results show that with the proposed mappings of the unitary constellations obtained from orthogonal designs, the asymptotic error performance of the iterative systems can closely approach a lower bound which is applicable to any unitary constellation and mapping.</p>

<p>This dissertation studies a state-of-the-art bandwidth-efficient coded modulation technique, known as bit interleaved coded modulation with iterative decoding (BICM-ID), together with various diversity techniques to dramatically improve the performance of digital communication systems over wireless channels.</p> <p>For BICM-ID over a single-antenna frequency non-selective fading channel, the problem of mapping over multiple symbols, i.e., multi-dimensional (multi-D) mapping, with 8-PSK constellation is investigated. An explicit algorithm to construct a good multi-D mapping of 8-PSK to improve the asymptotic performance of BICM-ID systems is introduced. By comparing the performance of the proposed mapping with an unachievable lower bound, it is conjectured that the proposed mapping is the global optimal mapping. The superiority of the proposed mapping over the best conventional (1-dimensional complex) mapping and the multi-D mapping found previously by computer search is thoroughly demonstrated.</p> <p>In addition to the mapping issue in single-antenna BICM-ID systems, the use of signal space diversity (SSD), also known as linear constellation precoding (LCP), is considered in BICM-ID over frequency non-selective fading channels. The performance analysis of BICM-ID and complex N-dimensional signal space diversity is carried out to study its performance limitation, the choice of the rotation matrix and the design of a low-complexity receiver. Based on the design criterion obtained from a tight error bound, the optimality of the rotation matrix is established. It is shown that using the class of optimal rotation matrices, the performance of BICM-ID systems over a frequency non-selective Rayleigh fading channel approaches that of the BICM-ID systems over an additive white Gaussian noise (AWGN) channel when the dimension of the signal constellation increases. Furthermore, by exploiting the sigma mapping for any M-ary quadrature amplitude modulation (QAM) constellation, a very simple sub-optimal, yet effective iterative receiver structure suitable for signal constellations with large dimensions is proposed. Simulation results in various cases and conditions indicate that the proposed receiver can achieve the analytical performance bounds with low complexity.</p> <p>The application of BICM-ID with SSD is then extended to the case of cascaded Rayleigh fading, which is more suitable to model mobile-to-mobile communication channels. By deriving the error bound on the asymptotic performance, it is first illustrated that for a small modulation constellation, a cascaded Rayleigh fading causes a much more severe performance degradation than a conventional Rayleigh fading. However, BICM-ID employing SSD with a sufficiently large constellation can close the performance gap between the Rayleigh and cascaded Rayleigh fading channels, and their performance can closely approach that over an AWGN channel.</p> <p>In the next step, the use of SSD in BICM-ID over frequency selective Rayleigh fading channels employing a multi-carrier modulation technique known as orthogonal frequency division multiplexing (OFDM) is studied. Under the assumption of correlated fading over subcarriers, a tight bound on the asymptotic error performance for the general case of applying SSD over all N subcarriers is derived and used to establish the best achievable asymptotic performance by SSD. It is then shown that precoding over subgroups of at least L subcarriers per group, where L is the number of channel taps, is sufficient to obtain this best asymptotic error performance, while significantly reducing the receiver complexity. The optimal joint subcarrier grouping and rotation matrix design is subsequently determined by solving the Vandermonde linear system. Illustrative examples show a good agreement between various analytical and simulation results.</p> <p>Further, by combining the ideas of multi-D mapping and subcarrier grouping, a novel power and bandwidth-efficient bit-interleaved coded modulation with OFDM and iterative decoding (BI-COFDM-ID) in which multi-D mapping is performed over a group of subcarriers for broadband transmission in a frequency selective fading environment is proposed. A tight bound on the asymptotic error performance is developed, which shows that subcarrier mapping and grouping have independent impacts on the overall error performance, and hence they can be independently optimized. Specifically, it is demonstrated that the optimal subcarrier mapping is similar to the optimal multi-D mapping for BICM-ID in frequency non-selective Rayleigh fading environment, whereas the optimal subcarrier grouping is the same with that of OFDM with SSD. Furthermore, analytical and simulation results show that the proposed system with the combined optimal subcarrier mapping and grouping can achieve the full channel diversity without using SSD and provide significant coding gains as compared to the previously studied BI-COFDM-ID with the same power, bandwidth and receiver complexity.</p> <p>Finally, the investigation is extended to the application of BICM-ID over a multiple-input multiple-output (MIMO) system equipped with multiple antennas at both the transmitter and the receiver to exploit both time and spatial diversities, where neither the transmitter nor the receiver knows the channel fading coefficients. The concentration is on the class of unitary constellation, due to its advantages in terms of both information-theoretic capacity and error probability. The tight error bound with respect to the asymptotic performance is also derived for any given unitary constellation and mapping rule. Design criteria regarding the choice of unitary constellation and mapping are then established. Furthermore, by using the unitary constellation obtained from orthogonal design with quadrature phase-shift keying (QPSK or 4-PSK) and 8-PSK, two different mapping rules are proposed. The first mapping rule gives the most suitable mapping for systems that do not implement iterative processing, which is similar to a Gray mapping in coherent channels. The second mapping rule yields the best mapping for systems with iterative decoding. Analytical and simulation results show that with the proposed mappings of the unitary constellations obtained from orthogonal designs, the asymptotic error performance of the iterative systems can closely approach a lower bound which is applicable to any unitary constellation and mapping.</p>

Recent attempts to increase the capacity of the compact disc (CD) and digital versatile disc (DVD) have explored the use of multilevel recording instead of binary recording. Systems that achieve an increase in capacity of about three times that of conventional CD have been proposed for production. Marks in these systems are multilevel and fixed-length as opposed to binary and variable length in CD and DVD. The main objective of this work is to evaluate the performance of multilevel ($M$-ary) runlength-limited (RLL) coded sequences in optical data storage. First, the waterfilling capacity of a multilevel optical recording channel ($M$-ary ORC) is derived and evaluated. This provides insight into the achievable user bit densities, as well as a theoretical limit against which simulated systems can be compared. Then, we evaluate the performance of RLL codes on the $M$-ary ORC. A new channel model that includes the runlength constraint in the transmitted signal is used. We compare the performance of specific RLL codes, namely $M$-ary permutation codes, to that of real systems using multilevel fixed-length marks for recording and the theoretical limits. The Viterbi detector is used to estimate the original recorded symbols from the readout signal. Then, error correction is used to reduce the symbol error probability. We use a combined ECC/RLL code for phrase encoding. We evaluate the use of trellis coded modulation (TCM) for amplitude encoding. The detection of the readout signal is also studied. A post-processing algorithm for the Viterbi detector is introduced, which ensures that the detected word satisfies the code constraints. Specifying the codes and detector for the $M$-ary ORC gives a complete system whose performance can be compared to that of the recently developed systems found in the literature and the theoretical limits calculated in this research.

The continuous phase frequency shift keying (CPFSK) is a modulation method with memory. The memory results from the phase continuity of the transmitted carrier phase from one signal interval to the next. For a specific form of phase, CPFSK becomes a special case of a general class of continuous phase modulation (CPM) signals. In this thesis, we extend the decomposition model of single-h CPM to the multi-h CPM decomposition model. Based on this decomposition model approach the multi-h CPFSK schemes are evaluated by searching the desired multi-h phase codes at a given number of states. Moreover, the trellis coded multi-h CPFSK schemes, which are the combination of the (binary) convolutional codes with the multi-h CPFSK schemes, are searching by optimization procedure via the matched encoding method. To further improve the performance, in terms of the coding gain, the ring convolutional codes are applied to the continuous phase encoder (CPE) of the proposed multi-h CPFSK schemes. Due to the fact that the code structure of the ring convolutional codes is similar to the CPE, this will result in having simple and efficient combination of the convolutional codes with the multi-h CPFSK signaling schemes.