Long-haul optical fiber communication system is an important infrastructure to support the latest broadband communication in the world. It is important to study a technology to improve the performance of such system, and the Return-to-Zero Differential Phase Shift Keying (RZ-DPSK) modulation attracts much attention because of its improved long distance transmission performance. One important technology of the current long-haul optical fiber communication system is the dispersion map, and it is widely deployed for already installed undersea optical fiber communication system in the world. Recently, a new dispersion map that was totally different from the map used for already deployed system was proposed, and it demonstrated advantageous performance of the long-haul RZ-DPSK transmission. Even though, the reason of the performance improvement is not investigated, and it is important to clarify the physical mechanism of the performance improvement, because it will contribute to improve the system design of the long-haul optical fiber communication systems in near future. In this master thesis, the performance of the RZ-DPSK format in the long-haul transmission system is studied. Both computer simulations and experiments are conducted to confirm the effects of various factors in the long-haul RZ-DPSK transmission system. From the theoretical study, it is pointed out that the Self-Phase Modulation (SPM) played a significant role to degrade the transmission performance of the conventional map, while it does not cause so significant degradation in the new map. The effects of the SPM and the Cross-Phase Modulation (XPM) with the conventional map are investigated through the experimental study.

Long-haul optical fiber communication system is an important technology to support the latest broadband communication in the world, and there is strong competition in optical long-haul transmission to achieve high channel bit rates and large transmission capacity. Therefore, it is important to study a technology to improve the performance of such system. As we have already known, return-to-zero differential phase shift keying (RZ-DPSK) is an attractive solution to improve the long distance transmission system performance compared to the conventional on-off keying (OOK) in a 10-Gb/s system, because it has a high nonlinear tolerance. The dispersion flattened fiber (DFF) is attractive for its ability to improve the system performance. Therefore, it is possible to improve the transmission performance by a combination of the RZ-DPSK and the DFF, and one important technology of the current long-haul optical fiber communication system is the dispersion map. And it is widely deployed for already installed undersea optical fiber communication system in the world. A previous study reported that the blockless type dispersion map showed a superior performance than the block type dispersion map, and some efforts to improve the transmission performance of the block type map were conducted. Fundamental idea to improve the transmission performance of the block type map is to reduce the zero crossing points, and one idea is to shift the map toward the positive or the negative cumulative dispersion to reduce the zero crossing points within the map, but it was not so successful. The other idea is to tilt the dispersion map and it was more successful but not good enough. In this master thesis, I continued the study to improve the long-haul RZ-DPSK system performance using the block type dispersion map. One new idea of the dispersion map shifting, the split shifting, was tried, and another new idea of the dispersion map tilting, the split tilting, was examined. The performance with different repeater output power and different compensation scheme within the dispersion map was simulated by a numerical simulator .The goal is, following previous research, to clarify improved dispersion map design of the long-haul RZ-DPSK based transmission and find the effective method to improve the transmission performance. In addition, I also investigate tolerance of the effective area of the transmission fiber theoretically for the long-haul RZ-DPSK system based on the DFF.

With the essence of robustness toward fiber nonlinearity owing to the increasingly required high-speed data rate from the country to country or country to the state, many useful methods are proposed upon the long-haul optical fiber transmission, such as modulation format, dispersion map and repeater spacing, etc. Return-to-zero differential phase shift keying (RZ-DPSK) format was chosen in this master thesis because of its high tolerance toward nonlinear effect in the wavelength-division multiplexing (WDM) system transmission. It has been realized that the system performance is wavelength-dependent by the combination of the WDM technology and the RZ-DPSK system with the commonly used block-type dispersion map, especially for the significant performance difference between including or excluding the self-phase-modulation (SPM) effect. Therefore, it is quite significant to investigate the unwanted fiber nonlinearity. In this master thesis, the influence relating to the cross-phase modulation (XPM) effect and the SPM effect with the conventional dispersion map after long distance transmission is the mainly concerned issue to be discussed. In this master thesis, both experiment and theoretical simulation are investigated. On the theoretical simulation part, the Q-factor of the system zero dispersion wavelength at 1543.8nm, 1550nm, 1556.2nm were degraded and their value were less than 10dB. The Q-factor was around 1.5dB less than the averaged value. However, the performance of the long-haul RZ-DPSK system based on the block-type dispersion map shows no significant performance by shifting the system zero dispersion wavelength out of the WDM signal wavelength band. On the experimental part, the impact of the XPM effect and the SPM effect on the long-haul optical fiber communication system is investigated quantitatively. For the XPM experiment, the system performance was just only 0.1 dB difference after 6000km transmission. On the contrary, for the SPM experiment, the Q-factor between best and worst performance was up to 1.3dB difference. At last, the experiment and the simulation support each other successfully in this master thesis.

Nowadays, the long-haul optical fiber communication system is one of the important way to convey the information, and there is strong competition of research in the optical long-haul transmission to achieve high channel bit rates and large transmission capacity. Therefore, it is important to study a technology to improve the performance of such system. The return-to-zero differential phase-shift keying (RZ-DPSK) and the return-to-zero differential quadrature phase-shift keying (RZ-DQPSK) have received renewed attention recently for the long-haul transmission systems, because they can improve the transmission performance of the long-haul system. Furthermore, the design of the dispersion map becomes significantly different from that of the conventional system using the intensity modulation direct detection (IM-DD) scheme. Besides, the RZ-DQPSK can transmit two bits per symbol, so it has twice the spectral efficiency of the RZ-DPSK. This study is focusing on the difference of the transmission performance of the long-haul RZ-DQPSK system due to the dispersion map using the numerical simulation.