Nowadays Optical fiber communication is one of the important way to convey information, and there is strong competition in optical long-haul transmission to achieve high channel bit rates and large transmission capacity. Therefore, it is important to study a technology to improve the performance of such system. As we have already known, the dispersion flattened fiber (DFF) and Return-to-zero differential phase shift keying (RZ-DPSK) modulation are attractive, because it can improve the transmission performance of the long-haul system, therefore, it is possible to improve the transmission performance by a combination of the RZ-DPSK and the DFF, and the performance improvement investigation is required, because it will contribute to improve the system design of the long-haul optical fiber communication systems in near future. In this master thesis, a theoretical study focused on the RZ-DPSK transmission system using DFF is conducted. One important technology of current long-haul optical fiber communication system is a dispersion map. In this master thesis, difference of the transmission performance is characterized with regard to number of dispersion blocks within the dispersion map, different repeater output power and different compensation scheme within dispersion map, etc., all of them calculated by a numerical simulator .The goal is to clarify the optimum dispersion map design of the long-haul DPSK based transmission and find the effective method to improve the transmission performance.