A novel residual prediction algorithm is proposed for high-bit-rate video coding in this work. We analysis the relationship between the residual data and different quantization parameters, according to the comparison results, we observe that the residual data is raised rapidly when the quality increases. Consequently, in order to reduce the bitrate, we propose a new residual prediction algorithm, it mainly reduce the residual data when the quantization parameter is finer. The proposed algorithm not only reduces the bitrate but also improves the video quality for high-bit-rate coding. Experimental results show that the proposed algorithm outperforms H.264/AVC. Compared to H.264/AVC, the proposed method decreases about 9.66% bitrate in average. The experimental results demonstrated that the second-order-residual prediction algorithm is efficiency for high-bit-rate coding.