In order to speedup the encoding process of H.264/AVC and Scalable Video Coding (SVC), Temporal and Spatial Correlation-based Merging and Splitting (TSCMS) fast mode decision algorithm and Coded Block Pattern (CBP)-based fast mode decision algorithm are proposed in this thesis. TSCMS and CBP-based fast mode decision algorithms are applied to H.264/AVC and SVC, respectively. In TSCMS, Temporal Correlation (TC) is used to predict the Motion Vectors (MVs) of 8¡Ñ8 blocks in each macroblock. In addition, the merging and splitting procedure is adopted to predict the motion vectors of other blocks. Afterwards, the spatial correlation is performed to merge 16¡Ñ16 blocks instead of the conventional merge scheme. CBP value is the syntax used at each Macroblock (MB) header to indicate whether an MB contains residual information or not in CBP-based fast mode decision algorithm. The proposed algorithm can exclude the invalid modes for the mode prediction of the current MB in Enhancement Layer (EL) through the CBP values and MB modes of adjacent MBs in EL and the co-located Base Layer (BL) MB modes. Experimental results show that the proposed algorithms reduce computations significantly with negligible PSNR degradation and bit increase when compared to JM 12.3, JSVM 9.12, and the other existing methods.