In this thesis, we study the scattering problem of a vertically offset dielectric slab waveguide, using continuous radiation modes. The calculation of radiation modes of an arbitrarily layered waveguide has been thoroughly investigated in the literature. Most approaches were based on launching two incident waves: one from above and one from below, resulting in two transmitted waves and two reflected waves. Radiation modes were obtained by algebraic adjustments of each incident wave¡¦s amplitude and phase. These radiation modes formed standing waves in both the substrates and superstrates. This implies that walls are located an infinite distance far from the first and the last interfaces. In addition to physical conflicts of simultaneous existence of the incident wave and the walls, the derivation details are complicated and non-intuitive. In our thesis, with a given propagation constant for an arbitrarily layered dielectric waveguide, we propose an intuitive method to obtain two independent radiation mode solutions. We also construct a specific procedure to orthogonalize and normalize these two radiation modes. The second part of this thesis is focused on applying these radiation modes into a customized coupled transverse mode integral equation formulation (CTMIE), to the study of vertically offset slab waveguides. CTMIE requires two artificial boundaries placed in the substrate and superstrate. We choose to compute discretized radiation modes with the periodic boundary conditions. Under these circumstances, modes correspond to different spatial frequencies and thereby do not inter-couple. This means the matrix of the overlap integral between these two groups of modes (slightly vertically shifted) are block-diagonally dominated. The off-diagonal elements are two orders of magnitude smaller than the diagonal ones. As a result, when the two artificial boundaries are pushed towards infinity in the CTMIE formulation, we may obtain an exact inverse of the Greene¡¦s matrix without relying on numerical inversion.