Ternary indium aluminum nitride (InXAl1-XN) semiconductor is an attractive material with a wide-range bandgap energy varied from ultraviolet (Eg(AlN): 6.2 eV) to near infrared (Eg(InN): 0.7 eV). With tuning composition, it can be widely used to many optoelectronic device applications. In this thesis, I have studied InXAl1-XN film deposited on Si (111) substrate using natural and isotopically enriched nitrogen as reactive gas by reactive magnetron sputter epitaxy (MSE). Four series of experiments were performed, which are I. InAlN presputtering, II. InAlN sputter deposition, III. InAlN direct deposition, and IV. InAlN direct deposition using isotopically enriched nitrogen. The samples were characterized by X-ray diffraction (XRD), transmission electron microscopy (TEM), and energy-dispersive X-ray spectroscopy (EDX). The θ-2θ XRD scan confirms that the designed composition x = 0.17 of InXAl1-XN film was obtained. TEM images shows that an amorphous interlayer with a thickness ranging from 1.2 nm to 1.5 nm was formed between Si substrate and InXAl1-XN film. However, high-resolution TEM shows that the interlayer actually contains partial crystalline structures. EDX line profile indicates that the chemical composition of the amorphous interlayer is silicon nitride (SiNX). By comparing d-spacing measurement of partial crystalline structures with EDX line profile, it reveals that partial SiNX crystal is formed in the interlayer. Nonetheless, the samples (IAD01, IAD02, IAD03, IAD04), grown without presputtering procedure, contain both crystalline SiNX and InXAl1-XN embedded in the amorphous interlayer. It means that SiNX and InXAl1-XN film can be directly grown on the substrate in the beginning of deposition. Moreover, the samples (IAD01, IAD03), quenched directly after deposition, have less crystalline structures in the interlayer then the samples (IAD02, IAD04), maintained at 800℃ for 20 min.