Context. Irregular satellites of Jupiter with their highly eccentric, inclined and distant orbits suggest that their capture took place after the giant planet migration. Aims. We aim to improve our understanding of the surface composition of irregular satellites of Jupiter to gain insight into a narrow time window when our solar system was forming. Methods. We observed three Jovian irregular satellites, Himalia (JVI), Elara (JVII), and Carme (JXI), using a medium-resolution 0.8-5.5 mu m spectrograph, SpeX on the NASA Infrared Telescope Facility (IRTF). Using a linear spectral unmixing model we have constrained the major mineral phases on the surface of these three bodies. Results. Our results confirm that the surface of Himalia (JVI), Elara (JVII), and Carme (JXI) are dominated by opaque materials such as those seen in carbonaceous chondrite meteorites. Our spectral modeling of NIR spectra of Himalia and Elara confirm that their surface composition is the same and magnetite is the dominant mineral. A comparison of the spectral shape of Himalia with the two large main C-type asteroids, Themis (D similar to 176 km) and Europa (D similar to 352 km), suggests surface composition similar to Europa. The NIR spectrum of Carme exhibits blue slope up to 1.5 mu m and is spectrally distinct from those of Himalia and Elara. Our model suggests that it is compositionally similar to amorphous carbon. Conclusions. Himalia and Elara are compositionally similar but di ff er significantly from Carme. These results support the hypotheses that the Jupiter's irregular satellites are captured bodies that were subject to further breakup events and clustered as families based on their similar physical and surface compositions.