This thesis focuses on improving the performance of the THz time-domain spectroscopy system using second-order nonlinear crystals for THz generation and detection in terms of bandwidth, sensitivity, and THz field strength. The theories for the THz generation based on optical rectification and detection technique, electro-optical sampling, based on Pockels effect are introduced in Chapter 2. In Chapter 3, some experiments are presented to characterize the performances of the THz system based on a 180 fs Yb:KGW femtosecond laser amplifier operating at 1035 nm. The Yb-based femtosecond laser is becoming increasingly popular due to its robustness, high repetition rate, and high average power. However, the NIR bandwidth of these femtosecond lasers is limited by the gain bandwidth of the gain medium, and achieving pulse durations shorter than 180 fs is challenging. Consequently, the full bandwidth of THz time-domain spectroscopy systems is constrained by such laser systems. In order to broaden the THz bandwidth of such THz time-domain spectroscopy systems, our work in Chapter 4 combines the Yb:KGW femtosecond laser amplifier with an argon-filled hollow-core photonic crystal fiber pulse shaper to spectrally broaden the near-infrared pulses from 3.5 to 8.7 THz, increasing the measured THz bandwidth correspondingly from 2.3 THz to 4.5 THz. This is one of the first works to have broadband THz system based on Yb-based femtosecond lasers in the year of 2018. In Chapter 5, the tilted-pulse-front phase matching in the THz generation and detection scheme is demonstrated using the same surface-etched phase gratings on the front surfaces of the 2 mm-thick GaP generation and detection crystals. This scheme overcomes the THz generation and detection bandwidth limit of thick crystals imposed by the traditional collinear phase matching, while allowing the long nonlinear interaction length. This results in a THz spectral range from 0.1 to 6.5 THz with a peak at 3 THz and a peak dynamic range of 90 dB. In the range between 1.1 and 4.3 THz, the system dynamic range exceeds 80 dB. Based on this contact grating-based THz generation, the next step involves generating high-field THz above 2 THz. For high-field THz generation, the most renowned technique is the tilted-pulse-front technique, which generates high-field THz below 2 THz in a LiNbO₃ crystal. Most nonlinear optics experiments in the THz regime rely on such THz sources. To generate high-field THz above 2 THz, one promising candidate is organic THz crystals. However, most organic crystals require a pump laser with a wavelength exceeding 1200 nm, necessitating a more complex laser system. Additionally, the low damage threshold of these crystals are susceptible to compromise the stability of the measurements. Other techniques, such as air plasma and metallic spintronics, can generate ultra-broadband high-field THz from 0.1 to 30 THz, but the pulse energy within certain frequency windows is relatively low, rendering these THz sources less effective for nonlinearly driving specific optical transitions. On the other hand, semiconductor crystals as THz generation crystals, have a high damage threshold and can achieve good phase matching at wavelength around 800 or 1000 nm. In Chapter 6, high-field THz generation with a peak field of 303 kV/cm and a spectral peak at 2.6 THz is achieved with a more homogenous grating on the surface of a 1 mm-thick GaP generation crystal in a configuration collimating the near-infrared generation beam with a pulse energy of 0.57 mJ onto the generation crystal. The experiments also show that the system operates significantly below the GaP damage threshold and THz generation saturation regime, indicating that the peak THz field strength can approach 1 MV/cm, with a 5 mJ near-infrared generation pulse. This is the first high-field THz source based on semiconductor crystals capable of generating high-field THz above 2 THz. With such a THz source, we can conduct nonlinear optics experiments above 2 THz, including the study of phonon-assisted nonlinearities, coherent control of Bose-Einstein condensation of excitons and polaritons in semiconductor cavities, and saturable absorption in molecular gases.