In the geometrical spin frustrated systems, order-disorder phenomena are interesting, for their intrinsic fluctuation, complicated completing interactions, and lattice structure leading to many intrigue physical behavior. To clarify the mechanism of such systems, experiments under extreme conditions (diverse magnetic fields and hydrostatic pressures) are powerful tools to meet the needs. In this dissertation, two kind of interesting materials are investigated. One is quasi-1D double chain antiferromagnet NaV2O4, the other is cubic like ferrimagnet Cu2OSeO3. In the polycrystalline compound NaV2O4, it exhibits an antiferromagnetic transition TN at 140 K, together with two field dependence subphases at 1 T≤H≤5 T. The two characteristic temperature TN1 and TN2 are associated to two subphases, which is determined by the peak position in the derivative of magnetization with respect to temperature. Under magnetic field, TN and TN1 remain almost unchanged (linear behavior), while TN2 acts in a nonlinear behavior with the application of magnetic field. Further, TN1 and TN2 are found to decrease roughly linear with applied pressure, while TN2 follows a nonlinear relation with applied pressure. On the other hand, the cubic single crystal Cu2OSeO3 exhibits a ferrimagnetic transition at 58 K, which is shifting to high temperature range with increasing magnetic fields. The peak values (from the mutual inductance measurements) associated with the ferrimagnetic transitions also increase with applied hydrostatic pressures. Moreover, the spin-flipped transitions are observed below transition temperature at ambient and applied pressure (12.67 kbar). The measurements above strongly suggested the ferrimagnetic spin configurations order earlier, i.e. transition temperatures increase with applied magnetic fields and pressures. In summary, the investigated frustrated spin systems (NaV2O4 and Cu2OSeO3) behave with the same trend with applied magnetic fields and hydrostatic pressures. It is possibly induced by the external DC magnetic field and the structure change and/or deformation under pressure.

An intriguing magnetodielectric (MD) behavior is observed in geometrically frustrated spin-tetrahedral systems Cu2Te2O5X2 (X = Cl and Br). While the phase transition observed in the Cl-system at TN=18.5 K is consistent with 3D antiferromagnetic ordering, the phase transition at To=11.5 K in the Br-system has several unusual features. Concomitantly, a pronounced ferroelectric ordering is observed coinciding with TN of Cl-system and To of Br-system. At the highest applied magnetic field 90 kOe, the temperature dependent dielectric behavior with almost frequency independent well defined at TN~18.5 K are decrease compared with that at zero field for Cl-system, and at the highest applied magnetic field 90 kOe, the temperature dependent dielectric behavior with almost frequency independent well defined at Tm~30 K are decrease compared with that at zero field for Br-system. The ferroelectricity is ascribed to the polarization of the Te4+ lone-pair electrons, while the MD effect is argued to be due to exchange interaction involving frustrated tetramer clusters and intercluster exchange bridges.

Order-disorder phenomena in geometrical frustrated systems are the attractive topics because of the intrinsic fluctuation. Among the geometrical frustrated systems, the material with spinel structure (AB2X4) is one of the appropriate candidates to investigate the long range ordering behavior. Corner sharing of tetrahedron and edge sharing of octahedron in the unique structural network of spinel structure are the characteristics for geometrical frustration. Hence, to study the 3d transition metal substituted in spinel system which leads to fruitful physical behavior becomes rapidly attractive. In this dissertation, long range ordering behavior in spin frustrated systems including three interesting materials LiTi2O4, NaxCoO2¡DyH2O, and CdCr2S4 were investigated. LiTi2O4 was found to show the highest superconducting transition temperature (Tc ~ 11 K) while first hydrated superconductor NaxCoO2¡DyH2O (Tc~ 4.5 K) was discovered in 2003. Superconductivity of LiTi2O4 and NaxCoO2¡DyH2O had been measured and analyzed by low temperature specific heat under magnetic field. According to the analyses of specific-heat results, isotropic (s-wave) and nodal (d-wave) gaps of superconducting pairing symmetry were proposed for LiTi2O4 and NaxCoO2¡DyH2O, respectively. Finally, LiTi2O4 was confirmed to be a typical BCS-like, fully gapped, and electron-phonon moderate-coupling type-II superconductor. Not like the case of LiTi2O4, the superconducting parameters of NaxCoO2¡DyH2O, such as Tc, HC2 and pairing symmetry, were strongly dependent on synthesized conditions. However, the evidence of nodal gap was found to be an intrinsic feature in this peculiar material NaxCoO2¡DyH2O. In the ferromagnetic insulator CdCr2S4, we first found several interesting features induced by external electric field in dielectric and magnetization measurements. Exchangestriction was proposed to be associated with the colossal change of dielectric constant value and suppression of magnetization under external electric and magnetic field in CdCr2S4. Therefore, our results supported that CdCr2S4 was a typical multiferroic material. In a conclusion, the intrinsic fluctuation of spin frustrated systems wasnecessary to pay more attention in the near future due to its fruitful physical properties and behind theoretical description.

An intriguing magnetodielectric behavior is observed in triangular or tetrahedral frustrated and low-dimensional system. Therefore, the spin-tetrahedral and low-dimensional compound copper-tellurides (Cu2Te2O5Br2) is suggested that has magnetodielectric behavior. Tetragonal Cu2Te2O5Br2 contains clusters of four Cu2+ (S = 1/2) in a planar coordination. These tetrahedral form weakly coupled sheets within the crystallographic a-b plane. Therefore, this system is ideal to study the interplay between the spin frustration on a tetrahedron with localized low-energy excitations and collective magnetism induced by inter-tetrahedral couplings. In this material a strongly reduced magnetic transition temperature To = 11.5 K in comparison with a dominant magnetic exchange of 30 K is found. Low-dimensional systems with triangular geometries are considered as prominent candidates for applications using novel magnetoelectric materials. At the highest applied magnetic field 90 kOe, the temperature dependent dielectric behavior with almost frequency independent well defined maxima at Tm ~ 30 K and To ~ 11.5 K are enhanced compared with that at zero field. We suggest that the observed magnetodielectric coupling can arise from exchange striction involving frustrated tetramer clusters and inter-cluster exchange bridges with polarizable lone-pair electrons on Te4+ ions.