There is no apparent, dominant interaction in heavy transition metal oxides (TMO), especially in 5d-TMO, where all relevant interactions are of comparable energy scales, and therefore strongly compete. In particular, the spin-orbit interaction (SOI) strongly competes with the electron-lattice and on-site Coulomb interaction (U). Therefore, any tool that allows one to tune the relative strengths of SOI and U is expected to offer an opportunity for the discovery and study of novel materials. BaIrO3 is a magnetic insulator driven by SOI whereas the isostructural BaRuO3 is a paramagnetic metal. The contrasting ground states have been shown to result from the critical role of the strong SOI in the iridate. This dissertation thoroughly examines a wide array of newly observed novel phenomena induced by adjusting the relative strengths of SOI and U via a systematic chemical substitution of the Ru4+(4d4) ions for Ir4+(5d5) ions in BaIrO3, i.e., in high quality single crystals of BaIr1-xRuxO3(0.0 < x < 1.0) . Our investigation of structural, magnetic, transport and thermal properties reveals that Ru substitution directly rebalances the competing energies so profoundly that it generates a rich phase diagram for BaIr1-xRuxO3 featuring two major effects: (1) Light Ru doping (0 < x < 0.15) prompts a simultaneous and precipitous drop in both the magnetic ordering temperature TC and the electrical resistivity, which exhibits metal-insulator transition at around TC. (2) Heavier Ru doping (0.41 < x < 0.82) induces a robust metallic and spin frustration state. For comparison and contrast, we also substituted Rh4+(4d5) ions for Ir4+(5d5) ions in BaIrO3, i.e. BaIr1-xRhxO3(0.0 < x < 0.10), where Rh only reduces the SOI, but without altering the band filling. Hence, this system remains tuned at the Mott instability and is very susceptible to disorder scattering which gives rise to Anderson localization.