Strongly correlated systems on geometrically frustrated lattices can stabilize a large number of interesting phases that includes a wide array of novel Mott insulators in both bosonic and electronic systems. Charge fluctuations in a Mott insulator are suppressed due to strong mutual interaction among the particles. The presence of frustration is of particular importance as the physics it offers is often rich, unexpectedly complicated, and continues to raise many open questions. The thesis elucidates some of these issues on a kagome lattice where strong interactions among the particles in the Mott phase impose non-trivial local constraints depending on the filling fraction on the lattice. These Mott insulators, in addition to featuring unusual magnetic and/or charge ordering, can also harbor topologically ordered states of quantum matter, e.g., resonating valence bond liquids realized in certain quantum dimer models on non-bipartite lattices. The dimer models can be regarded as low-energy effective theories for different types of bosonic models in the strong-coupling limit. Exploring this connection is a central theme of this thesis with the aim of realizing novel strongly correlated ground states. Past studies of these models have revealed the existence of various ordered and disordered phases with distinct signatures. Among these low-energy phases, the presence of a stable topological liquid at a particular point, known as Rokhsar-Kivelson point, in the phase diagram is notable. The classical versions of the dimer model are also known to have garnered a vast interest in various fields ranging from problems of pure mathematical origin to ones in physical chemistry as well as statistical physics. Pioneered by Kasteleyn, several analytical works came forward to exactly calculate the partition function of the problem from which other physical observables can be derived. Classical numerical methods are extensively applied to these models to verify the analytical predictions. We introduce a new classical algorithm here to compute the correlation functions of a classical dimer model on a square (bipartite) and a triangular (non-bipartite) lattice based on a tensor network construction. The method, called tensor network renormalization group, turns out to be a powerful tool for simulating short-ranged gapped systems as inferred from our results benchmarked against the classical Monte-Carlo technique and compared with past analytical studies. One should note that the quantum dimer model at the Rokhsar-Kivelson point can also be described as an infinite temperature canonical ensemble of classical dimers because of the particular structure of the ground state which is an equal weight superposition in the configuration manifold. The geometry of the lattice plays a pivotal role in deciding the nature of the phases that arise in the dimer models. Many physical properties of the dimer liquid phase can be extracted in the simple classical setting which certainly allows for a deep understanding of the classical models to be developed. The liquid phase is gapped on non-bipartite lattices and gapless on bipartite lattices, which is reflected in the decay of correlation functions with spatial distances. In general on non-bipartite lattices, the topological nature of the dimer liquid is characterized by a Z2 topological order which survives even when the model is perturbed away from the Rokhsar-Kivelson point. Stability of this liquid phase not only depends on the lattice geometries but notably on dimer concentrations also. In this context, we focus on a particular variant of the dimer model on a triangular lattice which is known as the quantum fully packed loop model. The model is composed of nonintersecting closed loops made of dimers and governed by the same Hamiltonian as the quantum dimer model. The loop model provides an effective low-energy description of a strongly correlated bosonic system at 1/3 filling on the kagome lattice. The corresponding Bose-Hubbard Hamiltonian consists of nearest-neighbor hopping and all possible repulsive interactions within a hexagonal plaquette. Conspicuous features of the zero-temperature phase diagram for this model include (i) presence of a stable Z2 liquid even without any Rokhsar-Kivelson potential term (in distinction to the standard quantum dimer model), and (ii) an unconventional phase transition from the liquid phase to a novel crystalline phase that has nematic order (dubbed lattice nematic). For a deeper understanding of the physics, a mapping to an Ising gauge theory is presented. The gauge theoretic description provides a useful way to predict the nature of the quantum phase transition to lie in the O(3) universality class. Finally a fermionic model at the same 1/3 filling is considered in which the ground state exhibits a number of exotic local orderings resulting from the spin-charge interplay of electrons. The Hamiltonian comprises nearest-neighbor hopping, strong on-site Coulomb interaction, and repulsive interaction terms only between nearest-neighbors. In the strong correlation limit, this fermionic problem maps to a two-color fully packed loop model – a model in which the loop segments carry an additional quantum number as color on a honeycomb lattice. The effective theory is governed by coherent three-particle ring exchanges and nearest-neighbor antiferromagnetic spin exchanges. The competition between these two leads to a phase diagram composed of a novel plaquette ordered state (known as the plaquette phase) that undergoes phase transition to a new kind of charge ordered state which we call a short loop phase. From our numerical analysis, we conclude that the plaquette phase features an unusual antiferromagnetic order with gapless spin excitations while the charge-ordered state is subjugated by spin fluctuations of localized electrons arranged in small hexagonal loops on the kagome lattice.

Strongly correlated systems on geometrically frustrated lattices can stabilize a large number of interesting phases that includes a wide array of novel Mott insulators in both bosonic and electronic systems. Charge fluctuations in a Mott insulator are suppressed due to strong mutual interaction among the particles. The presence of frustration is of particular importance as the physics it offers is often rich, unexpectedly complicated, and continues to raise many open questions. The thesis elucidates some of these issues on a kagome lattice where strong interactions among the particles in the Mott phase impose non-trivial local constraints depending on the filling fraction on the lattice. These Mott insulators, in addition to featuring unusual magnetic and/or charge ordering, can also harbor topologically ordered states of quantum matter, e.g., resonating valence bond liquids realized in certain quantum dimer models on non-bipartite lattices. The dimer models can be regarded as low-energy effective theories for different types of bosonic models in the strong-coupling limit. Exploring this connection is a central theme of this thesis with the aim of realizing novel strongly correlated ground states. Past studies of these models have revealed the existence of various ordered and disordered phases with distinct signatures. Among these low-energy phases, the presence of a stable topological liquid at a particular point, known as Rokhsar-Kivelson point, in the phase diagram is notable. The classical versions of the dimer model are also known to have garnered a vast interest in various fields ranging from problems of pure mathematical origin to ones in physical chemistry as well as statistical physics. Pioneered by Kasteleyn, several analytical works came forward to exactly calculate the partition function of the problem from which other physical observables can be derived. Classical numerical methods are extensively applied to these models to verify the analytical predictions. We introduce a new classical algorithm here to compute the correlation functions of a classical dimer model on a square (bipartite) and a triangular (non-bipartite) lattice based on a tensor network construction. The method, called tensor network renormalization group, turns out to be a powerful tool for simulating short-ranged gapped systems as inferred from our results benchmarked against the classical Monte-Carlo technique and compared with past analytical studies. One should note that the quantum dimer model at the Rokhsar-Kivelson point can also be described as an infinite temperature canonical ensemble of classical dimers because of the particular structure of the ground state which is an equal weight superposition in the configuration manifold. The geometry of the lattice plays a pivotal role in deciding the nature of the phases that arise in the dimer models. Many physical properties of the dimer liquid phase can be extracted in the simple classical setting which certainly allows for a deep understanding of the classical models to be developed. The liquid phase is gapped on non-bipartite lattices and gapless on bipartite lattices, which is reflected in the decay of correlation functions with spatial distances. In general on non-bipartite lattices, the topological nature of the dimer liquid is characterized by a Z2 topological order which survives even when the model is perturbed away from the Rokhsar-Kivelson point. Stability of this liquid phase not only depends on the lattice geometries but notably on dimer concentrations also. In this context, we focus on a particular variant of the dimer model on a triangular lattice which is known as the quantum fully packed loop model. The model is composed of nonintersecting closed loops made of dimers and governed by the same Hamiltonian as the quantum dimer model. The loop model provides an effective low-energy description of a strongly correlated bosonic system at 1/3 filling on the kagome lattice. The corresponding Bose-Hubbard Hamiltonian consists of nearest-neighbor hopping and all possible repulsive interactions within a hexagonal plaquette. Conspicuous features of the zero-temperature phase diagram for this model include (i) presence of a stable Z2 liquid even without any Rokhsar-Kivelson potential term (in distinction to the standard quantum dimer model), and (ii) an unconventional phase transition from the liquid phase to a novel crystalline phase that has nematic order (dubbed lattice nematic). For a deeper understanding of the physics, a mapping to an Ising gauge theory is presented. The gauge theoretic description provides a useful way to predict the nature of the quantum phase transition to lie in the O(3) universality class. Finally a fermionic model at the same 1/3 filling is considered in which the ground state exhibits a number of exotic local orderings resulting from the spin-charge interplay of electrons. The Hamiltonian comprises nearest-neighbor hopping, strong on-site Coulomb interaction, and repulsive interaction terms only between nearest-neighbors. In the strong correlation limit, this fermionic problem maps to a two-color fully packed loop model – a model in which the loop segments carry an additional quantum number as color on a honeycomb lattice. The effective theory is governed by coherent three-particle ring exchanges and nearest-neighbor antiferromagnetic spin exchanges. The competition between these two leads to a phase diagram composed of a novel plaquette ordered state (known as the plaquette phase) that undergoes phase transition to a new kind of charge ordered state which we call a short loop phase. From our numerical analysis, we conclude that the plaquette phase features an unusual antiferromagnetic order with gapless spin excitations while the charge-ordered state is subjugated by spin fluctuations of localized electrons arranged in small hexagonal loops on the kagome lattice.

In the first two chapters we presented the basics of density functional theory and semiempirical LSD+U approximation, which was implemented in the full-potential local-orbital (FPLO) minimal-basis calculation scheme. In the third chapter we tested the implemented version of LSDA+U on 3d transitional metal monoxides. Essential improvement of the spectroscopic properties was obtained. A simple model describing the value and direction of the magnetic moment of a transition metal ion was presented. The model visualizes the interplay of the spin-orbit coupling and crystal field splitting. In the fourth chapter we calculated the electronic spectrum of the single Zn impurity in CuO2 plane considered as a vacancy in Cu 3d states. The analytic solution for the states of different symmetry was obtained. Depending on the strength of perturbation induced by the impurity on the neighboring Cu ions, the states are either resonant or localized. The critical values of the perturbation were computed. In the fifth chapter we presented the calculations for three novel vanadates: MgVO3, Sb2O2VO3 and VOMoO4. The tight-binding parameters and the exchange integrals were computed. The magnesium and antimony vanadates appeared to be spin-½ one-dimensional systems, the latter having much stronger one-dimensional character and being probably the best realization of inorganic spin-Peierls system. The molybdenum vanadate was found to be two-dimensional spin-½ system. The Mo 4d orbitals play an important role in the electronic transfer.

In the last decade, a large number of studies have been devoted to the peculiarities of correlated physics found in the quasi-two-dimensional square lattice iridium oxides. It was shown that this 5d family of transition metal oxides has strong structural and electronic similarities to the famous 3d family of copper oxides. Moreover, a delicate interplay of on-site spin-orbit coupling, Coulomb repulsion and crystalline electric field interactions is expected to drive various exotic quantum states. Many theoretical proposals were made in the last decade including the prediction of possible superconductivity in square-lattice iridates emerging as a sister system to high-Tc cuprates, which however met only limited experimental confirmation. One can, therefore, raise a general question: To what extent is the low-energy physics of the quasi-two-dimensional square-lattice iridium oxides different from other transition metal oxides including cuprates? In this thesis we investigate some of the effects which are usually neglected in studies on iridates, focusing on quasi-two-dimensional square-lattice iridates such as Sr2IrO4 or Ba2IrO4. In particular, we discuss the role of the electron-phonon coupling in the form of Jahn-Teller interaction, electron-hole asymmetry introduced by the strong correlations and some effects of coupling scheme chosen to calculate multiplet structure for materials with strong on-site spin-orbit coupling. Thus, firstly, we study the role of phonons, which is almost always neglected in Sr2IrO4, and discuss the manifestation of Jahn-Teller effect in the recent data obtained on Sr2IrO4 with the help of resonant inelastic x-ray scattering. When strong spin-orbit coupling removes orbital degeneracy, it would at the same time appear to render the Jahn-Teller mechanism ineffective. We show that, while the Jahn-Teller effect does indeed not affect the antiferromagnetically ordered ground state, it leads to distinctive signatures in the spin-orbit exciton. Second, we focus on charge excitations and determine the motion of a charge (hole or electron) added to the Mott insulating, antiferromagnetic ground-state of square-lattice iridates. We show that correlation effects, calculated within the self-consistent Born approximation, render the hole and electron case very different. An added electron forms a spin-polaron, which closely resembles the well-known cuprates, but the situation of a removed electron is far more complex. Many-body configurations form that can be either singlets and triplets, which strongly affects the hole motion. This not only has important ramifications for the interpretation of angle-resolved photoemission spectroscopy and inverse photoemission spectroscopy experiments of square lattice iridates, but also demonstrates that the correlation physics in electron- and hole-doped iridates is fundamentally different. We then discuss the application of this model to the calculation of scanning tunneling spectroscopy data. We show that using scanning tunneling spectroscopy one can directly probe the quasiparticle excitations in Sr2IrO4: ladder spectrum on the positive bias side and multiplet structure of the polaron on the negative bias side. We discuss in detail the ladder spectrum and show its relevance for Sr2IrO4 which is in general described by more complicated extended t-J -like model. Theoretical calculation reveals that on the negative bias side the internal degree of freedom of the charge excitation introduces strong dispersive hopping channels encaving ladder-like features. Finally, we discuss how the choice of the coupling scheme to calculate multiplet structure can affect the theoretical calculation of angle-resolved photoemission spectroscopy and scanning tunnelling spectroscopy spectral functions.

In the last decade, a large number of studies have been devoted to the peculiarities of correlated physics found in the quasi-two-dimensional square lattice iridium oxides. It was shown that this 5d family of transition metal oxides has strong structural and electronic similarities to the famous 3d family of copper oxides. Moreover, a delicate interplay of on-site spin-orbit coupling, Coulomb repulsion and crystalline electric field interactions is expected to drive various exotic quantum states. Many theoretical proposals were made in the last decade including the prediction of possible superconductivity in square-lattice iridates emerging as a sister system to high-Tc cuprates, which however met only limited experimental confirmation. One can, therefore, raise a general question: To what extent is the low-energy physics of the quasi-two-dimensional square-lattice iridium oxides different from other transition metal oxides including cuprates? In this thesis we investigate some of the effects which are usually neglected in studies on iridates, focusing on quasi-two-dimensional square-lattice iridates such as Sr2IrO4 or Ba2IrO4. In particular, we discuss the role of the electron-phonon coupling in the form of Jahn-Teller interaction, electron-hole asymmetry introduced by the strong correlations and some effects of coupling scheme chosen to calculate multiplet structure for materials with strong on-site spin-orbit coupling. Thus, firstly, we study the role of phonons, which is almost always neglected in Sr2IrO4, and discuss the manifestation of Jahn-Teller effect in the recent data obtained on Sr2IrO4 with the help of resonant inelastic x-ray scattering. When strong spin-orbit coupling removes orbital degeneracy, it would at the same time appear to render the Jahn-Teller mechanism ineffective. We show that, while the Jahn-Teller effect does indeed not affect the antiferromagnetically ordered ground state, it leads to distinctive signatures in the spin-orbit exciton. Second, we focus on charge excitations and determine the motion of a charge (hole or electron) added to the Mott insulating, antiferromagnetic ground-state of square-lattice iridates. We show that correlation effects, calculated within the self-consistent Born approximation, render the hole and electron case very different. An added electron forms a spin-polaron, which closely resembles the well-known cuprates, but the situation of a removed electron is far more complex. Many-body configurations form that can be either singlets and triplets, which strongly affects the hole motion. This not only has important ramifications for the interpretation of angle-resolved photoemission spectroscopy and inverse photoemission spectroscopy experiments of square lattice iridates, but also demonstrates that the correlation physics in electron- and hole-doped iridates is fundamentally different. We then discuss the application of this model to the calculation of scanning tunneling spectroscopy data. We show that using scanning tunneling spectroscopy one can directly probe the quasiparticle excitations in Sr2IrO4: ladder spectrum on the positive bias side and multiplet structure of the polaron on the negative bias side. We discuss in detail the ladder spectrum and show its relevance for Sr2IrO4 which is in general described by more complicated extended t-J -like model. Theoretical calculation reveals that on the negative bias side the internal degree of freedom of the charge excitation introduces strong dispersive hopping channels encaving ladder-like features. Finally, we discuss how the choice of the coupling scheme to calculate multiplet structure can affect the theoretical calculation of angle-resolved photoemission spectroscopy and scanning tunnelling spectroscopy spectral functions.

In the first two chapters we presented the basics of density functional theory and semiempirical LSD+U approximation, which was implemented in the full-potential local-orbital (FPLO) minimal-basis calculation scheme. In the third chapter we tested the implemented version of LSDA+U on 3d transitional metal monoxides. Essential improvement of the spectroscopic properties was obtained. A simple model describing the value and direction of the magnetic moment of a transition metal ion was presented. The model visualizes the interplay of the spin-orbit coupling and crystal field splitting. In the fourth chapter we calculated the electronic spectrum of the single Zn impurity in CuO2 plane considered as a vacancy in Cu 3d states. The analytic solution for the states of different symmetry was obtained. Depending on the strength of perturbation induced by the impurity on the neighboring Cu ions, the states are either resonant or localized. The critical values of the perturbation were computed. In the fifth chapter we presented the calculations for three novel vanadates: MgVO3, Sb2O2VO3 and VOMoO4. The tight-binding parameters and the exchange integrals were computed. The magnesium and antimony vanadates appeared to be spin-½ one-dimensional systems, the latter having much stronger one-dimensional character and being probably the best realization of inorganic spin-Peierls system. The molybdenum vanadate was found to be two-dimensional spin-½ system. The Mo 4d orbitals play an important role in the electronic transfer.