In this thesis, we investigate strongly correlated phenomena in quantum spin systems. In the first part of this work, we study geometrically frustrated antiferromagnets (AFMs). Generalizing the SU(2) Heisenberg Hamiltonian to Sp(N) symmetry, we obtain, in the large-N limit, the mean-field phase diagrams for the planar pyrochlore and cubic AFMs. We then use gauge theories to consider fluctuation effects about their respective mean-field configurations. We find, in addition to conventional Neel states, a plethora of novel magnetically disordered phases: two kinds of spin liquids, Z2 in 2+1D and U(1)in 3+1D, and several valence bond solids such as two and three-dimensional plaquette and columnar singlet states. We use the same approach to study the diamond lattice AFM which possesses extended classical ground state degeneracy. We demonstrate that quantum and entropic fluctuations lift this degeneracy in different ways. In the second part of the thesis, we study ultracold spinor atoms confined in optical lattices. We first demonstrate the feasibility of experimental realization of rotor models using ultracold spin-one Bose atoms in a spin-dependent and disordered optical lattice. We show that the ground state of such disordered rotor models with quadrupolar interactions can exhibit biaxial nematic ordering in the disorder-averaged sense, and suggest an imaging experiment to detect the biaxial nematicity in such systems. Finally, using variational wavefunction methods, we study the Mott phases and superfluid-insulator transition of spin-three bosons in an optical lattice with an anisotropic two dimensional optical trap. We chart out the phase diagrams for Mott states with n = 1 and n = 2 atoms per lattice site. We show that the long-range dipolar interaction stabilizes a state characterized by antiferromagnetic chains made of ferromagnetically aligned spins. We also obtain the mean-field phase boundary for the superfluid-insulator transition, and show that inside the superfluid phase and near the superfluid-insulator phase boundary, the system undergoes a first order antiferromagnetic-ferromagnetic spin ordering transition.

In this thesis, we investigate strongly correlated phenomena in quantum spin systems. In the first part of this work, we study geometrically frustrated antiferromagnets (AFMs). Generalizing the SU(2) Heisenberg Hamiltonian to Sp(N) symmetry, we obtain, in the large-N limit, the mean-field phase diagrams for the planar pyrochlore and cubic AFMs. We then use gauge theories to consider fluctuation effects about their respective mean-field configurations. We find, in addition to conventional Neel states, a plethora of novel magnetically disordered phases: two kinds of spin liquids, Z2 in 2+1D and U(1)in 3+1D, and several valence bond solids such as two and three-dimensional plaquette and columnar singlet states. We use the same approach to study the diamond lattice AFM which possesses extended classical ground state degeneracy. We demonstrate that quantum and entropic fluctuations lift this degeneracy in different ways. In the second part of the thesis, we study ultracold spinor atoms confined in optical lattices. We first demonstrate the feasibility of experimental realization of rotor models using ultracold spin-one Bose atoms in a spin-dependent and disordered optical lattice. We show that the ground state of such disordered rotor models with quadrupolar interactions can exhibit biaxial nematic ordering in the disorder-averaged sense, and suggest an imaging experiment to detect the biaxial nematicity in such systems. Finally, using variational wavefunction methods, we study the Mott phases and superfluid-insulator transition of spin-three bosons in an optical lattice with an anisotropic two dimensional optical trap. We chart out the phase diagrams for Mott states with n = 1 and n = 2 atoms per lattice site. We show that the long-range dipolar interaction stabilizes a state characterized by antiferromagnetic chains made of ferromagnetically aligned spins. We also obtain the mean-field phase boundary for the superfluid-insulator transition, and show that inside the superfluid phase and near the superfluid-insulator phase boundary, the system undergoes a first order antiferromagnetic-ferromagnetic spin ordering transition.

Dans cette thèse nous abordons des aspects topologiques de la matière condensée. Les états topologiques sont insensibles à un large spectre des perturbations externes et au désordre – une propriété indispensable dans le domaine d'information quantique. L’effet des interactions dans des systèmes topologiques est pourtant loin d’être bien maîtrisé à ce jour. Dans ce travail, nous étudions la corrélation entre la description topologique et l'effet des interactions. Afin d'accomplir notre but, nous utilisons des méthodes analytiques et numériques. Nous nous intéressons aussi à des sondes expérimentales qui peuvent être utilisées pour vérifier nos prédictions théoriques. Tout d’abord, nous étudions la version bosonique en interactions du modèle de Haldane – le modèle célèbre qui décrit l’effet Hall anomal. Nous proposons son implémentation expérimentale dans des circuits quantiques, basée sur l’application de perturbation périodique dépendantes du temps – méthodologie qui s’appelle l’ingénierie de Floquet. En poursuivant ces idées, nous étudions la version bosonique du modèle de Kane-Mele d’un isolant topologique. Ce modèle possède un diagramme de phase très riche. En particulier, lorsque les interactions sont fortes, nous observons l’apparition d’un modèle de magnétisme frustrée présentant une variété d'états exotiques. La mise en œuvre de ces modèles dans des réseaux d'atomes ultra-froids ou des circuits quantiques permettra de sonder expérimentalement les propriétés exotiques que nous avons observées. Ensuite, nous abordons d’une manière plus détaillée la réalisation expérimentale des modèles topologiques dans des circuits quantiques, en considérant le cas particulier du modèle de Su-Schrieffer-Heeger en couplage fort. Nous testons aussi des nouvelles sondes qui peuvent être utilisées afin de mesurer la phase de Zak et en déduire la topologie du système. Finalement, nous nous intéressons aux sondes hors d’équilibre et des méthodes pour tester les propriétés spectrales de systèmes quantiques, en utilisant l’approche de purification, pertinent pour le numérique et les expériences. / In this thesis we study the topological aspects of condensed matter physics, that received a revolutionary development in the last decades. Topological states of matter are protected against perturbations and disorder, making them very promising in the context of quantum information. The interplay between topology and interactions in such systems is however far from being well understood, while the experimental realization is challenging. Thus, in this work we investigate analytically such strongly correlated states of matter and explore new protocols to probe experimentally their properties. In order to do this, we use various analytical and numerical techniques. First, we analyze the properties of an interacting bosonic version of the celebrated Haldane model – the model for the quantum anomalous Hall effect. We propose its quantum circuit implementation based on the application of periodic time-dependent perturbations – Floquet engineering. Continuing these ideas, we study the interacting bosonic version of the Kane-Mele model – the first model of a topological insulator. This model has a very rich phase diagram with an emergence of an effective frustrated magnetic model and a variety of symmetry broken spin states in the strongly interacting regime. Ultra-cold atoms or quantum circuits implementation of both Haldane and Kane-Mele bosonic models would allow for experimental probes of the exotic states we observed. Second, in order to deepen the perspectives of quantum circuit simulations of topological phases we analyze the strong coupling limit of the Su-Schrieffer-Heeger model and we test new experimental probes of its topology associated with the Zak phase. We also work on the out-of-equilibrium protocols to study bulk spectral properties of quantum systems and quantum phase transitions using a purification scheme which could be implemented both numerically and experimentally.