Galaxy redshift surveys are powerful probes of cosmology. Yet, in order to fully exploit the information contained in galaxy surveys, we need to improve upon our understanding of the structure formation in the Universe. Galaxies are formed/observed at late times when the density field is no longer linear so that understanding non-linearities is essential. In this thesis, we show that, at high redshifts, we can accurately model the galaxy power spectrum in redshift space by using the standard cosmological perturbation theory. Going beyond the power spectrum, we can use the three-point function, or the bispectrum, to gain important information on the early universe as well as on the galaxy formation via measurements of primordial non-Gaussianity and galaxy bias. We show that the galaxy bispectrum is more sensitive to primordial non-Gaussianities than previously recognized, making high-redshift galaxy surveys a particularly potent probe of the physics of inflation. Weak lensing offers yet another way of probing cosmology. By cross correlating the angular position of galaxies with the shear measurement from galaxy lensing or CMB lensing, we also show that one can obtain the information on cosmological distance scale, the galaxy bias, and the primordial non Gaussianity from weak lensing method. / text