The evolution to 5G and its use cases is driven by data-intensive applications requiring higher data rates over wireless channels. This has led to research in massive multiple input multiple output (MIMO) techniques and the use of the millimeter wave (mm wave) band. Because of the higher path loss at mm wave frequencies and the poor scattering nature of the mm wave channel (fewer paths exist), this thesis first proposes the use of the sphere decoding (SD) algorithm, and the semidefinite relaxation (SDR) detector to improve the performance of a uniform planar array (UPA) hybrid beamforming technique with large antenna arrays. The second contributions of this thesis consist of a low-complexity algorithm using the gradient descent for hybrid precoding and combining designs in mm wave systems. Also, in this thesis we present a low-complexity algorithm for hybrid precoding and combining designs that uses momentum gradient descent and Newton’s Method for mm wave systems which makes the objective function converge faster compared to other iterative methods in the literature; the two proposed low-complexity algorithms for hybrid precoding and combining do not depend on the antenna array geometry, unlike the orthogonal matching pursuit (OMP) hybrid precoding/combining approach. Moreover, these algorithms allow hybrid precoders/combiners to yield a performance very close to that of the optimal unconstrained digital precoders and combiners with a small number of iterations. Simulation results verify that the proposed hybrid precoding/combining scheme that uses momentum gradient descent and Newton’s Method outperforms previous methods that appear in the literature in terms of bit error rate (BER) and achievable spectral efficiency with lower complexity. Finally, an iterative algorithm that directly converts the hybrid precoding/combining in the full array (FA) architecture to subarray (SA) architecture is proposed and examined in this thesis. It is called direct conversion of iterative hybrid precoding/combining from FA to SA (DCIFS) hybrid precoding/combining. The proposed DCIFS design takes into consideration the matrix structure of the analog and baseband precoding and combining in the design derivation. Moreover, it does not depend on the antenna array geometry, unlike other techniques, such as the orthogonal matching pursuit (OMP) hybrid precoding/combining approach, nor does it assume any other constraints. Simulation results show that the proposed DCIFS hybrid design, when compared to the FA hybrid designs counterpart, can provide a spectral efficiency that is close to optimum while maintaining a very low complexity and better spectral efficiency than the conventional SA hybrid design with the same hardware complexity.