H-matrix techniques use a data-sparse tree structure to represent a dense or a sparse matrix. The leaves of the tree store matrix sub-blocks that are represented in full-matrix format or Rk-matrix (low rank matrix) format. H-matrix arithmetic is defined over the H-matrix representation, which includes operations such as addition, multiplication, inversion, and LU factorization. These H-matrix operations approximate results with almost optimal computational complexity. Based on the properties of H-matrices, the H-matrix preconditioner technique has been introduced. It uses H-matrix operations to construct preconditioners, which are used in iterative methods to speed up the solution of large systems of linear equations (Ax = b). To apply the H-matrix preconditioner technique, the first step is to represent a problem in H-matrix format. The approaches to construct an H-matrix can be divided into two categories: geometric approaches and algebraic approaches. In this thesis, we present our contributions to algebraic H-matrix construction approaches and H-matrix preconditioner technique. We have developed a new algebraic H-matrix construction approach based on matrix graphs and multilevel graph clustering approaches. Based on the new construction approach, we have also developed a scheme to build algebraic H-matrix preconditioners for systems of saddle point type. To verify the effectiveness of our new construction approach and H-matrix preconditioner scheme, we have applied them to solve various systems of linear equations arising from finite element methods and meshfree methods. The experimental results show that our preconditioners are competitive to other H-matrix preconditioners based on domain decomposition and existing preconditioners such as JOR and AMG preconditioners. Our H-matrix construction approach and preconditioner technique provide an alternative effective way to solve large systems of linear equations.