In order to perform a specific function, a protein needs to fold into the proper structure. Prediction the protein structure from its amino acid sequence has still been unsolved problem. The main focus of this thesis is to develop new approach on the protein structure modeling by means of differential geometry and integrable theory. The start point is to simplify a protein backbone as a piecewise linear polygonal chain, with vertices recognized as the central alpha carbons of the amino acids. Frenet frame and equations from differential geometry are used to describe the geometric shape of the protein linear chain. Within the framework of integrable theory, we also develop a general geometrical approach, to systematically derive Hamiltonian energy functions for piecewise linear polygonal chains. These theoretical studies is expected to provide a solid basis for the general description of curves in three space dimensions. An efficient algorithm of loop closure has been proposed.