Existing residential and small commercial buildings now represent the greatest opportunity to improve building energy efficiency. Building energy simulation analysis is becoming increasingly important because the analysis results can assist the decision makers to make decisions on improving building energy efficiency and reducing environmental impacts. However, manually measuring as-is conditions of building envelops including geometry and thermal value is still a labor-intensive, costly, and slow process. Thus, the primary objective of this research was to automatically collect and extract the as-is geometry and thermal data of the building envelope components and create a gbXML-based building geometry model. In the proposed methodology, a rapid and low-cost data collection hardware system was designed by integrating 3D laser scanners and an infrared (IR) camera. Secondly, several algorithms were created to automatically recognize various components of building envelope as objects from collected raw data. The extracted 3D semantic geometric model was then automatically saved as an industry standard file format for data interoperability. The feasibility of the proposed method was validated through three case studies. The contributions of this research include 1) a customized low-cost hybrid data collection system development to fuse various data into a thermal point cloud; 2) an automatic method of extracting building envelope components and its geometry data to generate gbXML-based building geometry model. The broader impacts of this research are that it could offer a new way to collect as is building data without impeding occupants’ daily life, and provide an easier way for laypeople to understand the energy performance of their buildings via 3D thermal point cloud visualization.