In recent years, many researches focused on the supervised classification, one of the machine learning methods, using Lidar and remotely sensed image to provide the four buildings, trees, roads, and grass categories of the ground features. However, buildings and trees are usually much closed or overlapped and this problem will lead buildings and nearby trees not easy to classify by single classification approach. The derived building outlines have many cracks which are not satisfactory for the requirement of GIS building vector map or building 3D modeling. To provide complete building outlines, this study develops an ¡§automatic detection of the overlapped areas of buildings and trees (ADOABT)¡¨ algorithm and an ¡§automatic linear feature recovery (ALFR)¡¨ approach to connect building outlines consequently. First, this research integrates Maximum Likelihood Classification (MLC) and Knowledge-Based Correction (KBC) to derive buildings and trees classification resultant images. Next, the ADOABT based on ¡§divide and conquer¡¨ principle was used to detect the overlapped areas of buildings and trees. Meanwhile, the building and tree edge images were detected using the Canny edge detector based on Lidar height image. Then, the intersection operator was applied to the detected areas and edge images to detect the crack of the building images. Afterward, vectorization and generalization of the intersection resultant images are applied to extract the straight line of the buildings. Finally, the automatic linear feature recovery procedure was performed to compensate the damage straight line effectively. According to the experiment results, the classification accuracy derived from integrated MLC and KBC classification method and the object-based classification (OBC) are similar. However, when applying the classification results to detect the overlapped areas of building and trees, because MLC and KBC has the procedure for handling temporal inconsistencies, the success rate of automatic detection is totally the same by artificial interpretation; the detection rate for the results of MLC and KBC is 100% whereas the one for the OBC only 67.7%. It can be concluded that the MLC and KBC approach is more suitable for the automatic detection for the overlapped areas of building and trees. Moreover, the ADOABT algorithm simplifies the workflow of the overlapped area detection. According to the result of edge detection and line detection, the Canny detector presents the clearest edge image. The lines extracted by Vectorized and generalization method are superior to the ones derived from Hough transform. The ALFR algorithm offers a way to connect building outline completely.