Real-time, minimal human intervention, and scalable classification of oak species, specifically Quercus petraea and Quercus robur, are crucial for forest management, biodiversity conservation, and ecological monitoring. Traditional methods are labor-intensive and costly, motivating the exploration of automated solutions. This study addresses the research problem of developing an efficient and scalable classification system using deep learning techniques. We developed a Convolutional Neural Network (CNN) from scratch and enhanced its performance with segmentation, fusion, and data augmentation techniques. Using a dataset of 649 oak leaf images, our model achieved a classification accuracy of 69.30% with a standard deviation of 2.48% and demonstrated efficient real-time application with an average processing time of 25.53 milliseconds per image. These results demonstrate the potential of deep learning to automate and improve the two oak species identification. This research provides a valuable tool for ecological studies and conservation efforts.