Purpose This project aims to explore the classification method of kinase inhibitors with five-channel cell painting image data based on the deep learning model. Methods A ResNet50 transfer learning model was used as the starting point to build the deep neural network (DNN) model, where different DNN parameters were selected to make the deep learning model more suitable for the cell painting data. Two different adaptive layers (adaptive average pooling 3D and convolution 2D) were added separately before the ResNet50 transfer learning model to adapt the five-layer cell painting image to the neural network. In addition, the skimage.transform.resize function was used to compress the five-layer cell painting image. Results The proposed deep learning model demonstrates the effectiveness in all three classification experiments. The proposed model performs particularly well in classifying among control, EGFR, PIKK and CDK kinase inhibitors families. It achieves an F1-score of 0.7764 on all four targets and has a 93\% accuracy rate in the PIKK kinase inhibitors family. The adaptive average pooling 3D layer successfully adapts the five-layer images to the model, resulting in an improved effect. The training time of the model is significantly reduced to one-fortieth by compressing the image size. Conclusion The proposed model achieved convincing effectiveness in classifying families, which showed progress in building the deep learning model to classify kinase inhibitors on five-channel cell painting data. This study also proved the feasibility of directly inputting five-channel cell painting images to DNN. In addition, the speed of the model increased sharply by compressing the image size without an obvious loss of data information.