This study examines (1) whether L2 learners’ oral fluency and accuracy improve through conversation practice with AI tutors in a Japanese language learning app, (2) when fluency increases, and (3) which learning cycle is most effective. Ten participants joined the study and practiced conversations with Kaizen Languages’ AI tutors for three weeks. Learners selected their preferred learning cycle: a group with fewer lessons to complete per day but more practice days per week or a group with more lessons to complete per day but less frequent practice days per week. After three weeks of self-study, participants took an oral test to show how much they improved their fluency and accuracy in new contexts. The results of this study show that not only did learners’ oral fluency improve with the use of the Kaizen app, but their oral accuracy also increased. In addition, learners who improved their fluency practiced repeatedly, and their fluency improved until three or four repetitive practices. Although this study did not find an effective learning cycle, it implies that the ideal learning cycle requires at least three times of practice to improve fluency, while five to six times of practice produces higher accuracy.