In this note we consider the autoregressive moving average recurrent neural network ARMA-NN(1, 1) process. We show that in contrast to the pure autoregressive process simple ARMA-NN processes exist which are not irreducible. We prove that the controllability of the linear part of the process is sufficient for irreducibility. For the irreducible process essentially the shortcut weight corresponding to the autoregressive part determines whether the overall process is ergodic and stationary. / Series: Working Papers SFB "Adaptive Information Systems and Modelling in Economics and Management Science"