This paper presents the results of analytical study of recurrent neural networks (RNN) and their summary classification, done from the position of dynamical systems with regard to a new type of RNN – reservoir RNN. Knowledge systematization in the subject field allowed to distinguish main dynamical regimes of RNN and to describe the most perspective trends in the development of training algorithms in terms of the detected advantages and drawbacks of existing approaches.