Development of Elements of Two-Level Biometric Protection Based on Face and Speech Recognition in the Video Stream

Abstract-this paper presents the results obtained when creating a prototype of a software complex that implements speech recognition in the video stream by the motion of the lips with the help of a neural network. This speech recognition based on lip’s motions is considered as a stage of two – level biometric authentication. The construction of a neural network model based on LSTM layers is also described here, which is basis of speech recognition’s realization. To train this model we assembled and processed database containing recordings of words from a given set of classes. To process and extract the words from the video, the lip model was developed and
optimized, described by the time variation of the geometric coordinates of the main points of the image of the lips. As result we achieved model that can recognize words from a given set of classes with an accuracy of 73.1%.