Speaker's flu symptoms recognition method fused with multiple end-to-end neural network structures

Speaker cold symptom recognition method integrating multiple end-to-end neural network structures

<heading id="h01"></heading>

1Technical field

2The present invention relates to the technical field of speech processing, and proposes a speaker cold symptom recognition method integrating multiple end-to-end deep learning structures

3Background technique

41. Speaker recognition, also known as voiceprint recognition, refers to the technology that automatically recognizes the speaker through the use of unique speaker information in the voice and uses pattern recognition technology. The current speaker technology achieves very good performance in experimental conditions, but in practice, speech will be affected by environmental noise and speaker health conditions, which reduces the robustness of existing speaker recognition technologies, and cold voice recognition methods By classifying the existing speech to judge whether it is a cold speech, by using a cold speech recognition method to judge whether the speech is a cold speech in advance, and then performing speaker recognition, the robustness of speaker recognition can be improved.

52. In the research of speech technology, researchers always hope to find the characteristics of the target type, and find the characteristics that clearly distinguish the normal speech from the recognition target speech for description. Speech feature extraction is to extract the speaker's speech characteristics and vocal tract characteristics. At present, the mainstream feature parameters include MFCC, LPCC, CQCC, etc., all of which are based on a single feature. There is insufficient information to characterize the speaker's cold symptoms, which affects the recognition accuracy. At the same time, a large amount of knowledge to distinguish the classification target speech is required. In the speech recognition algorithm, the method based on the vocal tract model and the knowledge of the speech model started earlier. However, due to the complexity of the model, it has not achieved good practical results. Matching methods such as dynamic time warping, hidden Markov models, vector quantization and other technologies have begun to exert good recognition effects. The separation of feature extraction and pattern classification is a common method of recognition research, but there are problems such as feature and model mismatch, training difficulties, and feature not easy to find. The classic recognition framework has the above problems.

63. With the development of deep learning in recent years, the recognition of images and speech based on deep neural networks has shown great energy, and a series of neural network structures have also been proposed, such as automatic coding networks, convolutional neural networks, and recurrent neural networks. Network, etc. Many scholars have found that learning speech through neural networks can better describe the hidden structural features of speech. Good recognition effect.

7Summary of the invention:

8According to the existing recognition technology, the feature and pattern classification are studied separately, and there are problems such as feature and model mismatch, difficulty in training, and difficulty in finding features. The present invention proposes a speaker cold symptom recognition method that integrates a variety of end-to-end deep learning structures. , We construct four different end-to-end deep learning networks, and finally merge four different end-to-end neural network structures to identify the speaker's cold symptoms.

9The four end-to-end deep learning structures are: 1. The input is speech, and the network is a multi-layer convolutional neural network and a long short-term memory network; 2. The input is a speech spectrum, and the network is a multi-layer convolutional neural network and a long-term memory network. Short-term memory network; 3. The input is a speech spectrum, and the network is a multi-layer convolutional neural network and a fully connected network; 4. The input is a Mel cepstrum coefficient and a constant Q cepstrum coefficient, and the network is a long and short-term memory network;

10The beneficial effects of the present invention are: based on the uncertainty of traditional features, the output we obtain through neural network training can better express the characteristics of the speaker's cold symptoms, and the input is relatively simple, without excessive feature processing . Because speech has time sequence information, we can achieve better results through long- and short-term memory network classification. By unifying feature learning and pattern classification, the whole speaker cold symptom recognition process is simpler and faster, and it has a wide range of application prospects.

11Description of the drawings

12Figure 1 shows the process of extracting Mel Cepstrum Coefficient (MFCC) from speech

13Figure 2 shows the process of extracting constant Q cepstral coefficient (CQCC) from speech

14Figure 3 shows the first end-to-end neural network, the input is speech, and the network is CNN+LSTM.

15Figure 4 is the second end-to-end neural network, the input is the speech spectrum, and the network is CNN+LSTM

16Figure 5 is the third end-to-end neural network, the input is the speech spectrum, and the network is CNN+ fully connected network

17Figure 6 shows the fourth end-to-end neural network. The input is Mel cepstrum coefficients or constant Q cepstrum coefficients, and the network is LSTM.

18Detailed ways:

19In order to make the technical solutions and advantages of the present invention clearer, the following describes the technical solutions of the invention clearly and completely in conjunction with the accompanying drawings:

20Step 1: Construct an end-to-end neural network with the input as speech and the network as CNN+LSTM, specifically: the input speech is divided into segments of the same size, such as 40ms, and then the mean is normalized, and the corresponding convolutional neural network Composed of 8 modules, each module is composed of a one-dimensional convolution layer, a ReLU activation layer, and a one-dimensional maximum pooling layer. Among them, the size of each convolution kernel is 32, and the size of the pooling kernel is 2. The pooling step is 2. Then use the long and short-term memory network for classification.

21Step 2: Construct the input as the speech spectrum, and the network is an end-to-end neural network of CNN+LSTM. Specifically: the input speech is divided into segments of the same size, and the fast Fourier transform is performed to obtain the spectrogram of the speech segment. Convolutional neural network is composed of 6 modules, each module is composed of a two-dimensional convolution layer, a ReLU activation layer, and a two-dimensional maximum pooling layer. Among them, the first convolution layer uses a 7*7 convolution layer, the second layer uses a 5*5 convolution kernel, the remaining 4 layers use a 3*3 convolution kernel, and all the maximum pooling layers use 3 *3 pooling core, pooling step size is 2. Finally, it is classified through the LSTM network.

22Step 3: Construct the input as the speech spectrum, and the network is an end-to-end neural network of CNN+LSTM. Specifically: the input speech is divided into segments of the same size, and the fast Fourier transform is performed to obtain the spectrogram of the speech segment. Convolutional neural network is composed of 6 modules, each module is composed of a two-dimensional convolution layer, a ReLU activation layer, and a two-dimensional maximum pooling layer. Among them, the first convolution layer uses a 7*7 convolution layer, the second layer uses a 5*5 convolution kernel, the remaining 4 layers use a 3*3 convolution kernel, and all the maximum pooling layers use 3 *3 pooling core, pooling step size is 2. It passes through a fully connected layer, and finally passes through Softmax for classification.

23Step 4: Construct the input as MFCC feature or CQCC feature, the network is an end-to-end neural network of LSTM, the MFCC feature pre-emphasizes the speech, windowing and framing, fast Fourier transform, and Mel scale triangular filter bank It is finally obtained after filtering, logarithmic operation, and discrete cosine transform. The CQCC feature is obtained by performing constant Q transformation on speech, finding the energy spectral density, taking logarithm operation, and cosine transform. Perform classification through a long- and short-term memory network. MFCC or CQCC features are extracted from the speech as the input of the neural network, and finally classified by the long and short-term memory network.

24Step 5: Integrate the above four networks together for speech recognition of the speaker's cold.