Pet recognition method combining face and voiceprint

A pet identification method combining face and voiceprintTechnical field

[0001] The present invention relates to Convolutional Neural Networks (CNN) and face recognition technology. The convolutional neural network uses 2DCNN and 3DCNN. The face recognition technology draws on the FaceNet network structure and loss function calculation ideas. Two recognition methods of face recognition and voiceprint recognition are used, and the two recognition methods are combined at the level of output results to obtain higher accuracy.

Background technique

[0002] With the continuous advancement of social technology and the urgent requirements for automatic identity verification in various aspects, biometric identification technology has been rapidly developed in recent decades. As an inherent attribute of living beings, biological characteristics have strong self-stability and individual differences, making it the most ideal basis for automatic identity verification. Among the many biometric technologies, facial recognition has more prominent direct, effective, and aspect characteristics than other recognition methods, and thus has been widely studied and applied.

[0003] In recent years, deep learning technology has sprung up, and breakthroughs have been made in the field of face recognition. Some models, such as FaceNet, Face++, DeepID2, etc., have a recognition rate of more than 99%, which is a human face. The large-scale application of recognition technology has laid a solid foundation in actual projects. At the same time, the application of deep learning technology in voiceprint recognition has also achieved great success.

[0004] However, it is worth noting that in real life, it is not only humans who need biometric technology. Cats, dogs, pigs, sheep, cows and other animals can be combined with biometric technology to optimize production and life. purpose. However, in these fields, there are few related researches and applications, and it is urgent for scientific researchers to develop them.

[0005] Patent 201410006204.6 and patent 201611032333.8 each propose a pet identification method, but both only involve image identification, and do not use voiceprint identification method. Contents of the invention

[0006] In order to overcome the low potential recognition accuracy of the existing pet recognition methods, in view of the above problems, the present invention combines facial recognition and voiceprint recognition technologies to propose a combination of facial recognition and voiceprint recognition with higher recognition accuracy. Pet identification method.

[0007] The technical solutions adopted by the present invention to solve the above technical problems are as follows:

[0008] A pet identification method combining face and voiceprint includes the following steps:

[0009] S1: Initialize the pet recognition classifier, including classifier structure initialization and classifier weight initialization;

[0010] S2: Obtain image data, and obtain voiceprint data;

[0011] S3: Classify and label the data;

[0012] S4: Voiceprint data processing;

[0013] S5: Iteratively update the classifier;

[0014] S6: Determine whether the classifier meets the accuracy requirements, if so, save the current parameters and end the program, if not, continue training.

[0015] Further, the step S1 includes the following steps:

[0016] Sl.l: Initialize the structure of the pet recognition classifier, the structure of the classifier in the facial recognition part is FaceNet, and the structure in the voiceprint recognition part is the 3D convolutional neural network;

[0017] S1.2: Initialize the weights of the classifier with an initialization function, and use the pre-training model trained on FaceNet on its own data set;

[0018] S1.3: The entire network structure is combined at the verification of facial recognition and voiceprint recognition, and the combination method is weighted addition of the calculated Euclidean distances of the two parts;

[0019] Still further, the step S2 includes the following steps:

[0020] S2.1: Supplement image data by using a method of manual field camera collection;

[0021] S2.2: Use a web crawler method to crawl image data corresponding to requirements from the Internet;

[0022] S2.3: The voiceprint data is stripped from the collected video image data.

[0023] The step S3 includes the following steps:

[0024] S3.1: Manually label and classify the collected image data and voiceprint data;

[0025] S3.2: Divide the data into training data and test data.

[0026] The step S4 includes the following steps:

[0027] S4.1: Use the SpeechPy package of python to extract the voiceprint features,

[0028] The calculation formula for MFEC features is:

[0029] £=Blood Worker$: :«=1

[0030] Where, E is the energy value, sn audio signal, and N is the number of signals.

[0031] S4.2: Convert the input tensor into a format of [2,80,40], where 80 is a temporal feature (this feature is related to the duration of the audio file), and 40 is MFEC features (this feature is extracted by SpeechPy).

[0032] The step S5 includes the following steps:

[0033] S5.1: Perform batch combination of the training set data in the collected pet pictures according to the unique triple loss function of FaceNet; the triple loss function is defined as:

[0034] J = --- £-%" + «]

[0035] where J represents the loss function value, m is the number of samples, ji* represents the distance between the feature of the target sample and the positive sample, Dia,in represents the distance between the feature of the anchor and the negative sample, and ia represents the i-th target sample , Ip represents the i-th positive sample, in represents the i-th negative sample, and α represents the supplement amount;

[0036] S5.2: Input the training set data in the pet picture into the pre-training model obtained in step S1 in batches to fine-tune the parameters.

[0037] S5.3: In the voiceprint recognition part, the corresponding voiceprint training set is combined in batches according to the batch combination of the previous picture training set, and the voiceprint training set is sent to the voiceprint training network Perform iterative training.

[0038] The step S6 includes the following steps:

[0039] S6.1: Use the test data set to test the accuracy of the classifier;

[0040] In the face recognition part, the accuracy test steps are as follows:

[0041] The first step is to send all the pictures in the gallery to FaceNet to extract and extract them into 128-dimensional feature vectors, and then store them in the comparison database.

[0042] The second step is to input a certain picture to be tested, and extract its feature vector.

[0043] In the voiceprint recognition part, the accuracy test steps are as follows:

[0044] The first step is to send all the audio files in the audio library into the trained 3D convolutional network to obtain a sound model, that is, a 128-dimensional feature vector, and store all the sound models in the database.

[0045] The second step is to input a certain audio file to be tested, and extract its feature vector.

[0046] In conjunction with the part, the accuracy test steps are as follows:

[0047] The first step is to subtract the feature vectors of the test picture and the test audio file from the feature vector of the same pet in the database and obtain two Euclidean distances.

[0048] The second step is to perform a weighted addition on the two Euclidean distances to obtain a distance score. By analogy, the test data and the data in the database are all matched.

[0049] The third step is to sort all distance scores from small to large, and take the topi tag as the final recognition result. If the tag and the test data tag are the same, the recognition is successful, otherwise, the recognition fails.

[0050] S6.2: If the accuracy meets the requirements, the program ends, and if the accuracy requirements are not reached, return to S5 to continue training.

[0051] The method of the present invention has the following beneficial effects:

[0052] (1) The method of the present invention has the advantage of identifying individuals for large quantities of pets, and has a good application prospect

[0053] (2) The method of the present invention combines two recognition methods of pet facial recognition and voiceprint recognition, and has the advantage of high recognition accuracy.

[0054] (3) The method of the present invention adopts a pre-training model with a small amount of parameters, and has a better performance experience in recognition speed.

Description of the drawings

[0055] FIG. 1 is a flowchart of a pet identification method combining face and voiceprint.

[0056] FIG. 2 is a specific flow chart of a training process of a pet recognition method combining face and voiceprint.

[0057] FIG. 3 is a specific flowchart of a test process of a pet identification method combining face and voiceprint.

Detailed ways

[0058] The present invention will be further described below in conjunction with the accompanying drawings.

[0059] Referring to Figure 1, Figure 2 and Figure 3, a pet identification method combining face and voiceprint includes the following steps:

[0060] S1: Initialize the pet recognition classifier, including classifier structure initialization and classifier weight initialization;

[0061] S2: Obtain image data, and obtain voiceprint data;

[0062] S3: Classify and label the data;

[0063] S4: Voiceprint data processing;

[0064] S5: Iteratively update the classifier;

[0065] S6: Determine whether the classifier meets the accuracy requirements, if so, save the current parameters and end the program, if not, continue training.

[0066] This example classifies and discriminates the picture data collected by itself, and the method includes the following steps:

[0067] S1: Initialize the structure of the pet recognition classifier

[0068] In the experiment, the face recognition part adopts the FaceNet structure (CNN structure, the loss function adopts the triple function), the voiceprint recognition part adopts the 3D neural convolution network, and the recognition network structure is built on the tensorflow platform, and

Enter the pre-trained model published by the FaceNet author as the initial weight parameter.

[0069] S2: Acquire image data and voiceprint data

[0070] The image data and voiceprint data used in the experiment come from a data set released by Google in 2017, and this data set is named Audioset.

[0071] S3: Classification and labeling of data

[0072] Manually classify and label the data, and divide the data into a training set and a test set in proportion.

[0073] S4: Voiceprint data processing

[0074] S4.1: Use the SpeechPy package of python to extract the voiceprint features,

[0075] The calculation formula for MFEC features is:

[0076] E=log ^5; n=1

[0077] Where, E is the energy value, sn audio signal, and N is the number of signals.

[0078] S4.2: Convert the input tensor into a format of [ξ,80,40], where 80 is temporal feature (this feature is related to the duration of the audio file), and 40 is MFEC features (this feature is extracted by SpeechPy)

[0079] S5: Iteratively update the classifier

[0080] S5.1: Perform batch combination of the training set data in the collected pet pictures according to the unique triple loss function of FaceNet; the triple loss function is defined as:

[0081] Eight Yu Yi[.For"-^Dl,: +« ].

2m j

[0082] Where, J represents the loss function value, m is the number of samples, Dia* represents the distance between the feature of the target sample and the positive sample, Dchong represents the distance between the feature of the target sample and the negative sample, and ia represents the i-th target Sample, ip represents the i-th positive sample, in represents the i-th negative sample, and α represents the supplement amount;

[0083] S5.2: Input the training set data in the pet picture into the pre-training model obtained in step S1 in batches to fine-tune the parameters.

[0084] S5.3: In the voiceprint recognition part, according to the batch combination of the previous picture training set, the corresponding voiceprint training set is combined in batches, and the voiceprint training set is sent to the voiceprint training network Perform iterative training.

[0085] S6: Determine whether the classifier meets the accuracy requirements

[0086] In the face recognition part, the accuracy test steps are as follows:

[0087] The first step is to send all the pictures in the gallery to FaceNet to extract and extract them into 128-dimensional feature vectors, and then store them in the comparison database.

[0088] The second step is to input a certain picture to be tested, and extract its feature vector.

[0089] In the voiceprint recognition part, the accuracy test steps are as follows:

[0090] The first step is to send all audio files in the audio library to the trained 3D convolutional network to obtain a sound model, that is, a 128-dimensional feature vector, and store all the sound models in the database.

[0091] The second step is to input a certain audio file to be tested, and extract its feature vector.

[0092] In conjunction with the part, the accuracy test steps are as follows:

[0093] The first step is to subtract the feature vectors of the test picture and the test audio file from the phasor and acoustic factor model of the same pet in the database to obtain two Euclidean distances.

[0094] The second step is to perform a weighted addition on the two Euclidean distances (here the weight of the Euclidean distance of the facial recognition part is

0.8, the Euclidean distance weight of the voiceprint recognition part is 0.2), and a distance score is obtained. By analogy, the test data and the data in the database are all matched.

[0095] The third step is to sort all distance scores from small to large, and take the tag of topi as the final recognition result. If the tag is the same as the tag of the test data, the recognition is successful, otherwise, the recognition fails.