Prediction guided sequential data learning method

STATEMENT AS TO RIGHTS TO INVENTIONS MADE UNDER FEDERALLY SPONSORED RESEARCH AND DEVELOPMENT

0001This work was supported by U.S. Government grant number 1R44NS097094-01A1, awarded by the NATIONAL INSTITUTE OF NEUROLOGICAL DISORDERS AND STROKE and U.S. Government grant number 5R43MH100780-02, awarded by the NATIONAL INSTITUTE OF MENTAL HEALTH. The U.S. Government may have certain rights in the invention.

BACKGROUND OF THE INVENTION

Field of the Invention

0002The present invention relates to machine learning and more particularly to a prediction guided sequential data learning method including semantic learning, update learning, and update and semantic learning.

Description of the Related Art

0000a. Description of Problem that Motivated Invention.

0003Machine learning, especially deep learning, powered by the tremendous computational advancement in graphics processing units (GPUs) and the availability of big data, has gained significant attention and is being applied to many new fields and applications. It can support end-to-end learning and learn hierarchical feature representation automatically. It is highly scalable and can achieve better prediction accuracy with more data. To handle large variations and dynamics inherent in sequential data, high capacity model is often required. It could be incredibly effective when trained with high capacity models (>10⁸parameters).

0004However, high capacity models require the training of huge labeled (annotated) datasets to avoid over-fitting. For example, the image database ImageNet contains 1.2 million images with 1000 categories for deep network training. In this highly connected mobile and cloud computing era, big datasets are becoming readily available. Therefore, the bottleneck is in acquiring the labels rather than the data. The situation is exacerbated with the ever increasing size of big databases.

0000b. How did Prior Art Solve Problem?

0005Prior art approaches use crowdsourcing such as AMT (Amazon Mechanical Turk) to get large training sets by having large numbers of people hand-label lots of data. There are also video games such as “Mozak”, “EVE Online: Project Discovery” designed to crowdsource the creation of labels by the game players. These approaches could be expensive and are hard to scale and the labeling quality is poor.

0006Because of the deficiencies of the prior art approaches, improved methods of machine learning, particularly for classifying sequential data, are urgently needed.

BRIEF SUMMARY OF THE INVENTION

0007The primary objective of this invention is to provide a computerized prediction guided sequential data learning method for efficient initial learning without labeling data and accurate semantic classification with a small number of labeled training data. The secondary objective of the invention is to provide a computerized prediction guided sequential data learning method for efficient initial learning without labeling data and update learning with a small number of labeled data for accurate data classification. The third objective of the invention is to provide a computerized prediction guided sequential data learning method for efficient initial learning without labeling data and semantic and update learning with a small number of labeled data for accurate semantic and data classification. The fourth objective of this invention is to provide a computerized self-supervised learning method to learn the rich internal representation for the sequential data without labeled data.

0008The current invention provides prediction guidance by self-supervised learning for sequential data. It first learns by inputting a stream of unlabelled data sequence and tries to predict a future input from the current and past inputs to generate an initial classifier. Since future inputs are also available in the data sequence, they can serve as labeled training data without explicit labeling. By learning to predict on a large amount of self-supervised data, the initial classifier creates a rich internal representation of high-order kinetic phenotypes to predict future inputs.

0009Afterwards, we can solve a specific classification task by prediction guided update learning. This is done by taking the learned feature representation embedded in the initial classifier and a small amount of labeled data for the targeted classification task, and apply supervised learning on that labeled data to solve the targeted classification task.

0010In brief, the methods according to the present invention includes a prediction learning, followed by a prediction guided learning, which may be semantic learning, update learning or update and semantic learning.

BRIEF DESCRIPTION OF THE DRAWINGS

0011<figref idref="DRAWINGS">FIG. 1</figref> shows the processing flow of one embodiment of the prediction guided sequential data learning method according to the present invention.

0012<figref idref="DRAWINGS">FIG. 2</figref> shows the processing flow of an alternative embodiment of the prediction guided sequential data learning module.

0013<figref idref="DRAWINGS">FIG. 3</figref> shows the processing flow of yet another embodiment of the prediction guided sequential data learning module.

0014<figref idref="DRAWINGS">FIG. 4</figref> shows the processing flow of one embodiment of the prediction learning module.

0015<figref idref="DRAWINGS">FIG. 5</figref> shows the processing flow of one embodiment of the supervised prediction learning module implemented as a deep network.

0016<figref idref="DRAWINGS">FIG. 6</figref> shows the processing flow of an embodiment of the supervised prediction learning module implemented as a recurrent network.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

0017The concepts and the preferred embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings.

0000I. Application Scenarios

0018<figref idref="DRAWINGS">FIG. 1</figref> shows the processing flow of one embodiment of the prediction guided sequential data learning method according to the present invention. A data sequence <b>100</b> is inputted to the memory of a computer hosting a computerized machine learning tool. A prediction learning module <b>102</b> processes the data sequence <b>100</b> by a computer program of the computerized machine learning tool to generate an initial classifier <b>104</b>. The prediction learning is performed without explicitly labeled data. A semantic label data <b>108</b> is inputted into the computer memory. The semantic label data <b>108</b> associates a subset of data sequence <b>100</b> with their semantic labels or it could be a new set of labeled data. A prediction guided semantic learning module <b>106</b> uses the data sequence <b>100</b>, the initial classifier <b>104</b> and the semantic label data <b>108</b> to generate an output classifier <b>110</b> and semantic classification <b>112</b> for the data sequence <b>100</b>. After learning, the output classifier <b>110</b> can be applied to new data sequences.

0019<figref idref="DRAWINGS">FIG. 2</figref> shows the processing flow of an alternative embodiment of the prediction guided sequential data learning method. A data sequence <b>100</b> is inputted to the memory of a computer hosting a computerized machine learning tool. A prediction learning module <b>102</b> processes the data sequence <b>100</b> by a computer program of the computerized machine learning tool to generate an initial classifier <b>104</b>. The prediction learning is performed without explicitly labeled data. A label data <b>202</b> is inputted into the computer memory. The label data <b>202</b> associates a subset of data sequence <b>100</b> with their truth labels. Additional data with labels could also be included. A prediction guided update learning module <b>200</b> uses the data sequence <b>100</b>, the initial classifier <b>104</b> and the label data <b>202</b> to generate an output classifier <b>110</b> and a data classification <b>204</b> for the data sequence <b>100</b>. After learning, the output classifier <b>110</b> can be applied to new data sequences.

0020<figref idref="DRAWINGS">FIG. 3</figref> shows the processing flow of a third embodiment of the prediction guided sequential data learning method. A data sequence <b>100</b> is inputted to the memory of a computer hosting a computerized machine learning tool. A prediction learning module <b>102</b> processes the data sequence <b>100</b> by a computer program of the computerized machine learning tool to generate an initial classifier <b>104</b>. The prediction learning is performed without explicitly labeled data. A semantic and label data <b>302</b> is inputted into the computer memory. The semantic and label data <b>302</b> associates a subset of data sequence <b>100</b> with their semantic labels and can include additional labeled data. A prediction guided update and semantic learning module <b>300</b> uses the data sequence <b>100</b>, the initial classifier <b>104</b> and the semantic and label data <b>302</b> to generate an output classifier <b>110</b>, a semantic classification <b>112</b> and a data classification <b>204</b> for the data sequence <b>100</b>. After learning, the output classifier <b>110</b> can be applied to new data sequences.

0000II. Data Sequence

0021The data sequence <b>100</b> consists of data ordered in a sequential fashion such as the information from languages, music, genomes, videos, plane slices of 3D images, etc. The essential property of the data sequence <b>100</b> is that data is ordered.

0000III. Semantic Labels

0022The semantic labels in the semantic label data <b>108</b> define the semantic meaning of the data sequence <b>100</b>. They can be cellular states and/or phenotypic regions of interest in a data sequence consisting of time-lapse cellular images. The semantic labels can also be objects of interest in a data sequence consisting of video clips. A person having ordinary skill in the art should recognize that other semantic labels such as the words contained in a speech clips or gene labels of a DNA sequence. They are within the scope of the current invention.

0000IV. Prediction Learning

0023<figref idref="DRAWINGS">FIG. 4</figref> shows the processing flow of one embodiment of the prediction learning module <b>102</b>. As shown in <figref idref="DRAWINGS">FIG. 4</figref>, the prediction learning module <b>102</b> performs a delay buffering <b>400</b> of the data sequence <b>100</b> to generate a delayed data sequence <b>402</b> which delays the data sequence <b>100</b> by an amount (for example D cycles) that is defined manually or automatically depending on the application scenarios underlying the data sequence <b>100</b>. The delayed data sequence <b>402</b> and the data sequence <b>100</b> are processed by a supervised prediction learning module <b>404</b> to generate the initial classifier <b>104</b>. The supervised prediction learning module <b>404</b> uses delayed data sequence <b>402</b> as the input to predict the input data sequence <b>100</b> which is effectively D cycles ahead of the delayed data sequence <b>402</b>. The supervised prediction learning module <b>404</b> uses data sequence from sequence index (or cycle) 1 to i-D for learning to predict the data sequence from i-D+1 to i, where i is the index for the current cycle of the data sequence <b>100</b>. The data at the sequences i-D+<b>1</b> to i are available from the data sequence <b>100</b> and are used directly as the truth data for the supervised prediction learning without the data labeling (annotation). Therefore a large number of training data can be made available through the prediction learning method of the current invention to sufficiently train the initial classifier.

0024By learning to predict a large amount of data through self-supervision, the initial classifier <b>104</b> could create a rich internal representation of high-order models for the sequential data <b>100</b>. Note that the prediction of future inputs may not have practical value as we will have them from input data. But it is used to force the classifiers such as deep network to learn to model the rich high-order application domain models.

0025In one embodiment of the invention, the supervised prediction learning <b>404</b> is implemented by a deep network. In another embodiment of the invention, the supervised prediction learning is implemented by a recurrent network. In yet a third embodiment of the invention, the supervised prediction learning is implemented by traditional machine learning methods.

0026These three implementations for the supervised prediction learning <b>404</b> are separately discussed below.

0027A. Deep Network

0028Deep network is rooted at artificial neural network facilitated by tremendous computational advancement (GPUs) and the availability of big data. The recent trend in deep layers of convolutional neural networks has dramatically changed the landscape in machine learning and pattern recognition. It uses a cascade of many layers of nonlinear processing units for feature extraction and transformation. Each successive layer uses the output from the previous layer as input. It learns multiple levels of features or representations of the data. Higher level features are derived from lower level features to form a hierarchical representation. It can be effectively scaled up to high capacity models. The traditional machine learning method is only partially trainable. They require hand-designed feature extraction followed by trainable classifier operating on hand-designed features. In contrast, the deep networks allow the learning of hierarchical feature representation automatically as well as the classifier.

0029<figref idref="DRAWINGS">FIG. 5</figref> shows the processing flow of an embodiment of the supervised prediction learning module <b>404</b> implemented as a deep network. The input data sequence <b>500</b> (corresponding to delayed data sequence <b>402</b>) is processed by at least one input layer <b>502</b>, followed by a plurality of hidden layers <b>504</b> and then the output layer <b>506</b>. In the training phase, the training data sequence <b>510</b> (corresponding to data sequence <b>100</b> in <figref idref="DRAWINGS">FIG. 4</figref>) is backpropagated from the output layer back to the hidden layers and then input layer to update their weights (see the dashed lines for the training flow). After training, the output classifier <b>512</b> (corresponding to initial classifier <b>104</b> in <figref idref="DRAWINGS">FIG. 4</figref>) contains the complete deep network architecture and their parameters (weights). In the classification phase, the input data sequence <b>500</b> is processed in a feed-forward fashion to generate the classification output <b>508</b>.

0030In addition to pattern classification, the deep network can also perform semantic segmentation. For example, in image data, the semantic segmentation provides per pixel labeling. To perform semantic segmentation, fully convolutional network can be used. These networks yield a coarse segmentation map for any given data, and it is followed by upsampling within the network to get dense predictions. This method enables an end-to-end training for the task of semantic segmentation of data. Typical fully convolutional networks include U-Net, deconvolution networks, SegNet, Dilated convolution network, Sharpmask and DeepLab, etc.

0031B. Recurrent Network

0032The conventional deep network such as Convolutional Neural Networks (CNN) are not designed to handle sequential data. The simplest way to include sequential data in CNN is to concatenate multiple cycles and feed it as a single input. Small variations of this method are used for context classification on one million youtube videos. However, it could not improve on single frame prediction by much which can indicate the inefficiency of this approach.

0033To handle sequential data, in another embodiment of the invention, recurrent network is used. Recurrent networks take as their input not just the current input data, but also the information extracted from previous cycles. Because the layers and cycles of deep networks relate to each other through multiplication, derivatives are susceptible to vanishing or exploding. The vanishing gradient problem emerged as a major obstacle to recurrent network performance. This problem is solved by a recurrent unit <b>600</b> such as Long Short-Term Memory Units (LSTMs).

0034LSTMs contain information outside the normal flow of the recurrent network in a gated cell. Information can be stored in, written to, or read from a cell, much like data in a computer's memory. The cell makes decisions about what to store, and when to allow reads, writes and erasures, via gates that open and close. Unlike the digital storage on computers, however, these gates are analog, implemented with element-wise multiplication by sigmoids, which are all in the range of 0-1. Analog has the advantage over digital of being differentiable, and therefore suitable for backpropagation.

0035<figref idref="DRAWINGS">FIG. 6</figref> shows the processing flow of an embodiment of the supervised prediction learning module <b>404</b> implemented as a recurrent network containing a recurrent unit <b>600</b>. The input to the recurrent unit includes the current data and previous cycle recurrent unit output, etc. The recurrent unit can be a LSTM or its variations such as Convolutional LSTM (CLSTM) and Bi-Directional Convolutional LSTM (BDC-LSTM). A person having ordinary skill in the art should recognize that other recurrent units such as Gated Recurrent Units (GRU) can be used. They are within the scope of the current invention.

0036C. Traditional Machine Learning Methods

0037Even though the traditional machine learning methods require hand-designed features, they can also be trained to predict future data. For prediction guidance, the prediction training can be performed to select good features from sample prediction training data set. The selected features can then be used for the next stage prediction guided learning.

0038The traditional machine learning methods that could be used include decision tree classifier, random forest classifier, support vector machine, kernel estimator, mixture of Gaussian classifier, nearest neighbor classifier, etc. A person having ordinary skill in the art should recognize that other traditional machine learning methods such as naive Bayes classifier, maximum likelihood classifier, Bayes linear and quadratic classifiers can be used and they are within the scope of the current invention.

0039V. Prediction Guided Learning

0040As shown in <figref idref="DRAWINGS">FIGS. 1-3</figref>, the prediction guided learning inputs the initial classifier <b>104</b> that is trained by prediction learning (<b>106</b>, <b>200</b> or <b>300</b>). The prediction guided learning performs update training of the classifier with new label data (<b>108</b>, <b>202</b>, <b>302</b>).

0041To train an entire classifier such as deep network from scratch with random initialization requires a large labeled dataset and is computationally demanding and time consuming. The prediction guided learning starts from initial classifier <b>104</b> that is trained by self-supervised prediction learning <b>102</b>. The prediction guided learning is then trained for the outcomes of interest by a small amount of labeled data through fine-tune learning.

0042The prediction guided learning can be considered a kind of transfer learning. In one embodiment of the invention, the initial classifier <b>104</b> is used as an initialization state for fine-tuning. In the fine-tune learning, the parameters such as the weights of deep network or recurrent network can be updated by continuing the learning with the labeled data. In one embodiment of the invention, the whole classifier is updated. In another embodiment of the invention, the earlier layers of deep network are fixed (due to overfitting concerns) and only higher-level portion of the network is updated by fine-tune learning.

0043In yet another embodiment of the invention, the initial classifier <b>104</b> is used as a fixed feature extractor for new applications. The last layer and/or higher-level portion of the network are removed, then the rest of the initial classifier <b>104</b> is treated as a fixed feature extractor, and a traditional machine learning method is trained for the new labeled data. This could be supplemented with conventional features as well.

0044A. Prediction Guided Semantic Learning

0045In the prediction guided semantic learning module <b>106</b> implemented in the embodiment shown in <figref idref="DRAWINGS">FIG. 1</figref>, the semantic outcomes of interest are not the future data that is predicted in the initial classifier <b>104</b>. However, the initial classifier <b>104</b> trained by prediction learning forms a model representation that captures the intrinsic characteristics and dynamics of the data sequence in deep network. In the case of prediction learning <b>102</b> using traditional machine learning methods, the initial classifier <b>104</b> extracts the discriminate features from the prediction learning <b>102</b> for use in the prediction guided semantic learning <b>106</b>.

0046In the case of prediction learning <b>102</b> using deep network or recurrent network, the prediction guided semantic learning <b>106</b> will use the same deep network or recurrent network. But rather than starting with random weights, the prediction guided semantic learning <b>106</b> starts with the parameters from the initial classifier <b>104</b>. Therefore the prediction guided semantic learning <b>106</b> could be trained with a small number of semantic label data <b>108</b> and can be trained with fewer iterations and can yield good accuracy for the output classifier <b>110</b> and semantic classification <b>112</b>.

0047In the case of prediction learning <b>102</b> using traditional machine learning methods, the prediction guided semantic learning <b>106</b> will use the same traditional machine learning methods. But rather than starting with all features, the prediction guided semantic learning <b>106</b> starts with the features extracted from the initial classifier <b>104</b>. Therefore the prediction guided semantic learning <b>106</b> could be trained with a small number of semantic label data <b>108</b> with fewer features and can yield good accuracy for the output classifier <b>110</b> and semantic classification <b>112</b>.

0048B. Prediction Guided Update Learning

0049In the prediction guided update learning module <b>200</b> implemented in the embodiment shown in <figref idref="DRAWINGS">FIG. 2</figref>, the outcomes of interest are in line with or can be derived from the future data that is predicted in the initial classifier <b>104</b>. Therefore, the prediction guided update learning <b>200</b> is simply a continued learning with explicit labeled data for fine-tune learning.

0050In the case of prediction learning <b>102</b> using deep network or recurrent network, the prediction guided update learning <b>200</b> will continue to use the same deep network or recurrent network. It will start with the parameters from the initial classifier <b>104</b> with additional training by a small number of label data <b>202</b> and can be trained with fewer iterations and can yield good accuracy for the output classifier <b>110</b> and data classification <b>204</b>, targeted at the intended data. In the case of prediction learning <b>102</b> using traditional machine learning methods, the prediction guided update learning <b>200</b> uses the same traditional machine learning methods. It starts with the features extracted from the initial classifier <b>104</b> with additional training by a small number of label data <b>202</b> and can be trained with fewer features yet can yield good accuracy for the output classifier <b>110</b> and data classification <b>204</b>, targeted at the intended data.

0051C. Prediction Guided Update and Semantic Learning

0052In the prediction guided update and semantic learning module <b>300</b> implemented in the embodiment shown in <figref idref="DRAWINGS">FIG. 3</figref>, the output classifier <b>110</b> can perform both semantic classification <b>112</b> and data classification <b>204</b>. In the case of prediction learning <b>102</b> using deep network or recurrent network, the prediction guided update and semantic learning <b>300</b> uses the same deep network or recurrent network. But rather than starting with random weights, the combined semantic and data learning starts with the parameters of the initial classifier <b>104</b>. Therefore the prediction guided update and semantic learning <b>300</b> could be trained with a small number of semantic and label data <b>302</b>. It can be trained with fewer iterations and can yield good accuracy for the output classifier <b>110</b> and both semantic classification <b>112</b> and data classification <b>204</b>.

0053In the case of prediction learning <b>102</b> using traditional machine learning methods, the prediction guided update and semantic learning <b>300</b> uses the same traditional machine learning methods. But rather than starting with all features, the prediction guided update and semantic learning <b>300</b> starts with the features extracted from the initial classifier <b>104</b>. Therefore the prediction guided update and semantic learning <b>300</b> could be trained with a small number of semantic and label data <b>302</b>. It can be trained with fewer features yet yield good accuracy for the output classifier <b>110</b> and both semantic classification <b>112</b> and data classification <b>204</b>.

0054The invention has been described herein in considerable detail in order to comply with the Patent Statutes and to provide those skilled in the art with the information needed to apply the novel principles and to construct and use such specialized components as are required. However, it is to be understood that the inventions can be carried out by specifically different equipment and devices, and that various modifications, both as to the equipment details and operating procedures, can be accomplished without departing from the scope of the invention itself.