Recommender system for identifying a new set of media items responsive to an input set of media items and knowledge base metrics

Description of equivalent WO 2006084102 A2

RECOMMENDER SYSTEM FOR IDENTIFYING A NEW SET OF MEDIA ITEMS

RESPONSIVE TO AN INPUT SET OF MEDIA ITEMS AND

KNOWLEDGE BASE METRICS

Related Applications

[0001] This application claims priority from U.S. Provisional Application No. 60/649,987 filed February 3, 2005, incorporated herein by this reference.

Technical Field

[0002] This invention relates generally to systems and methods for recommending media items to a user in a personalized manner. It particularly relates to "recommender" computer software systems for media items which are grouped by end users to define mediasets.

Background of the Invention

[0003] New technologies combining digital media item players with dedicated software, together with new media distribution channels through computer networks (e.g., the Internet) are quickly changing the way people organize and play media items. As a direct consequence of such evolution in the media industry, users are faced with a huge volume of available choices that clearly overwhelm them when choosing what item to play in a certain moment.

[0004] This overwhelming effect is apparent in the music arena, where people are faced with the problem of selecting music from very large collections of songs. However, in the future, we might detect similar effects in other domains such as music videos, movies, news items, etc.

[0005] In general, our invention is applicable to any kind of media item that can be grouped by users to define mediasets. For example, in the music domain, these mediasets are called playlists. Users put songs together in playlists to overcome the problem of being overwhelmed when choosing a song from a large collection, or just to enjoy a set of songs in particular situations. For example, one might be interested in having a playlist for running, another for cooking, etc. [0006] Different approaches can be adopted to help users choose the right options with personalized recommendations. One kind of approach is about using human expertise to classify the media items and then use these classifications to infer recommendations to users based on an input mediaset. For instance, if in the input mediaset the item x appears and x belongs to the same classification as y, then a system could recommend item y based on the fact that both items are classified in a similar cluster. However, this approach requires an incredibly huge amount of human work and expertise. Another approach is to analyze the data of the items (audio signal for songs, video signal for video, etc) and then try to match users preferences with the extracted analysis. This class of approaches is yet to be shown effective from a technical point of view.

Brief Summary of Presently Preferred Embodiments

[0007] This invention addresses the problem of assisting users in building their mediasets by recommending media items that go well together with an initial (or input) mediaset. The recommendation is computed using metrics among the media items of a knowledge base of the system. This knowledge base comprises collections of mediasets from a community of users. (As explained below, a mediaset is not a collection of media items or content. Rather, it is a list of such items, and may include various metadata.) Preferably, the methods of the present invention are implemented in computer software.

[0008] In commercial applications, the invention can be deployed in various ways. Recommender services can be provided, for example, to remote users of client computing machines via a network of almost any kind, wired or wireless. Here we use "computing machines" to include traditional computers, as well as cell phones, PDA's, portable music players etc. The knowledge base of the system, a database, can be local or remote from the user. It may be at one location or server, or distributed in various ways.

[0009] The invention in one aspect embodies a system for identifying a set of media items in response to an input set of media items. The system requires a knowledge base consisting of a collection of mediasets. Mediasets are sets of media items, which are naturally grouped by users. They reflect the users subjective judgments and preferences. The mediasets of the knowledge base define metrics among items. Such metrics indicate the extent of correlation among media items in the mediasets of the knowledge base.

[0010] Various different metrics between and among media items can be generated from the knowledge base of mediasets. Such metrics can include but are not limited to the follow examples: a) Pre-concurrency (for ordered mediasets) between two items is computed as the number of times a given item precedes the other item in the mediasets of the knowledge base. b) Post-concurrency (for ordered mediasets) between two items is computed as the number of times an item follows another item in the mediasets of the knowledge base. c) Co-concurrency between two items is computed as the number of times the items appear together in a mediaset. d) Metadata similarities may be computed as well by considering keywords associated with the media items such as artist, actor, date, etc. e) Combinations of the previous metrics can be useful. f) Combinations of the previous metrics applying transitivity. [0011] Such metrics can be represented in an explicit form that directly associates media items with other media items. For each media item of the input set, the system retrieves n media items with highest metrics. These media items are called candidates. Then, the recommended set of media items is a subset of the candidates that maximize an optimization criterion. Such criterion can be simply defined using the metrics of the knowledge base of the system. Furthermore, such criterion can also include filters including but not limited to: a) Filters that the user expresses to focus the recommendation only on a determined type of items. b) Filters that the user expresses to focus the recommendations on items that meet certain keyword-based criteria, such as a specific artist/s, year/s, genre/s, etc. c) Filters that personalize the recommendations to the user. This kind of filtering includes recommending only items that the user knows about, or only items that the user does not know about, etc. [0012] Additional aspects and advantages will be apparent from the following detailed description of preferred embodiments, which proceeds with reference to the accompanying drawings.

Brief Description of the Drawings

[0013] FIG. 1 A is a representation in matrix form of a metric describing the similarity values between collections of media items.

[0014] FIG. 1 B provides a weighted graph representation for the associations within a collection of media items. Each edge between two media items is annotated with a weight representing the value of the metric for the similarity between the media items.

[0015] FIG. 2 is a block diagram of one method for selecting a set of media items corresponding to an initial set of media items in accordance with an embodiment of the invention.

[0016] FIG. 3 is a simplified, conceptual diagram of a knowledge base or database comprising a plurality of mediasets.

Detailed Description of Preferred Embodiments

[0017] Reference is now made to the figures in which like reference numerals refer to like elements. For clarity, the first digit of a reference numeral indicates the figure number in which the corresponding element is first used. [0018] In the following description, certain specific details of programming, software modules, user selections, network transactions, database queries, database structures, etc. are omitted to avoid obscuring the invention. Those of ordinary skill in computer sciences will comprehend many ways to implement the invention in various embodiments, the details of which can be determined using known technologies.

[0019] Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In general, the methodologies of the present invention are advantageously carried out using one or more digital processors, for example the types of microprocessors that are commonly found in servers, PC's, laptops, PDA's and all manner of desktop or portable electronic appliances. Definitions

[0020] The system preferably comprises or has access to a knowledge base which is a collection of mediasets. A mediaset is a list of media items that a user has grouped together. A media item can be almost any kind of content; audio, video, multi-media, etc., for example a song, a book, a newspaper or magazine article, a movie, a piece of a radio program, etc. Media items might also be artists or albums. If a mediaset is composed of a single type of media items it is called a homogeneous mediaset, otherwise it is called a heterogeneous mediaset. A mediaset can be ordered or unordered. An ordered mediaset implies a certain order with respect to the sequence in which the items are used¹ by the user. Note again that a mediaset, in a preferred embodiment, is a list of media items, i.e. meta data, rather than the actual content of the media items. In other embodiments, the content itself may be included. Preferably, a knowledge base is stored in a machine- readable digital storage system. It can employ well-known database technologies for establishing, maintaining and querying the database.

[0021] In general, mediasets are based on the assumption that users group media items together following some logic or reasoning, which may be purely subjective, or not. For example, in the music domain, a user may be selecting a set of songs for driving, hence that is a homogeneous mediaset of songs. In this invention, we also consider other kinds of media items such as books, movies, newspapers, and so on. For example, if we consider books, a user may have a list of books for the summer, a list of books for bus riding, and another list of books for the weekends. A user may be interested in expressing a heterogeneous mediaset with a mix of books and music, expressing (impliedly) that the listed music goes well with certain books.

[0022] A set of media items is not considered the same as a mediaset. The difference is mainly about the intention of the user in grouping the items together. In the case of a mediaset the user is expressing that the items in the mediaset go together well, in some sense, according to her personal preferences. A common example of a music mediaset is a playlist. On the other hand, a set of media items does not express necessarily the preferences of a user. We use the term set of

1 Depending on the nature of the item, it will be played, viewed, read, etc. media items to refer to the input of the system of the invention as well as to the output of the system.

[0023] A metric M between a pair of media items / and / for a given knowledge base k expresses some degree of relation between i and j with respect to k. A metric may be expressed as a "distance," where smaller distance values (proximity) represent stronger association values, or as a similarity, where larger similarity values represent stronger association values. These are functionally equivalent, but the mathematics are complementary. The most immediate metric is the co- concurrency (i,j, k) that indicates how many times item / and item/ appear together in any of the mediasets of k. The metric pre-concurrency (i, j, k) indicates how many times item / and itemy appear together but i before j in any of the mediasets of k. The metric post-concurrency (i, j ,k) indicates how many times item / and item / appear together but only / after y in any of the mediasets of k. The previous defined metrics can also be applied to considering the immediate sequence of i and j. So, the system might be considering co/pre/post-concurrencies metrics but only if items i and j are consecutive in the mediasets (i.e., the mediasets are ordered). Other metrics can be considered and also new ones can be defined by combining the previous ones.

[0024] A metric may be computed based on any of the above metrics and applying transitivity. For instance, consider co-concurrency between item / and y, co(ij), and between j and k, co(j,k), and consider that co(i,k)=0. We could create another metric to include transitivity, for example d(i,k) = 1/co(i,j) + 1/co(j,k). These type of transitivity metrics may be efficiently computed using standard branch and bound search algorithms. This metric reveals an association between items / and k notwithstanding that / and k do not appear within any one mediaset in K. [0025] A matrix representation of metric M, for a given knowledge base K can be defined as a bidimensional matrix where the element M(i, j) is the value of the metric between the media item / and media item/

[0026] A graph representation for a given knowledge base k, is a graph where nodes represent media items, and edges are between pairs of media items. Pairs of media items i, j are linked by labeled directed edges, where the label indicates the value of the similarity or distance metric M(ij) for the edge with head media item i and tail media item j. Preferred Embodiments of the Invention

[0027] One embodiment of the invention is illustrated by the flow diagram shown in Fig 2. This method accepts an input set 301 of media items. Usually, this is a partial mediaset, i.e. a set of media items (at lease one item) that a user grouped together as a starting point with the goal of building a mediaset. A first collection of candidate media items most similar to the input media items is generated by process 302 as follows.

[0028] As a preliminary matter, in a presently preferred embodiment, a preprocessing step is carried out to analyze the contents of an existing knowledge base. This can be done in advance of receiving any input items. As noted above, the knowledge base comprises an existing collection of mediasets. This is illustrated in Fig. 3, which shows a simplified conceptual illustration of a knowledge base 400. In Fig. 3, the knowledge base 400 includes a plurality of mediasets, delineated by rectangles [or ovals] and numbered 1 through 7. Each mediaset comprises at least two media items. For example, mediaset 2 has three items, while mediaset 7 has five items. The presence of media items within a given mediaset creates an association among them.

[0029] Pre-processing analysis of a knowledge base can be conducted for any selected metric. In general, the metrics reflect and indeed quantify the association between pairs of media items in a given knowledge base. The process is described by way of example using the co-concurrency metric mentioned earlier. For each item in a mediaset, the process identifies every other item in the same mediaset, thereby defining all of the pairs of items in that mediaset. For example, in Fig. 3, one pair in set 1 is the pair M(1 ,1) + M(1 ,3). Three pairs are defined that include M(1 ,1 ). This process is repeated for every mediaset in the knowledge base, thus every pair of items that appears in any mediaset throughout the knowledge base is defined. [0030] Next, for each pair of media items, a co-concurrency metric is incremented for each additional occurrence of the same pair of items in the same knowledge base. For example, if a pair of media items, say the song "Uptown Girl" by Billy Joel and "Hallelujah" by Jeff Buckley, appear together in 42 different mediasets in the knowledge base (not necessarily adjacent one another), then the co-concurrency metric might be 42 (or some other figure depending on the scaling selected, normalization, etc. In some embodiments, this figure or co-concurrency "weight" may be normalized to a number between zero and one. [0031] Referring now to FIG. 1A, matrix 100 illustrates a useful method for storing the metric values or weights for any particular metric. Here, individual media items in the knowledge base, say mi, m₂, m₃ ...m_k are assigned corresponding rows and columns in the matrix. In the matrix, the selected metric weight for every pair of items is entered at row, column location x,y corresponding to the two media items defining the pair. In FIG. 1A, the values are normalized.

[0032] Now we assume an input set of media items is received. Referring again to process step 302, a collection of "candidate media items" most similar to the input media items is generated, based on a metric matrix like matrix 100 of FIG. 1A. For instance, for each media item, say (item m₂) in the input set 301 , process 302 could add to a candidate collection of media items every media item (Hi₁, 1Ω₃ ... m_k in Fig. 1A) that has a non-zero similarity value, or exceeds a predetermined threshold value, in the corresponding row 102 of metric matrix 100 for the media item m₂, labeling each added media item with the corresponding metric value (0.7, 0.4 and 0.1 , respectively). See the edges in Fig. 1 B. For each media item in the input set of size m, process 302 selects n media items as candidates; thus the aggregation of all the candidates produces a set of at most m * n media items. [0033] Process 303 receives the candidate set from process 302 which contains at the most m * n media items. This component selects p elements from the m * n items of the candidate set. This selection can be done according to various criteria. For example, the system may consider that the candidates should be selected according to the media item distribution that generated the candidate set. This distribution policy may be used to avoid having many candidates coming from very few media items. Also, the system may consider the popularity of the media items in the candidate set. The popularity of a media item with respect to a knowledge base indicates the frequency of such media item in the mediasets of the knowledge base. [0034] Finally, from the second collection of [p] media items, a third and final output set 305 of some specified number of media items is selected that satisfy any additional desired external constraints by a filter process 304. For instance, this step could ensure that the final set of media items is balanced with respect to the metrics among the media sets of the final set. For example, the system may maximize the sum of the metrics among each pair of media items in the resulting set. Sometimes, the system may be using optimization techniques when computation would otherwise be too expensive. Filtering criteria such as personalization or other preferences expressed by the user may also be considered in this step. In some applications, because of some possible computational constraints, these filtering steps may be done in the process 303 instead of 304. Filtering in other embodiments might include genre, decade or year of creation, vendor, etc. Also, filtering can be used to demote, rather then remove, a candidate output item. [0035] In another embodiment or aspect of the invention, explicit associations including similarity values between a subset of the full set of media items known to the system, as shown in graph form in FIG. 1B, may be used. To illustrate, if the similarity value between a first media item 202, generally denoted below by the index i, and a second media item, say 214, generally denoted below by the index j, is not explicitly specified, an implicit similarity value can instead be derived by following a directed path such as that represented by edges 210 and 212 from the first media item to an intermediate item, and finally to the second media item of interest, in this example item m_p. Any number of intermediate items can be traversed in this manner, which we call a transitive technique. The list of similarity values M(i, i+1), M(i+1 , i+2), ..., M(i+k, j) between pairs of media items along this path through the graph are combined in a manner such that the resulting value satisfies a definition of similarity between media item i and media item j appropriate for the application. For example, the similarity M(i,j) might be computed as:

M(ij) = min{M(i,i+1), M(i,i+2), ..., M(i+k,j)} or

M(ij) = M(i,i+1) * M(i,i+2) * ... * M(i+k,j)

[0036] Other methods for computing a similarity value M(ij) for the path between a first media item i and a second, non-adjacent media item j where the edges are labeled with the sequence of similarity values M(i, i+1), M(i+1 , i+2), ..., M(i+k, j) can be used. From the user standpoint, this corresponds to determining an association metric for a pair of items that do not appear within the same mediaset. [0037] Many modifications and other embodiments of the invention will come to mind to one skilled in the art to which this invention pertains having the benefit of the teachings presented in the foregoing descriptions and the associated drawings. Therefore, it is to be understood that the invention is not to be limited to the specific embodiments disclosed and that modifications and other embodiments are intended to be included within the scope of the appended claims. For example, one of ordinary skill in the art will understand that, while the above system and methods were described as embodied in a media recommendation system, it should be understood that the inventive system could be used in any system for recommending other items that can be grouped by users following some criterion. Although specific terms are employed herein, there are used in a generic and descriptive sense only and not for purposes of limitation.

It will be obvious to those having skill in the art that many changes may be made to the details of the above-described embodiments without departing from the underlying principles of the invention. The scope of the present invention should, therefore, be determined only by the following claims.