Generating and distributing playlists with music and stories having related moods
12 claims: 5 independent, 7 dependent
- 1第1の規定された順序の第1の複数のオーディオファイルへの参照を含む、クライアントデバイス用の第1のプレイリストをサーバデバイスによって生成することであって、前記第1の複数のオーディオファイルの再生が第1の時間長を有する、ことと、前記サーバデバイスによって前記第1のプレイリストをクライアントデバイスに送信することであって、前記クライアントデバイスにおける前記第1のプレイリストの受信により、前記クライアントデバイスのオーディオプレイヤアプリケーションに、前記第1の複数のオーディオファイルのうちの少なくともいくつかを前記第1の規定された順序で取り出して再生させる、ことと、前記クライアントデバイスが前記第1の複数のオーディオファイルのうちの現在のオーディオファイルを再生している間に、前記クライアントデバイスが目標時間長でスリープタイマーを作動させていることを規定する命令を前記サーバデバイスによって前記クライアントデバイスから受信することと、前記命令の受信に応答して、第2の規定された順序の第2の複数のオーディオファイルへの参照を含む第2のプレイリストを前記サーバデバイスによって生成することであって、前記第2の複数のオーディオファイルの再生が、前記第1の時間長よりも少ない、前記目標時間長の閾値時間長内の第2の時間長を有する、ことと、前記サーバデバイスによって前記第2のプレイリストを前記クライアントデバイスに送信することであって、前記クライアントデバイスにおける前記第2のプレイリストの受信により、前記オーディオプレイヤアプリケーションに、前記第1の複数のオーディオファイルからのオーディオファイルの再生を停止させ、前記第2の複数のオーディオファイルを前記第2の規定された順序で取り出して再生させる、こととを含むコンピュータにより実行される方法。
- 2前記第2の複数のオーディオファイルが、前記第1の規定された順序において前記現在のオーディオファイルよりも前の、前記第1の複数のオーディオファイルのうちのいずれのオーディオファイルも含んでいない、請求項1に記載のコンピュータにより実行される方法。
- 3前記第2の複数のオーディオファイルが、前記第1の規定された順序において前記現在のオーディオファイルよりも後の、前記第1の複数のオーディオファイルのうちの少なくともいくつかのオーディオファイルを含む、請求項1に記載のコンピュータにより実行される方法。
- 4前記第1の複数のオーディオファイルにおける前記オーディオファイルは、それぞれの優先度に関連付けられており、前記第2のプレイリストを生成することは、低い優先度のオーディオファイルよりも高い優先度のオーディオファイルを優先しながら、前記第1の複数のオーディオファイルのオーディオファイルから前記第2の複数のオーディオファイルを選択することを含む、請求項3に記載のコンピュータにより実行される方法。
- 5広告を含むオーディオファイルが、音楽を含むオーディオファイルよりも高い優先度に関連付けられる、請求項4に記載のコンピュータにより実行される方法。
- 6前記第2のプレイリストを生成することが、前記第1の規定された順序において前記現在のオーディオファイルよりも後の、前記第1の複数のオーディオファイルにおける前記オーディオファイルから、前記第2の複数のオーディオファイルを選択することを含む、請求項3に記載のコンピュータにより実行される方法。
- 7前記閾値時間長は3分以下である、請求項1に記載のコンピュータにより実行される方法。
- 8スリープタイマーの作動を理由に、前記第2の複数のオーディオファイルが、(i)ゆっくりしたテンポ、(ii)狭いダイナミックレンジ、または(iii)低い音量についての選好に基づいて選択される、請求項1に記載のコンピュータにより実行される方法。
- 9前記第2の時間長が、前記第1のプレイリストの残りの再生時間から前記現在のオーディオファイルの残りの再生時間を差し引いたものに基づく、請求項1に記載のコンピュータにより実行される方法。
- 10前記オーディオプレイヤアプリケーションが、前記現在のオーディオファイルの再生が完了した後に、前記第1の複数のオーディオファイルからのオーディオファイルの再生を停止する、請求項1に記載のコンピュータにより実行される方法。
- 11プログラム命令を記憶した非一時的なコンピュータ可読媒体を含む製造物品であって、前記プログラム命令が、サーバデバイスによる実行時に、前記サーバデバイスに、請求項1から10のいずれか1項に記載の動作を実行させる、製造物品。
- 12メモリと、1つまたは複数のプロセッサと、前記メモリに記憶され、請求項1から10のいずれか1項に記載の動作を実行するための、前記1つまたは複数のプロセッサによって実行可能なプログラム命令とを備えるコンピューティングデバイス。
Independent claims12
140 paragraphs, as filed
This application claims priority to U.S. Provisional Patent Application No. 15/043,059, filed February 12, 2016, which claims priority to U.S. Provisional Patent Application No. 62/274,591, filed January 4, 2016. , which are hereby incorporated by reference in their entirety.
To listen to radio, a listener typically tunes a receiver to a particular frequency (for example, an AM or FM frequency, or a satellite frequency) and listens to the music, news, or other broadcasts on that frequency by the radio station. Listen to audible content. Radio stations generate programs of audible content that are transmitted simultaneously to all listeners. These programs may be generated in advance, in real time, or in an ad hoc manner. In some cases, this may lead to situations where there is a mismatch, either thematically or emotionally, between the individual units of audible content that make up the playlist. Alternatively, a playlist may unintentionally include material that may be offensive to certain listeners or that may be considered insensitive based on current events.
<p>Embodiments herein generate playlists such that the audible content referenced in the playlists has some form of thematic association or is arranged to be approximately the exact length. disclose various mechanisms for For example, music, news, and/or advertising content may be selected based on a particular user's preferences or information known about that user. At least some of these units of audible content may be selected based on the nature of the preceding or following units such that there is thematic continuity from unit to unit of the playlist.</p><p>In some cases, this means that attributes of audio files containing music (e.g., artist name, song title, or lyric content) can be used to select audio files containing stories (e.g., news stories). It can mean Conversely, the content of the audio file containing the story can be used to select the audio file containing the music. In a playlist, references to audio files containing stories may be ordered before or after references to audio files containing music.</p><p>Accordingly, a first exemplary embodiment may include the server device selecting audio files that include music based possibly on a profile associated with the client device. A client device and a server device may be communicating with each other via a wide area network, and the client device may include an audio player application. Possibly based on attributes associated with audio files that include music, the server device may select audio files that include stories. The server device can generate a playlist for the client device, where the playlist includes (i) references to audio files that include music, and (ii) references to audio files that include stories. A server device can send a playlist to a client device over a wide area network. Receipt of the playlist at the client device may cause the audio player application to retrieve and play each of the audio files containing music and the audio files containing stories.</p><p>A second exemplary embodiment may include the server device selecting the text file containing the story based possibly on a profile associated with the client device. A client device and a server device may be communicating with each other via a wide area network, and the client device may include an audio player application. Possibly based on the text within the text file containing the story, the server device may select the audio file containing the music. An automatic text-to-speech system can convert a text file containing a story into an audio file containing a story. The server device can generate a playlist for the client device, where the playlist includes (i) references to audio files that include music, and (ii) references to audio files that include stories. A server device can send a playlist to a client device over a wide area network. Receipt of the playlist at the client device may cause the audio player application to retrieve and play each of the audio files containing music and the audio files containing stories.</p><p>In other cases, the audio file story may be parsed or scanned for keywords (eg, after being processed by an automatic speech-to-text system). The mood of the story (for example, happiness, sadness, humor, misery) may be determined based on the presence or absence of certain keywords. In some embodiments, semantic analysis may be performed on the story, in part or in whole, and the mood of the story may be based on this analysis.</p><p>The audio file containing the music may be chosen such that the audio file has the same or similar atmosphere, or at least a mood based on the atmosphere of the story. Conversely, the mood associated with audio files containing music can be used to select audio files containing stories. In a playlist, references to audio files containing stories may be ordered before or after references to audio files containing music.</p><p>Accordingly, a third example embodiment may include a server device obtaining an audio file that includes a story. A client device and a server device may be communicating with each other via a wide area network, and the client device may include an audio player application. Possibly based on attributes associated with the audio file containing the story and/or a semantic analysis of the story, the server device may determine the mood of the story. Possibly based on the mood of the story, the server device may select an audio file that includes music, and the audio file that includes music is associated with an attribute that is indicative of its mood. The server device can generate a playlist for the client device, where the playlist includes (i) references to audio files that include music, and (ii) references to audio files that include stories. A server device can send a playlist to a client device over a wide area network. Receipt of the playlist at the client device may cause the audio player application to retrieve and play each of the audio files containing music and the audio files containing stories.</p><p>A fourth example embodiment may include a server device obtaining an audio file that includes music. A client device and a server device may be communicating with each other via a wide area network, and the client device may include an audio player application. Possibly based on attributes associated with the audio file containing the music, the server device can determine the mood of the music. Based as much as possible on the mood of the music, the server device may select the audio file containing the story. Attributes associated with an audio file containing a story and/or semantic analysis of the story can indicate its atmosphere. The server device can generate a playlist for the client device, where the playlist includes (i) references to audio files that include music, and (ii) references to audio files that include stories. A server device can send a playlist to a client device over a wide area network. Receipt of the playlist at the client device may cause the audio player application to retrieve and play each of the audio files containing music and the audio files containing stories.</p><p>In further embodiments, the client device may be retrieving and playing audio files from a playlist generated by the server device. In response to input received by the client device, the client device may determine that playback should end within a certain amount of time. For example, a user may have activated a timer function on the client device so that the client device stops playing within a certain number of minutes. In response to this input, the client device can instruct the server device to generate a new playlist that ends after approximately a certain number of minutes. The server device can generate and provide this new playlist to the client device, and the client device can retrieve and begin playing audio files from the new playlist. In some embodiments, the timer may be a sleep timer, and the audio files referenced by the new playlist are selected and/or arranged such that the audio files aid sleep. Good too. In other embodiments, the timer can measure the amount of time remaining until a destination is reached or until some event occurs (e.g., the arrival of a bus or train, or the start of an appointment). .</p><p>Accordingly, a fifth example embodiment may include the server device generating a first playlist for the client device. The first playlist can include references to a first plurality of audio files in a first defined order. Playing the first plurality of audio files can have a first length of time. A client device and a server device may be communicating with each other via a wide area network, and the client device may include an audio player application. The server device can send the first playlist to the client device over the wide area network. Receipt of the first playlist at the client device may cause the audio player application to retrieve and play at least some of the first plurality of audio files in a first defined order. The server device may receive instructions from the client device while the client device is playing a current audio file of the first plurality of audio files. Possibly responsive to receiving the instructions, the server device may generate a second playlist. The second playlist may include references to a second plurality of audio files in a second prescribed order, and the playback of the second plurality of audio files is less than the first duration. can have a second time length. The server device can send the second playlist to the client device over the wide area network. Receipt of the second playlist at the client device may cause the audio player application to retrieve and play a second plurality of audio files in a second defined order.</p><p>In a sixth exemplary embodiment, an article of manufacture can include a non-transitory computer-readable medium having program instructions stored thereon, the program instructions being stored on the computing device when executed by the computing device. , cause operations to be performed in accordance with a first, second, third, fourth, and/or fifth exemplary embodiment.</p><p>In a seventh exemplary embodiment, a computing device can include at least one processor, as well as memory and program instructions. Program instructions may be stored in memory and, upon execution by at least one processor, may be transmitted to a computing device in accordance with a first, second, third, fourth, and/or fifth exemplary embodiment. Make the action take place.</p><p>In an eighth exemplary embodiment, the system includes various means for performing each of the operations of the first, second, third, fourth, and/or fifth exemplary embodiment. be able to.</p><p>Although the examples described herein attribute certain operations to certain devices, any device may perform those operations. For example, some or all of the operations attributed to "server device" above may be performed by different server devices and/or client devices.</p><p>These and other embodiments, aspects, advantages, and alternatives will become apparent to those skilled in the art upon reading the following detailed description, with reference, where appropriate, to the accompanying drawings. Furthermore, it is to be understood that this summary and other descriptions and figures provided herein are intended to illustrate embodiments by way of example only, and therefore, many variations are possible. be. For example, structural elements and process steps may be rearranged, combined, distributed, deleted, or otherwise modified while remaining within the scope of the claimed embodiments.</p>
<figref num="1">1 is a high-level diagram of a client-server computing system according to an example embodiment. FIG.</figref><figref num="2">1 is a schematic diagram of a computing device according to an example embodiment; FIG.</figref><figref num="3">1 is a schematic diagram of a networked server cluster according to an example embodiment; FIG.</figref><figref num="4">FIG. 2 is an illustration of a playlist according to an example embodiment.</figref><figref num="5A">FIG. 3 is a message flow diagram depicting playlist generation according to an example embodiment.</figref><figref num="5B">FIG. 2 is a message flow diagram representing playback of audio files referenced by a playlist according to an example embodiment.</figref><figref num="6">3 is a flowchart for playlist generation and distribution according to an example embodiment.</figref><figref num="7">3 is a flowchart for playlist generation and distribution according to an example embodiment.</figref><figref num="8">3 is a table of relationships between atmospheres in accordance with an exemplary embodiment;</figref><figref num="9">3 is a flowchart for playlist generation and distribution according to an example embodiment.</figref><figref num="10">3 is a flowchart for playlist generation and distribution according to an example embodiment.</figref><figref num="11">3 is a timing diagram according to an example embodiment; FIG.</figref><figref num="12">3 is a flowchart for playlist generation and distribution according to an example embodiment.</figref>
Exemplary methods, devices, and systems are described herein. It should be understood that the words "example" and "exemplary" are used herein to mean "serving as an example, instance, or illustration." . Any embodiment or feature described herein as "exemplary" or "example" is not necessarily to be construed as preferred or advantageous over other embodiments or features. isn't it. Other embodiments may be utilized and other changes may be made without departing from the scope of the subject matter presented herein.
Accordingly, the example embodiments described herein are not meant to be limiting. As generally described herein and illustrated in the figures, aspects of the present disclosure can readily be arranged, substituted, combined, separated, and designed in a wide variety of different configurations. It will be understood. For example, steps, blocks, or actions assigned to a particular device (eg, a server device) may be performed by components of another device (eg, a client device) in alternative embodiments.
Furthermore, unless the context suggests otherwise, the features illustrated in each of the figures may be used in combination with each other. Accordingly, the figures should be viewed generally as aspects constituting one or more general embodiments, with the understanding that not all illustrated features may be required for each embodiment. It is.
(1. Overview) Unlike traditional broadcast radio, online (eg, Internet-based) radio can transmit audio content to individual client devices or groups of client devices. This audio content can thus be personalized for different users or groups of users. For example, a server device may generate a playlist based on known attributes or preferences of one or more users. This playlist may include a series of references (eg, uniform resource locators, or URLs) to one or more remote audio files. Such a playlist may be sent to the user's client device. A client device can follow the playlist and retrieve and play audio files. In doing so, the client device may download one or more of the audio devices (from the server device or a different device) or receive the audio file as a stream prior to playing the audio file. Either can be done.
As used herein, downloading an audio file may refer to the client device retrieving the entire audio file from the server device before starting to play any part of the audio file. Streaming an audio file, on the other hand, may involve a client device simultaneously playing a previously received portion of the audio file while receiving another portion of the audio file.
As an example, after or when the server device generates a playlist, the server device can send the playlist to the client device, and the client device can follow the playlist entries. I can do it. In doing so, the client device can retrieve data representing each of the referenced audio files and play the audio files according to the sequential order defined by the playlist. Thus, for each sequential order of references defined by the playlist, (i) the client device may send a request to the server device (or a different device) for the referenced audio files, and (ii) the server the device may receive the transmitted request; and (iii) in response to the server device receiving the transmitted request, the server device transmits the stream of the requested audio file to the client. (iv) the client device may receive and begin buffering the stream of audio files; and (v) while still receiving the stream of audio files, the client device may begin buffering the received portion of the audio file. can be played. In this way, a user may be provided with substantially continuous playback of the audio files referenced by the playlist.
Despite the benefits of personalizing audio programming, computerized generation of playlists can introduce certain problems. For example, a computer-generated playlist may not exhibit thematic continuity from audio file to audio file. As a result, the selection and ordering of audio files may appear haphazard, even though these audio files could (in various ways) be selected based on the user's profile.
Embodiments in the present invention facilitate the selection of audio files for inclusion in a playlist, and their ordering, based on thematic associations. Thus, the topic of a news story can influence the selection of music that follows the news story in sequential order. Similarly, characteristics of the music (e.g., determined based on tempo, volume, artist name, song title, and/or lyrical content) can influence the selection of news stories that follow the music in sequential order. can. In some cases, more than one of such characteristics may be combined to determine a general similarity between the content of two or more audio files. Alternatively or additionally, the characteristics used may be different in different parts of the playlist. For example, artist name may be used as the similarity characteristic between the first three entries, song title may be used as the similarity characteristic between the next four entries, and so on.
Additionally, the mood of the news story (eg, happy, sad, tragic, funny) can influence the selection of music that follows the news story in sequential order. Similarly, the mood of the music can influence the selection of news stories that follow the music in sequential order. In some cases, the server device may attempt to match the mood of the news story and the music, or the server device may attempt to match the mood of the news story and the music so that the moods of the news story and the music do not conflict with each other. You can choose. In other cases, the server device may select the atmospheres such that these atmospheres are in conflict with each other. Generally speaking, a server device can select one of the news stories and music such that its mood is based in some way on the mood of the other.
In various embodiments, a news story may initially exist in a text format (eg, a flat text file or a markup language file). When a news story is selected for inclusion in a playlist, the server device can determine whether an audio version of that news story exists, and if not, the server device can determine whether an audio version of that news story exists, and if not, the server device can perform a text-to-speech (e.g. , audio generation) technology can be used to automatically generate audio files for news stories. Conversely, to determine the tone of a news story in an audio file format, the server device may automatically generate a text file of the news story using speech-to-text (e.g., speech recognition) technology. . The tone of the news story may then be determined based on the content of the text file.
In some embodiments, a server device may be generating a playlist and a client device may be playing audio files referenced in the playlist. A server device may receive an indication from a client device that the client device is instructed to stop playing audio at some future point. For example, a user of a client device may have activated a sleep timer feature on the client device. In response, the server device can generate a new playlist that approximately matches the amount of time remaining before the client device is scheduled to stop playing. The media referenced by this new playlist may be selected and ordered to help the listener fall asleep as playback nears its end. For example, the dynamic range of the media may become progressively narrower and/or the tempo of the audio may become progressively slower. In other embodiments, the timer can measure the amount of time remaining until a destination is reached or until some event occurs (e.g., the arrival of a bus or train, or the start of an appointment). .
Any of the embodiments described above or elsewhere in the invention may be combined in various ways. Accordingly, in some implementations, a new playlist generated in response to activation of the timer function described above may be populated with thematically related media.
Although the embodiments described herein focus on generating a playlist of audio files for audio playback, the same or similar embodiments may include audio/video playlists for audio/video playback. It may be used to generate playlists of video files (eg, television news programs, entertainment programs, commercials, movies, etc.). Therefore, the described embodiments are not limited to audio files, but can be applied to other types of media.
Regardless of how the embodiments herein may be implemented, they may utilize one or more computing devices. These computing devices may include, for example, a client device under the control of a user and a server device that interacts directly or indirectly with the client device. Such devices are described in the following sections.
2. Exemplary Computing Devices and Cloud-Based Computing Environments FIG. 1 illustrates an exemplary communication system 100 for performing one or more of the embodiments described herein. . Communication system 100 may include a computing device. As used herein, "computing device" refers to either a client device, a server device (e.g., a standalone server computer or a cluster of networked server equipment), or any other type of computerized platform. I can do it.
Client device 102 is a personal computer, laptop computer, configured to send data 106 to and/or receive data 108 from server device 104 in accordance with embodiments described herein. It may be any type of device, including a wearable computing device, a wireless computing device, a head-mounted computing device, a mobile phone, or a tablet computing device, etc. For example, in FIG. 1, client device 102 may communicate with server device 104 via one or more wired or wireless interfaces. In some cases, client device 102 and server device 104 may communicate with each other via a local area network. Alternatively, client device 102 and server device 104 may each reside in different networks and may communicate via a wide area network, such as the Internet.
Client device 102 may include a user interface, a communication interface, a main processor, and data storage (eg, memory). The data storage performs one or more operations, such as operations related to data sent to or received from server device 104, and/or other operations disclosed herein. may contain instructions executable by the main processor. The user interface of client device 102 may include buttons, a touch screen, a microphone, and/or any other elements for receiving input, as well as speakers, one or more displays, and/or for communicating output. May contain any other elements.
Server device 104 may be any entity or computing device arranged to perform server operations as described herein. Additionally, server device 104 may be configured to send data 108 to client device 102 and/or receive data 106 from client device 102.
Data 106 and data 108 can take a variety of formats. For example, data 106 and 108 may represent packets sent by client device 102 or server device 104, respectively, as part of one or more communication sessions. Such communication sessions may include packets sent on the signaling plane (e.g., session setup, management, and teardown messages) and/or on the media plane (e.g., text, graphics, audio, and/or video data). ). For example, data 106 may include sending requests for playlists and audio file streams, and data 108 may include sending playlists and audio file streams in response to these requests, respectively. I can do it.
Regardless of the exact architecture, the operations of client device 102, server device 104, and any other operations associated with the architecture of FIG. 1 may be performed by one or more computing devices. These computing devices may be organized in a stand-alone fashion, in a cloud-based (networked) computing environment, or in other arrangements.
FIG. 2 is a simplified block diagram illustrating a computing device 200 showing some of the functional components that may be included in a computing device arranged to operate in accordance with embodiments herein. Exemplary computing device 200 may be a client device, a server device, or some other type of computerized platform. For purposes of simplicity, this specification may at times equate computing device 200 with a server. Nevertheless, the description of computing device 200 may apply to any components used for the purposes described herein.
In this example, computing device 200 includes a processor 202, data storage 204, a network interface 206, and input/output functionality 208, all of which may be coupled by a system bus 210 or similar mechanism. Processor 202 may include one or more general purpose processors and/or one or more special purpose processors (e.g., application specific integrated circuits (ASICs), digital signal processors (DSPs), network processors, etc.). Can contain one or more CPUs.
Data storage 204, in turn, can include volatile and/or non-volatile data storage and can be integrated, in whole or in part, with processor 202. Data storage 204 holds program instructions executable by processor 202, and data that can be manipulated by these instructions to perform various methods, processes, or operations described herein. I can do it. Alternatively, these methods, processes, or operations may be defined by hardware, firmware, and/or any combination of hardware, firmware, and software.
By way of example, data in data storage 204 may include program instructions executable by processor 202, possibly stored on a non-transitory computer-readable medium, to implement the methods, processes, and methods disclosed herein or in the accompanying drawings. or perform any of the following actions. Data storage 204 may also include one or more playlists and/or audio files possibly referenced by the playlists.
Network interface 206 can take the form of a wireline connection, such as an Ethernet, Token Ring, or T-carrier connection. Network interface 206 may also take the form of a wireless connection, such as IEEE 802.11 (WiFi), BLUETOOTH, or a wide area wireless connection. However, other forms of physical layer connections and other types of standard or proprietary communication protocols may be used on network interface 206. Further, network interface 206 can include multiple physical interfaces.
Input/output functionality 208 can facilitate user interaction with example computing device 200. Input/output functionality 208 may include multiple types of input devices such as a keyboard, mouse, touch screen, and others. Similarly, input/output functionality 208 may include multiple types of output devices, such as a screen, monitor, printer, or one or more light emitting diodes (LEDs). In addition, or in the alternative, the example computing device 200 may be configured to connect via the network interface 206 or to another interface, such as a Universal Serial Bus (USB) or High Definition Multimedia Interface (HDMI) port. (not shown) may support remote access from another device.
In some embodiments, one or more computing devices may be arranged in a networked architecture. The exact physical location, connectivity, and configuration of the computing device may be unknown and/or unimportant to the client device. Accordingly, computing devices are sometimes referred to as "cloud-based" devices that may be housed in various remote locations.
FIG. 3 depicts a cloud-based server cluster 304, according to an example embodiment. In FIG. 3, the functionality of a server device such as server device 104 (as exemplified by computing device 200) may be distributed among server device 306, cluster data storage 308, and cluster router 310; All may be connected by a local cluster network 312. The number of server devices, cluster data storage, and cluster routers in server cluster 304 may depend on the computing tasks and/or applications assigned to server cluster 304.
For example, server device 306 may be configured to perform various computing tasks for computing device 200. Accordingly, computing tasks may be distributed among one or more of server devices 306. To the extent that these computing tasks can be performed in parallel, such distribution of tasks can reduce the total time to complete these tasks and return results. For purposes of brevity, both server clusters 304 and individual server devices 306 may be referred to as "server devices." It should be understood that this name suggests that one or more different server devices, data storage devices, and cluster routers may be involved in server device operation.
Cluster data storage 308 is a data storage array that includes a disk array controller configured to manage read and write access to a group of disk drives (e.g., hard drives with rotating platters, or solid state drives). It's okay. The disk array controller, alone or in conjunction with server devices 306, also detects disk drive failures or other types of failures that prevent one or more of server devices 306 from accessing a unit of cluster data storage 308. Cluster data storage 308 may be configured to maintain backup or redundant copies of data stored in cluster data storage 308 to protect against data loss. As an example, cluster data storage 308 may include one or more playlists and/or audio files possibly referenced by the playlists.
Cluster router 310 may include networking equipment configured to provide internal and external communications to the server cluster. For example, cluster router 310 may provide network communication between (i) server devices 306 and cluster data storage 308 via cluster network 312, and/or (ii) server cluster 304 via communication link 302 to network 300. It may include one or more packet switching devices and/or routing devices configured to provide network communications with other devices.
In addition, the configuration of cluster router 310 depends on the data communication requirements of server devices 306 and cluster data storage 308, the latency and throughput of local cluster network 312, the latency, throughput, and cost of communication links 302, and/or the system architecture. It can be based at least in part on other factors that may contribute to cost, speed, fault tolerance, resiliency, efficiency, and/or other design goals.
As possible examples, cluster data storage 308 may include any type of database, such as a Structured Query Language (SQL) database. Various types of data structures can store information in such databases, including, but not limited to, tables, arrays, lists, trees, and tuples. Additionally, any database in cluster data storage 308 may be monolithic or distributed across multiple physical devices.
Server device 306 may be configured to send data to and receive data from cluster data storage 308. This sending and retrieval may each take the form of an SQL query or other type of database query, and the output of such a query. Additional text, images, video, and/or audio may be included as well. Further, server device 306 can organize the received data into a web page representation. Such representation may take the form of a markup language such as Hypertext Markup Language (HTML), Extensible Markup Language (XML), or some other standardized or proprietary format. . Additionally, the server device 306 can support, but is not limited to, Perl, Python, PHP hypertext preprocessor (PHP), Active Server May have the ability to run various types of computerized scripting languages, such as Pages (ASP), JavaScript, and others. Computer program code written in these languages can facilitate the provision of web pages to client devices, as well as client device interaction with web pages.
3. Exemplary Playlist and Streaming Environment FIG. 4 depicts an exemplary playlist 400 that includes a series of two or more references to audio files. Playlist 400 can take the form of a computer file, such as a plain text file, an extensible markup language (XML) file, a hypertext markup language (HTML) file, a binary file, or some other file format. . Playlists can also take other forms and include other types of content.
In some cases, playlist 400 may be generated by a server device, such as server device 104, for a given client device, such as client device 102. In doing so, server device 104 may take into account information in client device 102's profile stored on or accessible to server device 104. For example, this profile may include preferences about the user of client device 102, such as preferred types of news stories, preferred style of music, preferred bit rate and/or audio quality of streamed audio files, etc. It can include expressions of. In other cases, playlist 400 may be generated for a group of one or more client devices based possibly on information in the profiles of at least some of these devices.
Further, as described in more detail below, server device 104 may generate playlist 400 such that the audio files referenced in the playlist are thematically consistent with respect to content and/or atmosphere. Alternatively, server device 104 may generate playlist 400 such that the audio files referenced in the playlist are at least thematically consistent.
Each entry in playlist 400 may include an identifier of an audio file and/or a reference to the audio file. Audio files mentioned in playlist 400 may be stored in database 402. So, for example, news story 1 referenced in playlist 400 contains a URL pointing to the location of the audio file for news story 1, while song 1 referenced in playlist 400 contains an audio file for song 1. Contains a URL pointing to a location, etc. Nevertheless, playlists can take other forms, including a comprehensive sequential order of files or references to files.
Generally, database 402 may contain hundreds, thousands, tens of thousands, or even more audio files. Accordingly, database 402 may represent one or more physical devices that store these files. Such physical devices may be located at the same physical location or distributed over a communication network (eg, the Internet). In some cases, audio files may be obtained via a third-party file download or streaming service.
Playlist 400 includes references to five songs and four generally interleaved news stories. However, a playlist may include more or fewer entries. Additionally, the pattern of news stories and some music may vary. In some cases, the playlist may primarily contain references to news stories, with fewer references to music; in other cases, the playlist may contain fewer references to news stories, and in other cases, the playlist may contain fewer references to news stories, It may also primarily include references to music.
The mix of news stories and music referenced in playlist 400 can represent a style of content called "Morning Drive Time" that mixes spoken material with music. Accordingly, the "news stories" referred to in playlist 400 may include information about current events, weather, traffic, humorous stories, joke exchanges, advertisements, and the like. Other arrangements of playlist content are also possible.
In some embodiments, music and/or stories may be repeated to some extent. However, repeating a story is generally less desirable than repeating music. On the other hand, repeating stories that include advertisements may be desirable.
FIG. 5A is a message flow diagram 500 representing example generation and distribution of a playlist. Flowchart 500 includes a client device 102, a server device 104, a profile database 502, a text story database 504, and an audio database 506. In some embodiments, one or more of server device 104, profile database 502, text story database 504, and audio database 506 are combined into fewer components or divided into more components. Good too. As just two possible examples, server device 104 may include the functionality of profile database 502, text story database 504, and audio database 506, or profile database 502, text story database 504, and audio Each of databases 506 may be distributed on multiple physical computing devices. Other arrangements are also possible.
The client device 102 can include an audio player application that requests a playlist, loads a playlist, parses a playlist, and/or sends an audio file referenced in a playlist. Streams can be requested. In some embodiments, an audio player application may be triggered to request and begin streaming audio files by receiving a playlist.
At step 510, client device 102 may send a playlist request to server device 104. This request may not explicitly seek to obtain a particular playlist, but rather may request streaming of personalized radio stations for the user of client device 102. At step 512, server device 104 may send a profile request to profile database 502 for a profile of client device 102. At step 514, profile database 502 may send a representation of this profile to server device 104.
The profile may include information about the user of client device 102 and the user's preferences for radio programming. For example, a profile may include information about demographic information about the user, the user's news interests, e-commerce transactions, the user's favorite music styles, favorite sports teams, etc. Based at least in part on the information in the profile, server device 104 can select stories and music for the playlist.
Accordingly, at step 516, server device 104 may evaluate and select a text story from text story database 504. Similarly, at step 518, server device 104 may evaluate and select audio files from audio database 506. Text story database 504 may include, for example, text files containing news stories. Audio database 506 may include news stories and/or music audio files. In some embodiments not shown, audio files containing news stories may be stored in one physical or logical database, while audio files containing music may be stored in a different physical or logical database. Sometimes.
In step 520, server device 104 (or another device) may use text-to-speech (TTS) technology if one or more text files containing news stories are selected for inclusion in the playlist. You can convert these text files to audio files with . Such TTS technology can automatically turn words and/or syllables in a text file into synthetic speech. At step 522, any converted news story audio files may be stored in audio database 506.
At step 524, server device 104 may generate a playlist. As discussed in the context of playlist 400, the generated playlist may include a series of references to audio files stored in audio database 506. The playlist may then be sent to the client device 102 at step 526.
FIG. 5B is a message flow diagram 528 representing an example playback of an audio file referenced in a playlist. Continuing from where FIG. 5A left off, at step 530, client device 102 may send a request to audio database 506 for a first entry of a playlist. For example, client device 102 can parse the playlist, determine the URL of the first entry, and request the content at that URL from audio database 506. In response, at step 532, audio database 506 may send the stream of audio files associated with this entry to client device 102. Then, at step 534, client device 102 may play the stream.
Among other things, client device 102 may receive and buffer a portion (eg, 5-10 seconds) of the stream before beginning playback of the stream. In this manner, if there is jitter in the delivery of subsequent portions of the stream to the client device 102, this jitter may be hidden from the user by buffering the stream. Client device 102 may attempt to maintain this buffer throughout playback of all audio files.
At step 536, client device 102 may send a request to audio database 506 for a second entry in the playlist. Client device 102 may send this request while still playing the portion of the audio file associated with the first entry. In this manner, client device 102 may be able to smoothly transition from playing the audio file associated with the first entry to playing the audio file associated with the second entry. For example, an audio player application may fade out the end of an audio file associated with a first entry while fading in the beginning of an audio file associated with a second entry.
Regardless, at step 538, audio database 506 may send to client device 102 the stream of audio files associated with the second entry. Then, at step 540, client device 102 may play the stream.
5A and 5B are merely exemplary embodiments of playlist generation and audio file streaming. Other embodiments can be used to achieve the same or similar results.
(4. Generating Playlists with Associated Music and Stories) One possible drawback of computer-generated playlists is that the Stories and music selected for inclusion in a given playlist may be Being able to be relevant. However, the generation of these playlists provides an opportunity to thematically relate the entries in the playlists. For example, attributes or characteristics of each entry in the playlist may be used as criteria for selecting other entries in the playlist or to order preselected entries.
For example, text in a text file containing stories can be used to select other entries (stories or music) in a playlist. Additionally, an audio file that includes music may include or be associated with one or more attributes. These attributes can exist as metadata that can be included in the audio file or in another file paired with the audio file. Such attributes include the name of the artist who wrote and/or performed the music, the title of the music (e.g., song title), the lyrics of the music, the length of the music, the publisher of the music (e.g., record label), the music may include various audio file descriptors and/or descriptors of the music contained in the audio file, such as where the audio file was recorded, etc. Any of these attributes can be used to select other entries (stories or music) in the playlist.
In some embodiments, the playlists have a predefined theme (e.g., a "classic rock" theme for a music-focused playlist, a "classic rock" theme for a playlist that includes music and/or interviews by a particular artist, etc.). ``Artist'' theme, or a ``Soccer'' theme for stories and/or music related to a particular soccer event or team). When a playlist is associated with such a theme, the selection of entries in the playlist can be based not only on other entries in the playlist but also on that theme.
When an audio file is selected for a specific reason (for example, it has been determined that it is related to the content of a previous audio file), this reason is displayed to the user of the client device playing the audio file. It's okay to be. Additionally, users may be given the ability (eg, via a user interface) to express preferences for certain types of relationships between audio files. For example, a user may indicate that a used relationship is (i) acceptable, (ii) should not be used again, or (iii) should be replaced with a different relationship.
(A. Selection of Stories Based on Music) Audio files containing music may be selected for inclusion in a playlist based on information in the user profile. For example, a user profile may indicate that the user for whom the playlist is being generated has previously expressed an interest in the style of music or the artists performing the music.
Any of the attributes of an audio file can be used to select the story that precedes or follows the audio file in the playlist. As an example, the artist name may be used to find a news story about the artist, and a reference to this news story may be placed in a playlist before or after a reference to the audio file containing the music. can. If the user is interested in the artist, the selected story is also likely to be of interest to the user. For example, a selected story may include the artist's tour dates or news of an upcoming album release by the artist.
In some cases, stories may also be selected based on information in the user's profile, such as the user's location and/or purchase history. For example, stories that include tour dates may be selected because one or more of the tour dates are geographically close to the user's location (eg, within a 50 mile threshold distance from the user's home). Alternatively, or in addition, news of an upcoming album release may be selected because the user's purchase history includes purchases of other albums or songs by that artist.
Other examples of selecting stories based on attributes associated with audio files containing music are also possible. For example, titles or lyrics may be used as criteria for selecting stories. In particular, the server device can identify keywords or key phrases that appear in the title or lyrics, and then use these keywords or key phrases as input to a search engine. A search engine can return one or more stories related to a keyword or key phrase. As an example, if the title or lyrics include the words "car," "truck," "drive," or "driving," search engines search for one or more stories about cars, driving, roads, or traffic conditions. can return a reference to. Additionally, the search engine may also return references to one or more stories containing advertisements about car sales.
In case the selected story exists in a text file, the server device uses TTS technology to convert the text file into an audio file containing the story, stores this audio file in the database, and stores the stored audio. References to files can be included in playlists.
FIG. 6 is a flowchart illustrating an exemplary embodiment. The process illustrated by FIG. 6 may be performed by a server device, such as computing device 200, and/or a cluster of computing devices, such as server cluster 304. However, the processes may be performed by other types of devices or device subsystems.
Block 600 may include selecting audio files including music based possibly on a profile associated with the client device. A client device and a server device may be communicating with each other via a wide area network, and the client device may include an audio player application.
Block 602 may include selecting the text file containing the story based possibly on attributes of the audio file containing the music. Block 604 may include converting the text file containing the story into an audio file containing the story by an automated text-to-speech system. In some cases, blocks 602 and 604 may be replaced with directly selecting the audio file that contains the story. This may be done if the audio file is associated with one or more attributes that describe its content.
Block 606 may include generating a playlist for the client device, the playlist including (i) references to audio files that include music, and (ii) references to audio files that include stories. . Block 608 may include transmitting the playlist to the client device over the wide area network. Receipt of the playlist at the client device may cause the audio player application to retrieve and play each of the audio files containing music and the audio files containing stories. The audio player application can retrieve each of the audio files containing music and the audio files containing stories from the server device.
In some cases, the text file or audio file containing the story may also be selected based on the text file or audio file containing the story being available from a third party media server affiliated with the server device. good. In other words, the text or audio files containing the stories may be selected from a media server that is not operated or owned by the entity that operates or owns the server device that generates the playlist.
The attribute may be one of an artist name associated with the music, a song title associated with the music, and/or lyrics associated with the music. However, other attributes are also possible. A text file or audio file containing a story may be selected based on the attribute being within or associated with a text file or audio file containing a story. Alternatively, or in addition, the attribute may be associated with a particular geographic area, such that the text file or audio file containing the story contains text related to or associated with the particular geographic area. A text file or an audio file containing a story may be selected based on being associated with a story.
In other embodiments, the profile associated with the client device may include an indication of the location of the client device, and the audio files containing music may be selected such that the attributes are related to the location of the client device. Alternatively, an audio file that includes music based on the artist associated with the music performing within a predetermined distance of the client device's location within an upcoming time period (e.g., within the next few days or within the next few weeks) may be selected.
In some embodiments, references to audio files containing music may be ordered within the playlist before references to audio files containing stories. Thus, the audio player application can retrieve and play at least some of the audio files that include music before retrieving and playing the audio files that include the story. Alternatively, references to audio files containing music may be ordered in the playlist after references to audio files containing stories. In this case, the audio player application may retrieve and play at least some of the audio files that include music after retrieving and playing the audio files that include the story.
In some embodiments, the profile associated with the client device can include a preferred language, such that the audio file containing the music is in the preferred language of at least some singing therein. May be selected. The attribute may indicate that at least some of the sung portion of the music is in the preferred language, and the text file or audio file containing the story is based on at least some of the sung portion of the story being in the preferred language. May be selected.
(B. Music Selection Based on Stories) Instead of or in addition to selecting stories based on attributes associated with the audio files, audio files may be selected based on the content of the stories. . This may include selecting a text file containing a story based on the user's profile, and then selecting an audio file containing music based on the content of the text file. For example, text files may be selected based on user demographics, interests, location, purchase history, and/or other factors.
The server device can identify keywords or key phrases that appear within the text file and then use these keywords or key phrases as input to a search engine. A search engine can return references to one or more audio files containing music related to a keyword or key phrase. For example, a search engine can search for keywords or key phrases in attributes associated with audio files. As an example, if a text file contains the words "car," "truck," or "drive" or "driving," a search engine will search for one or more text files with artist names, song titles, or lyrics about cars and driving. Can return references to multiple audio files. In some embodiments, semantic analysis of a text file can be performed to determine meanings or themes of words in the text file. Semantic analysis can include associating syntactic structures such as phrases, clauses, sentences, paragraphs, etc. with their language-independent meanings. In this way, words in a text file or audio file may be considered in the context of other words therein. The results of this analysis can be used to select one or more moods of the text.
Additionally, the server device may use TTS technology to convert the text file into an audio file containing a story, store this audio file in a database, and include a reference to the stored audio file in a playlist. can.
FIG. 7 is a flowchart illustrating an exemplary embodiment. As in FIG. 6, the process illustrated by FIG. 7 may be performed by a server device, such as computing device 200, and/or a cluster of computing devices, such as server cluster 304. However, the processes may be performed by other types of devices or device subsystems.
Block 700 may include selecting a text file containing a story based possibly on a profile associated with the client device. A client device and a server device may be communicating with each other via a wide area network, and the client device may include an audio player application.
Block 702 may include selecting an audio file containing music based as possible on the text within the text file containing the story. Audio files containing music may be selected based on text that matches one of the artist name, song title, or lyrics associated with the music. Block 704 may include converting the text file containing the story into an audio file containing the story by an automated text-to-speech system.
In some cases, at least a portion of blocks 700, 702, and 704 may be replaced with directly selecting an audio file that includes a story. This may be done if the audio file is associated with one or more attributes that describe its content. In these cases, a separate transform block may not be necessary.
Block 706 may include generating a playlist for the client device. A playlist can include (i) references to audio files that include music, and (ii) references to audio files that include stories. Block 708 may include transmitting the playlist to the client device over the wide area network. Receipt of the playlist at the client device may cause the audio player application to retrieve and play each of the audio files containing music and the audio files containing stories.
The embodiments of FIGS. 6 and 7 can be simplified by removing any one or more of the features shown therein. Additionally, these embodiments can be combined with each other and with any features, aspects, and/or implementations of the preceding figures or otherwise described herein.
(5. Generating playlists with associated music and story vibes) Another possible drawback of computer-generated playlists is that the stories and music selected for inclusion in a given playlist However, they may have different atmospheres. Transitions between moods may seem discordant to the listener. For example, if a news story in a playlist is about the benefits of quiet mediation, but the previous or next audio file referenced in the playlist contains loud, pounding music, the sequential order of these entries may seem inappropriate.
Even worse, computer-generated playlists may place news stories about a tragedy before or after music with lyrics that are insensitive in the context of the tragedy. As an example, if a news story is reporting on a shooting or the trial of a shooter, the music that follows this news story with lyrics that glorify or make fun of guns, death, or lawlessness will not emphasize the tragedy. It would seem like a mockery. Such an arrangement of entries in a playlist can be unpleasant for listeners.
However, computerized generation of playlists provides the opportunity to select entries in the playlist such that the mood of the sequentially ordered music and story is relevant. In some cases, this means that the mood of an entry in the playlist is based on the mood of one or more previous entries in the playlist. For example, the mood of entry i in the playlist may be similar or the same as the mood of entry i-1 in the playlist, or the mood of entry i may be contrary to the mood of entry i-1. good. In doing so, the textual content of the news story as well as the musical content can be taken into account.
The mood of a story may be determined by the presence or absence of certain keywords and/or key phrases. As noted above, semantic analysis of a text file can be performed to determine the meaning or theme of the words therein. The results of this analysis can be used to select one or more moods of the text. For audio stories, characteristics of the speaker's voice may alternatively, or as well, be used, such as prosodic information (eg, vocal intonation, stress on certain syllables or words).
In some cases, an audio file containing a story may be converted to a text file containing a story using speech-to-text (speech recognition) technology. These text files can then be analyzed for keywords and key phrases and/or semantic information associated with one or more moods. In some cases, one may manually determine the tone or moods of a story. The audio file containing the story may then be associated with the identified atmosphere. For example, atmosphere may be included as metadata in audio files or in separate files associated with each audio file.
The mood of music may be determined by the presence or absence of certain keywords and/or key phrases in the music's artist name, song title, and/or lyrics, as well as the music's tempo, dynamic range, and/or volume. As noted above, audio files containing music may be associated with metadata that includes textual representations of these characteristics.
Similar to the textual representation of a story, this metadata can be parsed for keywords and key phrases associated with one or more moods. In some cases, one may manually determine one or more moods of the music. Audio files containing music may then be associated with the identified atmosphere. For example, atmosphere may be included as metadata in audio files or in separate files associated with each audio file.
Based on the identified mood of the story, appropriate audio files containing music may be selected. Alternatively, or in addition, appropriate audio files containing stories may be selected based on the identified mood of the music. The relationship between various atmospheres that can promote these processes is shown in Figure 8.
Table 800 lists multiple moods: happy, sad, funny, miserable, angry, frustrated, anxious, lonely, confused, excited, hopeful, and relaxed. For each of these atmospheres, an entry provides an indication of the relationship of the particular atmosphere to other atmospheres. If the entry contains a check mark, the two moods match each other. If the entry contains an x, the two moods are mutually exclusive. If the entry is blank, the two moods will neither match nor contradict each other.
For example, joy matches itself as well as fun. Happiness is also opposed to sadness, misery, anger, frustration, and loneliness. Therefore, if the goal is to match the mood of a story with the mood of music associated with joy, only stories associated with a mood of joy or fun are candidates for selection. On the other hand, if the goal is to choose stories that have a mood that is consistent with the musical mood associated with happiness, choose stories that are associated with moods of sadness, misery, anger, frustration, and loneliness. will be excluded from selection. Furthermore, if the goal is to select stories that have moods that conflict with the musical moods associated with happiness, only stories that are associated with moods of sadness, misery, anger, frustration, and loneliness. will be selected.
The relationships shown by table 800 are presented for purposes of example. Different relationships between atmospheres may be used, and therefore table 800 may be written differently. Additionally, more or fewer atmospheres may be used in such tables.
In some situations, music with a certain mood or content may be intentionally omitted from the playlist. Accordingly, audio files containing music associated with characteristics embodying a particular mood may be excluded from the playlist. This may be done, for example, in response to current events or because of the current date. For example, music associated with sadness or anger may not be placed in a playlist generated for a public holiday.
Additionally, playlists generated for a client device will not include music that is associated with a certain mood or associated with certain keywords, key phrases, or meanings based on the client device's profile. There is. This may be done to avoid offending users with known sensitivities.
As noted above, playlists may have a predefined theme (e.g., a "Classic Rock" theme for a music-focused playlist, a "Classic Rock" theme for a playlist containing music and/or interviews by a particular artist, etc.). a "Soccer" theme, or a "Soccer" theme for stories and/or music related to a particular soccer event or team. When a playlist is associated with such a theme, the selection of entries in the playlist can be based not only on the mood of other entries in the playlist, but also on that theme.
When an audio file is selected for a specific reason (for example, it has been determined that it is related to the content of a previous audio file), this reason is displayed to the user of the client device playing the audio file. It's okay to be. Additionally, users may be given the ability (eg, via a user interface) to express preferences for certain types of relationships between audio files. For example, a user may indicate that a used relationship is (i) acceptable, (ii) should not be used again, or (iii) should be replaced with a different relationship.
(A. Selection of Music Based on the Atmosphere of the Story) FIG. 9 is a flowchart illustrating one embodiment for selecting audio files containing music based on the atmosphere of the story. 6 and 7, the process illustrated by FIG. 9 may be performed by a server device, such as computing device 200, and/or a cluster of computing devices, such as server cluster 304. However, the processes may be performed by other types of devices or device subsystems.
Block 900 may include obtaining an audio file that includes a story. A client device and a server device may be communicating with each other via a wide area network, and the client device may include an audio player application. Block 902 may include converting the audio file containing the story into a text file containing the story by an automated speech-to-text system.
Block 904 may include determining the tone of the story based possibly on the presence or absence of one or more keywords in the text file containing the story. This block may include semantic and/or prosodic analysis. The mood may be one of happy, sad, funny, miserable, angry, frustrated, anxious, lonely, confused, excited, hopeful, relaxed, or some other mood.
In some cases, audio files containing stories may already be associated with an atmosphere. In such situations, some or all of blocks 902 and 904 may be omitted. For example, determining the mood of a story can be based on one or more story attributes associated with an audio file containing the story.
Block 906 may include selecting audio files including music based as possible on the mood of the story. An audio file containing music may be associated with one or more musical attributes that indicate its mood. In some cases, story attributes may match music attributes. Alternatively, or in addition, selecting an audio file that includes music may include (i) the artist name associated with the music, (ii) the title associated with the music, or (iii) the lyrics associated with the music. , can be based on the presence or absence of one or more keywords.
Block 908 may include generating a playlist for the client device. A playlist can include (i) references to audio files that include music, and (ii) references to audio files that include stories. Generating a playlist to include (i) references to audio files containing music, and (ii) references to audio files containing stories can be done based on mood, date, and/or time of day. can be based on.
Block 910 may include transmitting the playlist to a client device over a wide area network. Receipt of the playlist at the client device may cause the audio player application to retrieve and play each of the audio files containing music and the audio files containing stories. The audio player application can retrieve each of the audio files containing music and the audio files containing stories from the server device. Alternatively, at least one of the audio files containing music or the audio files containing stories may be retrieved from a third party media server affiliated with the server device.
In some embodiments, references to audio files containing music may be ordered in the playlist before references to audio files containing stories, and the audio player application retrieves the audio files containing stories. At least some of the audio file containing the music can be retrieved and played before being played. Alternatively, references to audio files containing music may be ordered in the playlist after references to audio files containing stories, and the audio player application, after retrieving and playing the audio files containing stories, At least some of the audio files containing music can be retrieved and played.
Indicating a mood by one or more musical attributes may include that the musical attributes express a mood. In some embodiments, the atmosphere may be a first atmosphere, and the musical attribute indicating the first atmosphere includes the musical attribute representing a second atmosphere based on the first atmosphere. be able to. A musical attribute indicating an atmosphere means that a particular tempo, a particular beat pattern, or a particular chord progression (i) is associated with an atmosphere, and (ii) is present in the music. The attribute may further include what it indicates.
(B. Selection of Stories Based on Musical Atmosphere) FIG. 10 is a flowchart illustrating one embodiment for selecting an audio file containing a story based on the musical atmosphere. 6, 7, and 9, the process illustrated by FIG. 10 may be performed by a server device, such as computing device 200, and/or a cluster of computing devices, such as server cluster 304. However, the processes may be performed by other types of devices or device subsystems.
Block 1000 may include obtaining an audio file that includes music. A client device and a server device may be communicating with each other via a wide area network, and the client device may include an audio player application.
Block 1002 may include determining the mood of the music, possibly based on one or more musical attributes associated with an audio file that includes the music. The music attribute can indicate the atmosphere. For example, a music attribute may indicate that a particular tempo, a particular beat pattern, or a particular chord progression (i) is associated with a mood, and (ii) is present in the music.
Block 1004 may include selecting a text file containing a story based as much as possible on the mood of the music. One or more keywords, or their meanings, in a text file containing a story can indicate its mood. Block 1006 may include converting the text file containing the story into an audio file containing the story by an automated text-to-speech system. In some cases, at least a portion of blocks 1004 and 1006 may be replaced with directly selecting an audio file that includes a story. This may be done if the audio file containing the story is associated with one or more story attributes that describe its content.
Block 1008 may include generating a playlist for the client device. A playlist can include (i) references to audio files that include music, and (ii) references to audio files that include stories.
Block 1010 may include transmitting the playlist to a client device over a wide area network. Receipt of the playlist at the client device may cause the audio player application to retrieve and play each of the audio files containing music and the audio files containing stories.
The embodiments of FIGS. 9 and 10 can be simplified by removing any one or more of the features shown therein. Additionally, these embodiments can be combined with each other and with any features, aspects, and/or implementations of the preceding figures or otherwise described herein.
6. Generating Playlists Based on Time Constraints Media playback devices such as radios and televisions, and media player applications on various types of computing devices may support sleep timers. If the sleep timer function is activated during audio (or video) playback, the playback continues for the specified period of time and then ends abruptly at the end of the length of time defined by the sleep timer. The goal of such a feature is to allow the user to be lulled to sleep while playback is occurring, and once the user falls asleep, to turn off playback, thereby saving energy and creating a quieter The goal is to provide an environment that is more conducive to sleep.
However, the media that is played does not change between when the sleep timer function is activated and when playback ends due to activation of the sleep timer function; that is, the media that is played does not change between when the sleep timer function is activated and when playback ends; It's the same as if it wasn't activated. Therefore, when a user is trying to sleep, some media (eg, loud music and/or music with high dynamic range) may be played that is not conducive to sleep. Also, the natural termination of playback may be disturbing to the user and may even wake up a sleeping user. Computer generation of playlists can be used to overcome these two shortcomings of current technology.
Nevertheless, embodiments described herein can be used for purposes other than supporting improved sleep timer functionality. For example, these embodiments can be used whenever a user desires to modify an existing playlist or to generate a playlist of a fixed length and/or a fixed atmosphere. For example, a positioning unit (eg, Global Positioning System (GPS)) in a vehicle or on a wireless communication device can be used to at least partially determine the time to reach a destination. Timers herein may be set based on such time and may also be updated when the time changes. Alternatively, the timer herein can be based on a predetermined schedule, such as an event taking place in a personal or group calendar application, or even on a train or bus schedule. Other possibilities also exist.
FIG. 11 depicts a timing diagram for generating a new playlist in response to activation of a timer function. At time T0, the client device begins playing entry n-1 of playlist 1. Playlist 1 may be a playlist generated for a client device and may have been generated according to any of the embodiments herein. Not unlike the embodiment of FIG. 5B, for each entry in playlist 1, the client device can (i) sequentially request and receive a stream of the entry, and (ii) play the stream.
At time T1, the client device begins playing entry n of playlist 1. This entry is almost 4 minutes long. At time T2, approximately three minutes into the playback of entry n of playlist 1, the client device receives a timer function activation indicating that the requested timer duration is 30 minutes. The timer may be activated by the user of the client device, the user may select the length of time, or the length of time may be automatically selected based on a predetermined schedule or a trigger from another software unit. may have been done.
In response to receiving the timer function activation and understanding that there is approximately 1 minute remaining in the playback of entry n of playlist 1, at time T3, the client device may request a 29 minute playlist. can. This request may be sent to a server device, eg, the server device that provided playlist 1 to the client device.
In response to receiving such a request, the server device may generate playlist 2. Entries for this playlist may be selected such that the total time length of the playlist is approximately 29 minutes long. Additionally, if the timer is a sleep timer, these entries may be selected such that they aid in sleep. For example, music with a slow tempo, narrow dynamic range, and/or low volume may be selected. Additionally, the entries in the playlist may be arranged so that any music or stories within the playlist become progressively quieter over the length of the playlist. Therefore, playlist 2 can be organized to lull the listener to sleep.
At time T4, the client device may receive playlist 2. At time T5, the playback of entry n of playlist 1 has finished, so the client device can begin playing entry 1 of playlist 2. Accordingly, the client device can switch from playing the entries in Playlist 1 to playing the entries in Playlist 2. At time T6, the client device may continue playing Entry 2 of Playlist 2 by beginning playing Entry 2 of Playlist 2.
FIG. 12 is a flowchart illustrating one embodiment for switching from playing one playlist to playing another playlist. 6, 7, 9, and 10, the process illustrated by FIG. 12 is performed by a server device, such as computing device 200, and/or a cluster of computing devices, such as server cluster 304. good. However, the processes may be performed by other types of devices or device subsystems.
Block 1200 may include generating a first playlist for a client device. The first playlist can include references to a first plurality of audio files in a first defined order. Playing the first plurality of audio files can have a first length of time. A client device and a server device may be communicating with each other via a wide area network, and the client device may include an audio player application.
Block 1202 may include transmitting the first playlist to a client device over a wide area network. Receipt of the first playlist at the client device may cause the audio player application to retrieve and play at least some of the first plurality of audio files in a first defined order.
Block 1204 may include receiving instructions from the client device while the client device is playing a current audio file of the first plurality of audio files. Block 1206 may include generating a second playlist, possibly in response to receiving the instructions. The second playlist can include references to a second plurality of audio files in a second defined order. Playback of the second plurality of audio files can have a second time length that is shorter than the first time length.
Additionally, the second plurality of audio files may not include any audio file of the first plurality of audio files that precedes the current audio file in the first defined order. . Thus, the second plurality of files may be selected to avoid repeating media that has already been played by the client device. Alternatively, or in addition, the second plurality of audio files includes at least some audio files of the first plurality of audio files that are subsequent to the current audio file in the first defined order. May include. In some cases, generating the second playlist includes selecting a second playlist from audio files in the first plurality of audio files that are subsequent to the current audio file in the first defined order. may include selecting an audio file. Thus, the second plurality of files may be selected to include at least some media that would otherwise have been played by following the first playlist. Nevertheless, the second plurality of files may have no files in common with the first plurality of files.
In some embodiments, audio files in the first plurality of audio files are associated with respective priorities. Generating a second playlist allows you to select multiple audio files from the first multiple while prioritizing higher priority audio files over lower priority audio files. This may include selecting. Audio files containing advertisements may be associated with a higher priority than audio files containing music, or vice versa.
Block 1208 may include transmitting the second playlist to the client device over the wide area network. Receipt of the second playlist at the client device may cause the audio player application to retrieve and play a second plurality of audio files in a second defined order. Receiving the second playlist at the client device may also cause the audio player application to play the second plurality of audio files after the audio player has completed playing the current audio file.
In some embodiments, the instructions define a target time length and generating the second playlist includes a second plurality of audio files such that the second time length is within a threshold of the target time length. This may include selecting files. The threshold may be, for example, 3 minutes or less. Therefore, the threshold may be 15 seconds or less, 30 seconds or less, 1 minute or less, 2 minutes or less, etc. Additionally, if the instructions specify that the client device is operating a sleep timer for a target length of time, the second plurality of audio files may have (i) a slow tempo, (ii) a narrow dynamic range, or ( iii) may be selected based on a preference for one or more of: low volume;
The embodiment of FIG. 12 can be simplified by removing any one or more of the features shown therein. Additionally, this embodiment may be combined with any features, aspects, and/or implementations of previous figures or otherwise described herein.
7. Conclusion This disclosure is not to be limited with respect to the particular embodiments described in this application, which are intended as illustrations of various aspects. Many modifications and variations can be made without departing from the scope of the invention, as will be apparent to those skilled in the art. Functionally equivalent methods and apparatus within the scope of the disclosure, in addition to those enumerated herein, will be apparent to those skilled in the art from the above description. Such modifications and variations are intended to fall within the scope of the appended claims. The above detailed description explains various features and functionality of the disclosed systems, devices, and methods with reference to the accompanying figures. The illustrative embodiments described herein and in the figures are not meant to be limiting. Other embodiments may be utilized and other changes may be made without departing from the scope of the subject matter presented herein. As generally described herein and illustrated in the figures, aspects of the present disclosure may be arranged, substituted, combined, separated, and designed in a wide variety of different configurations, all of which are described herein. It will be readily understood that this is explicitly contemplated in the book.
With respect to any or all of the message flow diagrams, scenarios, and flowcharts in the figures and discussed herein, each step, block, and/or communication is in accordance with an example embodiment. May represent processing of information and/or transmission of information. Alternative embodiments are included within the scope of these exemplary embodiments. In these alternative embodiments, functions described, e.g., as steps, blocks, transmissions, communications, requests, responses, and/or messages, may be performed substantially simultaneously or in reverse order, depending on the functionality involved. may be performed in any order out of that shown or discussed. Additionally, more or fewer blocks and/or functions may be used with any of the ladder diagrams, scenarios, and flowcharts discussed herein, and these ladder diagrams, scenarios, and flowcharts may be , can be combined with each other, partially or completely.
The steps or blocks representing processing of information may correspond to circuits that may be configured to implement defined logical functions of the methods or techniques described herein. Alternatively or additionally, steps or blocks representing processing of information may correspond to modules, segments, or portions of program code (including associated data). The program code may include one or more instructions executable by a processor to implement determined logical functions or actions in a method or technique. Program code and/or related data may be stored on any type of computer-readable medium, such as a storage device including a disk, hard drive, or other storage medium.
Computer-readable media may also include non-transitory computer-readable media, such as computer-readable media that store data for short periods of time, such as register memory, processor cache, and random access memory (RAM). can. Computer-readable media can also include non-transitory computer-readable media that store program code and/or data for longer periods of time. Thus, computer-readable media may include secondary or permanent long-term storage, such as, for example, read-only memory (ROM), optical or magnetic disks, compact disk read-only memory (CD-ROM), etc. I can do it. The computer-readable medium may also be any other volatile or non-volatile storage system. A computer readable medium can be considered, for example, a computer readable storage medium or a tangible storage device.
Furthermore, the steps or blocks representing one or more information transmissions may correspond to information transmission between software modules and/or hardware modules in the same physical device. However, other information transmissions may be between software modules and/or hardware modules in different physical devices.
The particular arrangements shown in the figures are not to be viewed as limiting. It should be understood that other embodiments may include more or less of each element shown in a given figure. Additionally, some of the illustrated elements may be combined or omitted. Furthermore, example embodiments may include elements not shown in the figures.
Although various aspects and embodiments have been disclosed herein, other aspects and embodiments will be apparent to those skilled in the art. The various aspects and embodiments disclosed herein are for the purpose of illustration and are not intended to be limiting, the true scope thereof being indicated by the following claims.
13 sheets
Sheet 1 Sheet 2 Sheet 3 Sheet 4 Sheet 5 Sheet 6 Sheet 7 Sheet 8 Sheet 9 Sheet 10 Sheet 11 Sheet 12 Sheet 13
Every citation, both ways
| Document | Relation | Office |
|---|---|---|
| JP2007095239A | Cites | Japan |
| JP2010169760A | Cites | Japan |
| JP2016026427A | Cites | Japan |
| JP2005526340A | Cites | Japan |
76 members in 8 offices
Priority claims6
| Document | Office | Kind | Date |
|---|---|---|---|
| 201662274591 | United States of America | P | |
| 62274591 | United States of America | – | |
| 15043059 | United States of America | – | |
| 201615043059 | United States of America | A | |
| 2020025923 | Japan | A | |
| 2021067528 | Japan | A |
Members76
| Document | Office | Kind | |
|---|---|---|---|
| US2017193101A1 | United States of America | A1 | |
| US2017193102A1 | United States of America | A1 | |
| US2017195398A1 | United States of America | A1 | |
| WO2017120008A1 | World Intellectual Property Organization (WIPO) | A1 | |
| WO2017120009A1 | World Intellectual Property Organization (WIPO) | A1 | |
| US9959343B2 | United States of America | B2 | |
| KR20180082615A | Republic of Korea | A | |
| KR20180082616A | Republic of Korea | A | |
| US2018218072A1 | United States of America | A1 | |
| GB201810391D0 | United Kingdom | D0 | |
| GB201810393D0 | United Kingdom | D0 | |
| CN108475187A | China | A | |
| CN108604242A | China | A | |
| DE112016006153T5 | Germany | T5 | |
| DE112016006155T5 | Germany | T5 | |
| GB2561754A | United Kingdom | A | |
| GB2562913A | United Kingdom | A | |
| JP2019504353A | Japan | A | |
| JP2019506632A | Japan | A | |
| US10261963B2 | United States of America | B2 | |
| US10261964B2 | United States of America | B2 | |
| US2019138550A1 | United States of America | A1 | |
| US10311100B2 | United States of America | B2 | |
| US2019205337A1 | United States of America | A1 | |
| HK1254557A1 | Hong Kong, China | A1 | |
| US2019236100A1 | United States of America | A1 | |
| US10579671B2 | United States of America | B2 | |
| JP6665302B2 | Japan | B2 | |
| JP6665303B2 | Japan | B2 | |
| KR102098104B1 | Republic of Korea | B1 | |
| KR20200037884A | Republic of Korea | A | |
| KR102100212B1 | Republic of Korea | B1 | |
| KR20200040312A | Republic of Korea | A | |
| US2020151213A1 | United States of America | A1 | |
| JP2020096362A | Japan | A | |
| US10706099B2 | United States of America | B2 | |
| JP2020109518A | Japan | A | |
| US10740390B2 | United States of America | B2 | |
| US2020293573A1 | United States of America | A1 | |
| US2020334289A1 | United States of America | A1 | |
| KR20210035323A | Republic of Korea | A | |
| KR102244919B1 | Republic of Korea | B1 | |
| KR20210047378A | Republic of Korea | A | |
| JP6869391B2 | Japan | B2 | |
| US11017021B2 | United States of America | B2 | |
| JP6883679B2 | Japan | B2 | |
| US11061960B2 | United States of America | B2 | |
| JP2021103336A | Japan | A | |
| US2021240763A1 | United States of America | A1 | |
| CN108475187B | China | B | |
| JP2021121869A | Japan | A | |
| US2021294844A1 | United States of America | A1 | |
| CN113641850A | China | A | |
| US11216507B2 | United States of America | B2 | |
| KR102364122B1 | Republic of Korea | B1 | |
| GB2561754B | United Kingdom | B | |
| KR20220025916A | Republic of Korea | A | |
| US2022083593A1 | United States of America | A1 | |
| GB2562913B | United Kingdom | B | |
| KR102393704B1 | Republic of Korea | B1 | |
| KR20220061276A | Republic of Korea | A | |
| JP7135159B2 | Japan | B2 | |
| JP7150927B2 | Japan | B2 | |
| US11494435B2 | United States of America | B2 | |
| KR102469541B1 | Republic of Korea | B1 | |
| KR20220156987A | Republic of Korea | A | |
| US2023019226A1 | United States of America | A1 | |
| JP2023015026A | Japan | A | |
| CN108604242B | China | B | |
| KR102536152B1 | Republic of Korea | B1 | |
| KR20230074633A | Republic of Korea | A | |
| US11868396B2 | United States of America | B2 | |
| JP7429271B2This record | Japan | B2 | |
| US11921779B2 | United States of America | B2 | |
| US2024134906A1 | United States of America | A1 | |
| KR102717981B1 | Republic of Korea | B1 |
6 legal events, as the office reported them to INPADOC
Over the term
Point at a mark for the eventEvents
| Event | Code | |
|---|---|---|
| First payment of annual fees (during grant procedure)JAPANESE INTERMEDIATE CODE: A61A61 | A61 | |
| Certificate of patent or registration of utility modelJAPANESE INTERMEDIATE CODE: R150R150 | R150 | |
| Written decision to grant a patent or to grant a registration (utility model)JAPANESE INTERMEDIATE CODE: A01A01 | A01 | |
| Decision of grant or rejection writtenTRDD | TRDD | |
| Request for written amendment filedJAPANESE INTERMEDIATE CODE: A523A521 | A521 | |
| Written request for application examinationJAPANESE INTERMEDIATE CODE: A621A621 | A621 |
Numbers
- Publication
- 7429271
- Application
- 154883
Titles2
- Japanese
- 関連した音楽およびストーリを有するプレイリストの生成および配信
- English
- Generation and distribution of playlists with related music and stories
Classification
- CPC, 20
- G06F16/635
- G06F16/639
- G06F16/951
- H04L67/60
- G06F16/9535
- G06F3/165
- H04L67/06
- H04N21/44224
- G06F16/00
- G06Q30/02
- G10L13/00
- H04N21/8113
- H04N21/439
- H04L67/62
- H04N21/4532
- H04N21/44213
- H04N21/44222
- H04L67/01
- G06F16/9538
- H04L67/10
- IPC, 6
- G10K15 02
- H04N21 262
- H04N21 438
- G06F16 635
- G06F16 68
- G06F16 687
