Method and apparatus for creating and distributing real-time interactive media content through wireless communication networks and the internet

【0001】

Field of Invention The present invention relates to the formation and distribution of messages using computers and networks, and more particularly to the operator of messages having auditory or visual content formed by an operator using a handheld device such as a mobile telephone. With respect to methods and systems that allow distribution.

【0002】

Background of the Invention The use of mobile devices such as cellular telephones, so-called personal digital assistants (PDAs) and handheld computers has expanded at a rate that far exceeds even the most optimistic expectations of just a few years ago. There is. Cellular phones are widely accepted because they are inexpensive and allow individuals to easily carry them around to reach friends and entertainment sources. Other mobile devices, such as playing and recording music or video, have also been widely accepted as they provide entertainment and entertain individuals.

【0003】

These mobile devices are used worldwide by individuals of all ages, but are especially prevalent in those who embrace the trends and lifestyles of "pop culture." Pop culture makes up a large proportion of the total population (especially the younger generation) and is quickly accepted around the world.

【0004】

Music and videos recorded or distributed by professional sources are an important part of pop culture. However, there is a growing hobby for individuals to form their own auditory or visual content and share it with friends. Unfortunately, the formation and distribution of auditory and visual content such as music and video requires the use of devices that are not small, i.e. not easily portable, such as cellular phones. The ability to form and distribute at least one or both of hearing and vision using mobile devices such as cellular phones is required.

【0005】

Disclosure of the Invention An object of the present invention is to provide the formation and distribution of content such as music or moving images using a mobile device.

【0006】

According to one aspect of the invention, the operator uses a handheld device that is instructed and presented with a representation of the first content, through which at least one recipient is identified and controlled by the operator. According to the temporary relationship, the presentation of the first content is given the second content that overlaps in time, the remote server is given the representation of the second content and the recipient identification, and the remote server is at least Have one recipient send a message representing the first and second content arranged according to a temporary relationship.

【0007】

According to another aspect of the invention, the system includes a handheld device, the device having a radio and a processing circuit, the processing circuit having an output to the handheld device to indicate instructions to the operator. An input that gives a representation of the first content and represents the identification of at least one recipient, and a second content that temporally overlaps the presentation of the first content according to a temporary relationship controlled by the operator. Is received from the operator, the representation of the second content and the identification of at least one recipient are sent through the radio, and the representation of the second content and the identification of at least one recipient are received and stored. Sends a message to at least one recipient, including the subsystem, that represents the first and second content arranged according to a temporary relationship.

【0008】

According to another aspect of the invention, the handheld device comprises a server system that receives at least one signal from the handheld device generated under the control of the operator of the handheld device, wherein the handheld device is the first. The server carries the content identifier of the content, the second content that overlaps the presentation of the first content by the handheld device in time according to a temporary relationship controlled by the operator, and the identification of at least one recipient. The system obtains information that identifies the temporary relationship and receives at least one recipient a message representing the first content and the second content arranged according to the temporary relationship.

【0009】

The various features of the invention and its preferred practice are well understood by reference to the following description and accompanying drawings, in which similar reference numerals indicate similar elements in some of the accompanying drawings. To do. The content in the following description and accompanying drawings is provided by way of illustration only and should not be understood to limit the object of the present invention.

【0010】

Embodiment A of the present invention In FIG. 1, the operator of the mobile device 10 interacts with the server 30 to obtain original content and pre-existing content for at least one or both of auditory content and visual content. It is a schematic diagram of a system capable of generating messages having a combination with and controlling the distribution of these messages to at least one recipient, such as recipients 42, 52, 62.

【0011】

The following description specifically refers to cellular phones, public switched telephone networks (PSTNs), and the Internet. Although these facilities are important examples of how the present invention is utilized, the present invention may simply be implemented and used in a wider range of facilities than these particular examples. Cellular phones, Na just an example of a device that may be used as the apparatus 10 has. Other examples of suitable devices include PDAs, portable computers (including handheld computers), and two-way radios. The teachings of the present invention may be applied to stationary or desktop telephones and stationary or desktop computers. Cellular technology that meets existing or prospective standards is an example of a communication technology that can be used in device 10. The Internet and PSTN are examples of equipment that can be used to provide communication between system components. Basically any equipment or technology includes those that carry ultrasonic, wireless, and optical signals in the air or through transmission media such as electric wires and optical fibers, such as radio broadcasting, point-to-point, and multipoint communication channels.

【0012】

1. Applications The two applications of the present invention are referred to herein as Song Mail and Music DIY (do it yourself). SongMail and MusicDIY allow the operator of device 10 to form a message containing pre-existing content (eg background music, additional content given by the operator, etc.) and then receive this message. Allows transmission to at least one recipient in a form that can be heard by a person. In the SongMail application, the operator gives auditory content such as voice utterances. In a MusicDIY application, the operator provides auditory content in a manner similar to playing a music device by activating at least one input device on device 10.

【0013】

Although SongMail and MusicDIY relate to auditory content, the present invention may be used to create messages that include the presentation of essentially any content, including visual content perceptible to the recipient. In addition, the SongMail and Music DIY applications themselves can be extended to include, for example, visual content. These applications will be described in detail below, but they are shown merely as illustrations of the applications that can be provided by the present invention.

【0014】

2. Communication With reference to FIG. 1, a system incorporating various features of the present invention is a communication facility between server 30 and device 10, server 30 and potential receiver (eg computer system 42, telephone 52, mobile). Includes communication equipment with (equipment 62, etc.). The system also includes communication equipment between the server 30 and the service or content provider, which are indicated by computer system 41 and telephone 51. For example, the server 30 can obtain pre-existing content from the computer system 41, which content is not available from the server's own content database on the storage device 33.

【0015】

The communication facility between the server 30 and the device 10 is used by the operator of the device 10 to compose a message and send it to the server 30, which continuously sends the message to at least one recipient. Send. The communication equipment between the server 30 and the recipient is used by the server to distribute the message to the recipient. Communication equipment between the server 30 and the provider of the service or content (eg, pre-recorded music or video) is used by the server to acquire these service or content.

【0016】

A wide range of communication techniques, techniques and protocols can be used. Although not restricted to any particular communication facility, the facility shown in FIG. 1 is expected to be of commercial importance for many implementations of the present invention.

【0017】

3. Server The server 30 can be run by normal hardware including a processor 31, a storage device 33, a connection to communication equipment such as PSTN50, a mobile phone system, and a network 40. The network 40 may be a global network such as the Internet, or a more limited network such as an in-house intranet.

【0018】

Environments that support the various applications described below may be provided by known operating systems such as Linux, Unix®, or various versions of Windows®.

【0019】

The practice of the present invention does not require a particular software environment. The display of server 30 in FIG. 1 suggests one computer system with a single processor 31, even if server 30 is run by at least one computer system, each with a single or multiple processors. Good. These systems may be placed close to each other or widely separated from each other. It is not limited to a specific execution or configuration.

【0020】

Server 30 provides a number of services that can be used to perform the SongMail and Music DIY described above. Some examples of services that can be provided by the server 30 in various executions of the present invention are e-mail, Interactive Voice Response (IVR), Short Message Services: Includes SMS (SMS) or Multimedia Messaging Service (MMS), Wireless Access Protocol (WAP), and Hypertext Transfer Protocol (HTTP). These services are examples of currently known services and will probably be replaced by more evolving and corresponding future services. According to the teachings of the present invention, the server 30 may be given or operate with existing and future versions of these types of services.

【0021】

The storage device 33, which may include multiple devices, stores the operating system and application system software executed by the processor 31 as well as the data used by the software. Three examples of data are an operator profile, a content database, and a message formed by an operator. The operator / profile may include data such as accounting information and personal preferences associated with a particular operator. An example of an operator's preference can be a display of the type of preference of the music or recording artist. The operator / profile can also include an "address book" of potential recipients. The content database includes pre-existing content that the operator can choose to include messages with the operator's own content. Preferably, the database is arranged to facilitate selection by artist, title, content format or style carried by the content, or by the basic message or mood carried by the content.

【0022】

Server 30 may use essentially any form of communication with other components of the system. In the execution shown in FIG. 1, server 30 communicates with mobile devices 10,62 through radio channel 60, telephones 51,52 through PSTN50, and computer systems 41,42 through network 40. The signal is transmitted and received by the receiver 35 and the transmitter 36, respectively, through the radio channel 60, the receiver 35 and the transmitter 36 are connected to at least one antenna 37, and are connected to the processor 31 by the communication equipment 34. The receiver 35 and the transmitter 36 may be placed close to or widely separated from the processor 31. In one run, the communication facility 34 provides connectivity to a cellular phone system that includes multiple receivers 35, multiple transmitters 36, and multiple antennas 37.

【0023】

For the sake of brevity, the following description will refer to the implementation in which the server 30 is implemented by a single computer system with one processor 31. The processor 31 is connected to one storage device 33, PSTN50, network 40, receiver 35 and transmitter 36. Various variations in implementation and configuration are possible.

【0024】

4. Handheld device The illustration of mobile device 10 in FIG. 1 suggests a cellular phone, but other types of devices may be used as described above. In the particular embodiment illustrated, device 10 has multiple input and multiple output devices. The input device includes an array of keypad-like switches 21 that can be activated by the operator and an auditory input transducer 22 that allows the operator to speak or provide other auditory inputs. Output devices include an auditory output transducer 23, such as a loudspeaker, and a visible display 24, such as a liquid crystal display (LCD) panel, which can display visible information. In other equipment not shown, device 10 is a PDA, which has an array of switches 21 for inputs and a visible display 24 for outputs, but without an auditory input or output transducer. May be good.

【0025】

Figure 2 schematically shows the virtual implementation of device 10 as a cellular phone. In this implementation, processor 12 represents a circuit system that provides computing resources. Memory 13 provides volatile or non-volatile information storage, such as random access memory and various forms of read-only memory. The input / output control 18 represents an interface to an input / output device such as an array of switches 21 (keypad) and an auditory output transducer 26 (piezoelectric device). The display control 17 represents an interface to the display device 24 (LCD panel). Codec16 shows a circuit system that provides encoding and decoding of the auditory signal received from the auditory input transducer 22 (microphone) and sent to the auditory output transducer (loud speaker). The communication processor 14 shows an interface to the transmitter 15 and the receiver 19, and the transmitter 15 and the receiver 19 are connected to the antenna 25, and the antenna 25 transmits / receives a signal through a radio channel. In the illustrated example, various components of device 10 are connected to bus 11, which can represent more than one physical bus. Other implementations may be achieved using architectures other than the bass architecture.

【0026】

An alternative implementation of a PDA, for example, eliminates various components such as Codec16, auditory input transducer 22 and auditory output transducer 23 and replaces other components such as communication processor 14, transmitter 15, receiver 19 and antenna 25 with device 10. It may be provided by a separate device outside the. Many variants are possible.

【0027】

At least one function of these components can be achieved by a wide range of methods including discrete logic components and at least one or both of one or more ASICs and program control processors. The form of realization is not limited.

【0028】

Some components that may be important in practical implementation, such as antenna transmission / reception switches, power management circuits, etc., are not important in the principles of the present invention and are omitted in the drawings for clarity of illustration. There is.

【0029】

B. Applications Most of the implementation details for SongMail and Music DIY applications depend in part on the characteristics of device 10 and the communication technology that links device 10, server 30, and recipients. The explanations in this section give a conceptual overview of each application and omit detailed considerations that may be necessary for practical implementation. Additional considerations will be given below with respect to the description of the various implementation techniques.

【0030】

1. The SongMail operator may form a SongMail message using a telephone or other device with auditory input and output transducers. Preferably, the phone is rasterized with an array of switches 21 (which can be activated by pressing the corresponding array of buttons), an auditory input transducer 22 (eg, a microphone), and an auditory output transducer 23 (eg, a loudspeaker). It has a (rasterized) visible display device 24 (eg, an LCD panel). The operator starts the SongMail process, for example, by pressing at least one button on the device 10. The device 10 uses at least one of its output devices to represent prompts and other information, which presents auditory information, for example through an auditory output transducer 23, or preferably visible information through a display device 24. By displaying, the operator is guided during the SongMail process. If device 10 is a typical mobile phone, for example, the visible display device 24 gives only the normal display of the telephone number entered by the operator and the operator's voice as described below. Supports the setting of communication routes. In response, the operator uses at least one input device in device 10 to control message formation and distribution. For example, the audio input can be provided through an auditory input transducer 22, or preferably a tactile input by pressing a button to activate at least one switch in the array of switches 21.

【0031】

a) Settings Figure 3 shows each step in one conceptual implementation of the SongMail process. In step 101 of this process, the operator initiates the SongMail application in any manner as appropriate for a particular implementation. In step 102, the operator selects a language for the system used to provide prompts and other information to guide the operator during processing. In step 103, the operator (1) forms a new message, (2) listens to the already formed message, (3) deletes the already formed message, and (4) sends the already formed message. , Or (5) abandonment may be selected. In a preferred practice, the option to listen, delete, or send a message is not offered unless the already formed message exists. If the operator chooses to form a new message, this process continues at step 114. If the operator chooses to listen to the already formed message, this process continues at step 124. If the operator chooses to delete the already formed message, the process continues to step 134. If the operator chooses to send the already formed message, this process continues to step 144. If the operator chooses to abandon, this process continues to step 104, which performs the actions necessary to stop the process.

【0032】

b) In formation step 114, the operator selects "background music" for the message. In a preferred embodiment, the operator can select background music by title, artist, type of music, or the message or mood carried by the music. The system may also show the operator only some of the content databases stored in the storage device 33 that are filtered and placed according to the operator's preferences.

【0033】

The term "background music" is used to refer to pre-existing content given by the system to "operator content" given by the operator. This pre-existing content does not have to be background music, but such music will be a popular choice. In one embodiment, the server 30 stores the pre-existing content in a database on the storage device 33 and transmits the content selected by the operator to the device 10. In other embodiments, the pre-existing content is stored in device 10 by, for example, a removable solid-state memory device.

【0034】

In step 115, the system presents the performance of the selected background music through the auditory output transducer 23 and receives the operator content from the operator through the auditory input transducer 22. Step 116 allows the operator to sing, for example, while listening to background music. This allows the operator to temporally superimpose the operator content on the presentation of background music and control the temporary relationship of this superposition.

【0035】

Steps 115 and 116 are repeated until step 117 determines that the formation of the operator content is complete. This process then continues to step 103. Alternatively, the method may continue to a step that allows the operator to identify at least one recipient and send the newly formed operator content along with the message to that recipient. One method of sending a message is shown below. Preferably, if the method goes directly to a step that allows the operator to send a message, one step is given to allow the operator to decline the transmission of the message.

【0036】

Preferably, the server 30 stores in the storage device 30 a representation of a message that includes operator content but does not include background music selected by the operator. Server 30 only stores the identification of the selected background music and an indicator of the temporary relationship between these two contents. Background music itself is otherwise stored in the content database. The message is sent to the recipient or to the operator for review, and the representation of the background music and the operator content is between the two contents observed by the operator when these operator contents are given. Can be combined in a way that effectively preserves the temporary relationships of.

【0037】

Generally, when a particular technique, such as a cellular phone system, is used to send selected music to device 10 to present to the operator and to receive operator content from device 10. , There is a non-negligible delay in its transmission. If these delays cannot be determined, Server 10 cannot determine the temporary relationship between the operator content and the presentation of background music and cannot save this relationship when sending the message to the recipient. .. Some methods that can determine these delays will be described below. In principle, the method of determining these delays is not important to the present invention.

【0038】

If the operator uses a transmission service such as a cellular phone service and a provider of that service, a transmission delay estimate can be given, and the operator content gives this estimated round trip delay. It can be used to estimate the temporary relationships that the operator perceives when they are asked.

【0039】

If the service provider is unable to give an estimate of the transmission delay, the server 30 can measure the delay by sending a signal to the device 10 that communicates some recognizable event back to the server 30. Since the device 10 includes several features that automatically respond to a particular signal, the server 30 can measure this delay without the involvement of an operator. In another method, the server 30 is the time between transmitting an instruction to the device 10 so that the operator performs some action (for example, pressing a button) and receiving from the device 10 in which the operator performs the action. The delay can be measured by observing the intervals.

【0040】

If the delay is not known accurately, the server 30 cannot accurately determine the temporary relationship perceived by the operator, so that when a message is sent to the operator for recipient or review, this temporary relationship Can prevent the system from accurately preserving relationships. However, if the server 30 can determine the delay with an accurate degree, it can substantially store the temporary relationships observed by the operator given the operator content.

【0041】

c) In listening step 124, the operator selects the already formed message for review. In step 125, the device 10 presents a representation of the operator content and the background music selected for the message. The presentation is a method that effectively saves the temporary relationships observed by the operator when the operator content is first given, and combines the representation of the operator content with the representation of the selected background music. Overlay. This presentation continues until step 126 determines whether the presentation has been completed, or whether the operator has requested the end of the presentation, for example by pressing a button. This process continues to step 103.

【0042】

d) In delete step 134, the operator chooses to delete the already formed message. At step 135, request confirmation of the deletion. If the deletion is confirmed, the message is deleted in step 136 and the process continues in step 103. If the deletion is not confirmed, the process continues to step 103.

【0043】

e) In transmission step 144, the operator is selected to transmit the already formed message. In step 145, the operator identifies at least one recipient. In a preferred practice with a cellular phone, the operator either presses at least one button on the phone to identify one phone number, or a phone number already established by the operator and stored in the storage device 33 by the server 30. Alternatively, one recipient can be selected from the list of email addresses. In an alternative practice, the operator can identify the email address by pressing a button on the phone according to known techniques for identifying letters and numbers. Selective step 146 allows the operator to select some additional message to be included in the message (eg, text or visual image that introduces the message to the recipient). This process continues to step 103.

【0044】

In step 147, the server 30 sends a representation of the message to each of the recipients specified in step 145 by an appropriate delivery method to each recipient. The expression of the message can be communicated using various methods. The "direct" method transmits the oral content of the message directly to the recipient in almost the same manner as the method of transmitting so-called voice mail (voice mail) to the voice mail subscriber. This direct method is suitable for transmission to ordinary telephones or other devices that have an auditory output transducer and have little or no processing power. The "notification" method conveys only notifications that have instructions for retrieving the message. The notification method is suitable for transmission to basically any type of device including a normal telephone, but particularly suitable for transmission to a cellular phone by SMS method and transmission to a computer by e-mail method, for example. is there. Voice notifications can be sent to landlines or cellular phones. The operator of the recipient device can search for the actual message content by following the instructions contained in the notification. Hereinafter, the transmission method will be described in more detail.

【0045】

The transmission method may be explicitly specified in step 145, but in some cases it may be possible to infer the correct method from the recipient authentication. For example, it may be possible to infer the authentication method from the recipient email address or from the telephone number of the mobile device.

【0046】

Regardless of the transmission method used, when the recipient finally receives the auditory content of the message, the presentation of the message content is the temporary observed by the operator when the operator content is first given. Includes representations of operator content with representations of selected background music in a manner that substantially retains the relationship.

【0047】

Additional considerations are described below.

【0048】

2. The MusicDIY operator may form a MusicDIY using a telephone, a PDA, or other device having an array of input devices and an auditory output transducer such as a switch. Preferably, the device includes an array of switches 21 (which can be activated by pressing the corresponding array of buttons), an auditory output transducer 23 (eg loudspeaker), and a rasterized visible display device 24 (eg). LCD panel) and. The operator initiates the Music DIY process, for example by pressing at least one button on device 10. Device 10 uses at least one of its output devices to represent prompts and other information, which guides the operator during Music DIY processing, for example by displaying visible information through display device 24. .. If device 10 is a typical mobile phone, for example, the visible display device 24 gives only the normal display of the telephone number entered by the operator and the operator's voice as described below. Supports the setting of communication routes. In response, the operator uses at least one input device in device 10 to control message formation and distribution. Preferably, a tactile input can be provided by pressing a button to activate at least one switch in the array of switches 21.

【0049】

The method described above in connection with SongMail, shown in FIG. 3, can also be used to illustrate one conceptual implementation of Music DIY processing. Since the description of each step described above for SongMail can be applied to Music DIY in a corresponding manner, there is no need to explain other than explaining the difference in step 115.

【0050】

In step 115, the system gives the performance of the selected background music through the auditory output transducer, as was done in SongMail, but for Music DIY, instead of giving the operator direct auditory content through the auditory input transducer. , The auditory content is indirectly given by activating at least one switch in the array of switches 21 in the same manner as the playback of a music device. The device 10 produces a sound in response to the activation of the switch and presents this sound to the operator through the auditory output transducer 23. This allows the operator to listen to the music formed by the device 10 and control the temporary relationship between the presentation of background music and the music formed by the operator.

【0051】

The device 10 produces a routine annotation or sound in response to the activation of a particular switch as is done by a normal music device, but in a preferred practice, the device 10 is of a particular switch. Change the sound produced in response to activation. Preferably, the sound changes according to the characteristics of the background music, and activation of any switch in the array of switches causes the device 10 to produce a sound that follows the desired musical rules. In other words, the switch array is adapted to the sound positioning to make the overall effect of the operator content combined with the background music enjoyable, regardless of which switch is activated. .. In one implementation, background music is given to the control information, which shows how and when the position of the switch tone was changed. This will be described in more detail below.

【0052】

C. Communication Equipment The above-mentioned processing shown in Fig. 3 omits some considerations that are directly related to the implementation using various communication technologies. These considerations are described below.

【0053】

1. Server-operator communication In principle to the present invention, no particular communication equipment is important for device 10, but for many applications communication via a step 1 cellular phone system is commercially important. Is expected to be. The techniques that can be used for several types of cellular phone systems are described below. It is clear that many of these techniques can be used in a wide variety of combinations.

【0054】

a) Continuous use of data and voice communication equipment The first technology can be used for at least one communication system that provides data and voice communication equipment. According to this first technique, the operator sets up an application using the data communication equipment, then stops using the data communication equipment and uses the voice communication equipment that forms the message content. This first technique can be used, for example, in cellular phone systems that do not provide simultaneous access to data and voice communications.

【0055】

Figure 4 shows one method in which the first technology can be incorporated. In step 201, a data communication path (if this path is not yet available and available to carry operator choices such as desired application (eg SongMail or MusicDIY), language, and background music). By starting, the application "session" is established. In an alternative implementation, step 201 causes the operator to identify at least one recipient for the message that will be formed immediately.

【0056】

At step 202, the use of the data communication path ends. The server 30 stores information on a storage device 33, which stores any choices or other data given by the operator during application setup. This information is used to run the rest of the application session.

【0057】

At step 203, the session is continued by initiating a voice communication path (if this path is not yet available). Some examples of how to do this are given below.

【0058】

In the first example, prior to ending the use of the data communication path, the server 30 sends information to device 10, which identifies the session and identifies the telephone number for the operator to make a call. Includes code. For example, SMS may be used to send this information, if available. In response to receiving the phone call, the server 30 asks the operator if he / she wants to enter the session code, perhaps by pressing a button on the phone. Alternatively, device 10 stores session-identifying information and automatically identifies the session by sending an audible signal (eg, a DTMF tone) to server 30. In either case, the server 30 can obtain the session code from, for example, a voice communication channel using IVR. When server 30 receives a valid session code, the server continues the application session using the voice communication path.

【0059】

In the second example, prior to ending the use of the data communication path, the server 30 obtains the telephone number used to call the operator. This number may be obtained directly from the operator or directly from the profile information stored on the storage device 33 for this operator. Server 30 may ask the operator to confirm this number. This voice communication path is established when the server 30 reaches the operator using this telephone number. The server 30 may then ask the operator to enter some code that confirms the operator's identity, or confirms the identity of a particular session in a manner similar to that described above. When the server 30 successfully contacts the operator, the server continues the application session using the voice communication path.

【0060】

In a third example, the application running on device 10 uses services such as those provided through the Wireless Telephony Application Interface (WTAI) to call server 30, establish a voice communication path, and use WAP. Sessions are identified by sending audible signals such as DTMF tones using such protocols.

【0061】

In the fourth example, the device 10 sends information including the user identifier to the server 30 prior to ending the use of the data communication path. In response to receiving a phone call that initiates a voice communication path, the server 30 automatically determines the user identifier to identify the session. For example, if device 10 is associated with a telephone number, device 10 may send the associated telephone number to server 30, which stores other information related to the session. Server 30 may automatically determine the user identifier by using a telephone system feature such as "Caller ID" or automatic number identification (ANI).

【0062】

In step 204, the server 30 can give prompts or instructions as desired, and the server automatically starts presenting background music, or the operator presses a button on the telephone, for example, to start a command. You can either wait until you give.

【0063】

In step 205, the server 30 receives the operator content and stores it in the storage device 33.

【0064】

Steps 204 and 205 repeat the continuation of background music performance and presentation of operator content until step 206 determines that the formation process is complete. It can detect, for example, a time interval where there is no operator input beyond a certain time threshold, or receive a specific command from the operator, for example by pressing a specific button on the phone, or in the background. -It may be determined when the music performance is finished.

【0065】

The rest of the application in step 207 can be implemented in various ways. In one implementation, step 207 sends the formed message to a recipient identified during application setup. In another embodiment, step 207 allows the operator to hear a message and performs other functions in a manner similar to that described below for techniques that use only voice communication.

【0066】

In another embodiment, step 207 continues the application session by ending the use of the voice communication path and initiating the use of the data communication path. The operator may send a message in a manner similar to that described below for techniques using simultaneous data and voice communication equipment.

【0067】

In the alternative implementation of the first technique, steps 202 and 203 are performed in different ways. In step 202, the information indicating the operator's preferences and choices is stored in the device 10 instead of the server 30. In step 203, the application running within device 10 utilizes services such as those provided through WTAI to server calls, establishes voice communication paths, and delivers audible signals such as DTMF tones using protocols such as WAP. Identify the operator's preferences and choices by sending.

【0068】

b) Use of voice communication equipment only The second technology can be used with any communication system that provides communication equipment. According to the second technique, the operator sets up the application and forms the message content using only the voice communication equipment.

【0069】

Figure 3 shows one method in which the second technique can be implemented. In step 101, a voice communication path is established (if it is not already available). The rest of the steps are performed by letting the operator form a choice and enter commands through the voice communication path. This can be done, for example, by voice utterance or by pressing a button on the phone. The server 30 may use speech recognition software that interprets speech utterances. A convenient practice uses the usual dual-tone multiple-frequency (DTMF) signals to indicate operator selection and commands. Letters and numbers can be entered via buttons using known techniques.

【0070】

If voice recognition is used, this feature must be suppressed or carefully controlled while the operator provides operator content for the SongMail application. This can be achieved, for example, with respect to speech recognition by limiting the limited memorized words to those that are unlikely to be spoken by the operator as operator content. Speech recognition is preferably suppressed during this stage of processing and only the DTMF signal is recognized as a command.

【0071】

With DTMF recognition, this feature must be suppressed or carefully controlled while the operator provides operator content for the Music DIY application. Preferably, if device 10 makes this possible, all DTMF production is suppressed during the Music DIY formation phase. However, one button, such as a hash (#) button or a star (*) button, may be reserved to represent a command to stop the formation stage. Alternatively, speech recognition may be available to recognize commands that stop this stage, with all buttons available for inputting operator content.

【0072】

c) Simultaneous use of data and voice communication equipment The third technology can be used with any communication system that provides simultaneous access to data and voice communication equipment. According to this third technique, the operator sets up the use of an application that uses the data communication equipment and then forms a message using the voice communication equipment without intermittent use of the data communication equipment.

【0073】

Figure 3 shows one method in which the third technique can be implemented. In step 101, the data and voice communication paths are established as needed if neither is already available. The rest of the steps are performed by having the operator make a choice, enter commands through the data communication path, and receive background music through the voice communication path.

【0074】

d) Use of data communication equipment only The fourth technology uses features such as "voice on IP" that allow operator content to be sent over the data communication path. This allows both setup and message formation applications to be performed using data communication paths.

【0075】

Figure 3 shows one method in which the fourth technique can be implemented. In step 101, if the data communication path is no longer available, establish it. The rest of the steps are performed by having the operator make a choice and enter commands through the data communication path, receive background music, and provide operator content through the data communication path.

【0076】

2. Server-Recipient Communication Server 30 can deliver message content directly to the recipient, but the preferred practice is to deliver only notifications in place of the message, which includes instructions to search for the message. It can also include introductions and explanations from the formed operator. This can be done in a variety of ways. No particular method is important to the present invention.

【0077】

One way to send notifications is by using the messaging service of a telephone system such as SMS or MMS. The content of an SMS message is, for example, text only due to length restrictions, but device 10 may mark the text in the message (eg, a phone number) to the user, and this marked text may be used. It may be used for telephone calls. Other text, such as message code or session code, can also be marked and sent to server 30 to identify the message to be retrieved. These features reduce the effort required for message retrieval.

【0078】

Another way to send notifications is by email, which gives more space than SMS. The e-mail message contains a Uniform Resource Locator (URL) that links to a file on network 40 that contains the message to search for. Using a regular browser or various email software, recipients can search for messages simply by using a pointing device (eg, a mouse) that clicks on a URL link. E-mail messages also include additional content, such as graphical images, selected by the operator who formed the message.

【0079】

Other ways to send notifications are by auditory information. Using speech synthesis software, for example, the server 30 can generate notifications that can be sent by phone to the recipient or stored directly in the recipient's voice mailbox. In one practice, this notification includes a phone number for the call and a code that identifies the message to search for. In other practices, the notification explains to the recipient that the operator identified by name formed the message, and then asks the recipient if he or she wishes to receive it. If the recipient affirms the delivery, the server 30 delivers the message over the phone. The server 30 may be able to return a voice message to the recipient or send this message to others.

【0080】

The message itself may be stored or transported in essentially any format capable of presenting content.

【0081】

D. Additional Features In a preferred implementation of a Music DIY application, device 10 is adapted to synchronize with background music, and any switch activation in the switch array causes device 10 to sound according to the desired music rules. It is designed to be generated. This feature can be used so that the overall effect of the operator content combined with the background music is pleasing, regardless of which switch is activated.

【0082】

The device 10 can be adapted to a particular sound or note by changing the mapping associated with each switch in the switch array. At least one mapping relationship is referred to herein as a "map," which can be a priori stored in device 10 or obtained from information sent with background music.

【0083】

1. Sync The maps that are valid at a particular time change as background music is presented to the operator, or are selected from a set of maps. The identification of the map to be used and the time to use this map are specified by the control information, which is received during or prior to the time the background music is received or presented to the operator. To. In general, the mapping fits the time corresponding to a key change or chord change in background music.

【0084】

If control information is received prior to presentation, it must specify the map to use and when to use this map. It may specify the desired map and the corresponding time by associating the map identifier with the time in the background music that is staggered with respect to the start of the music. For example, the control information that specifies 7@37.2 should start with the 7th map from the set of maps already stored at the time when device 10 is 37.2 seconds after the start of background music. Is shown.

【0085】

If the control music is received during the presentation of the background music, the time is indicated by the time when the control information was received, so even if you specify only the map to be used. Good.

【0086】

2. Mapping In a preferred practice, the mapping is based on standard music theory and based on five jazz / pop music chords: major, dominant 7th, minor (durian), half diminish (locrian) and diminish. It is a predetermined scale structure.

【0087】

The chord represents a series of tones called scales. The scales of the other chords are ordered according to their "sound" or pitch of consonant or non-consonant to the bass tone of a particular chord. It often has a scale of 5 degrees or more that is relatively compatible with any particular code.

【0088】

The table below shows two examples of a given code (referred to herein as the "base code"), and the other codes are in harmony with the base code. The bass chord is shown in the first column, which shows the chord that occurs in the background music. The base code notes are shown in the second column. Other codes that are compatible with the base code are shown in the third column. In both examples, the consonant code is chosen to be identical to the base code. The notes on the scale of the consonant code are shown in the fourth column.

【0089】

[table 1]<img file="JP2004536498A_D0001.tif" /> 【0090】

Device 10 is a mobile phone with a regular 12 keypad (with buttons labeled 1-9, *, 0, #), and the two Kyowa codes are mapped to the keypad as follows: it can.

【0091】

C Major C Dominant 7th C DECD EF GAFG AB CD Bb C DE FGEFG For each time, the bass chord changes in the background music and the mapping changes so that there are no keys forming non-consonant or "wrong" notes. I have to let you.

【0092】

For example, assuming the first four chords of background music, they are F major, E minor 7th (Em7), A7th, and D minor 7th (Dm7). Furthermore, the consonant scale selected for each base code is assumed to be the scale of the bass code itself. The keys for these four codes can be mapped as follows:

【0093】<img file="JP2004536498A_D0002.tif" />In this method, the map generates a sequence of notes F / E / A / D each time the operator presses only the "1" key.

【0094】

Unfortunately, skilled musicians do not play on scales in this format. Instead, the musician chooses the most appropriate set of notes as the chords in the background music change in response to chord / scale relationships, rhythmic notes, and pitch substitutions. Simple mapping of keys according to the chord structure described above is not sufficient to give an unskilled operator the ability to match the practice of experience.

【0095】

The practice of experience is greatly enhanced by controlling the logical flow from map to map. Two different mapping relationships can be used in combination to achieve this. The first mapping relationship is the code / scale mapping relationship described above. This mapping is pre-established by musicians. A particular relationship can be selected by the musician according to which one the musician finds most suitable for the particular song. Instead, the musician can establish several relationships depending on the different musical styles: jazz, blues, folk, pop, and the operator is given the opportunity to choose the desired style. This establishes the first code / scale mapping relationship.

【0096】

The second mapping relationship controls the change in flow between different maps. This second flow relationship can also be established by the musician or is automatically given by the server 30 or device 10.

【0097】

Using the example above, a code / scale map has been established for the four codes described above. Changes in flow between these maps are controlled by adapting to the notes of each scale assigned to the "1" key. The note assigned to the "1" key for a particular map is a note with a chromatic scale, which is assigned the "1" key for the preceding map. This gives the following sequence of maps.

【0098】<img file="JP2004536498A_D0003.tif" />In this method, the map generates a sequence of notes F / F # / F # / F each time the operator presses only the "1" key. By pressing the "4" key, the operator generates a sequence of notes Bb / B / B / B.

【0099】

By controlling the flow of mapping changes, the operator can better predict notes that are generated by pressing a particular key and that naturally form the corresponding music as the pressed key progresses. ..

【0100】

Alternative relationships can be used to control changes in flow between maps. For example, the third, fifth, or seventh note of a chord can be assigned the "1" key for a particular map, and similar notes can be assigned the "1" key for subsequent maps within the chromatic scale. If the operator agrees that the mapping flow relationship is consistent with the subsequent map on the 5 key, the map for the 4 codes shown above will make the "1" key note C, C #, C # and C. It becomes.

[Simple explanation of drawings]

【0101】

FIG. 1 is a schematic block diagram of a system in which a handheld telephone operator can interact with servers connected to various networks and communication paths.

FIG. 2 is a schematic block diagram of key components in one virtual practice of a handheld telephone.

FIG. 3 shows a process that allows an individual to form and distribute a message with auditory content using a device such as a cellular phone or PDA.

FIG. 4 shows a process that allows an individual to form and distribute a message with auditory content using mismatched data and voice communication pathways.