Mediaplayer with 3D audio rendering based on individualised HRTF measured in real time using earpiece microphones.

Technical Field

0001The present invention relates to a realistic three-dimensional audio service and, more particularly, to a three-dimensional (3D) audio signal process apparatus and method for providing the most realistic three-dimensional audio signals by generating three-dimensional audio signals based on an Head Related Transfer Function (HRTF) modeled according to physical characteristics of an individual user and a highly realistic multimedia playing system using the same.

0002This work was supported by the IT R&D program for MIC/IITA [2007-S-004-01, "Development of Glassless Single-User 3D Broadcasting Technologies"].

Background Art

0003Recently, the number of people watching multimedia data through diverse multimedia playing systems such as a MP3 player, a Portable Multimedia Player (PMP), a cell phone, and a Digital Multimedia Broadcasting (DMB) player is increasing.

0004<figref idref="f0001">Fig. 1</figref> is a diagram of a typical multimedia playing system.

0005Referring to <figref idref="f0001">Fig. 1</figref>, a multimedia playing system 10 includes a demultiplexer 11, a video decoder 12, an audio decoder 13, and a three-dimensional (3D) audio signal processor 14.

0006When the demultiplexer 11 divides the multimedia data into video data and audio data, the video decoder 12 decodes the divided video data to restore thereinto original video signals. The audio decoder 13 decodes the divided audio data to restore thereinto original audio signals.

0007The three-dimensional audio signal processor 14 gives three-dimensional stereophonic sound effect to the audio signals restored by the audio decoder 13 to generate three-dimensional audio signals. Herein, the three-dimensional stereophonic sound forms sound source at a certain place in a virtual space through a headphone or a speaker. Thus, the user feels senses of direction, distance, and space as if the sound actually comes from a location where the virtual sound source is.

0008When users uses the multimedia playing system illustrated in <figref idref="f0001">Fig. 1</figref>, particularly a portable playing system (portable device), they usually listen to audio signals through a headphone or an earphone. At this time, an Inside-the-Head Localization (IHL) phenomenon occurs. That is, sound image is localized in head of the listener.

0009The IHL phenomenon can be a cause for reduced sense of space and reality. Thus, various methods have been developed for listeners to feel the three-dimensional effect, for instance, Sound Retrieval System (SRS), Digital Natural Sound Engine (DNSe), and Baseband Booster Effect (BBE). The SRS recovers the reality of the sound damaged in typical stereo. The DNSe is an automatic adjustment method that amplifies low sound to make the listener feel as if he is at the concert hall just with a small MP3 player.

0010Researches on development of the three-dimensional audio technology have been conducted. It is reported that audio signal processing based on an individualized HRTF is the best way for playing the realistic audio.

0011In the audio signal processor using a typical HRTF, a microphone is put inside the ears of a human being or a dummy, for instance, a torso. Then, the audio signals are recorded to acquire impulse response. When impulse signals are applied to the audio signals, the user can feel the location of the audio signals in the three-dimensional space.

0012The HRTF indicates a transfer function generated between the sound source and the ears of the human being. The HRTF is different according to not only directions and height of the sound source but also physical characteristics such as shape and size of the head and the ears. That is, each listener has their own HRTF.

0013However, up to now, the HRTF measured by various kinds of models, for instance, a dummy head, which is non-individualized HRTF is used for the three-dimensional audio signal processing. Thus, it is difficult to provide the same three-dimensional sound effect to listeners each having different physical characteristics.

0014Furthermore, the typical multimedia playing system does not employ a module applying different HRTF according to the physical characteristics of each user, the three-dimensional audio signals optimized for the individual cannot be provided.

Disclosure of Invention

Technical Problem

0015When a typical multimedia playing system plays three-dimensional audio, physical characteristics, for instance, shape and size of head, and shape of ears, of a user is not considered. Thus, the user may feel insufficient reality of three-dimensional audio signals. Thus, the object of the present invention is to solve this problem.

0016An embodiment of the present invention is directed to providing an apparatus.

0017This invention provides to a three-dimensional audio signal process apparatus for providing the most realistic three-dimensional audio signals by generating three-dimensional audio signals using an HRTF modeled by physical characteristics of individual user and a high realistic multimedia playing system using it.

0018The objects of the present invention are not limited to the above-mentioned ones. Other objects and advantages of the present invention can be understood by the following description, and become apparent with reference to the embodiments of the present invention. Also, it is obvious to those skilled in the art of the present invention that the objects and advantages of the present invention can be realized by the means as claimed and combinations thereof.

Technical Solution

0019The present invention is defined by the independent claims 1 and 4. The dependent claims define embodiments thereof.

Advantageous Effects

0020In the present invention, three dimensional audio signals are generated according to Head Related Transfer Function (HRTF) based on individual physical characteristics of user. Thus, the most realistic three-dimensional audio signals can be provided to each user.

0021That is, in this invention, a module receiving the individualized HRTF is added to the multimedia player. When the users play audio data through their own multimedia player, each user can play the high realistic three-dimensional audio optimized for themselves.

Brief Description of the Drawings

0022<ul id="ul0001" list-style="none" compact="compact"><li><figref idref="f0001">Fig. 1</figref> is a diagram of a typical multimedia playing system.</li><li><figref idref="f0002">Fig. 2</figref> is a diagram showing a high realistic multimedia playing system using an individualized Head Related Transfer Function (HRTF) in accordance with an embodiment of the present invention.</li><li><figref idref="f0003">Fig. 3</figref> is a flowchart showing a method for processing signals in the highly realistic multimedia playing system illustrated in <figref idref="f0002">Fig. 2</figref>.</li></ul>

Best Mode for Carrying Out the Invention

0023The advantages, features and aspects of the invention will become apparent from the following description of the embodiments with reference to the accompanying drawings, which is set forth hereinafter. Therefore, those skilled in the field of this art of the present invention can embody the technological concept and scope of the invention easily. In addition, if it is considered that detailed description on a related art may obscure the points of the present invention, the detailed description will not be provided herein. The preferred embodiments of the present invention will be described in detail hereinafter with reference to the attached drawings.

0024Three-dimensional sound technology is for understanding a mechanism about detecting a location of sound source using only sense of hearing and technologically applying the mechanism. Generally, the three-dimensional location can be represented by three variables. To estimate the three variables, three independent variables should be measured.

0025Human or animal, particularly an owl, can accurately estimate not only directions (front, back, left, right, up, and down) of the sound source but also distance from the sound source through two signals measured by ears. Because spectrum of sound source reaching both ears changes according to directions of the sound source because of diffusion or rotation of the sound wave caused by a head, a trunk, and external ears. The changed spectrum of the sound wave is transferred to internal ears. Brain can estimate the accurate location of the sound source.

0026If the mechanism of detecting the location of the sound source can be accurately understood and reproduced, listeners can listen virtual sound source (embodiment of the virtual sound field) can estimate the location of real sound source by reversely applying the mechanism, i.e., through signals measured by two or more microphones (estimation of the sound source location). This technology can add hearing virtual reality to a typical visual-focused virtual system to increase an immersion of the listener. Also, 5.1 channel surround sound system effect can be achieved by two TV front speakers. Furthermore, a robot can estimate and deal with the location of an unseen person or a noise source. As a result, human feels intimateness about the robot.

0027To accurately figure out the mechanism of detecting the location of the sound source, the Head Related Transfer Function (HRTF) should be understood. The HRTF is a transfer function between sound waves diffused from the sound source at a head-related certain location and sound waves reached both eardrums. The HRTF is different according to direction and height of the sound source. The HRTF is also changed according to shapes of head and external ears so that individuals have their own HRTF.

0028When the HRTF optimized for the physical characteristics of the individuals (which is individualized HRTF) is multiplied by audio signals (original sound) in a convolution form and played, the listener can hear the highly realistic three-dimensional audio signals.

0029Thus, this invention generates the three-dimensional audio signals by using the HRTF corresponding to the physical characteristics of individuals to provide the high realistic three-dimensional audio signals optimized for the individuals.

0030<figref idref="f0002">Fig. 2</figref> is a diagram showing a high realistic multimedia playing system using an individualized HRTF in accordance with an embodiment of the present invention.

0031Referring to <figref idref="f0002">Fig. 2</figref>, a high realistic multimedia playing system 20 includes a demultiplexer 21, a video decoder 22, an audio decoder 23, and a three-dimensional audio generator 24. The audio decoder 23 and the three-dimensional audio generator 24 are called a three-dimensional audio signal processor 25.

0032When the multiplexer 21 divides data into video data and audio data, the video decoder 22 restores the divided video data into original video data. The audio decoder 23 decodes the divided audio data and restores the divided audio data into original audio signal (stereo signals not added with three-dimensional effect).

0033The three-dimensional audio generator 24 generates three-dimensional audio signals corresponding to the audio signals restored in the audio decoder 23 by using the HRTF optimized for an individual. Herein, the three-dimensional audio generator 24 includes an individual HRTF providing unit 241 and a three-dimensional audio signal processing unit 242.

0034The individual HRTF providing unit 241 receives and stores an HRTF modeled by individual physical characteristics, for instance, size/shape of head, shape of ears, of users to provide it to the three dimensional audio signal processing unit 242.

0035Hereinafter, a method for acquiring the HRTF corresponding to physical characteristics of the user, that is, the individualized HRTF, will be described in detail.

0036First, the individualized HRTF can be acquired by measuring the body of the user. That is, the HRTF can be estimated based on the physical characteristics to acquire the individualized HRTF.

0037Second, the individualized HRTF can be acquired by transforming the HRTF measured through the human model.

0038Third, the individualized HRTF can be acquired by using an ear microphone. The ear phone is equipped with a small microphone to measure the HRTF in real time to apply to the three-dimensional audio signal processing.

0039The individual HRTF providing unit 241 stores different types of HRTF samples. The HRTF may be inputted by the user later and then stored. When the user selects a certain HRTF, the selected HRTF may be provided to the three-dimensional audio signal processing unit 242.

0040The three-dimensional audio signal processing unit 242 generates the three-dimensional audio signals (three-dimensional audio signal optimized for the individual user) corresponding to the audio signals restored in the audio decoder 23 by using the HRTF provided by the individual HRTF providing unit 241. For instance, the three-dimensional audio signal processing unit 242 convolutes the audio signals restored in the audio decoder 23 to generate the three dimensional audio signals.

0041To sum up, in this invention, the multimedia playing system has the three-dimensional audio generator 24, i.e., an individualized three-dimensional audio signal processor, for increasing the three-dimensional effect by performing the audio signal process using the individualized HRTF. Thus, the user can listen more realistic three-dimensional audio. Also, in this invention, since the user can input the HRTF optimized for his own physical characteristics into the multimedia playing system, same product (multimedia playing system) can process the signals corresponding to the physical characteristics of the user.

0042<figref idref="f0003">Fig. 3</figref> is a flowchart describing a method for processing signals in the high realistic multimedia playing system illustrated in <figref idref="f0002">Fig. 2</figref>, particularly a method for processing the three-dimensional audio signals by using the individualized HRTF.

0043The high realistic multimedia playing system in this invention demultiplexes the multimedia data into the video data and the audio data S300.

0044The high realistic multimedia playing system decodes the video data to restore the original video data S302.

0045The highly realistic multimedia playing system decodes the audio data to restore the original audio data S304. Then, the three-dimensional audio signals corresponding to the restored audio signals is generated using the HRTF optimized for the physical characteristics of the user, i.e., the individualized HRTF S306.

0046While the present invention has been described with respect to certain preferred embodiments, it will be apparent to those skilled in the art that various changes and modifications may be made without departing from the scope of the invention as defined in the following claims.