Data processing device

<b>TECHNICAL FIELD</b>

0001The present invention relates to the technique of converting high definition video (HD video) into standard definition video (SD video).

<b>BACKGROUND ART</b>

0002Recently, DVDs have become very popular storage media. Among other things, a huge number of DVD-Video discs (which will be simply referred to herein as "DVDs") are already on the market as packaged software in which movies or any other type of content is stored.

0003FIG. <b>1</b> shows the data structure of a DVD. As shown in the lower portion of FIG. <b>1</b>, logical addresses are assigned to the area between the innermost lead-in area and the outermost lead-out area of the DVD, on which logic data is written based on the logical addresses. In the data area that can be accessed based on the logical addresses (i.e., logical address space), the volume information of a file system is recorded after its header and then application data such as video or audio data is written there.

0004The file system refers to ISO 9660 or UDF (universal disc format) and is designed as a system that manages the data on a disc based on some unit, which is called either "directory" or "file". Even in a PC (personal computer) people use daily, the data stored on the hard disk is normally processed by a file system called "FAT" or "NTFS" so as to be represented on the computer with a data structure such as directory or file. In this manner, the usability of such data can be increased.

0005A DVD uses both of these file systems UDF and ISO 9660 (which are sometimes called a "UDF bridge" collectively) such that the data can be read out by a file system driver that complies with either UDF or ISO 9660. Naturally, data can be physically read from, written on, and erased from, a rewritable DVD medium such as a DVD-RAM, a DVD-R or a DVD-RW by way of such a file system.

0006The data stored on a DVD can be viewed as a directory or files as shown in the upper left tree of FIG. 1 by way of the UDF bridge. More specifically, a directory called "VIDEO_TS" is put just below a root directory "ROOT" so as to store the application data of the DVD. The application data is stored as a plurality of files, which includes: <ul id="ul0001" list-style="none"><li>VIDEO_TS.IFO: disc playback control information file;</li><li>VTS_01_0.IFO: video title set #1 playback control information file; and</li><li>VTS_01_0.VOB: video title set #1 stream file as main files.</li></ul>

0007The file system defines two types of extensions. Specifically, the extension "IFO" indicates a file on which playback control information is stored, while the extension "VOB" indicates a file on which an MPEG stream is stored as AV data. Examples of the playback control information include information needed to achieve interactivity, which allows the user to change the playback dynamically by his or her manipulation, as adopted in a DVD, information such as meta-data accompanying the title, and information accompanying the stream. Also, in a DVD, the playback control information may be generally called "navigation information".

0008The playback control information files include the VIDEO_TS.IFO file that manages the overall disc and the VTS_01_0.IFO file that contains playback control information for each of a plurality of video tiles in a set. It should be noted that multiple titles (e.g., different movies or different versions of the same movie) can be stored on a single DVD. In this case, "01" included in the file name body represents the number of the video title set. Thus, a video title set #2 is identified by "VTS_02_0.IFO", for example.

0009On the upper right corner of FIG. 1, shown is a DVD navigation space on the DVD application layer, which is a logical structure space where the playback control information is expanded. Specifically, the information included in the VIDEO_TS.IFO file is expanded as video manager information (VMGI) and the playback control information included in either the VTS_01_0.IFO file or any other video title set is expanded as video title set information (VTSI) in the DVD navigation space.

0010In the VTSI, program chain information (PGCI), which is information about a playback sequence called "program chain (PGC)", is described. The PGCI consists of a group of cells and a sort of programming information called "command". The cell itself is a collection of some or all of the video objects (VOBs) representing an MPEG stream. Thus, playing back a cell means playing back a section specified by the cell consisting of the VOBs.

0011The command is a concept comparable to Java™ script to be executed on a browser, and is processed by a DVD's virtual machine. The Java™ script can perform not only logical operations but also window and browser controls (e.g., newly opening the window of another browser). However, the DVD command can perform nothing but AV title playback controls (e.g., specifying a chapter to be played back) other than the logical operations.

0012Each cell includes, as its internal information, the start and end addresses of the VOBs stored on the disc (i.e., their logical addresses recorded on the disc). The player reads and writes data in accordance with the VOB start and end address information described by the cell.

0013FIG. <b>2</b> shows various types of information included in the navigation information embedded in a stream. The interactivity, which is one of the key features of the DVD, is not achieved just by the navigation information stored in VIDEO_TS.IFO, VTS_01_0.IFO and other files described above. In addition, some pieces of important information are multiplexed with video data and audio data in a VOB using a dedicated carrier called "navigation pack" (which will be referred to herein as either "navi pack" or "NV_PCK").

0014A menu will be described as a simple example of interactivity. A menu picture includes a number of buttons, for each of which a type of processing to be done when the button is selected and pressed down is defined. Also, one of those buttons is selected on the menu (i.e., highlighted with a semi-transparent color overlaid thereon to indicate the user that the button is now being selected), and the user can scroll upward, downward, rightward or leftward using the arrow keys of the remote controller to change the buttons being selected. That is to say, when the user shifts the highlight, using the arrow keys of the remote controller, to his or her desired button to be selected and pressed down and makes a decision with the enter key pressed down, a program of its associated command is executed. For example, a title or chapter associated with the button selected is played back in response to the command.

0015In the NV_PCK, highlight color information, individual button information and other information are included. The highlight color information describes color palette information, which specifies the semi-transparent color of the highlight to be overlaid. The button information describes rectangular area information (i.e., information about the location of each button), information about to what button the highlight should be shifted from the current button (i.e., information designating the destination buttons in response to the user's up, down, right and left key manipulations) and button command information (i.e., information about the command to be executed when the button is pressed down).

0016The highlight on the menu is produced as a picture to be overlaid on the menu picture. More specifically, an area corresponding to the rectangular area specified by the button information is colored in accordance with the color palette information, and then superposed on the rectangular area specified by the button information of the menu, thereby producing the highlight on the menu. This overlaid picture is synthesized with the background picture and the resultant synthetic picture is presented on the screen.

0017A portion of the navigation data is embedded as NV_PCK in a stream for the following purposes. Firstly, synchronization can be established more easily even if the menu information is dynamically updated synchronously with the stream (e.g., in a situation where the menu should be displayed only during 5 to 10 minutes after a movie started to be played back) or even if the given application often causes synchronization timing errors. Secondly, the machine will come in handier to the user (e.g., AV data can be decoded and played back smoothly even during a special playback operation such as a fast-find or rewind playback operation) if special playback support information is stored in the NV_PCK.

0018FIGS. <b>3A, 3B</b> and <b>3C</b> show how to generate a VOG and FIG. <b>3D</b> shows a configuration for a player for playing back the VOG generated. The moving pictures, sound and subtitles shown in FIG. <b>3A</b> are packetized and packed in compliance with MPEG system standard ISO/IEC 13818-1 as shown in FIG. <b>3B</b>. In this case, the video is high definition video (which will be abbreviated herein as "HD video") and packetized and packed while maintaining its video quality. Examples of HD videos include various types of high-definition quality videos such as motion pictures, still pictures, characters and graphics. These types of data are multiplexed together, thereby generating a single MPEG program stream as shown in FIG. <b>3C</b>. In this case, NV_PCK including button commands to realize the interactivity is also included in the mix.

0019When an MPEG data stream is broken down into the individual types, the data representing the motion pictures, sound or subtitles are sequentially arranged as a bit stream in the order of decoding. However, when the motion pictures, sound and subtitles are multiplexed together by an MPEG system, not all of the data included in the bit stream are arranged in the order of playback (i.e., in the order of decoding) among the respective types. For example, even if a motion picture data pack and a subtitle data pack are arranged back to back, that motion picture data and that subtitle data pack are not always decoded or played back at the same timing.

0020These data should be multiplexed in this manner because there is a restriction that an MPEG stream needs to be encoded so as to be decoded by a predetermined decoder model (which is a so-called "system target decoder (STD)"). That is to say, a number of decoder buffers associated with respective elementary streams are defined for a decoder model. One of those buffers associated with an elementary stream has a different capacity from that of another one of the buffers associated with another elementary stream. More specifically, a buffer associated with a stream of motion pictures has a capacity of 232 KB, a buffer associated with a stream of sound has a capacity of 4 KB and a buffer associated with a stream of subtitles has a capacity of 52 KB. Also, the video and audio data are temporarily stored until it is time to decode them. The timing of decoding changes with the specific buffer capacity. Accordingly, even if two data are multiplexed together back to back, those data may still be decoded at different times.

0021An MPEG program stream to be played back is input to the player shown in FIG. <b>3D.</b> A demultiplexer <b>3100</b> receives the MPEG program stream, separates a pack including motion picture data, subtitle data and audio data, and supplies it to a buffer/decoder <b>3108</b>. The buffer/decoder <b>3108</b> buffers the packed data and decodes the motion picture data, subtitle data and audio data separately, thereby generating respective streams. More specifically, the buffer <b>3108</b> includes a video buffer, a subtitle buffer and an audio buffer and stores the motion picture data, subtitle data and audio data in the video, subtitle and audio buffers, respectively. Thereafter, the buffer/decoder <b>3108</b> makes motion picture data, subtitle data and audio data based on the packed data and outputs them. No decoders are particularly shown in FIG. <b>3D.</b> The buffer/decoder <b>3108</b> outputs the motion picture data, subtitle data and audio data of HD video. After having been decoded, the audio data is output to a loudspeaker, for example, and reproduced as sound.

0022The HD video data is processed as follows. The HD video data (including motion picture data, still picture data and character/graphics data) is output from the buffer/decoder <b>3108</b> to respective processing paths. Specifically, the motion picture data is converted by a video converting section <b>3101</b> into NTSC- or PAL-compliant interlaced video (i.e., SD video) and then has its resolution and aspect ratio modified by a scaling section <b>3102</b>. If the motion picture has been MPEG encoded, then noise unique to the MPEG compression (such as block noise and mosquito noise) is removed effectively by a first image quality correcting section <b>3103</b>. As a result, the image quality is improved. On the other hand, the still picture data is subjected to image quality correction unique to a still picture encoding method and down-converted by a second image quality correcting section <b>3104</b>. And the still and motion pictures are output alternately. The character/graphics data is subjected to a down conversion suitable for characters and graphics by a third image quality correcting section <b>3105</b>.

0023On the other hand, the subtitle data is output to a fourth image quality correcting section <b>3107</b>. Specifically, the subtitle data is subjected to necessary color conversion processing by way of a color conversion table CLUT <b>3106</b> and then down-converted into SD video by the fourth image quality correcting section <b>3107</b>. Thereafter, the SD video subtitle data and the SD video motion picture data are synthesized together, and the resultant synthetic video is output to a TV that can present SD video.

0024After the video data has been output from the buffer/decoder <b>3108</b>, the resolution, aspect ratio, frame rate and other parameters are converted from HD video grades into SD video grades. This is because if the motion picture, still picture and subtitles can be processed individually through their best filters, then the deterioration in image quality can be minimized during the down conversion. Processing of this type is disclosed in Japanese Laid-Open Publication No. 2001-292341, for example.

0025It is expected that there will be more and more occasions where HD video will be recorded without decreasing its frame rate. For example, as the capacities of storage media and write rates go on increasing, film materials would be recorded at the originally intended 24 frames per second and 720 P (or 1080i) more and more often. Meanwhile, it is also imaginable that TVs that can display nothing but SD video will be continuously used for a rather long time. Accordingly, there will be very high demand for a player that is intended to play HD video storage medium but can also convert HD video into SD video. However, the HD video cannot be written on any DVD on the market without decreasing its frame rate. For that reason, no DVD players have ever been designed so as to perform such a conversion as well.

0026Also, if multiple separate streams, including motion picture data, subtitle data and other data, respectively, are multiplexed together as in an MPEG stream, then the motion pictures, subtitles and so on need to be played back just as intended by the content provider during the authoring process when HD video is converted into SD video.

0027An object of the present invention is to convert the HD grade video, subtitle and other data into SD grade video, subtitle and other data while maintaining the originally intended playback timings.

<b>DISCLOSURE OF INVENTION</b>

0028A data processor according to the present invention includes: a receiving section that receives a data stream including data representing first primary video to be presented by switching a plurality of pictures one after another at a first vertical scanning frequency and data representing first auxiliary video to be presented synchronously with the first primary video; and a converting section for converting the data representing the first primary video and the first auxiliary video into data representing synthetic video to be presented by switching the pictures at a second vertical scanning frequency, which is different from the first vertical scanning frequency. The data stream includes timing information defining respective times to present the first primary video and the first auxiliary video. The converting section associates second auxiliary video, having the same contents as the first auxiliary video on a picture of the first primary video, with a picture of second primary video, having the same contents as the counterpart of the first primary video, thereby generating synthetic video composed of the second primary video and the second auxiliary video.

0029The converting section may include: a video converting section for converting the data representing the first primary video into data representing the second primary video by changing the vertical scanning frequencies; a subtitle converting section for converting the data representing the first auxiliary video into data representing the second auxiliary video, identifying a picture of the first primary video to be presented synchronously with letters of the first auxiliary video, and associating letters of the second auxiliary video, corresponding to the letters of the first auxiliary video, with a picture of the second primary video corresponding to the identified picture of the first primary video; and a synthesizing section for generating data representing the synthetic video by synthesizing together the second primary video data and the second auxiliary video data that have been associated with each other.

0030Alternatively, the converting section may include: a synthesizing section for generating superposed video, in which the first primary video and the first auxiliary video are synchronized with each other in accordance with the timing information and superposed one upon the other; and a video converting section for converting data representing the superposed video into data representing the synthetic video.

0031As another alternative, the converting section may convert the data representing the first primary video and the first auxiliary video and having a predetermined resolution into data representing the synthetic video that has a resolution different from the predetermined resolution.

0032In that case, the converting section may convert the first primary video data and the first auxiliary video data of a film material into NTSC- or PAL-compliant synthetic video data.

0033More specifically, the converting section may convert one frame of the first primary video and the first auxiliary video of the film material into at least two fields of the PAL-compliant synthetic video with the same field inserted at regular intervals a number of times during the conversion into the fields.

0034The data stream may include conversion permissibility information indicating whether or not the converting section may do conversion, and the converting section may do the conversion when the conversion permissibility information permits it.

0035The data stream may include conversion designating information designating the type of conversion to do, and the converting section may do the type of conversion that has been designated by the conversion designating information.

0036The data process may further include an input section, to which conversion designating information, designating the type of conversion to do, is input, and the converting section may do the type of conversion that has been designated by the conversion designating information.

0037In that case, the converting section may generate PAL-compliant synthetic video by performing either a conversion of presenting each picture a single time or a conversion of presenting a particular picture a number of times in accordance with the conversion designating information.

0038A data processing method according to the present invention includes the steps of: receiving a data stream that includes data representing first primary video to be presented by switching a plurality of pictures one after another at a first vertical scanning frequency and data representing first auxiliary video to be presented synchronously with the first primary video; and converting the data representing the first primary video and the first auxiliary video into data representing synthetic video to be presented by switching the pictures at a second vertical scanning frequency, which is different from the first vertical scanning frequency. The data stream includes timing information defining respective times to present the first primary video and the first auxiliary video. The converting step includes the step of associating second auxiliary video, having the same contents as the first auxiliary video on a picture of the first primary video, with a picture of second primary video, having the same contents as the counterpart of the first primary video, thereby generating synthetic video composed of the second primary video and the second auxiliary video.

0039The converting step may include: a video converting step for converting the data representing the first primary video into data representing the second primary video by changing the vertical scanning frequencies; a subtitle converting step for converting the data representing the first auxiliary video into data representing the second auxiliary video, identifying a picture of the first primary video to be presented synchronously with letters of the first auxiliary video, and associating letters of the second auxiliary video, corresponding to the letters of the first auxiliary video, with a picture of the second primary video corresponding to the identified picture of the first primary video; and a synthesizing step for generating data representing the synthetic video by synthesizing together the second primary video data and the second auxiliary video data that have been associated with each other.

0040Alternatively, the converting step may include: a synthesizing step for generating superposed video, in which the first primary video and the first auxiliary video are synchronized with each other in accordance with the timing information and superposed one upon the other; and a video converting step for converting data representing the superposed video into data representing the synthetic video.

0041As another alternative, the converting step may include the step of converting the data representing the first primary video and the first auxiliary video and having a predetermined resolution into data representing the synthetic video that has a resolution different from the predetermined resolution.

0042In that case, the converting step may include the step of converting the first primary video data and the first auxiliary video data of a film material into NTSC- or PAL-compliant synthetic video data.

0043More specifically, the converting step may include the step of converting one frame of the first primary video and the first auxiliary video of the film material into at least two fields of the PAL-compliant synthetic video with the same field inserted at regular intervals a number of times during the conversion into the fields.

0044The data stream may include conversion permissibility information indicating whether or not the converting section may do conversion, and the converting step may include the step of doing the conversion when the conversion permissibility information permits it.

0045The data stream may include conversion designating information designating the type of conversion to do, and the converting step may include the step of doing the type of conversion that has been designated by the conversion designating information.

0046The data processing method may further include the step of inputting conversion designating information designating the type of conversion to do, and the converting step may include the step of doing the type of conversion that has been designated by the conversion designating information.

0047In that case, the converting step may include the step of generating PAL-compliant synthetic video by performing either a conversion of presenting each picture a single time or a conversion of presenting a particular picture a number of times in accordance with the conversion designating information.

<b>BRIEF DESCRIPTION OF DRAWINGS</b>

0048<ul id="ul0002" list-style="none" compact="compact"><li>FIG. <b>1</b> shows the data structure of a DVD.</li><li>FIG. <b>2</b> shows various types of information included in the navigation information embedded in a stream.</li><li>FIGS. <b>3A, 3B</b> and <b>3C</b> show how to generate a VOG and FIG. <b>3D</b> shows a configuration for a player for playing back the VOG generated.</li><li>FIG. <b>4</b> shows a BD <b>104</b> as a disk storage medium and the structure of the data <b>101, 102</b> and <b>103</b> stored thereon.</li><li>FIG. <b>5</b> shows the directory/file structure of the logical data stored on the BD.</li><li>FIG. <b>6</b> shows a schematic functional configuration for a player for reading the VD data from the BD <b>201</b> and playing back the video.</li><li>FIG. <b>7</b> shows a detailed functional block arrangement for the player.</li><li>FIG. <b>8</b> shows the concept of an application space on the BD.</li><li>FIG. <b>9</b> shows a relationship between video pictures and the data structure of an MPEG stream.</li><li>FIG. <b>10</b> shows the data structure of a pack.</li><li>FIG. <b>11</b> shows VOB data files, their associated files, and a functional block arrangement for a player for reading these files.</li><li>FIG. <b>12</b> shows a VOB data continuous supply model using a track buffer.</li><li>FIG. <b>13</b> shows the internal structure of the VOB management information file <b>YYY.VOBI</b>.</li><li>FIG. <b>14</b> shows correspondence between the playback duration and the data size of a VOBU.</li><li>FIG. <b>15</b> shows the concept of processing for obtaining an address based on the playback duration of a VOBU.</li><li>FIG. <b>16</b> shows the data structure of the play list information.</li><li>FIG. <b>17</b> shows the data structure of an event handler table <b>XXX.PROG.</b></li><li>FIG. <b>18</b> shows the data structure of information about an overall BD.</li><li>FIG. <b>19</b> shows the program table <b>BD.PROG</b> of a global event handler.</li><li>FIG. <b>20</b> shows the concept of a time event.</li><li>FIG. <b>21</b> shows the concept of a user event of menu selection.</li><li>FIG. <b>22</b> shows the concept of a global event.</li><li>FIG. <b>23</b> shows the arrangement of functional blocks associated with the program processor.</li><li>FIG. <b>24</b> is a list of the system parameters <b>SPRM</b>.</li><li>FIG. <b>25</b> shows an exemplary program for a menu that has two selection buttons.</li><li>FIG. <b>26</b> shows an exemplary program for an event handler to process the user event of "menu selection".</li><li>FIG. <b>27</b> shows the flow of the basic processing to be done before AV data is played back.</li><li>FIG. <b>28</b> shows the flow of processing to be done since the playback of a play list is started and until the playback of a VOB is started.</li><li>FIG. <b>29</b> shows the flow of event handling to be done after the AV data started being read.</li><li>FIG. <b>30</b> shows the flow of subtitle processing.</li><li>FIG. <b>31</b> shows a configuration for a player <b>100</b> for performing down conversion processing according to this preferred embodiment.</li><li>FIG. <b>32</b> shows the procedure of the down conversion processing.</li><li>FIG. <b>33</b> shows correlations between pictures of respective grades when pictures of HD video (i.e., film material video) are converted into pictures of SD video (i.e., video complying with the NTSC or PAL standard).</li><li>FIG. <b>34</b> shows correlations between the pictures in a situation where pictures of HD video (i.e., film material video) are converted into pictures of SD video.</li><li>FIG. <b>35</b> shows the procedure of second exemplary down conversion processing.</li><li>FIG. <b>36</b> shows the data structure of conversion designating information that describes a recommended conversion method.</li><li>FIG. <b>37</b> shows a procedure in which SD video is played back with a down conversion method determined in accordance with the conversion designating information.</li></ul>

<b>BEST MODE FOR CARRYING OUT THE INVENTION</b>

0049Hereinafter, preferred embodiments of the present invention will be described. In the preferred embodiments of the present invention, HD video is written on a high-density storage medium such as Blu-ray Disc (which will be referred to herein as a "BD"), of which the storage capacity is several times as high as that of a DVD. Thus, in the following description of preferred embodiments, with the high-density storage medium supposed to be a BD, the configuration and operation of a player that can convert the HD video on a BD into SD video will be described.

0050First, a number of technical terms that will be used to describe the preferred embodiments of the present invention will be defined. <ul id="ul0003" list-style="none" compact="compact"><li>● "Video" is herein supposed to be a group of pictures presented by switching one of those pictures after another at a predetermined vertical scanning frequency. The "video" includes motion pictures, still pictures, characters such as subtitles, and graphics. The motion and still pictures will be sometimes referred to herein as "primary video" and the subtitles and so on to be synchronized with, and superposed on, the primary video will be sometimes referred to herein as "auxiliary video";</li><li>● "Picture" consists of two fields including odd-numbered lines and even-numbered lines, respectively, in an interlacing method and consists of just one frame in a progressive method. "Picture" is a concept that may mean either a frame or a field;</li><li>● "SD video" refers herein to standard grade video (i.e., 480i video according to the NTSC standard and 576i video according to the PAL standard); and</li><li>● "HD video"refers herein to high definition video (e.g., so-called "High Vision" video in Japan) other than the SD video.</li></ul>

0051The following Tables 1 and 2 show more specific examples of SD video motion pictures and HD video motion pictures. The 24p or 23.976p HD video in Table 2 is called "film material video" : <tables id="tabl0001" num="0001"><table frame="all"><title>Table 1:</title><tgroup cols="3" colsep="1" rowsep="1"><colspec colnum="1" colname="col1" colwidth="52.50mm" /><colspec colnum="2" colname="col2" colwidth="52.50mm" /><colspec colnum="3" colname="col3" colwidth="52.50mm" /><thead valign="top"><row><entry namest="col1" nameend="col3" align="center">Resolution (number of lines)+type of scanning</entry></row><row><entry namest="col1" nameend="col1" /><entry namest="col2" nameend="col2" align="center">NTSC</entry><entry namest="col3" nameend="col3" align="center">PAL</entry></row></thead><tbody valign="top"><row><entry namest="col1" nameend="col1" align="center">SD</entry><entry namest="col2" nameend="col2" align="center">480i</entry><entry namest="col3" nameend="col3" align="center">576i</entry></row><row><entry namest="col1" nameend="col1" morerows="2" align="center">HD</entry><entry namest="col2" nameend="col2" align="center">480p</entry><entry namest="col3" nameend="col3" align="center">576p</entry></row><row><entry namest="col2" nameend="col2" align="center">720p</entry><entry namest="col3" nameend="col3" align="center">720p</entry></row><row><entry namest="col2" nameend="col2" align="center">1080i</entry><entry namest="col3" nameend="col3" align="center">1080i</entry></row><row rowsep="1"><entry namest="col1" nameend="col3" align="justify">i: interlaced scanning,   p: progressive scanning</entry></row></tbody></tgroup></table></tables><tables id="tabl0002" num="0002"><table frame="all"><title>Table 2:</title><tgroup cols="3" colsep="1" rowsep="1"><colspec colnum="1" colname="col1" colwidth="52.50mm" /><colspec colnum="2" colname="col2" colwidth="52.50mm" /><colspec colnum="3" colname="col3" colwidth="52.50mm" /><thead valign="top"><row><entry namest="col1" nameend="col3" align="center">Frequency (Hz)+type of scanning</entry></row><row><entry namest="col1" nameend="col1" /><entry namest="col2" nameend="col2" align="center">NTSC</entry><entry namest="col3" nameend="col3" align="center">PAL</entry></row></thead><tbody valign="top"><row><entry namest="col1" nameend="col1" align="center">SD</entry><entry namest="col2" nameend="col2" align="center">59.94i (≒60i)</entry><entry namest="col3" nameend="col3" align="center">50i</entry></row><row><entry namest="col1" nameend="col1" morerows="2" align="center">HD</entry><entry namest="col2" nameend="col2" align="center">59.94p (≒60p)</entry><entry namest="col3" nameend="col3" align="center">50p</entry></row><row><entry namest="col2" nameend="col2" align="center">24p</entry><entry namest="col3" nameend="col3" align="center">24p</entry></row><row><entry namest="col2" nameend="col2" align="center">23.976p</entry><entry namest="col3" nameend="col3" align="center">23.976p</entry></row><row rowsep="1"><entry namest="col1" nameend="col3" align="justify">i: interlaced scanning, p: progressive scanning</entry></row></tbody></tgroup></table></tables>

0052It should be noted that the 480p and 576p videos shown in Table 1 could be classified as non-HD videos. However, these videos are also treated herein as HD videos. The reason is as follows. Specifically, these videos can be converted by an interlaced-progressive (IP) conversion technique into NTSC-compliant 480i and PAL-compliant 576i, respectively, and can be displayed on a standard grade TV (SDTV). Thus, in terms of the applicable method of processing, these videos can be said to be equivalent to 720p and 1080i, which are generally treated as HD videos.

0053The following description will be itemized to make this invention more easily understandable. Following is the list of specific items to be described: <ul id="ul0004" list-style="none" compact="compact"><li>(1) Logical data structure on BD</li><li>(2) Player configuration</li><li>(3) Application space on BD</li><li>(4) Specifics of VOB</li><li>(5) VOB interleaved writing</li><li>(6) Navigation data structure</li><li>(7) Play list data structure</li><li>(8) How to generate events</li><li>(9) Virtual player machine</li><li>(10) Typical programs</li><li>(11) Flow of processing to be done by virtual player and</li><li>(12) Player performing down conversion processing</li></ul> It should be noted that one of the key processing techniques of the present invention is applied to down-converting HD video into SD video.

(1) Logical data structure on BD

0054FIG. <b>4</b> shows a BD <b>104</b> as a disk storage medium and the structure of the data <b>101, 102</b> and <b>103</b> stored thereon. Specifically, the data stored on the BD <b>104</b> include AV data <b>103</b>, BD management information <b>102</b> such as AV data management information and AV playback sequence, and a BD playback program <b>101</b> contributing to interactivity. The following description of the preferred embodiment will be focused on the AV application of the BD (i.e., playing back the AV contents of a movie). Naturally, however, the BD may be used not only in this particular application but also in other applications as well.

0055FIG. <b>5</b> shows the directory/file structure of the logical data stored on the BD described above. Just like other optical discs (e.g., DVDs and CDs), the BD too has a spiral recording area extending from its innermost portion toward its outermost portion. Also, the BD, as well as the DVD, has a data area that can be accessed with a logical address specified (i.e., a logical address space). In addition, a burst cutting area (BCA), which is a special-purpose area that can be read only by the drive, is provided inside of the lead-in area thereof. This area is unreadable through an application and is sometimes used for the purpose of copyright protection, for example.

0056The logical address space is headed by file system information (or volume), which is then followed by application data such as video data. As already described for the background section, the file system is UDF, ISO 9660, or the like, and allows the user to read the stored logical data using the directory or file structure just like a normal PC.

0057In the directory/file structure on the BD of this preferred embodiment, a BDVIDEO directory is defined right under a root directory ROOT. This is a directory in which the data to be handled with the BD (such as the AV contents and management information, or the data <b>101, 102</b> and <b>103</b> shown in FIG. <b>4</b>) is stored.

0058Under the BDVIDEO directory, the following seven types of files (a) through (g) are recorded: <ul id="ul0005" list-style="none" compact="compact"><li>(a) BD.INFO (of which the file name is fixed), which is a type of "BD management information" and which is a file that contains information about the overall BD. A BD player must read this file first;</li><li>(b) BD.PROG (of which the file name is fixed), which is one of "BD playback programs" and which is a file that contains playback control information about the overall BD;</li><li>(c) XXX.PL (where XXX is changeable but the extension "PL" is fixed), which is a type of "BD management information" and which is a file that contains play list information as a scenario (playback sequence). One file is included for each play list;</li><li>(d) XXX.PROG (where XXX is changeable but the extension "PL" is fixed), which is one of "BD playback programs" and which is a file that contains playback control information for each of the play lists described above. The association between this file and the play list is identified by the file body name (because "XXX" should be the same when the file and play list are associated with each other);</li><li>(e) YYY.VOB (where YYY is changeable but the extension "VOB" is fixed), which is a piece of "AV data" and which is a file that contains a VOB (that is the same as VOB as described for the background section). One file is included for each VOB;</li><li>(f) YYY.VOBI (where YYY is changeable but the extension "VOBI" is fixed), which is a type of "BD management information" and which is a file that contains stream management information for VOB as AV data. The association between this file and the VOB is identified by the file body name (because "YYY" should be the same when the file and VOB are associated with each other); and</li><li>(g) ZZZ.PNG (where ZZZ is changeable but the extension "PNG" is fixed), which is another piece of "AV data" and which is a file that contains image data PNG (which is an image format standardized by W3C and which reads like "ping") to compose the subtitles and menu. One file is included for one PNG image.</li></ul>

(2) Player configuration

0059Next, the configuration of a player for playing back the BD will be described with reference to FIGS. <b>6</b> and <b>7</b>.

0060FIG. <b>6</b> shows a schematic functional configuration for a player for reading the VD data from the BD <b>201</b> and playing back the video. The player includes an optical pickup <b>202</b>, various memories including a program storage memory <b>203</b>, a management information storage memory <b>204</b> and an AV storage memory <b>205</b>, a program processing section <b>206</b> and a management information processing section <b>207</b>.

0061The optical pickup <b>202</b> reads the data from the BD <b>201.</b>

0062The program processing section <b>206</b> receives event information, including information about the play lists to be presented and information about the timing to execute the program, from the management information processing section <b>206</b>, thereby processing the program. Also, the play lists to be presented can be changed dynamically in a program. To do so, a play list presenting instruction may be issued to the management information processing section <b>207</b>. The program processing section <b>206</b> receives an event from the user (i.e., his or her request submitted by tapping remote controller keys) and executes a program corresponding to the user event if any.

0063In accordance with the instruction given by the program processing section <b>206</b>, the management information processing section <b>207</b> analyzes the specified play lists and the management information of the VOBs associated with the play lists and instructs a presentation processing section <b>208</b> to play back the target AV data. Also, the management information processing section <b>207</b> receives reference time information from the presentation processing section <b>208</b> and instructs the presentation processing section <b>208</b> to stop playing back the AV data in accordance with the time information. Furthermore, the management information processing section <b>207</b> generates an event that shows the program processing section <b>206</b> the timing to execute the program.

0064The presentation processing section <b>208</b> includes three decoders for motion pictures, sound, and subtitles or images (i.e., still pictures), respectively, and decodes and outputs the AV data in accordance with the instruction given by the management information processing section <b>207</b>. The motion picture data and the subtitles/images will be drawn on their dedicated planes, i.e., a video plane <b>210</b> and an image plane <b>209</b>, respectively, after having been decoded.

0065A synthesis processing section <b>211</b> performs a video synthesizing process and outputs the resultant synthetic video to a TV or any other display device.

0066The processing is done by the player in the following flow. First, the data on the BD <b>201</b> is read out by the optical pickup <b>202</b> and then transferred to its dedicated memory according to the type of the data. Specifically, the BD playback program (i.e., the contents of the BD.PROG or XXX.PROG file) is transferred to the program storage memory <b>203</b>. The BD management information (i.e., BD.INFO, XXX.PL or YYY.VOBI) is transferred to the management information storage memory <b>204</b>. And the AV data (i.e., YYY.VOB or ZZZ.PNG) is transferred to the AV storage memory <b>205</b>.

0067Then, the BD playback program stored in the program storage memory <b>203</b> is processed by the program processing section <b>206</b>. The BD management information stored in the management information storage memory <b>204</b> is processed by the management information processing section <b>207</b>. And the AV data stored in the AV storage memory <b>205</b> is processed by the presentation processing section <b>208</b>.

0068The presentation processing section <b>208</b> decodes and outputs the motion pictures, sound and subtitles/images (i.e., still pictures). As a result, the motion picture data and subtitles /images are drawn on the video plane <b>210</b> and image plane <b>209</b>, respectively. Then, the synthesis processing section <b>211</b> synthesizes the motion pictures and subtitles or images together and outputs the resultant synthetic video.

0069As can be seen, the BD player shown in FIG. <b>6</b> is designed based on the structure of the data stored on the BD as shown in FIG. <b>4</b>.

0070FIG. <b>7</b> shows a detailed functional block arrangement for the player. The image memory <b>308</b> and track buffer <b>309</b> shown in FIG. <b>7</b> correspond to the AV storage memory <b>205</b> shown in FIG. <b>6</b>. The program processor <b>302</b> and UOP manager <b>303</b> shown in FIG. <b>7</b> correspond to the program processing section <b>206</b>. The scenario processor <b>305</b> and presentation controller <b>306</b> shown in FIG. <b>7</b> correspond to the management information processing section <b>207</b> shown in FIG. <b>6</b>. And the clock <b>307</b>, demultiplexer <b>310</b>, image processor <b>311</b>, video processor <b>312</b> and sound processor <b>313</b> shown in FIG. <b>7</b> correspond to the presentation processing section <b>208</b> shown in FIG. <b>6</b>.

0071The MPEG stream (VOB data) and image data (PNG), which have been read out from the BD <b>201</b>, are stored in the track buffer <b>309</b> and image memory <b>308</b>, respectively. In accordance with the time shown by the clock <b>307</b>, the demultiplexer <b>310</b> extracts the VOB data from the track buffer <b>309</b> and sends out the video data to the video processor <b>312</b> and the audio data to the sound processor <b>313</b>, respectively. In compliance with the MPEG system standards, the video processor <b>312</b> and sound processor <b>313</b> are each made up of a decoder buffer and a decoder. That is to say, the video or audio data supplied from the demultiplexer <b>310</b> is temporarily stored in the decoder buffer and then decoded by the decoder in accordance with the time shown by the clock <b>307</b>.

0072The PNG stored in the image memory <b>308</b> may be processed by one of the following two methods.

0073If the image data is subtitles, then the presentation controller <b>306</b> specifies the timing of decoding. The scenario processor <b>305</b> once receives the time information from the clock <b>307</b> and instructs the presentation controller <b>306</b> whether or not to display the subtitles when it's time to start or stop displaying the subtitles so as to show the subtitles appropriately. On receiving the decoding/presenting instruction from the presentation controller <b>306</b>, the image processor <b>311</b> extracts the specified PNG data from the image memory <b>308</b>, decodes the data, and then draws it on the image plane <b>314</b>.

0074On the other hand, if the image data is a menu, then the program processor <b>302</b> specifies the timing of decoding. However, it depends on the specific BD program being processed by the program processor <b>302</b> exactly what time the program processor <b>302</b> instructs the image decoding, and therefore, that timing is subject to change at any time.

0075As already described with reference to FIG. <b>6</b>, the image data and video data are decoded, output to the image plane <b>314</b> and video plane <b>315</b>, respectively, and then synthesized together and output by the synthesis processing section <b>316</b>.

0076The management information (i.e., scenario information and AV management information) that has been read out from the BD <b>201</b> is stored in the management information storage memory <b>304.</b> Thereafter, the scenario information (BD.INFO and XXX.PL) is fed into, and processed by, the scenario processor <b>305</b>. Meanwhile, the AV management information (YYY.VOBI) is fed into, and processed by, the presentation controller <b>306</b>.

0077The scenario processor <b>305</b> analyzes the play list information and tells the presentation controller <b>306</b> what VOBs are referred to by the play lists and where those VOBs should be played back. On the other hand, the presentation controller <b>306</b> analyzes the management information of the target VOB (i.e., YYY.VOBI) and instructs a drive controller <b>317</b> to read out the target VOB.

0078In accordance with the instruction given by the presentation controller <b>306</b>, the drive controller <b>317</b> moves the optical pickup, thereby reading out the target AV data. The read AV data is once stored in either the image memory <b>308</b> and track buffer <b>309</b> as described above.

0079Also, a system processor <b>305</b> monitors the time shown by the clock <b>307</b> and transfers an event to the program processor <b>302</b> at the timing specified by the management information.

0080The BD program (i.e., BD.PROG or XXX.PROG) stored in the program storage memory <b>301</b> is executed and processed by the program processor <b>302</b>. The program processor <b>302</b> processes the BD program when an event is transferred from either the scenario processor <b>305</b> or the UOP manager <b>303</b>. When the user transmits his or her request by tapping remote controller keys, the UOP manager <b>303</b> generates an event for the program processor <b>302</b>.

(3) Application space on BD

0081FIG. <b>8</b> shows the concept of an application space on the BD.

0082In the BD application space, a play list <b>PlayList</b> is used as a playback unit. Each play list has a static scenario, which is a playback sequence obtained by coupling a plurality of cells (<b>Cell</b>) together and defined by the order of coupling, and a dynamic scenario described by a program. Unless there is such a dynamic scenario described by the program, each play list defines only a procedure in which the respective cells are sequentially played back in their order. The playback by the play list finishes when all of those cells are played back. On the other hand, according to the program, a playback route across multiple play lists can be described. Thus, the playback targets can be changed dynamically according to the user's choice or the player status. Menu is a typical example of such dynamically changeable playback targets. And the menu can be defined as a scenario to be played back according to the user's choice. Thus, playback of a menu means dynamically selecting play lists according to a program.

0083Hereinafter, the program will be described in further detail. As used herein, the "program" refers to the handler of an event to be executed as either a time event or a user event.

0084The time event is generated in accordance with the time information embedded in a play list. The event transmitted from the scenario processor <b>305</b> to the program processor <b>302</b> as already described with reference to FIG. <b>7</b> is one such event. When the time event is issued, the program processor <b>302</b> executes its associated event handler, which is identified by its ID. As described above, the program to be executed can specify the playback of another play list. In that case, the playback of the current play list is stopped and switched into the playback of that specified play list.

0085The user event is generated by the user's manipulations of remote controller keys. The user events are roughly classified into the following two types.

0086A user event of a first type is a menu selection event to be generated by manipulating particular keys. Examples of the particular keys include BD "play" key, the cursor keys (including "up", "down", "right" and "left" keys) of the body or the remote controller, and "enter" key. An event handler corresponding to the menu selection event is effective only within a limited period of the play list. The play list information includes the effective periods of respective event handlers. When a particular key is pressed down, an effective event handler is searched for. If an effective event handler is found, then the event handler is executed. Otherwise, the menu selection event is ignored.

0087A user event of a second type is a menu calling event to be generated by manipulating particular keys. Examples of the particular keys include BD "play" key and the "menu" key of the body or the remote controller. When the menu calling event is generated, a global event handler is called. The global event handler is an event handler that is always effective without depending on the play list. By using this function, a DVD menu call (i.e., the function of calling an audio or subtitle menu while a title is being played back, changing the audio or subtitle, and then resuming the playback of the title from the suspended point) can be executed.

0088A cell <b>Cell</b> as a unit that makes up a static scenario in a play list refers to part or all of the playback section of a VOB (or MPEG stream). The cell carries the information about the VOB playback section as start/end time information. The VOB management information <b>VOBI,</b> which is paired with each VOB, includes a time map <b>TM</b>, which is write address table information associated with data reading time. By using this time map, the VOB playback start and end times can be derived from the reading start and end addresses within the VOB (i.e., within the target file "YYY.VOB"). The time map will be described in further detail later.

(4) Specifics of VOB

0089FIG. <b>9</b> shows a relationship between video pictures and the data structure of an MPEG stream. The VOB data structure shown at the bottom includes at least one video object unit (VOBU). Each VOBU includes video packs <b>V_PCK</b> and audio packs <b>A_PCK,</b> each of which is of the same size as one sector (e.g., 2 KB in this preferred embodiment).

0090A VOBU is one playback unit of a multiplexed stream, which is based on a group of pictures (GOP) in an MPEG video stream and additionally includes audio data and so on as well. A VOBU contains data corresponding to a video playback time ranging from 0.4 to 1.0 second (usually about 0.5 second). That is to say, in NTSC, about 15 frames are stored in one GOP in many cases.

0091FIG. <b>10</b> shows the data structure of a pack. The pack includes a pack header, a PES packet header and a PES packet payload. The packet header and payload together makes up one PES packet.

0092The payload is the data storage area of the PES packet. Elementary data such as video data or audio data is sequentially stored in the payload from the beginning. In the packet header, an ID <b>stream_ID</b> for use to identify what data stream is stored in the payload and a decoding time stamp <b>DTS</b> and a presentation time stamp <b>PTS</b>, which are time stamps carrying information about the decoding and presentation times of the payload, are stored. It should be noted that PTS and DTS are not stored in every packet header.

0093In the pack header, a time stamp system clock reference <b>SCR</b>, representing when the pack will pass the demultiplexer and be input to the decoder buffer of each elementary stream, is recorded.

(5) VOB interleaved writing

0094Next, the processing of reading a VOB file, which was written by an interleaving technique, will be described with reference to FIGS. <b>11</b> and <b>12</b>.

0095FIG. <b>11</b> shows VOB data files, their associated files, and a functional block arrangement for a player for reading these files.

0096The lower part of FIG. <b>11</b> shows how VOB data files and PNG files are written on a BD so as to be interleaved with each other. As shown in FIG. <b>11</b>, the VOB data file is divided into multiple sections, which are arranged discretely on the disc. To jump from one continuous interval to another, a seek operation needs to be carried out and data reading must be stopped during the seek operation. That is to say, the supply of data might be stopped.

0097On a normal ROM such as a CD-ROM or a DVD-ROM, AV data to be a series of continuous playback units are written continuously. This is because as long as the data was written there continuously, the drive just needs to read the data and pass them to the decoder sequentially, i.e., its processing is simplified. For that reason, the VOB data file is preferably written in a continuous area. Even so, a data structure like this is adopted because there are some additional data (e.g., subtitle data) to be played back synchronously with the video data in the VOB and because the subtitle data, as well as the VOB files, need to be read out from the BD in some way or other. That is why the VOB data file needs to be divided into several blocks, among which the image data for subtitles (i.e., the PNG files) need to be arranged.

0098According to an alternative method of reading subtitle data, the overall image data representing all subtitles (i.e., PNG files) are read out at a time before the VOB starts being read out. However, this method is impractical because a huge memory would be needed in that case.

0099Thus, by appropriately interleaving the VOB files and image data with each other, the image data can be stored in an image memory at a required timing without storing a huge amount of data in a single memory, albeit temporarily, as in the alternative method just mentioned.

0100This player functional block represents a part of the player described above. The data on the BD is read out by an optical pickup. The VOB data files (i.e., an MPEG stream) are input to a track buffer, while PNG (i.e., image data) files are input to an image memory.

0101The track buffer is a first-in, first-cut (FIFO) buffer, which passes the incoming VOB data to a demultiplexer sequentially (i.e., in the order of input). In this case, each pack is extracted from the track buffer according to the SCR described above, and its data is transferred to a video processor or a sound processor by way of the demultiplexer. As to the image data on the other hand, it is the presentation controller that instructs which image should be drawn. The image data that has been used for drawing subtitles is deleted from the image memory simultaneously. Meanwhile, the image data for a menu is left as it is in the image memory while the menu is being drawn. This is because the menu drawing depends on the user's operation. That is to say, a portion of the menu may be re-displayed or replaced with another image in accordance with the user's instruction. Thus, it would be easier to decode the image data of the portion to be re-displayed if the menu image data were left there as it is.

0102FIG. <b>12</b> shows a VOB data continuous supply model using a track buffer. First, a VOB continuous storage area is supposed to range from a logical address <b>a1</b> to a logical address <b>a2</b> as shown in the upper part of FIG. <b>12</b>. The following range from <b>a2</b> to <b>a3</b> is supposed to be an interval in which the image data is written and from which no VOB data can be read out.

0103The VOB data is temporarily stored in the track buffer. Accordingly, if there is a difference between the rate <b>Va</b> of inputting the data to the track buffer and the rate <b>Vb</b> of outputting the data from the track buffer (where Va>Vb) and as long as the data continues being read out from the BD, the amount of data accumulated in the track buffer continues to increase. The lower part of FIG. <b>12</b> shows how the amount of data stored in the track buffer changes. The abscissa represents the time and the ordinate represents the amount of data accumulated in the track buffer. The time <b>t1</b> is a point in time when the data starts being read from the VOB continuous storage area at the logical address <b>a1</b>. As of this time <b>t1</b>, the data will be accumulated in the track buffer at a rate of Va - Vb, which is the difference between the track buffer input and output rates. The time <b>t2</b> is a point in time when the data has been read out fully from the continuous storage area at the logical address <b>a2</b>. In the interval between the times <b>t1</b> and <b>t2</b>, the amount of data in the track buffer increases at the rate Va-Vb. And the amount of data accumulated at the time <b>t2</b> (B (t2)) is given by:<maths id="math0001" num="(1)"><math display="block"><mrow><mtext>B (t2)=(Va-Vb)×(t2-t1)</mtext></mrow></math><img file="EP1517554A1_D0001.tif" /></maths>

0104Thereafter, the image data lasts until the address <b>a3</b> on the BD, and therefore, the input to the track buffer becomes zero and the amount of data in the track buffer decreases at the rate of -Vb, which is the output rate. This state continues until the readout position <b>a3</b> (i.e., the time <b>t3</b>).

0105It should be noted that if the amount of data accumulated in the track buffer became equal to zero before the time <b>t3</b>, there would be no VOB data anymore to supply to the decoder and the playback of the VOB might stop. In other words, if the data still remained in the track buffer at the time <b>t3</b>, then the VOB can be played back continuously without stopping.

0106This condition is represented by the following inequality:<maths id="math0002" num="(2)"><math display="block"><mrow><mtext>B (t2) ≧ -Vb× (t3-t2)</mtext></mrow></math><img file="EP1517554A1_D0002.tif" /></maths>

0107That is to say, the arrangement of the image data (i.e., non-VOB data) needs to be determined so as to satisfy the inequality (2).

(6) Navigation data structure

0108The BD navigation data (i.e., BD management information) structure will be described with reference to FIGS. <b>13</b> through <b>19</b>.

0109FIG. <b>13</b> shows the internal structure of the VOB management information file <b>YYY.VOBI</b>. The VOB management information includes the stream attribute information <b>Attribute</b> of the VOB and the time map <b>TMAP</b> thereof. The stream attribute includes a video attribute <b>Video</b> and respective audio attributes <b>Audio</b> #<b>0</b> through <b>Audio #m</b>. In an audio stream among other things, a VOB can have multiple audio streams at the same time. Accordingly, the number of audio streams <b>Number</b> indicates whether or not there are any data fields.

0110The following is the fields that the video attribute <b>Video</b> has and their possible values: <ul id="ul0006" list-style="none" compact="compact"><li>Compression method <b>Coding</b>: <ul id="ul0007" list-style="none" compact="compact"><li>MPEG 1</li><li>MPEG 2</li><li>MPEG 4 and</li><li>MPEG 4-AVC (advanced video coding);</li></ul></li><li>Resolution <b>Resolution:</b><ul id="ul0008" list-style="none" compact="compact"><li>1,920×1,080</li><li>1,280×720</li><li>720×480 and</li><li>720×565;</li></ul></li><li>Aspect ratio <b>Aspect</b>: <ul id="ul0009" list-style="none" compact="compact"><li>4: 3 and</li><li>16: 9;</li></ul></li><li><b>Frame rate Framerate:</b><ul id="ul0010" list-style="none" compact="compact"><li>60</li><li>59.94 (60/1.001)</li><li>50</li><li>30</li><li>29.97 (30/1.001)</li><li>25</li><li>24 and</li><li>23.976 (24/1.001)</li></ul></li></ul>

0111The following is the fields that the audio attribute <b>Audio</b> has and their possible values: <ul id="ul0011" list-style="none" compact="compact"><li>Compression method <b>Coding:</b><ul id="ul0012" list-style="none" compact="compact"><li>AC3</li><li>MPEG 1</li><li>MPEG 2 and</li><li>LPCM</li></ul></li><li>The number of channels <b>Ch:</b><ul id="ul0013" list-style="none" compact="compact"><li>1 through 8</li><li>Language attribute <b>Language:</b></li></ul></li></ul>

0112The time map <b>TMAP</b> is a table containing information about each VOBU, and includes the number of VOBUs <b>Number</b> that the VOB has and information about the respective VOBUs <b>VOBU #1</b> through <b>VOBU #n.</b> And the information about each VOBU includes the playback duration <b>Duration</b> and data size <b>Size</b> of the VOBU.

0113FIG. <b>14</b> shows correspondence between the playback duration and the data size of a VOBU. According to AC3, which is an audio compression standard, the compression is done at a fixed bit rate, and therefore, the time-address relation can be derived by a linear equation. As for MPEG video, however, the time-address relation cannot be represented by a linear equation.

0114This is because an MPEG video stream may be generated by being subjected to a variable bit rate compression for the purpose of high-quality recording and there is no simple correlation between the playback duration and the data size thereof. In MPEG video data, each frame has a fixed presentation time. In an NTSC signal, for example, each frame has a presentation time of 1/29.97 second. However, the data size of each frame after the compression changes significantly with the property of the picture or the picture type used for the compression (i.e., I-, P- or B-picture).

0115In the same way, in an MPEG system stream obtained by multiplexing the MPEG video data (i.e., a VOB), the time-data size relation cannot be represented by a linear equation, either. Thus, information about the time-address correlation within a VOB is needed. The time map <b>TMAP</b> is provided for that purpose. As shown in FIG. <b>14,</b> the time map <b>TMAP</b> has an entry that correlates the number of frames in each VOBU with the number of packs in the VOBU (i.e., the data size).

0116The time map <b>TMAP</b> will be described more fully with reference to FIG. <b>15</b>, which shows the concept of processing for obtaining an address based on the playback duration of a VOBU. First, when time information is given, a VOBU to which the time belongs is searched for. To do so, the numbers of frames of VOBUs on the time map may be added together, thereby finding a VOBU in which the sum of the numbers of frames is equal to or greater than a particular number of frames representing that time by conversion. Next, the data sizes of the respective VOBUs on the time map are also added together up to the previous VOBU for the given VOBU. This value is used to obtain the address of the pack to read for the purpose of playing back a frame including the given time.

(7) Play list data structure

0117Next, the data structure of the play list information <b>XXX.PL</b> will be described with reference to FIG. <b>16</b>. FIG. <b>16</b> shows the data structure of the play list information. The play list information consists of a cell list <b>CellList</b> and an event list <b>EventList.</b>

0118The cell list <b>CellList</b> shows the sequence of cells to play back from the play list. That is to say, the cells will be played back in the order described in this list. The contents of the cell list <b>CellList</b> include the number of cells <b>Number</b> and information about respective cells <b>Cell #1</b> through <b>Cell #n.</b>

0119The information about each cell <b>Cell #</b> includes the file name of the VOB <b>VOBName,</b> the start time <b>In</b> and end time <b>Out</b> within the VOB, and a subtitle table <b>SubtitleTable.</b> The start and end times <b>In</b> and <b>Out</b> are represented by the numbers of frames within the VOB. By using the time map <b>TMAP</b> described above, the address of the VOB data required for playback can be obtained.

0120The subtitle table <b>SubtitleTable</b> contains information about the subtitles to be played back synchronously with the VOB in question. The subtitles, as well as the sound, can have multiple versions in different languages. Thus, the top information of the subtitle table <b>SubtitleTable</b> is the number of languages <b>Number</b> included, which is followed by tables for the respective languages <b>Language #1</b> through <b>Language #k.</b>

0121The table for each language <b>Language #</b> consists of language information <b>Lang</b>, the number of subtitle information <b>Number</b> of the subtitles to be presented, and the subtitle information <b>Speech #1</b> through <b>Speech #j</b> of the subtitles to be presented. Each subtitle information <b>Speech #</b> includes the file name <b>Name</b> of its associated image data, subtitle presentation start time <b>In</b> and end time <b>Out</b> and subtitle display position <b>Position</b>.

0122The event list <b>EventList</b> is a table that defines the events to occur within the play list. The event list includes the number of events <b>Number</b>, which is followed by respective events <b>Event #1</b> through <b>Event #m.</b> And each event <b>Event #</b> consists of the type of the event <b>Type</b>, the ID of the event <b>ID,</b> the time of occurrence of the event <b>Time</b> and the duration of the event <b>Duration.</b>

0123FIG. 17 shows the data structure of an event handler table <b>XXX.PROG.</b> The event handler table <b>XXX.PROG</b> includes event handlers (i.e., a time event and a user event for menu selection) for each play list.

0124The event handler table includes the number of event handlers/programs defined <b>Number</b> and the respective event handlers/programs <b>Program #1</b> through <b>Program #n.</b> The description of each event handler/program <b>Program #</b> includes the definition of the beginning of the event handler (i.e., <event_handler> tag) and an event handler ID <b>ID</b> to be paired with the event ID described above. Thereafter, the program in question is also described between the parentheses "{" and "}" following Function. The events <b>Event #1</b> through <b>Event #m</b> stored in the event list <b>EventList</b> of XXX.PL described above are identified by the event handler ID <b>ID</b> of XXX.PROG.

0125Next, the internal structure of information <b>BD.INFO</b> about the overall BD will be described with reference to FIG. <b>18</b>. FIG. <b>18</b> shows the data structure of the information about the overall BD.

0126The information about the overall BD includes a title list <b>TitleList</b> and an event table <b>EventList</b> for global events.

0127The title list <b>TitleList</b> consists of the number of titles <b>Number</b> in the disc and respective title information <b>Title #1</b> through <b>Title #n</b> following it. Each title information <b>Title #</b> includes the play list table <b>PLTable</b> included in the title and a chapter list <b>ChapterList</b> within the title. The play list table <b>PLTable</b> includes the number of play lists <b>Number</b> in the title and the file names of the play lists <b>Name</b>.

0128The chapter list <b>ChapterList</b> consists of the number of chapters <b>Number</b> included in the title and respective chapter information <b>Chapter #1</b> through <b>Chapter #n</b>. Each chapter information <b>Chapter #</b> includes the table of cells <b>CellTable</b> included in the chapter. The cell table <b>CellTable</b> consists of the number of cells <b>Number</b> and respective cells' entry information <b>CellEntry #1</b> through <b>CollEntry #k</b>. Each cell entry information <b>CellEntry #</b> is described by the name of the play list including that cell and the number of that cell in the play list.

0129The event list <b>EventList</b> includes the number of global events <b>Number</b> and respective global events' information. It should be noted that the global event to be defined first is called FirstEvent that is the event to be read first after the BD has been loaded into a player. Each global event information includes only an event type <b>Type</b> and an event ID <b>ID</b>.

0130FIG. <b>19</b> shows the program table <b>BD.PROG</b> of a global event handler. This table has the same contents as the event handler table that has already been described with reference to FIG. <b>17</b>.

(8) How to generate events

0131Next, it will be described with reference to FIGS. <b>20</b> through <b>22</b> how to generate events.

0132FIG. <b>20</b> shows the concept of a time event. As described above, a time event is defined by the event list <b>EventList</b> of the play list information <b>XXX.PL</b>. If the given event is defined as a time event (i.e., if the event type <b>Type</b> is TimeEvent), a time event having an ID <b>Ex1</b> is passed from the scenario processor to the program processor when it is the time <b>t1</b> to generate the event. The program processor looks for an event handler with the event ID <b>Ex1</b> and executes the event handler in question. For example, in this preferred embodiment, the program processor may draw an image representing two buttons.

0133FIG. <b>21</b> shows the concept of a user event of menu selection.

0134As described above, the user event of menu selection is also defined by the event list <b>EventList</b> of the play list information <b>XXX.PL.</b> If the given event is defined as a user event (i.e., if the event type <b>Type</b> is UserEvent), the user event gets ready when it is the time <b>t1</b> to generate the event. At this point in time, however, the event itself has not been generated yet. That event is ready to enter its effective period described by duration information <b>Duration.</b>

0135As shown in FIG. <b>21</b>, when the user presses down the "up", "down", "left", "right" or "enter" key of the remote controller, first, an UOP event is generated by a UOP manager and passed to the program processor. The program processor passes the UOP event to the scenario processor. In response, the scenario processor determines whether or not there is any valid user event at the time of receipt of the UOP event. If there is such a user event, the scenario processor generates the user event and passes it to the program processor. The program processor searches for an event handler with the event ID <b>Ev1</b> and then executes the event handler in question. For example, in this preferred embodiment, the program processor starts playing back the play list #2.

0136The user event to be generated includes no information about what remote controller key has been pressed down by the user. Instead, the information about the selected remote controller key is conveyed by the UOP event to the program processor and then stored and retained in a register SPRM 8 that the virtual player has. The event handler program can execute branch processing by checking out this register value.

0137FIG. <b>22</b> shows the concept of a global event. As described above, the global event is defined by the event list <b>EventList</b> of the information <b>BD.INF</b>O about the overall BD. If the given event is defined as a global event (i.e., if the event type <b>Type</b> is GlobalEvent), that event is generated only when the user presses a remote controller key.

0138For example, if the user has pressed down the "menu" key, a UOP event is generated by the UOP manager and then passed to the program processor. The program processor passes the UOP event to the scenario processor. In response, the scenario processor generates a corresponding global event and passes it to the program processor. The program processor searches for an event handler with the event ID <b>Menu</b> and then executes the event handler in question. For example, in this preferred embodiment, the program processor starts playing back the play list #3.

0139In this example, the "menu" key is supposed to be single and called "the" menu key. Optionally, a plurality of menu keys may be provided as in a DVD player. In that case, IDs just need to be defined for the respective menu keys.

(9) Virtual player machine

0140The functions of the program processor will be described with reference to FIG. <b>23</b>. FIG. <b>23</b> shows the arrangement of functional blocks associated with the program processor. The program processor is a processing module including a virtual player machine inside. The virtual player machine is a functional model defined with respect to a BD and never depends on any specific implementation of a BD player. Thus, the virtual player machine guarantees that the same functions are achievable by any BD player. For example, the configuration shown in FIGS. <b>6</b> and <b>7</b> suppose that the virtual player machine has the functions to be described below.

0141The virtual player machine has two primary functions: programming function <b>a, b</b> and <b>c</b> and player variables (registers). The programming function may define the following two functions as BD proper functions based on Java® Script, for example: <ul id="ul0014" list-style="none" compact="compact"><li>(a) Link function: stop the current playback and start playback from a specified play list, cell or time    Format: Link (PL #, Cell #, time)       where PL # stands for a play list name,       Cell # stands for a cell number, and       Time stands for a playback start time within the cell</li><li>(b) PNG drawing function: draw specified PNG data on an image plane    Format: Draw (File, X, Y)       where File stands for a PNG file name,       X represents an X coordinate, and       Y represents a Y coordinate.</li><li>(c) image plane clear function: clear the specified range on the image plane    Format: Clear (X, Y, W, H)       where X represents an X coordinate;       Y represents a Y coordinate,       W represents the width in the X direction, and       H represents the width in the Y direction</li></ul>

0142The player variables include system parameters <b>SPRM</b> representing a condition of the player and general parameters <b>GPRM</b>, which can be used for general purposes.

0143FIG. <b>24</b> is a list of the system parameters <b>SPRM</b>: <ul id="ul0015" list-style="none" compact="compact"><li>SPRM (0): language code</li><li>SPRM (1): audio stream number</li><li>SPRM (2): subtitle stream number</li><li>SPRM (3): angle number</li><li>SPRM (4): title number</li><li>SPRM (5): chapter number</li><li>SPRM (6): program number</li><li>SPRM (7): cell number</li><li>SPRM (8): key name</li><li>SPRM (9): navigation timer</li><li>SPRM (10): current playback time</li><li>SPRM (11): player audio mixing mode for Karaoke</li><li>SPRM (12): country code for parental management</li><li>SPRM (13): parental level</li><li>SPRM (14): player configuration for video</li><li>SPRM (15): player configuration for audio</li><li>SPRM (16): language code for AST</li><li>SPRM (17): language code ext. for AST</li><li>SPRM (18): language code for STST</li><li>SPRM (19): language code ext. for STST</li><li>SPRM (20): player region code</li><li>SPRM (21): reserved</li><li>SPRM (22): reserved</li><li>SPRM (23): player status</li><li>SPRM (24): reserved</li><li>SPRM (25): reserved</li><li>SPRM (26): reserved</li><li>SPRM (27): reserved</li><li>SPRM (28): reserved</li><li>SPRM (29): reserved</li><li>SPRM (30): reserved</li><li>SPRM (31): reserved</li></ul>

0144In this preferred embodiment, the programming function of the virtual player is supposed to be based on Java® Script. Alternatively, any other programming function such as B-Shell or Perl Script used in UNIX® OS, for example, may be used, too. In other words, the present invention is never limited to Java® Script.

(10) Exemplary programs

0145Exemplary programs for the event handler will be described with reference to FIGS. <b>25</b> and <b>26</b>.

0146FIG. <b>25</b> shows an exemplary program for a menu that has two selection buttons.

0147At the beginning of the cell (PlayList #1. Cell #1), the program starting with <event_handler ID="pre"> shown in FIG. <b>25</b> is executed by using a time event. In this case, GPRM (0), which is one of general parameters, is set equal to "1" first. GPRM (0) is used to identify the button being selected in the program. In the beginning, a state in which the left-hand side button <b>1</b> is selected is defined as an initial value. Next, PNG images are drawn for the buttons <b>1</b> and <b>2</b>, respectively, by using a drawing function <b>Draw</b>. Specifically, as for the button <b>1</b>, a PNG image "lblack.png" is drawn from a point with coordinates (10, 200) as a start point (i.e., the left corner). On the other hand, as for the button <b>2</b>, a PNG image "2white.png" is drawn from a point with coordinates (330, 200) as a start point (i.e., the left corner).

0148Also, at the end of this cell, the program on the righthand side of FIG. <b>25</b> is executed by using a time event. In this case, it is instructed that the cell be played back all over again from the beginning by using a Link function.

0149FIG. <b>26</b> shows an exemplary program for an event handler to process the user event of "menu selection".

0150The event handler describes the program so as to tell what to do if any of the remote controller keys "left", "right" and "enter" has been pressed down. If the user has pressed down one of those remote controller keys, then the user event is generated as already described with reference to FIG. <b>21</b> to activate the event handler shown in FIG. <b>26</b>. This event handler carries out branch processing based on the value of GPRM (0) that identifies the selected button and SPRM (8) that identifies the selected remote controller key. The branch and execute processing is carried out as follows depending on the conditions: <ul id="ul0016" list-style="none" compact="compact"><li>Condition 1): If the button <b>1</b> is currently selected and if the selected key is the "right" key, then GPRM (0) is modified into "2", the button being selected is changed into the righthand side button <b>2</b> and the images of the buttons <b>1</b> and <b>2</b> are rewritten;</li><li>Condition 2): If the selected key is the "enter (OK)" and if the button <b>1</b> is currently selected, then the play list #2 starts being played back; and</li><li>Condition 3): If the selected key is the "enter (OK)" and if the button <b>2</b> is currently selected, then the play list #3 starts being played back.</li></ul>

(11) Processing flow of the virtual player

0151Next, the flow of the processing to be done by the player will be described with reference to FIGS. <b>27</b> through <b>30</b>. FIG. <b>27</b> shows the flow of the basic processing to be done before AV data is played back.

0152When a BD is inserted (S101), the BD player reads and analyzes a BD.INFO file (S102) and then reads BD.PROG (S103). Both BD.INFO and BD.PROG are temporarily stored in a management information storage memory and are analyzed by a scenario processor.

0153Subsequently, in accordance with the first event information <b>FirstEvent</b> in the BD.INFO file, the scenario processor generates the first event <b>(</b>S104). The first event generated is received by a program processor, which executes an event handler corresponding to the event (S105).

0154The event handler corresponding to the first event is expected to contain information about the play list to play back first. If the playback of no play lists (i.e., PL playback) were instructed, then the player would just wait to accept a user event (S201) without playing back anything. On receiving a user's command through the remote controller, the BD player makes an UOP manager issue a UOP event to the program manager (S202).

0155The program manager determines whether or not the UOP event is a menu key (S203). If the answer is YES, the program manager supplies the UOP event to the scenario processor, thereby making the scenario processor generate a user event (S204). Then, the program processor executes an event handler corresponding to the user event generated (S205).

0156FIG. <b>28</b> shows the flow of processing to be done since the playback of the play list is started and until the playback of a VOB is started.

0157As described above, either the first event handler or the global event handler starts playing back the play list (S301). To collect information needed to play back the target play list, the scenario processor reads and analyzes the play list information <b>XXX.PL</b> (S302) and then reads program information <b>XXX.PROG</b> associated with the play list (S303). Subsequently, the scenario processor instructs playing back a cell in accordance with the information about the cells registered with the play list (S304). To play back a cell means that the scenario processor sends a command to the presentation controller. Thus, the presentation controller starts playing back the AV data (S305).

0158When the playback of the AV data is started (S401), the presentation controller reads and analyzes a VOB information file <b>XXX.VOBI</b> associated with the cell to play back (S402). By using the time map, the presentation controller specifies the VOB to start playback at and its address and tells the drive controller a readout address. In response, the drive controller reads the target VOB data (S403). Then, the VOB data is supplied to the decoder and starts being read (S404).

0159The VOB is continuously played back until its playback interval ends (S405). When the playback finishes, the next cell starts being played back (S304). If there is no next cell anymore, then the playback operation ends (S406).

0160FIG. <b>29</b> shows the flow of event handling to be done after the AV data started being read. The BD player is an event driven type. When the play list starts being played back, event handling processes, including time event, user event and subtitle display lines, are started up. And these three lines of event handling processes are carried out in parallel with each other.

0161The processing steps <b>S501</b> through <b>S505</b> show the flow of the "time event" line of processing. After the play list started being played back (S501), it is determined whether or not the play list has been played back completely (S502). Then, the scenario processor sees if it is the time to generate a time event (S503). If the answer is YES, the scenario processor generates the time event (S504). Then, the program processor receives the time event and executes the event handler (S505).

0162If the answer to the query of Step <b>S503</b> is NO (i.e., if it is not yet the time to generate the time event) or after the event handler has been executed in Step <b>S504,</b> the process returns to Step <b>S502</b> and the same processing is repeatedly performed all over again. On the other hand, if it is confirmed in Step <b>S502</b> that the play list has been played back fully, the time event line of processing is brought to a forced end.

0163The processing steps <b>S601</b> through <b>S608</b> show the flow of the "user event" line of processing. After the play list started being played back (S601), it is determined whether or not the play list has been played back completely (S602). Then, the process advances to the step of seeing if an UOP has been accepted (S603). If an UOP has been accepted, then the UOP manager generates a UOP event (S604). Next, the program processor, having received the UOP event, checks whether the UOP event is a menu call or not (S605). If the answer is YES, then the program processor makes the scenario processor generate an event (S607). And the program processor executes the event handler (S608).

0164On the other hand, if it is determined in Step <b>S605</b> that the UOP event is not a menu call, then the UOP event should be an event specified by either a cursor key or the "enter" key. In that case, the scenario processor determines whether or not the current time falls within the duration of the user event (S606). If the answer is YES, then the scenario processor generates the user event (S607) and the program processor executes the event handler in question (S608).

0165If the answer to the query of Step <b>S603</b> is NO (i.e., if no UOP has been accepted) or if the answer to the query of Step <b>S606</b> is NO (i.e., if the current time falls out of the user event duration) or after the event handler has been executed in Step <b>S608,</b> the process returns to Step <b>S602</b> and the same processing is repeatedly performed all over again. On the other hand, if it is confirmed in Step <b>S602</b> that the play list has been played back fully, the user event line of processing is brought to a forced end.

0166FIG. 30 shows the flow of subtitle processing. After the play list started being played back (S701), it is determined whether or not the play list has been played back completely (S702). Then, the process advances to the step of seeing if it is the time to start drawing the subtitle (S703). If the answer is YES, then the scenario processor instructs the presentation controller to draw the subtitle, and the presentation controller, in turn, instructs the image processor to draw the subtitle (S704). On the other hand, if the answer to the query of Step <b>S703</b> is NO, then it is determined whether or not it is the time to finish drawing the subtitle (S705). If the answer is YES, then the presentation controller instructs the image processor to erase the subtitle, thereby erasing the subtitle being drawn from the image plane (S706).

0167After the subtitle drawing has been instructed in Step <b>S704</b> or after the subtitle erasure has been instructed in Step <b>S706</b> or if the answer to the query of Step <b>S705</b> is NO (i.e., if it is not yet the time to finish drawing the subtitle), the process returns to Step <b>S702</b> and the same processing is repeatedly performed all over again. On the other hand, if it is confirmed in Step <b>S702</b> that the play list has been played back fully, the processing is brought to a forced end.

(12) Player performing down conversion processing

0168FIG. <b>31</b> shows a configuration for a player <b>100</b> for performing down conversion processing according to this preferred embodiment. The player <b>100</b> is constructed based on a transport stream related system target decoder (T-STD) model according to an MPEG standard. The player <b>100</b> can down-convert HD video into SD video.

0169In FIG. <b>31</b>, moving picture data, subtitle data, still picture data and audio data are separately input to this decoder model. These data are obtained after an MPEG transport stream has been read by an optical head from a BD, received by a demultiplexer, divided into packets storing the moving picture data, subtitle data and so on, and output. The demultiplexer may have the same functions as the demultiplexer <b>3100</b> shown in FIG. <b>3</b>, for example.

0170The player <b>100</b> may be roughly grouped into a buffer <b>3121</b>, a decoder <b>3122</b>, a signal processing system for respective data types, and a converting section <b>3123</b>.

0171The buffer <b>3121</b> includes a plurality of buffers for storing each of the moving picture data, subtitle data, still picture data and audio data on multiple stages. The decoder <b>3122</b> decodes the video and audio data, which has been compression-coded so as to comply with an MPEG standard, in compliance with the same MPEG standard. The signal processing system performs a different type of processing according to the specific type of the given data such as the moving picture data. This system will be described in further detail later. The converting section <b>3123</b> down-converts HD video into SD video. The buffer <b>3121</b>, decoder <b>3122</b> and signal processing system may be implemented as a single decoder chip. Alternatively, only the decoder <b>3122</b> and signal processing system may be implemented as a single chip.

0172Hereinafter, the functions of the respective components, as well as the processing flow of the moving picture data, subtitle data and still picture data, which are all included in the video data, will be described.

0173First, the moving picture data is transferred to a video decoder line, passed through a transport buffer (<b>TB</b>) <b>3201</b> and multiplexing buffer (<b>MB</b>) <b>3202</b> and then temporarily stored in an elementary buffer (<b>EB</b>) <b>3203.</b> The pictures, which have been stored in the <b>EB 3203</b>, are transferred to, and decoded by, a video decoder <b>3204</b> when the time specified by the decoding time stamp <b>DTS</b> has come. Among these pictures, I- and P-pictures to be referred to by other pictures are transferred to a reorder buffer <b>3205</b> and used for decoding the other pictures. Each of these pictures is sent to, and presented by, an image display device at a time defined by its presentation time stamp <b>PTS</b>. If an HDTV is connected, the HD video decoded is output to the HDTV <b>3206</b> at the PTS time. If down conversion is needed to present it on an SDTV, then the HD video decoded is transferred to, and converted into SD video by, a video down converter (V-DC) <b>3207</b>.

0174The subtitle data is transferred through a transport buffer (TB) <b>3208</b> to a buffer (B) <b>3209</b> and temporarily stored there. And as in the video, at the time specified by the decoding time stamp <b>DTS</b>, the subtitle data is also transferred to, and decoded by, the subtitle decoder (D) <b>3210</b> instantaneously. The subtitle data decoded is once expanded on an image buffer (IB) <b>3212</b> and a specified subtitle is output to an HDTV by way of a color conversion table (CLUT) <b>3213</b> at the PTS time in response to a request from an image controller IC <b>3211</b>. The subtitle is presented on an HDTV display area that has been specified by the image controller IC <b>3211</b>. If down conversion is needed to present it on an SDTV, then the HD grade subtitle decoded is transferred to, and converted into SD grade subtitle by, a subtitle down converter (S-DC) <b>3215</b>.

0175The still picture data, representing a still picture such as a menu, is also decoded and output to an HDTV or an SDTV. The still picture data is processed similarly to the subtitle data. Since the statement about the subtitle data also applies to the still picture data, the description thereof will be omitted herein.

0176The down-converted moving picture data, subtitle data and still picture data are synthesized together to produce SD video to be output to an SDTV <b>3229</b>. First, the moving picture and subtitle are α-blended (i.e., synthesized) together. An SD grade moving picture, obtained as a result of the synthesis, is further α-blended (synthesized) with a still picture such as a menu. In this manner, SD video, in which the subtitle is superimposed on the background image and in which the still picture such as a menu is further superposed on the subtitle, is completed. The SD video is output to the SDTV <b>3229</b>.

0177The audio data is processed in the following flow. Specifically, the audio data is transferred through a transport buffer (TB) <b>3224</b> to, and temporarily stored in, a buffer (B) <b>3225</b>. Thereafter, at the time specified by the decoding time stamp <b>DTS</b>, the audio data is instantaneously output to, and decoded by, an audio decoder <b>3226</b>. It should be noted that the decoding time of the audio data is often identical with the presentation time. That is to say, in almost every sound, its decoding time stamp <b>DTS</b> is its presentation time stamp <b>PTS</b>. Accordingly, after having been decoded, the sound is output to a loudspeaker <b>3227</b> immediately.

0178If no HDTV is connected to the video output and if the HD video needs to be down-converted into SD video, then the audio data is transferred to an audio down converter (A-DC) <b>3228</b> and subjected to sampling and other decimation processing there. As a result, sound is output through a loudspeaker <b>3230</b> synchronously with the SD video. The audio down converter (A-DC) <b>3228</b> is provided for the following reason. Specifically, SD video, which has been subjected to a down conversion such as DEFT conversion complying with the PAL standard (to be described later), may be presented at a different rate than its original HD video. Thus, to adjust the playback rates, the audio down conversion is sometimes needed such that the resultant sound matches the SD video.

0179It should be noted that if the playback rate of the down-converted SD video is different from that of the original HD video, then it is not always necessary to decode it precisely as specified by the decoding time stamp <b>DTS</b> as will be described later.

0180Next, the process of down-converting HD video into SD video will be described in detail with reference to FIGS. <b>32</b> and <b>33</b>. In this example, the HD video is supposed to be progressive scanned film material video (i.e., moving pictures with subtitles) with a frame rate of 24 Hz. On the other hand, the SD video is supposed to be either interlaced scanned NTSC video with a frame rate of 59.94 Hz or interlaced scanned PAL video with a frame rate of 50 Hz. In any case, since the original HD video is moving pictures with subtitles, the down-converted video is also supposed to be moving pictures with subtitles.

0181FIG. <b>32</b> shows the procedure of the down conversion processing. First, the player <b>100</b> reads an MPEG data stream from an optical disc (S801). The demultiplexer (not shown) receives the MPEG data stream, separates it into moving picture data, subtitle data, still picture data and audio data, and then sends these data to the decoder <b>3122</b> by way of the buffer <b>3121</b>. In response, the decoder <b>3122</b> decodes the moving picture data, subtitle data, still picture and so on of the HD video (S802).

0182Next, the converting section <b>3123</b> acquires the presentation time stamp <b>PTS</b> of the HD video (S803). More specifically, the video down converter <b>3207</b> acquires the PTS of the moving pictures and the subtitle down converter <b>3215</b> acquires the PTS of the subtitles. As already described with reference to FIG. <b>10</b>, the presentation time stamp <b>PTS</b> is obtained from the packet header of one of the packets that make up the MPEG data stream.

0183Subsequently, the converting section <b>3123</b> identifies the moving picture data and the data of the subtitle to be presented on that picture by reference to the PTS (S804), and converts the HD video picture data identified into SD video picture data and the HD video subtitle data identified into SD video subtitle data, respectively (S805).

0184Hereinafter, the processing to be carried out in Steps <b>S804</b> and <b>S805</b> will be described in further detail with reference to FIG. <b>33</b>. FIG. <b>33</b> shows correlations between pictures of respective grades when pictures of the HD video (i.e., film material video) are converted into pictures of the SD video (i.e., video complying with the NTSC or PAL standard). The film material HD video, extracted from an MPEG data stream, is obtained by switching pictures <b>P1, P2</b> and so on, which are progressive scan signals with a playback period of 1/24 second, one after another such that those pictures are presented continuously.

0185The converting section <b>3123</b> converts this HD video into SD video. The converted SD video (complying with the NTSC standard) has a playback period of 1/59.94 second and is obtained by switching fields, which are interlaced scan signals, one after another such that those fields are presented continuously. Meanwhile, the converted SD video (complying with the PAL standard) has a playback period of 1/50 second and is obtained by switching fields, which are interlaced scan signals, one after another such that those fields are presented continuously.

0186In converting the HD video into the NTSC-compliant SD video, the conversion is done on the following rule. Specifically, a frame <b>P1</b> of the film material is converted into two fields <b>P1t</b> and <b>P1b</b> and a frame <b>P2</b> of the film material is converted into three fields <b>P2t</b>, <b>P2b</b> and <b>P2t.</b> Speaking more generally, an odd-numbered HD video picture is converted into two SD video fields, while an even-numbered HD video picture is converted into three SD video fields. This conversion is called "2: 3 pull down" and is a technique of converting two frames of HD video into five fields of SD video. In the converted NTSC-compliant SD video, however, the field <b>P2t,</b> for example, is displayed twice with the field <b>P2b</b> interposed, and therefore, the video looks blurred.

0187In converting HD video into PAL-compliant SD video, a technique called "DEFT conversion" may be used. In the DEFT conversion, a frame <b>Pn</b> of the film material is converted into two fields <b>Pnt</b> and <b>Pnb.</b> Thus, unlike the NTSC-compliant video, the same field is never presented twice, and video blurring can be minimized. On the other hand, each field is presented at 50 Hz (i.e., each frame is presented at 25 Hz). That is to say, the 24 Hz HD video is converted into 25 Hz SD video. Thus, compared with the original HD video, the SD video is presented 1 Hz faster per frame (i.e., two fields).

0188As described above, if the HD video with a frame rate of 24 Hz (or 23.976 Hz) such as a film material is down-converted into SD video, then each picture is presented at a different timing from the original timing defined by 24 Hz. Accordingly, in a situation where subtitles need to be superimposed on moving pictures, it must be carefully determined on what pictures the subtitles should be superimposed.

0189In superimposing the subtitles on both a moving picture and a still picture, the presentation time stamps <b>PTS</b> may be provided separately for the moving picture and still picture. However, the SDTV has a presentation time grid (of 59.94 Hz or 50 Hz) for each picture. For that reason, the output time on the SDTV is preferably matched to this grid.

0190First, according to the MPEG-2, the DTS and PTS of each access unit can be set at a time accuracy of 90,000 Hz. Accordingly, as for the 24 Hz film material, the difference in PTS between two adjacent moving pictures that should be presented back to back is 3,750. As for the NTSC on the other hand, the PTS difference between two fields of a moving picture is either 1,501 or 1,502. And in the PAL, the PTS difference between two fields is 1,800. In each presentation format, the PTS of every picture needs to comply with associated one of these grids.

0191When multiplexed according to the MPEG-2, the subtitle data can also have its presentation time stamp PTS set at a time accuracy of 90,000 Hz. However, the subtitle data should not be defined at an arbitrary point in time. On an SD video picture obtained from its associated HD video picture, a subtitle, having the same contents as the counterpart on the HD video picture, needs to be always presented. For example, if the data is simultaneously output to an HDTV and to an SDTV, then the same scene of a moving picture being presented on the HDTV and SDTV should not have different subtitles.

0192In FIG. <b>33</b>, the pictures <b>P4</b>, <b>P5</b> and <b>P6</b> illustrated as solid black bars are supposed to have subtitles. To present the same subtitles even on the SD video versions of these pictures <b>P4, P5</b> and <b>P6</b>, the converting section <b>3123</b> identifies the picture data of a moving picture and the subtitle data to be superimposed on that picture by reference to the PTS of the moving picture and the PTS of the subtitle (i.e., Step <b>S804</b> shown in FIG. <b>32</b>).

0193Specifically, supposing the PTS of the pictures <b>P4, P5</b> and <b>P6</b> are identified by t4, t5 and t6 (where t5=t4+3750 and t6=t5+3750), respectively, subtitles, of which the PTS are also t4, t5 and t6, are identified. Then, the data of the pictures <b>P4, P5</b> and <b>P6</b> of the identified HD video is converted into SD video picture data, and the subtitle data of the identified HD video is converted into subtitle data of SD video (i.e., Step <b>S805</b> shown in FIG. <b>32).</b> Then, the converting section <b>3123</b> synthesizes together the picture data and subtitle data of the SD video generated and outputs the synthetic data (S806).

0194As a result, in the SD video obtained by this conversion, subtitles having the same contents as the subtitles with the PTS value of t4 are presented in SD grade on the fields <b>P4b, P4t</b> and <b>P4b</b> corresponding to the picture <b>P4</b>. On the other hand, subtitles having the same contents as the subtitles with the PTS value of t5 are presented in SD grade on the fields <b>P5t</b> and <b>P5b</b> corresponding to the picture <b>P5</b>. The same statement as that applied to the picture <b>P4</b> also applies to the picture <b>P6</b>. Accordingly, even after the film material has been converted into an NTSC- or PAL-compliant picture, the correlation between the subtitles and the moving pictures on which the subtitles are superimposed is maintained. The same statement applies to the PAL pictures, too.

0195However, according to the DEFT conversion applied to the PAL picture, frames are updated at a higher rate than the original frame display rate (i.e., the vertical scanning frequency) of the film material. Accordingly, the sound to be synchronized with those frames needs to be reproduced at an increased rate and has to have a higher pitch, which is a problem. In that case, such an inconvenience can be eliminated by the following method.

0196FIG. <b>34</b> shows correlations between the pictures in a situation where pictures of HD video (i.e., film material video) are converted into pictures of SD video. In this example, the processing of converting the pictures of the HD video (film material video) shown on the first row of FIG. <b>34</b> into the pictures of the SD video shown on the third row will be described. It should be noted that the second row of FIG. <b>34</b> shows DEFT-converted PAL-compliant pictures as in the third row of FIG. <b>33</b>. These pictures also are shown in FIG. <b>34</b> so as to be easily compared with the converted pictures shown on the third row of FIG. <b>34.</b>

0197As described above, if the 24 Hz HD video is converted into the 25 Hz PAL-compliant SD video by the DEFT conversion technique, then the resultant SD video is presented faster than the HD video by one frame per second. Thus, in this preferred embodiment, although the DEFT conversion is carried out most of the time, a particular picture (i.e., a field or a frame) is displayed a number of times at regular intervals during the conversion, thereby adjusting the number of pictures to be presented per second and eventually preventing the video presentation rate from becoming too high. This conversion will be referred to herein as a "field doubling conversion".

0198Hereinafter, it will be described with reference to FIG. <b>34</b>. In FIG. <b>34</b>, fields obtained by subjecting a picture <b>P5</b> of the HD video to the DEFT conversion are identified by <b>P5t</b> and <b>P5b,</b> and fields resulting from another picture <b>P6</b> of the HD video are identified by <b>P6t</b> and <b>P6b</b>. In the field doubling conversion, the video is converted such that after the fields <b>P5t</b> and <b>P5b</b> have been presented, the field <b>P5t</b> is presented again. When the field <b>P5t</b> has been presented for the second time, the field <b>P6b</b> and then the field <b>P6t</b> are presented in this order. By presenting the field <b>P5t</b> twice, the presentation time of the next field <b>P6b</b> is delayed by 1/50 second. As a result, the fast forwarding can be reduced by one field. This processing is similarly applicable to not just a moving picture but also a subtitle and other video to be presented synchronously with that moving picture. As to the subtitle data, for example, the subtitle converter <b>3215</b> shown in FIG. <b>31</b> may perform the conversion such that a subtitle field to be presented with the field <b>P5b</b> is followed by a subtitle field to be presented with the field <b>P5t</b> and then a subtitle field is presented synchronously with the field <b>P6b.</b>

0199Furthermore, if another field is presented twice and if two particular fields are presented twice a second, then a picture consisting of two fields immediately after them can have its presentation timing matched with that of its associated picture of the film material. According to the field doubling conversion, the same field is presented a number of times, and therefore, the SD video might look blurred when the field is presented for the second time. However, since the presentation timing of a picture can be matched with that of HD video, the audio data to be reproduced synchronously with that picture can be reproduced as it is without any distortion. Accordingly, if a content, which is a film material (i.e., with a rate of 24 Hz or 23.976 Hz) and in which the sound plays an important role, is down-converted into a PAL-compliant picture, this field doubling down conversion processing is preferred.

0200As can be seen from FIG. <b>31</b>, the player <b>100</b> down-converts the HD grade moving picture data, subtitle data and still picture data independently of each other, thereby generating respective types of data of the SD grade. Thereafter, the player 100 synthesizes them together to obtain SD video. Alternatively, the player <b>100</b> may synthesize together HD grade moving picture data, subtitle data and still picture data and then down-convert the synthesized HD video into SD video to generate SD synthetic video. To carry out such processing, the converting section <b>3123</b> shown in FIG. <b>31</b> needs to be provided with a component for synthesizing together the HD grade moving picture data, subtitle data and so on and a down converter for converting the HD synthetic video output of that component into SD video.

0201FIG. <b>35</b> shows the procedure of second exemplary down conversion processing. This processing may be carried out in place of the processing shown in FIG. <b>32</b>. Although the configuration of a system for executing this processing is not shown, the system may be made up of the same components as those of the player <b>100</b> shown in FIG. <b>31</b> except for the internal configuration of the converting section <b>3123</b>. Thus, the following example will also be described with reference to the player <b>100</b> shown in FIG. <b>31</b>.

0202First, the player <b>100</b> reads an MPEG data stream from an optical disc (S901). The demultiplexer (not shown) receives the MPEG data stream, separates it into moving picture data, subtitle data, still picture data and audio data, and then sends these data to the decoder <b>3122</b> by way of the buffer <b>3121</b>. In response, the decoder <b>3122</b> decodes the moving picture data, subtitle data, still picture and so on of the HD video (S902).

0203Next, the converting section <b>3123</b> acquires the picture data of the HD video and the presentation time stamp PTS added to the subtitle data (S903). Then, the converting section <b>3123</b> identifies the picture data of the HD video and the data of the subtitle to be superimposed on that picture by reference to the PTS (S904).

0204Thereafter, the converting section <b>3123</b> synthesizes the identified picture data and subtitle data of the HD video together, thereby generating synthetic HD video data (S905). Subsequently, the converting section <b>3123</b> converts the picture data of the synthetic HD video into picture data of SD video and outputs it (S906).

0205The processing of generating the synthetic HD video and then converting it into SD video may be carried out by one of the conversion methods that have been described with reference to FIGS. <b>33</b> and <b>34</b>. For example, the HD video may be converted into NTSC-compliant SD video by the 2: 3 pull down conversion technique or into PAL-compliant SD video by either the DEFT conversion technique or the field doubling conversion technique. Also, if it is expected that the video being down-converted will be panned and scanned on a TV set with an aspect ratio of 4 to 3, then the conversion may be carried out such that the subtitle display area (or its width among other things) will not exceed the 4 to 3 TV screen after the down conversion.

0206On whether the DEFT conversion or the field doubling conversion is to adopt, the viewer may instruct the player 100 every time the conversion needs to be carried out or one of these two techniques may be selected in advance.

0207Alternatively, conversion designating information, describing a recommended conversion method, may be added to the stream and the player <b>100</b> may pick a conversion method in accordance with that conversion designating information.

0208FIG. <b>36</b> shows the data structure of the conversion designating information that describes a recommended conversion method. The conversion designating information may be described in the BD management information <b>YYY.VOBI</b> and/or in <b>YYY.VOB</b> of an MPEG stream, for example. If that information is described in the stream management information <b>YYY.VOBI</b>, then appropriate down conversion processing may be selected on a VOB basis. As another alternative, that information may also be stored in a separate area on the disc (e.g., BD.INFO). In this description, a broadcaster on the transmitting end is supposed to transmit an MPEG transport stream with the conversion designating information described therein as a matter of principle. If necessary, however, the recorder may describe the conversion designating information in the management information <b>YYY.VOBI</b> accompanying an MPEG transport stream during a writing operation.

0209More specifically, the conversion designating information is defined by the down conversion information <b>DownCon</b> within the attribute information <b>Attribute</b>. The down conversion information includes a flag <b>DC_prohibit_flag</b> and information <b>DC_PAL</b> designating a recommended conversion method.

0210The flag <b>DC_prohibit_flag</b> shows whether or not this content may be down-converted so as to output SD video. If the down conversion is prohibited, then "1b" is added to the flag. On the other hand, if the down conversion is permitted, then "0b" is added to the flag. The information <b>DC_PAL</b> designating a recommended conversion method shows how to carry out a down conversion into PAL-compliant SD video and designates a conversion method. In this preferred embodiment, "00b" is added to DC_PAL if there is no recommended conversion method, "01b" is added to DC_PAL if the DEFT conversion is recommended, and "10b" is added to DC_PAL if the field doubling (FD) conversion is recommended as shown in FIG. <b>36.</b> If no conversion method is particularly recommended, then "11b" is added thereto to indicate that either conversion method will do.

0211FIG. <b>37</b> shows a procedure in which SD video is played back with a down conversion method determined in accordance with the conversion designating information. This processing may be carried out by the player <b>100</b>, for example.

0212First, on receiving a viewer's command to start the playback operation, the player <b>100</b> determines whether or not the display device (e.g., a TV set) connected can display the HD video content to be presented without down-converting it (S1002). If the answer is YES, then the player <b>100</b> outputs the HD video as it is to the display device (S1005) to get the HD video presented on the display device.

0213On the other hand, if the HD video needs to be down-converted into SD video, then the player <b>100</b> acquires the conversion designating information (S1002) and determines by reference to the flag <b>DC_prohibit_flag</b> whether or not the down conversion is permitted (S1003). If the answer is NO, then the playback of this content is ended.

0214However, if the down conversion is permitted, then the player <b>100</b> identifies a recommended down conversion method by reference to the information <b>DC_PAL</b> that designates the recommended conversion method, and down-converts the HD video by that method (S1004). Then, the player <b>100</b> outputs the SD video, obtained by the down conversion, to the display device (S1005). Thereafter, until the playback interval of that content has been played back fully, the player <b>100</b> continues performing the down conversion processing and outputting and playing the SD video.

0215Optionally, while presenting the information about the recommended conversion method on an on-screen display, the player <b>100</b> may pick a down conversion method in accordance with the viewer's command.

0216In the preferred embodiment described above, only HD grade subtitles or still picture data is stored. Alternatively, SD grade subtitles or still picture data may be separately stored for future down conversion such that the player or the user can make a selection at the time of down conversion. Also, although only the HD grade subtitles or still picture data is supposed to be stored, the subtitles to be synthesized during the down conversion may be generated with a vector image font such as True Type in order to guarantee sufficient video quality at the time of the down conversion.

0217Also, in the preferred embodiment described above, the method of defining the subtitle display area has not been described in detail. Optionally, a number of different subtitle display areas may be separately defined with the possible down conversion taken into account. Examples of those areas include an area for HD subtitles, an area for down-converted NTSC-compliant letterbox subtitles, an area for down-converted NTSC-compliant panned and scanned subtitles, an area for down-converted PAL-compliant letterbox subtitles, and an area for down-converted PAL-compliant panned and scanned subtitles. Alternatively, the display area may be changed according to the user's preference.

0218In the preferred embodiment described above, while a down conversion is carried out as the field doubling conversion, a particular field is supposed to be presented again as it is. However, to reduce the degree of video blurring during the down conversion, an intermediate field picture may be generated from the previous and next pictures and may be inserted to perform the field doubling conversion. In the example illustrated in FIG. <b>34</b>, for instance, the second picture <b>P5t</b> to be presented again in 50 Hz-FD (PAL) on the third row may be generated from the two pictures <b>P5</b> and <b>P6</b> in 24 Hz (film).

0219Furthermore, in the preferred embodiment described above, the HD grade video data and subtitle data are supposed to be converted into SD grade video data. However, even when SD grade video is converted into HD grade video and presented on an HDTV, HD grade data, in which the video and subtitle are synthesized together, can be obtained by performing processing similar to that described above. Thus, the present invention can be used to convert video and subtitle data with a certain resolution into synthetic video data with a different resolution from the former resolution.

0220The present invention has been described as being applied to a model based on a T-STD for an MPEG-TS. However, this is just an example, and the present invention is equally applicable for use to process an MPEG-PS or any other video/audio multiplexed stream.

<b>INDUSTRIAL APPLICABILITY</b>

0221According to the present invention, in down-converting HD video into SD video and viewing the SD video, a best down conversion method for the given content may be selected either automatically or according to the user's preference.

0222Also, an SD grade subtitle, having the same contents as the counterpart appearing on an HD grade video picture, is associated with an SD grade video picture, thereby generating and outputting synthetic video. As a result, SD video with quite the same contents as HD video can be presented. Furthermore, by defining a synchronous presentation model for a subtitle (i.e., a still picture) and a moving picture, compatibility is guaranteed among various types of equipment.