Storage medium recording text-based subtitle stream, reproducing apparatus and reproducing method for reproducing text-based subtitle stream recorded on the storage medium

A storage medium for recording a text-based subtitle stream, and a reproduction device and reproduction method for reproducing the text-based subtitle stream recorded on the storage medium

The present invention relates to the reproduction of a multimedia image, and more particularly to a storage medium for recording a multimedia image stream and a text subtitle stream, and the reproduction of the text subtitle stream recorded on the storage medium. Device and regeneration method.

In order to provide high-density multimedia images, video streams, audio streams, playback graphics streams that provide subtitles, and interactive graphics streams that provide buttons and menus to interact with users are multiplexed to a main stream Stream and record on the storage medium, where this main stream is also known as the audio visual "AV" data stream. In particular, the graphics stream that provides subtitles also provides bitmap images in order to display subtitles or titles on the images.

In addition to its large size, the bitmap title data will have problems in the production of subtitles or title data, and it will be quite difficult to edit the created title data. This is because the title data is multiplexed with other streaming data (such as video, audio, and interactive graphics streams). Furthermore, another problem is that the output style of the title data cannot be changed in various ways, that is, the output style of one title cannot be changed to another output style.

The object of the present invention is to provide a storage medium for recording a text-based subtitle stream, and a reproduction device and method for reproducing text-based subtitle data recorded on the storage medium.

The object of the present invention is to provide a device for reproducing data from a storage medium storing image data and text subtitles, which displays a title on an image based on the image data. The device includes: a video decoder, which is used for Decode image data; and a subtitle decoder, which is used to convert playback information into bitmap images according to style information, and control the output of converted playback information to synchronize with decoded image data, where text-based data includes display titles The playback information of the unit and the style information of the output style of the specified title.

The subtitle decoder decodes the text subtitles recorded separately from the image data and outputs the subtitle data, which overlays the subtitle data on the decoded image data. The pattern information and playback information are formed in units of packetized elementary streams (PESs), and the subtitle decoder parses and processes the pattern information and playback information in the units of PESs.

The style information is formed by a PES and recorded in the front part of the subtitle data, and several playback information items will be recorded after the style information in units of PESs, and the subtitle decoder will apply a style information item to these playback information items.

In addition, the playback information includes text information indicating the content of the title and combination information that controls the output of the bitmap image obtained by converting the text information, and the subtitle decoder will perform the conversion when outputting the converted text information by referring to the combination information. Control this time.

The playback information will specify one or more window areas, where the window area is the area where the title is output on the screen, and the subtitle decoder will output the converted text information in one or more windows at the same time.

The output start time and output end time of the playback information in the combined information are defined as time information on the global time axis, where the global time axis is used in the playlist, and the playlist is the reproduction unit of the video data, and the subtitles are decoded The converter synchronizes the output of the converted text information with the output of the decoded image data by referring to the output start time and output end time.

If the output end time of the currently reproduced play information item is the same as the output start time of the next play information item, the subtitle decoder will continuously reproduce the two play information items.

If the next playback information does not require continuous reproduction, the subtitle decoder will reset the interval buffer between the output start time and the output end time, and if continuous reproduction is required, the interval buffer will be reserved without resetting .

The style information is a collection of output styles, and the output style is predefined by the manufacturer of the storage medium and applied to the playback information, and the subtitle decoder will convert the playback information items recorded after the style information to a bitmap image according to the style information .

In addition, the text information in the playback information includes the text to be converted into a bitmap image and the inline style information to be applied to the only part of the text, and the subtitle decoder application applies the only part of the inline style information of the text To provide a function to enhance the text part through the style information predefined by the manufacturer.

The subtitle decoder uses the relative value of the predefined font information or the part of the predefined absolute value to the text included in the style information predefined by the manufacturer as the inline style information.

In addition, the style information further includes user-changeable style information, and after receiving from the user the selection information of a style among the changeable style information items, the subtitle decoder will use the generated pre-defined style information, and then apply In-line style information, and the user who finally applies the corresponding selection information can change the style information item to the text.

The subtitle decoder applies the relative value of the predefined font information to the text among the style information items predefined by the manufacturer, as the user can change the style information.

If the storage medium allows the predefined style information to be defined in the reproduction device in addition to the style information predefined by the manufacturer, the subtitle decoder will apply the predefined style information to the text.

In addition, the style information includes a set of color palettes to be applied to the playback information, and the subtitle decoder converts all playback information items after the style information into bitmap images according to the colors defined in the color palette.

The playback information further includes a set of color palettes and a color update flag, which are separate from the set of color palettes included in the style information, and if the color update flag is set to "1", the subtitle decoder The set of color palettes included in the playback information will be applied, and if the color update flag is set to "0", the subtitle decoder will apply the original set of color palettes included in the style information.

By setting the color update flag to "1" and gradually changing the transparency value of the color palette included in the continuous playback information item, the subtitle decoder implements the fade-in/fade-out effect, and when the fade-in/fade-out effect is completed , The subtitle decoder will reset the color look-up table (CLUT) according to the original set of color palettes included in the style information.

In addition, the style information includes area information indicating the position of the window area, which is used for converted playback information to be output on the image, and font information for converting the playback information into a bitmap image, and subtitle decoding The device converts the converted playback information into a bitmap image by using regional information and font information.

The font information includes the output start position, output direction, sorting, line spacing, font identification, font size, or color of at least one converted playback information, and the subtitle decoder will convert the playback information into a bitmap image based on the font information .

The subtitle decoder refers to the instruction information on the font file as the font identifier. The font file is included in the clip information file, which is the attribute information of the recording unit storing the image data.

In addition, the subtitle decoder buffers the subtitle data and the font file referenced by the subtitle decoder before reproducing the video data.

In addition, if several subtitle data items supporting several languages are recorded on the storage medium, the subtitle decoder will receive the selection information of the expected language from the user, and reproduce the subtitle data corresponding to the selected information among the subtitle data items.

Another object of the present invention is to provide a method for reproducing data from a storage medium storing image data and text subtitles, which displays the title on the image based on the image data. The method includes: decoding the image data; reading style information and Play information; convert the play information into bitmap images according to the style information; and control the output of the converted play information to synchronize with the decoded image data. The textual data includes playback information indicating the unit displaying the title and style information specifying the output style of the title.

Another object of the present invention is to provide a storage medium, which stores: image data; and text subtitle data, which is used to display the title on the image according to the image data, wherein the subtitle data includes: a style information, which is used To indicate the output style of the title; and several playback information items, which are the unit for displaying the title, and the subtitle data is separated from the video data and recorded separately.

Other objects and advantages of the present invention will be described in detail below and learned through the embodiments of the present invention.

In order to make the above and other objects, features, and advantages of the present invention more comprehensible, a preferred embodiment will be described in detail below in conjunction with the accompanying drawings.

Please refer to FIG. 1, according to an exemplary embodiment of the present invention, a storage medium (such as the medium 230 shown in FIG. 2) is constructed in a multi-layer manner to manage a multimedia data structure 100 in which a multimedia image stream is recorded. The multimedia data structure 100 includes a clip 110, a playlist 120, a movie object 130, and a table of contents 140. The clip 110 is the recording unit of the multimedia image, the playlist 120 is the reproduction unit of the multimedia image, and the movie object 130 includes a device for reproducing the multimedia image. The navigation command and the table of contents 140 are used to specify the movie object to be reproduced first and the title of the movie object 130.

The clip 110 will be implemented as an object, which includes the clip AV stream for the audio-visual (AV) data stream of the high-quality movie and the clip information 114 corresponding to the AV data stream. For example, the AV data stream can be compressed according to a standard such as Motion Picture Experts Group (MPEG). However, this clip 110 does not need to compress the AV data stream 112 for the purpose of the present invention. In addition, the clip information 114 includes the audio/video attributes of the AV data stream 112, the entry point map, etc., wherein the information about the location of the random access entry point is recorded in the entry point map in units of pre-defined magnetic regions.

The playlist 120 is a collection of playback intervals of these clips 110, and each playback interval is regarded as a play item 122. The movie object 130 is composed of navigation programs, and these navigation programs start the reproduction of the playlist 120, exchange between the movie objects 130, or manage the reproduction of the playlist 120 according to the needs of the user.

The table of contents 140 is the top table of the storage medium. It is used to define several titles and menus. The table of contents 140 includes the starting position information of all the titles and menus so that it can be reproduced through user operations (such as title search or Menu call) the selected title and menu. The table of contents 140 also includes the title and the start position information of the menu that are automatically reproduced for the first time when the storage medium is placed in the reproducing device.

In these projects, the data structure of the clip AV stream of the compression-encoded multimedia video will be described in detail in conjunction with FIG. 2. FIG. 2 is a schematic diagram illustrating an exemplary data structure of the AV data stream 210 and the text subtitle stream 220 of FIG. 1 according to an embodiment of the present invention.

Please refer to FIG. 2, in order to solve the above-mentioned problem with bitmap title data, the text subtitle stream 220 is provided separately from the clip AV data stream 210 recorded on the storage medium 230 according to the embodiment of the present invention. , Such as a digital versatile disc (DVD). The AV data stream 210 includes a video stream 202, an audio stream 204, a playback graphics stream 206 for providing subtitle data, and an interactive graphics stream 208 for providing buttons and menus for interaction with the user. The animation main stream is multiplexed and recorded in the storage medium 230, where the animation main stream is a well-known audio-visual (AV) data stream.

According to the embodiment of the present invention, the textual subtitle data 220 represents the subtitle and title data used to provide multimedia images to be recorded in the storage medium 230, and is implemented using a markup language, such as Extensible Markup Language (XML). ). However, the subtitles and titles of the multimedia images are provided using binary data. After that, the text subtitle data 220 that uses binary data to provide the subtitles and titles of the multimedia images is simply regarded as a "text subtitle stream". The playback graphics stream 206 for providing subtitle data also provides bitmap subtitle data to display subtitles (or titles) on the screen.

Since the text subtitle stream 220 is recorded separately from the AV data stream 210 and will not be multiplexed with the AV data stream 210, the size of the text subtitle stream 220 is not limited to this. Therefore, subtitles and titles can be provided in several languages. Furthermore, the text-based subtitle stream 220 can be continuously reproduced and effectively edited without any difficulty.

After that, the text subtitle stream 220 is converted into a bitmap graphic image, and output to overlay a multimedia image on the screen. The process of converting the text subtitle data into an image bitmap image is regarded as rendering. The text subtitle stream 220 includes information required for rendering the title text.

The text subtitle stream 220 including rendering information will be described in detail below in conjunction with FIG. 3. FIG. 3 is a schematic diagram illustrating the data structure of the text subtitle stream 220 according to an embodiment of the present invention.

Referring to FIG. 3, the text subtitle stream 220 according to an embodiment of the present invention includes a dialog style unit (DSU) 310 and a number of dialog presentation units (DPU) 320 to 340. DSU 310 and DPU 320 to 340 can also be regarded as dialogue units. Each of the dialogue units 310 to 340 forming the text subtitle stream 220 is stored in the form of packetized elementary streams (PESs) or simply known PES packets 350. Similarly, the PES of the text subtitle stream 220 is recorded and transmitted in units of transport packets (TP) 362. The continuous TP can be regarded as a transport stream (TS).

However, as shown in FIG. 2, according to the embodiment of the present invention, the text subtitle stream 220 will not be multiplexed with the AV data stream 210 and separate TS will be recorded on the storage medium 230.

Referring to FIG. 3, a dialogue unit is recorded in a PES packet 350 included in the text subtitle stream 220. The text subtitle stream 220 includes a DSU 310 arranged at the front and a number of DPUs 320 to 340 connected behind the DSU 310. The DSU 310 includes information describing the output style of the dialogue in the title displayed on the screen, where the multimedia image is reproduced on this screen. Meanwhile, several DPUs 320 to 340 include text information items on the dialog content to be displayed and information on individual output items.

FIG. 4 is a schematic diagram illustrating a text subtitle stream 220 having the data structure of FIG. 3 according to an embodiment of the present invention.

Please refer to FIG. 4, the text subtitle stream 220 includes a DSU 410 and a plurality of DPUs 420.

In the exemplary embodiment of the present invention, several DPUs are defined as num_of_dialog_presentation_units. However, several DPUs will not be specified separately. The example case is to use a syntax like while(processed_length<end_of_life).

DSU information and DPU junction structure will be described in detail with FIG. FIG. 5 is a schematic diagram illustrating the dialogue style unit in FIG. 3 according to an embodiment of the present invention.

Referring to FIG. 5, a set dialog_styleset() 510 of dialog style information items is defined in the DSU 310, in which the output style information items of the dialog to be displayed as a title are collected. The DSU 310 includes information about the position of the area where the dialog is displayed in the title, information required for rendering the dialog, information about the style that the user can control, and so on. The details will be explained below.

FIG. 6 is a schematic diagram illustrating an exemplary data structure of a dialog style unit (DSU) according to an embodiment of the present invention.

Please refer to FIG. 6, the DSU 310 includes a palette collection 610 and a regional style collection 620. The palette set 610 is a set of several color palettes, which are used to define the colors used in the title. The color combination and color information (such as transparency) included in the palette set 610 can be applied to all the DPUs arranged behind the DSU.

A region style collection (region style collection) 620 is a collection of output style information of individual dialogs forming a title. Each area style includes area information 622 indicating the position of the dialogue displayed on the screen, text style information 624 indicating the output style to be applied to each dialogue text, and a user changeable style collection indicating the style. ) 626, where the user can arbitrarily change the style applied to each dialogue text.

FIG. 7 is a schematic diagram illustrating an exemplary data structure of a dialogue style unit according to another embodiment of the invention.

Please refer to FIG. 7. The difference from FIG. 6 is that the palette set 610 is not included. That is, the color palette set is not defined in the DSU 310, but the palette set 610 is defined in the DPU described in FIGS. 12A and 12B. The data structure of each area style 710 is the same as the data structure described in FIG. 6.

FIG. 8 is a schematic diagram illustrating the example dialogue style unit in FIG. 6 or FIG. 7 according to an embodiment of the present invention.

Referring to FIGS. 8 and 6, DSU 310 includes palette sets 860 and 610 and several regional styles 820 and 620. As described above, the palette set 610 is a set of several color palettes, which are used to define the colors used in the title. The color combination and color information (such as transparency) included in the palette set 610 can be applied to all the DPUs arranged behind the DSU.

Meanwhile, each area style 820 and 620 includes area information 830 and 622, which indicate information on the window area, where the title in the window area is to be displayed on the screen, and the area information 830 and 622 include X, Y coordinates, width, and Information about the window area such as high background color, in which the title of the window area is to be displayed on the screen.

Similarly, each area style 820 and 620 includes text style information 840 and 624, which indicate the output style to be applied to each dialog text. That is to say, it includes the X and Y coordinates of the position where the dialog text is to be displayed in the above window area, the output direction (for example, from left to right or from top to bottom), sorting, line spacing, font recognition characters to be referred to, and font style (for example Bold or italic), font size and font color information, etc.

Furthermore, each of the regional styles 820 and 620 also includes user-changeable style sets 850 and 626, which indicate styles that the user can change arbitrarily. However, it is not necessary for the user to change the style sets 850 and 626. The user-changeable style sets 850 and 626 may include change information such as the position of the window area, the position of the text output, and the font size line interval among the text output style information items 840 and 624. Each change information item can be expressed as a relative increase or decrease value of related information on the output styles 840 and 625 to be applied to each dialogue text.

In summary, there are three types of style-related information: the style information (region_style) 620 defined in the regional styles 820 and 620, the inline style information (inline_style) 1510 (explained later) that is used to enhance the title, and the user The style information (user_changeable_style) 850 can be changed, and the sequence of applying this writing information item is as follows: 1) Basically, the regional style information 620 defined in the regional style is applied.

2) If the in-line style information is applied, the in-line style information 1510 is applied to cover the application part of the regional style information and strengthen the part of the title text.

3) If there is a user who can change the style information 850, then this information will be applied last. It is not necessary for the user to change the presentation of the style information.

Meanwhile, among the text style information items 840 and 624 to be applied to each dialogue text, the font file information referenced by the recognized word of the font (font_id) 842 can be defined as follows.

FIG. 9A is a schematic diagram illustrating an example clip information file 910 including several font sets referenced by the font information 842 in FIG. 8 according to an embodiment of the present invention.

9A, FIG. 8, FIG. 2 and FIG. 1, according to the present invention, StreamCodingInfo () 930 includes various information recorded on the stream of the storage medium, where StreamCodingInfo () 930 refers to the information included in the clip information file 910 And the stream encoding information structure in 110. That is, the information on the video stream 202, audio stream, playback graphics stream, interactive graphics stream, and text subtitle stream is included. In particular, it includes information (textST_language_code) 932 on the language of the title to be displayed, which is related to the text subtitle stream 220. Similarly, the font name 936 and the file name 938 of the file storing font information can also be defined, which correspond to the font_id842 and 934 indicating the identification characters of the font to be referred to and displayed in FIG. 8. The method for finding the font file of the recognized character of the font to be referred to and defined here will be described in detail in conjunction with FIG. 10.

FIG. 9B is a schematic diagram illustrating an example clip information file 940 including several font sets referenced by the font information 842 of FIG. 8 according to another embodiment of the present invention.

Referring to FIG. 9B, the structure ClipInfo() is defined in the clip information files 910 and 110. In this structure, several font sets referred to by the font information 842 in FIG. 8 are defined. That is, specify the font file name 952 corresponding to font_id 842 in detail, where font_id 842 indicates the identification of the font to be referred to and displayed in FIG. 8. The method for finding the font file of the recognized character of the font to be referred to and defined here will be described in detail below.

10 is a schematic diagram showing the positions of several font files referenced by font file names 938 and 952 in FIGS. 9A and 9B.

Please refer to FIG. 10, which shows the directory structure of files recorded on multimedia according to an embodiment of the present invention. In particular, due to the use of a directory structure, it is easy to find the location of the font file, such as 11111.font 1010 or 99999.font 1020 stored in the auxiliary data (AUXDATA) directory.

Meanwhile, the structure of the DPU forming the dialogue unit will be described in detail in conjunction with FIG. 11.

FIG. 11 is a schematic diagram illustrating an exemplary data structure of the DPU 320 in FIG. 3 according to another embodiment of the present invention.

Please refer to Figures 11 and 3, the DPU 320 that includes the text information to be output of the dialog content and the information on the display time includes the time information 1110 indicating the time for outputting the dialog on the screen, and the color palette to be referenced specifically The palette reference information 1120 and the dialogue area information 1130 for the dialogue to be output on the screen. In particular, the dialogue area information 1130 for the dialogue to be output on the screen includes style reference information 1132 indicating the output style to be applied to the dialogue and dialogue text information 1134 indicating the text actually output on the screen. In this case, it is assumed that the color palette set indicated by the palette reference information 1120 is defined in the DSU (please refer to 610 in FIG. 6).

Meanwhile, FIG. 12A is a schematic diagram illustrating an exemplary data structure of the DPU 320 in FIG. 3 according to an embodiment of the present invention.

12A and 3, the DPU 320 includes time information 1210 indicating the time for outputting the dialogue on the screen, a palette set 1220 that defines the color palette set, and the dialogue for the dialogue to be output on the screen. Area information 1230. In this case, the palette set 1220 will not be defined in the DSU as shown in FIG. 11, but will be directly defined in the DPU 320.

Meanwhile, FIG. 12B is a schematic diagram illustrating an exemplary data structure of the DPU 320 in FIG. 3 according to an embodiment of the present invention.

12B, the DPU 320 includes time information 1250 indicating the time for outputting the dialogue on the screen, a color update flag 1260, a color palette set 1270 required when the color update flag is set to 1, and a user To output the information 1280 of the dialogue area to be dialogue on the screen. In this case, the color palette set 1270 is also defined in the DSU as shown in FIG. 11 and stored in the DPU 320. In particular, in order to indicate the use of continuous reproduction fade-in/fade-out, in addition to the basic palette defined in the DSU, the palette set 1270 used to indicate the fade-in/fade-out will be defined in the DPU 320 and the color update flag 1260 Will be set to 1. This will be explained in detail in conjunction with Figure 19.

FIG. 13 is a schematic diagram showing the DPU 320 in FIG. 11 to FIG. 12B according to an embodiment of the present invention.

Referring to FIGS. 13, 11, 12A, and 12B, the DPU includes dialogue start time information (dialog_strat_PTS) and dialogue end time information (dialog_end_PTS) 1310 as time information 1110 indicating the time for the dialogue to be output on the screen. Similarly, the dialog palette identifier (dialog_palette_id) is included as palette reference information 1120. In the case of FIG. 12A, the color palette set 1220 may be included in place of the palette reference information 1120. The dialog text information (region_subtitle) 1334 is included as the dialog region information 1230 for the dialog to be output, and in order to specify the output style applied to it, the region style identifier (region_style_id) 1332 is also included. The example in FIG. 13 is only an embodiment of the DPU, and the DPU having the data structure shown in FIGS. 11 to 12B can be modified in various ways to be implemented.

FIG. 14 is a schematic diagram illustrating an example data structure of the dialog text information (region_subtitle) in FIG. 13.

Referring to FIG. 14, the dialogue text information (1134 in FIG. 11, 1234 in FIG. 12A, 1284 in FIG. 12B, and 1334 in FIG. 13) includes in-line information 1410 and dialogue text 1420 as output styles to enhance the dialogue.

FIG. 15 is a schematic diagram illustrating the dialog text information 1334 of FIG. 13 according to an embodiment of the present invention. As shown in FIG. 15, the dialogue text information 1334 is implemented by the inline style information (inline_style) 1510 and the dialogue text (text_string) 1520. Likewise, it is preferable that the information indicating the end of the in-line style is included in the embodiment of FIG. 15. Unless the end part of the in-line style is defined, the specified in-line style may be applied later, which will be the opposite of the manufacturer's setting.

Meanwhile, FIG. 16 is a schematic diagram for explaining the limitation of continuously reproducing dialog presentation units (DPUs).

Please refer to FIG. 16 and FIG. 13, when it is necessary to continuously regenerate the above-mentioned DPUs, the following restrictions are required.

1) When the dialog object starts to be output on the graphics plane (graphic plane, GP), the dialog start time information (dialog_start_PTS) 1310 defined in the DPU will indicate a time, and the graphics plane (graphic plane, GP) will cooperate in the following Figure 17 illustrates in detail.

2) The dialog start time information (dialog_start_PTS) 1310 defined in the DPU indicates a time to reset the text-based subtitle decoder for processing text-based subtitles. The text-based subtitle decoder will be described in detail in conjunction with FIG. 17 below.

3) When it is necessary to continuously regenerate the aforementioned DPUs, the dialogue end time information (dialog_end_PTS) of the current DPU should be the same as the dialogue start time information (dialog_start_PTS) of the next continuously regenerated DPU. That is, in FIG. 16, in order to continuously reproduce DPU#2 and DPU#3, the dialogue end time information included in DPU#2 should be the same as the dialogue start time information included in DPU#3.

Meanwhile, it is best that the DSU according to the present invention satisfies the following restrictions.

1) The text subtitle stream 220 includes a DSU.

2) Several user-changeable style information items (user_control_style) included in all region styles (region_style) should be the same.

Meanwhile, it is best that the DPU according to the present invention satisfies the following restrictions.

1) The window area for at least two titles should be defined.

The structure of an exemplary reproduction device based on the data structure of the text subtitle stream 220 recorded on the storage medium according to the embodiment of the present invention will be described below with reference to FIG. 17.

FIG. 17 is a schematic diagram illustrating an exemplary reproduction device for text subtitle streaming according to an embodiment of the present invention.

Referring to FIG. 17, the playback device 1700 (the so-called recording and playback device) includes a buffer unit and a text subtitle decoder 1730. The buffer unit includes a font preloading buffer (font preloading buffer, FPB) 1712 for storing font files and a subtitle preloading buffer (SPB) 1710 for storing text subtitle files, and the text subtitles The decoder 1730 uses a graphics plane (GP) 1750 and a color look-up table (CLUT) 1760 to decode and reproduce the text subtitle stream previously recorded on the storage medium as output.

In particular, the subtitle preloading buffer (SPB) 1710 will preload the text subtitle data stream 220 and the font preloading buffer (FPB) 1712 will preload the font information.

The text-based subtitle decoder 1730 includes a text-based screen processor 1732, a dialog composition buffer (DCB) 1734, a dialog buffer (DB) 1736, a text-based subtitle conversion (rendering) device 1738, and a dialog player A controller 1740 and a bitmap object buffer (BOB) 1742.

The text-based screen processor 1732 receives the text-based subtitle data stream 220 from the subtitle preloading buffer (SPB) 1710, converts the format of the information included in the DSU and the dialogue output time information included in the DPU to The dialog composition buffer (DCB) 1734 and converts the dialog text information included in the DPU to the dialog buffer (DB) 1736.

The dialogue playback controller 1740 controls the text-based subtitle conversion (rendering) 1738 by using the style about the information included in the dialogue composition buffer (DCB) 1734, and controls the user by using the dialogue output time information. At the time of rendering the bitmap image in the bitmap object buffer (OBO) 1742, it is output to the graphics plane (GP) 1750.

According to the control of the dialogue playback controller 1740, the text subtitle conversion (rendering) device 1738 converts (that is, performs the conversion (rendering)) of the dialogue text information into a bitmap image, which is applied to the font preload buffer ( font preloading buffer (FPB) 1712 preloads font information items corresponding to the dialog text information stored in the dialog buffer (DB) 1736 to the dialog text information. The rendered bitmap image is stored in the bitmap object buffer (OBO) 1742 and output to the graphics plane (GP) 1750 according to the control of the dialog player controller 1740. At this time, the color specified in the DSU is applied by referring to the color look-up table (CLUT) 1760.

The information defined by the manufacturer in the DSU can be used as style-related information applied to the dialog text, and style-related information predefined by the user can also be applied. The reproducing device 1700 shown in FIG. 17 applies the user-defined style information prior to the style-related information defined by the manufacturer.

As shown in Figure 8, the region style information (region_style) defined by the manufacturer in the DSU is basically applied to the style-related information applied to the dialog text, and if the in-line style information (inine_style) is included in the DPU, where The DPU includes the dialog text to which the regional style information is applied, and the inline style information (inline_style) is applied to the corresponding part. Similarly, if the manufacturer additionally defines user-changeable styles in the DSU and one of the user-defined user-changeable styles is selected, the regional style or in-line style will be applied, and then the user-changeable style will be applied finally. Change information. Similarly, as shown in FIG. 15, it is preferable that the information indicating the end of applying the in-line style is included in the content of the in-line style.

Furthermore, the manufacturer can specify whether to use the style-related information defined in the reproducing device itself, which is separate from the style-related information defined by the manufacturer and recorded on the storage medium.

FIG. 18 is a schematic diagram illustrating the pre-loading procedure of the text-based subtitle stream 220 in an exemplary reproduction device 1700 (for example, as shown in FIG. 17) according to an embodiment of the present invention.

Referring to FIG. 18, the text subtitle stream 220 shown in FIG. 2 is defined in the sub-path of the above-mentioned playlist. In the subpath, several text-based subtitle streams 220 supporting several languages can be defined. Similarly, the font file applied to the text-based subtitles can be defined in the clip information file 910 or 940 as shown in FIGS. 9A and 9B. A maximum of 255 text-based subtitle streams 220 that can be included in one storage medium can be defined in each playlist. Similarly, it is also possible to define up to 255 font files included in a storage medium. However, to ensure uninterrupted playback, the size of the text subtitle stream 220 should be less than or equal to the size of the preload buffer 1710 of the playback device 1700 (for example, as shown in FIG. 17).

FIG. 19 is a schematic diagram illustrating a DPU regeneration procedure in an exemplary regeneration device according to an embodiment of the present invention.

Please refer to Figure 19, Figure 13 and Figure 17 to show the process of regenerating the DPU. The playback controller 1740 controls the time for the rendering dialog to be output on the graphics plane (GP) 1750 by using the dialog start time information that specifies the output time 1310 of the dialog included in the DPU ( dialog_start_PTS) and dialog end time information (dialog_end_PTS). At this point, when the conversion of the converted (rendering) dialog bitmap image stored in the bitmap object buffer (BOB) 1742 to the graphics plane (GP) 1750 is completed, the bitmap object A bitmap object buffer (BOB) 1742 is included in the text subtitle decoder 1730, and the dialogue start time information specifies a time. That is, if the dialogue start time is defined in the DPU, the bitmap information required to construct the dialogue will be ready to be used after the conversion of the information to the graphics plane (GP) 1750 is completed. Similarly, when the regeneration of the DPU is completed, the dialogue end time information will specify a time. At this time, the text subtitle decoder 1730 and the graphics plane (graphics plane, GP) 1750 will be reset. The best is that regardless of whether it is continuously reproduced in the text subtitle decoder 1730 buffer (such as bitmap object buffer (BOB) 1742), it will be between the start time and the end time of the DPU. Was reset.

However, when several DPUs are required for continuous reproduction, the text subtitle decoder 1730 and graphics plane (GP) 1750 will not be reset and are stored in each buffer (such as a dialog composition buffer). , DCB) 1734, dialog buffer (DB) 1736 and bitmap object buffer (OBO) 1742) will be retained. That is, when the dialogue end time information of the currently reproduced DPU is the same as the dialogue start time information of the subsequent continuously reproduced DPU, the content of each buffer is retained without resetting.

In particular, there is a fade-in/fade-out effect as an example of continuous reproduction using several DPUs. The fade-in/fade-out effect can be implemented by changing the color look-up table (CLUT) 1760 of the bitmap object, where the bitmap object is converted to the graphics plane (GP) 1750. That is, the first DPU includes combination information, such as color, style, and output time, and subsequent consecutive DPUs have the same combination information as the first DPU, but only update the color palette information. In this case, the fade-in/fade-out effect is implemented by gradually changing the transparency (from 0% to 100%) in the color information item.

In particular, when the data structure of the DPU shown in FIG. 12B is used, the fade-in/fade-out effect can be effectively implemented using the color update flag 1260. That is, if the dialog playback controller 1740 checks and confirms that the color update flag 1260 included in the DPU is set to "0", that is, if the fade-in/fade-out effect is generally not required, it will basically The color information included in the DSU shown in FIG. 6 is used. However, if the dialogue playback controller 1740 checks and confirms that the color update flag 1260 included in the DPU is set to "1", that is, if a fade-in/fade-out effect is required, it will use the color information 1270 (instead of Figure 6). The color information in the DSU shown) to implement the fade-in/fade-out effect. At this time, simply implement the fade-in/fade-out effect by adjusting the transparency of the color information 1270 included in the DPU.

After displaying the fade-in/fade-out effect, it is best to update the color look-up table (CLUT) 1760 to the original color information included in the DSU. This is because unless the color look-up table (CLUT) 1760 is updated, the color information once specified can be applied continuously, contrary to the expectations of the manufacturer.

20 is a schematic diagram for explaining the synchronization and output process of text subtitle stream and animation data in an exemplary playback device according to an embodiment of the present invention.

Please refer to Fig. 20, the dialogue start time information and dialogue end time information of the DPU included in the text subtitle data stream 220 should be defined as the time points on the global time axis used in the playlist in order to communicate with the AV data of the multimedia image. The output time of the stream is synchronized. Therefore, the discontinuity between the system time clock (STC) of the AV data stream and the dialog output time (PTS) of the text subtitle data stream 220 can be avoided.

FIG. 21 is a schematic diagram for explaining a process of outputting a text subtitle stream to a screen in an exemplary playback device according to an embodiment of the present invention.

Please refer to FIG. 21, which shows the process of applying the rendering information 2101 including style-related information, the process of converting text information 2140 into a bitmap image 2106, and the output position information included in the combined information 2108 ( For example, region_horizontal_position and region_vertical_position) output the converted bitmap image to the corresponding position on the graphics plane (GP) 1750.

The rendering information 2102 presents style information, such as the width and height of the area, the color of the foreground, the color of the background, the name of the font, and the size of the font.

As described above, the combination information 2108 indicates the start time and end time of the playback, the horizontal and vertical position information of the window area, and so on. The title in the window area is output on the graphics plane (GP) 1750.

FIG. 22 is a schematic diagram illustrating the process of rendering the text subtitle data stream 220 in the playback device 1700 (as shown in FIG. 17) according to an embodiment of the present invention.

Please refer to Figures 22, 21 and 8, the window area specified by using region_horizontal_position, region_vertical_position, region_width, and region_height is designated as a region where the title is displayed on the graphics plane (GP) 1750, where region_horizontal_position, region_vertical_position, region_width And region_height is used to define the position information 830 of the window area of the title of the DSU. The bitmap image of the rendered dialog is displayed from the starting point position specified by region_horizontal_position and region_vertical_position, where region_horizontal_position and region_vertical_position are the output position 840 of the dialog in the window area.

Meanwhile, the reproducing device according to the present invention stores the style information (style_id) selected by the user in the system temporary storage area. FIG. 23 is a schematic diagram illustrating an exemplary status register configured in an exemplary reproduction device for reproducing a text subtitle data stream according to an embodiment of the present invention.

Please refer to Figure 23, the status register (playing status register, hereinafter referred to as PSRs) stores the style information selected by the user in the 12th register (the style 2310 has been selected). Therefore, for example, if the user presses the style information change button even after performing a menu call or other operation on the playback device 1700 (as shown in FIG. 17), the style information previously selected by the user will first be applied by referring to the PSR 12. The register for storing information will be changed.

The method of reproducing the text subtitle data stream 220 according to the storage medium for recording the text subtitle data stream 220 and the reproduction device for reproducing the text subtitle data stream 220 will be described as follows with reference to FIG. 24. FIG. 24 is a flowchart of a method for reproducing text subtitle data stream 220 according to an embodiment of the present invention.

In step 2410, the text subtitle data stream 220 including DSU information and DPU information is read from the storage medium 230 (as shown in FIG. 2), and in step 2420, according to the rendering included in the DSU information The information converts the title text included in the DPU information into a bitmap image. In step 2430, the converted bitmap image is output on the screen according to time information and location information, where the time information and location information are combined information included in the DPU information.

As described above, the present invention provides a storage medium that separates the text subtitle data stream from the video data. The present invention also provides a reproducing device and a method for reproducing the text subtitle data stream, so that the production of subtitle data and the editing of the created subtitle data can be easier. At the same time, because the number of subtitle data items is not limited, titles in several languages can be provided.

In addition, since the subtitle data is formed by one style information item and several playback information items, the output style applied to all the playback data can be defined in advance and can be changed in various ways, and the in-line information to enhance the title part can also be defined And the user can change the style.

Furthermore, by using several adjacent playback information items, continuous reproduction of titles can be turned on and can be used to implement fade-in/fade-out effects.

The present invention can be implemented as a program code on a computer-readable recording medium, which can be read by a general computer. Computer-readable recording media include all kinds of recording media that can store computer-readable data. Computer-readable recording media include magnetic storage media (such as ROM, floppy disk, hard disk), optical storage media (such as CD-ROM, DVD), and Carrier wave (that is, transmission through the Internet). At the same time, the computer-readable recording medium can be shared in the computer system through the network and the computer-readable code can be stored and executed in a distributed manner.

Although the present invention has been disclosed as above in the preferred embodiment, it is not intended to limit the present invention. Anyone familiar with the art can make some changes and modifications without departing from the spirit and scope of the present invention. For example, any computer-readable media or data storage device can be used to record the text subtitle data and the AV data separately. In addition, the text subtitle data can be configured in different ways as shown in FIG. 3 and FIG. 4. Furthermore, the reproducing device of FIG. 17 may be implemented as a part of a recording device or a device that performs a single recording and/or reproducing function. Similarly, the CPU can be implemented as a chip with firmware or a general-purpose or special-purpose programmed computer to execute the method described in FIG. 24. Therefore, the scope of protection of the present invention is not limited to the disclosed embodiments, but shall be subject to what is defined by the attached patent application scope.

<p>100. . . Multimedia data structure</p><p>110. . . Edit</p><p>112. . . AV data streaming</p><p>114. . . Clip information</p><p>120. . . Playlist</p><p>122. . . Play item</p><p>130. . . Movie object</p><p>140. . . Table of Contents</p><p>202. . . Video streaming</p><p>204. . . Audio streaming</p><p>206. . . Play graphics stream</p><p>208. . . Interactive graphics streaming</p><p>210. . . AV data streaming</p><p>220. . . Text subtitle data</p><p>230. . . Storage media</p><p>310. . . Dialogue style unit (DSU)</p><p>320, 330, 340. . . Dialogue presentation units (DPU)</p><p>350. . . PES packet</p><p>362. . . Transport packets (transport packets, TP)</p><p>410. . . DSU</p><p>420. . . DPU</p><p>610. . . Palette collection</p><p>620. . . Regional style collection</p><p>622. . . Regional Information</p><p>624. . . Text style information</p><p>626. . . User can change the style collection</p><p>710. . . Area style</p><p>820. . . Area style</p><p>830. . . Regional Information</p><p>840. . . Text style information</p><p>850. . . User can change the style collection</p><p>860. . . Palette collection</p><p>910, 940. . . Clip Information File</p><p>1110. . . Time information</p><p>1120. . . Color palette reference information</p><p>1130. . . Dialogue area information</p><p>1132. . . Style reference information</p><p>1134. . . Dialog text information</p><p>1210. . . Time information</p><p>1220. . . Palette collection</p><p>1230. . . Dialogue area information</p><p>1232. . . Style reference information</p><p>1234. . . Dialog text information</p><p>1250. . . Time information</p><p>1260. . . Color update flag</p><p>1270. . . Color palette collection</p><p>1280. . . Dialogue area information</p><p>1282. . . Style reference information</p><p>1284. . . Dialog text information</p><p>1410. . . In-line style information</p><p>1420. . . Dialogue text</p><p>1700. . . Regeneration device</p><p>1710. . . Subtitle preloading buffer (SPB)</p><p>1712. . . Font preloading buffer (font preloading buffer, FPB)</p><p>1730. . . Text subtitle decoder</p><p>1732. . . Text screen processor</p><p>1734. . . Dialogue composition buffer (DCB)</p><p>1736. . . Dialog buffer (DB)</p><p>1738. . . Text subtitles converter (rendering)</p><p>1740. . . Dialogue playback controller</p><p>1742. . . Bitmap object buffer (BOB)</p><p>1750. . . Graphics plane (GP)</p><p>1760. . . Color look-up table (color look-up table, CLUT)</p>

FIG. 1 is a schematic diagram illustrating the structure of multimedia data recorded on a storage medium according to an embodiment of the present invention.

FIG. 2 is a schematic diagram illustrating an exemplary data structure of the clip AV stream and the text subtitle stream of FIG. 1 according to an embodiment of the present invention.

FIG. 3 is a schematic diagram illustrating the data structure of a text subtitle stream according to an embodiment of the present invention.

FIG. 4 is a schematic diagram illustrating a text subtitle stream having the data structure of FIG. 3 according to an embodiment of the present invention.

FIG. 5 is a schematic diagram illustrating the dialogue style unit in FIG. 3 according to an embodiment of the present invention.

FIG. 6 is a schematic diagram illustrating an exemplary data structure of a dialogue style unit according to an embodiment of the present invention.

FIG. 7 is a schematic diagram illustrating an exemplary data structure of a dialogue style unit according to another embodiment of the invention.

FIG. 8 is a schematic diagram illustrating the example dialogue style unit in FIG. 6 or FIG. 7 according to an embodiment of the present invention.

9A and 9B are schematic diagrams illustrating example clip information files including several font sets referenced by font information according to an embodiment of the present invention.

FIG. 10 is a schematic diagram showing the positions of several font files referenced by font file information (shown in FIGS. 9A and 9B).

FIG. 11 is a schematic diagram illustrating an exemplary data structure of the dialogue playing unit in FIG. 3 according to another embodiment of the present invention.

12A and 12B are schematic diagrams illustrating an exemplary data structure of the dialogue playing unit in FIG. 3 according to another embodiment of the present invention.

Fig. 13 is a schematic diagram illustrating the dialogue playing unit in Figs. 11 to 12B according to an embodiment of the present invention.

FIG. 14 is a schematic diagram for explaining an exemplary data structure of the dialog text information in FIG. 13.

FIG. 15 is a schematic diagram illustrating the dialog text information of FIG. 13 according to an embodiment of the present invention.

FIG. 16 is a schematic diagram for explaining the limitation of continuous dialogue presentation units (DPUs) in continuous reproduction.

FIG. 17 is a schematic diagram illustrating an exemplary reproduction device for text subtitle streaming according to an embodiment of the present invention.

FIG. 18 is a schematic diagram illustrating the pre-loading procedure of a text-based subtitle stream in an exemplary playback device according to an embodiment of the present invention.

FIG. 19 is a schematic diagram illustrating a playback procedure of a dialog presentation unit (DPU) in an exemplary playback device according to an embodiment of the present invention.

20 is a schematic diagram for explaining the synchronization and output process of text subtitle stream and animation data in an exemplary playback device according to an embodiment of the present invention.

FIG. 21 is a schematic diagram for explaining a process of outputting a text subtitle stream to a screen in an exemplary playback device according to an embodiment of the present invention.

FIG. 22 is a schematic diagram for explaining a process of rendering a text subtitle stream in an exemplary playback device according to an embodiment of the present invention.

FIG. 23 is a schematic diagram illustrating an example status register configured in an example reproduction device for reproducing a text subtitle stream according to an embodiment of the present invention.

Fig. 24 is a flowchart of a method for reproducing a text subtitle stream according to an embodiment of the present invention.