Playback apparatus, playback method, recording medium, recording apparatus, recording method for recording a video stream and graphics with window information over graphics display

ES 2 536 680 T3

DESCRIPTION

Reproduction apparatus, reproduction method, recording medium, recording apparatus, recording method for recording a video stream and graphics having decoding time indication with window information about graphics display

Technique field

The present invention relates to a recording medium such as a BD-ROM, and a reproduction apparatus and, more specifically, to a subtitling technique by means of the reproduction of a digital stream that is constituted by multiplexing a stream of video and graphics stream.

Background of the technique

The subtitling that is obtained by modeling streams of graphics is an important technique to allow people in different linguistic areas to appreciate a film produced in a language other than their native languages. An example of a conventional subtitling technique is a memory allocation scheme for a Pixel Buffer based on the ETSI EN 300 743 standard set out by the European Telecommunications Standards Institute (ETSI). The Pixel Buffer is a memory for temporarily storing decompressed graphics, and a playback apparatus writes the graphics in the Pixel Buffer to a display memory called a Graphics Plane, and thus the graphics are displayed. In the memory allocation scheme, a definition of a Region is included in the Pixel Buffer, and a part of the decompressed graphics that corresponds to the Region is written to the Graphics Plane. For example, when a subtitle "Goodbye ..." is contained in the Pixel Buffer and a position and a size of the Region are defined in order to include an "A" part, then the "A" part is writes on the Graphics Plane and is displayed on the screen. Similarly, when the position and size of the Region are defined in order to include an Adi ”part, then the Adi” part is displayed on the screen.

By repeating the definition of the Region and writing on the Graphics Plane, the subtitle "Goodbye ..." is gradually displayed on the screen, that is, first "A", then "Adi" , then "Goodbye" and, finally, the entire subtitle "Goodbye ..." is displayed. By rendering a subtitle in such a way, it is possible to obtain an entrance wipe effect.

The ETSI EN 300 743 standard, however, does not consider at all to guarantee synchronization between a graphics display and an image display when a load for writing on the Graphics Plane is high. Graphics that are written to the Graphics Plane are not compressed and consequently the load for writing to the Graphics Plane increases as a graphics resolution becomes higher. A size of the graphics to be written to the Graphics Plane is up to 2 Mbytes when rendering the graphics at 1920 x 1080 resolution, which is a proposed normalized resolution for a BD-ROM, and a higher bandwidth. High for a graphics data transfer from the Pixel Buffer to the Graphics Plane is required in order to render graphics as large as 2 Mbytes synchronously with the image display. However, demanding a high bandwidth for data transfer for writing the graphics on the Graphics Plane hampers an attempt to reduce costs in manufacturing the reproducing apparatus. It is possible to reduce the bandwidth required in writing on the Graphics Plane by making the playback apparatus always perform a "reasonable writing", where only a difference from a previous display is written on the Graphics Plane. . However, requiring that the playback apparatus always perform "reasonable writing" restricts the software applicable to the playback apparatus.

As described above, the high load for writing to the Graphics Plane demands that the playback apparatus operate in the high bandwidth or perform reasonable writing, and as a result, restricts the product development of the devices. reproduction apparatus.

Disclosure of the invention

An object of the present invention is to provide a recording medium with which graphics can be updated synchronously with an image display even when an amount of data to be written on a Graphics Plane is large, an apparatus and a method of reproduction, and a respective recording apparatus and method.

Brief description of the drawings

Figure 1 illustrates an example of use of a recording medium according to the present invention.

Figure 2 illustrates a structure of a BD-ROM.

Fig. 3 is a diagram schematically illustrating a structure of an AVClip.

Figure 4A illustrates a structure of a presentation graphics stream.

ES 2 536 680 T3

Figure 4B illustrates a PES packet that is obtained after functional segments are converted.

Figure 5 illustrates a logical structure that is made up of various types of functional segments.

Fig. 6 illustrates a relationship between a subtitle display position and an Epoch.

Figure 7A illustrates a syntax for defining a Graphics Object in an Object Definition Segment (ODS).

Figure 7B illustrates a syntax for a Palette Definition Segment (PDS).

Figure 8A illustrates a syntax for a Window Definition Segment (WDS).

Figure 8B illustrates a syntax for a Presentation Composition Segment (PCS).

Figure 9 illustrates an example of a description of a Display Set for captioning.

Figure 10 illustrates an example of a description of the WDS and PCS on a DS1.

Figure 11 illustrates an example of a description of the PCS in a DS2.

Figure 12 illustrates an example of a description of the PCS in a DS3.

Figure 13 is an example of a description of a Display Set when performing a Cut of

Input / Output, illustrated along a timing diagram.

Figure 14 is an example of a description of a Display Set when Fade In / Fade Out, illustrated along a timing diagram.

Figure 15 is an example of a description of a Display Set when Scrolling, which is illustrated along a timing diagram.

Figure 16 is an example of a description of a Display Set when an Entry / Exit Wipe is performed, which is illustrated along a timing diagram.

Figure 17 is a diagram comparing two cases: a window has four Graphics Objects, and a window has two Graphics Objects.

Figure 18 illustrates an example of an algorithm for calculating a decoding duration.

Figure 19 is a flow chart of the algorithm of Figure 18.

Figures 20A and B are flow charts of the algorithm of Figure 18.

Figure 21A illustrates a case where each window has an Object Definition Segment.

Figures 21B and C are timing diagrams showing orders of the numbers referred to in Figure 18.

Figure 22A illustrates a case where each window has two Object Definition Segments.

Figures 22B and C are timing diagrams showing orders among the numbers referred to in Figure 18.

Figure 23A describes a case where each of two Windows includes an ODS.

Fig. 23B illustrates a case where a decoding period (2) is longer than a total of an erase period (1) and a write period (31).

Fig. 23C illustrates a case in which a total of the erase period (1) and the write period (31) is longer than the decode period (2).

Figure 24 illustrates deviations in time of the update that is described in an example in the present specification.

Figure 25A illustrates four Display Sets that are described in order to perform the update explained above.

Figure 25B is a timing diagram showing DTS and PTS settings of the functional segments that are included in the four Display Sets.

Figure 26 illustrates an internal structure of a reproduction apparatus according to the present invention.

Figure 27 illustrates sizes of the write rates Rx, Rc, and Rd, Graphics Plane 8, Coded Data Buffer 13, Object Buffer 15, and Composition Buffer 16.

FIG. 28 is a timing diagram illustrating pipeline processing by the playback apparatus.

FIG. 29 illustrates a timing diagram in a pipeline processing of a case in which the decoding of the ODS ends before the erasure of the Graphics Plane is completed.

Fig. 30 is a flow chart illustrating a process of a functional segment load operation. Figure 31 shows an example of multiplexing.

Figure 32 illustrates one way in which a DS10 is loaded into Coded Data Buffer 13.

Figure 33 illustrates the loading of a DS1, DS10, and DS20 in normal playback.

Figure 34 illustrates the loading of the DS1, DS10, and DS20 in normal playback as shown in Figure 33.

Figure 35 illustrates a flow chart showing a process that is performed by the Graphic Controller 17.

Figure 36 illustrates a flow chart showing the process that is performed by the Graphic Controller 17.

Figure 37 illustrates a flow chart showing the process that is performed by the Graphic Controller 17.

Fig. 38 illustrates a pipeline process of the playback apparatus based on the PTS of the PDS.

Fig. 39 is a diagram describing an importance of END in the duct process of the reproduction apparatus.

Fig. 40 illustrates an internal structure of the reproduction apparatus according to a second embodiment.

Figure 41 schematically illustrates an acquisition and writing operation on the Graphics Planes constituting a double buffer.

ES 2 536 680 T3

Fig. 42 is a flow chart illustrating the BD-ROM manufacturing process according to a third embodiment.

Best way to carry out the Invention

[First realization]

A first embodiment of a recording medium according to the present invention is explained hereinafter.

Figure 1 illustrates an example of use of the recording medium. In the drawing, a BD-ROM 100 is the recording medium according to the present invention. The BD-ROM 100 is used to provide motion picture data to a structured Home Theater System by means of a playback apparatus 200, a television 300, and a remote controller 400.

The recording medium according to the present invention is manufactured by an improvement in an application layer of a BD-ROM. Figure 2 illustrates a structure of the BD-ROM.

In the drawing, the BD-ROM is shown in a lower part of the drawing, and a track on the BD-ROM is shown above the BD-ROM. The track is actually in a spiral shape on the disc, but it is shown in a line in the drawing. The track includes an entry area, a volume area, and an exit area. The volume area in this drawing has a physical layer, a file system layer, and an application layer. In a top part of the drawing, an application format of the BD-ROM is illustrated using a directory structure. As illustrated in the drawing, the BD-ROM has a BDMV directory under the root directory, and the BDMV directory contains a file to store an AVClip with an M2TS extension (XXX.M2TS), a file to store administrative information for the AVClip with a FPIC extension (XXX.CLPI), and a file to define a logical Playlist (PL, Play List) for the AVClip with an MPLS extension (YYY.MPLS). By forming the above application format, it is possible to manufacture the recording medium according to the present invention. In a case where there is more than one file for each type, it is preferable to provide three directories named STREAM, CLIPINF and PLAYLIST under the BDMV to store the files with the same extension in one directory. Specifically, it is desirable to store the files with the M2TS extension in the STREAM, the files with the FPIC extension in the CLIPINF, and the files with the MPLS extension in the PLAYLIST.

An explanation about the AVClip (XXX.M2TS) in the above application format is given hereinafter.

The AVClip (XXX.M2TS) is a digital stream in MPEG-TS format (TS is Transport Stream) that is obtained by multiplexing a video stream, at least one audio stream, and a streaming stream. presentation graphics. The video stream represents movie images, the audio stream represents movie sound, and the presentation graphics stream represents movie subtitles. Figure 3 is a diagram schematically illustrating a structure of the AVClip.

The AVClip (XXX.M2TS) is structured as follows. Each of the video streams made from a plurality of video frames (picture pj1, pj2 and pj3), and the audio stream made from a plurality of audio frames (the top row of the drawing) are converted into a line of PES packets (the second row in the drawing), then in a line of TS packets (the third row in the drawing). The presentation graphics stream (the bottom row of the drawing) is converted to PES packages (the second row from the bottom of the drawing) and then to TS packages (the third row from the bottom of the drawing). Three lines of PS packets are multiplexed and thus the AVClip (XXX.M2TS) is constituted.

In the drawing, only one stream of presentation graphics is multiplexed. However, in a case where the BD-ROM supports a plurality of languages, a display graphics stream for each language is multiplexed to constitute the AVClip. The AVClip that is constituted in the above way is divided into more than one extension, like ordinary computer files, and stored in a few areas in the BD-ROM.

The following explains the presentation graphics flow. Fig. 4A illustrates a presentation graphics stream structure. A top row indicates the line of TS packets to be multiplexed on the AVClip. A second row from the top indicates the PES packet line that constitutes a graphics stream. The PES packet line is structured by recovering payloads from among TS packets having a previously determined PID, and connecting the recovered payloads.

A third row from the top indicates the structure of the graphics flow. The graphics stream is made up of functional segments called Presentation Composition Segment (PCS), Window Definition Segment (WDS), Palette Definition Segment (PDS), Object Definition Segment (ODS), and END. of the Display Set Segment (END). Among the above functional segments, PCS is called screen composition segment, and WDS, PDS, ODS, and END are called definition segments. The PES package and each of the functional segments correspond one to one, or one to many. In other words, a functional segment is either written to the BD-ROM after it has been converted into a package.

PES ES 2 536 680 T3, or after it has been divided into fragments and converted into more than one PES packet.

Figure 4B illustrates the PES packet that is obtained by converting the functional segments. As shown in the drawing, the PES packet is made up of a packet header and payload, and the payload is a substantial body of a functional segment. Hereinafter, the packet header includes a DTS and a PTS that correspond to the functional segment. The DTS and the PTS that are included in the packet header are referred to as the DTS and the PTS of the functional segment.

The foregoing has described various types of functional segments constituting a logical structure as illustrated in Figure 5. Figure 5 illustrates the logical structure that is made of the various types of functional segments. In the drawing, a top row illustrates Epochs, a middle row illustrates Display Sets (DS), and a bottom row illustrates functional segments.

Each of the DS shown in the middle row is a group of functional segments that make up graphics for a screen, out of the totality of the plurality of functional segments that make up the graphics stream. The dashed lines in the drawing indicate the DS to which the functional segments in the bottom row belong, and show that a number of the functional segments of the PCS, WDS, PDS, ODS, and END constitute a DS. The playback apparatus is capable of generating graphics for a screen by reading the functional segments that make up the DS.

The Epochs displayed in the top row indicate periods of time, and memory management is consecutive in time along a timing diagram of AVClip playback in an Epoch. An Epoch also represents a group of data that is assigned to the same time period. The memory referred to in the present case is the Graphics Plane that stores the graphics for a screen, and an Object Buffer that stores decompressed graphics data. Consecutive memory management means a flash of the Graphics Plane or the Object Buffer does not take place in the Epoch, and the removal and rendering of the graphics are only performed in a previously determined rectangular area on the Plane. Graphics (the flashing in the present case indicates the erasure of all the contents of the data stored in a map or a buffer memory). A size and a position of the rectangular area are fixed during an Epoch. As long as the removal and rendering of the graphics are only performed in the predetermined rectangular area in the Graphics Plane, a synchronized reproduction between the image and the graphics is guaranteed. In other words, the Epoch is a unit in the playback timing diagram, and in this unit, the image and graphics are guaranteed to be reproduced synchronously. When moving the area, in which the graphics are removed and rendered, to a different position, it is necessary to define a point on the timing diagram to move the area, and a period after the point becomes a new Epoch . Synchronized playback is not guaranteed on a border between two Epochs.

When watching a real movie, an Epoch is a period of time in which subtitles are displayed in the same rectangular area on the screen. Figure 6 illustrates a relationship between the position of the subtitles and the Epochs. In an example illustrated by the drawing, the positions in which the five subtitles "Actually ...", "I was hiding", "my feelings.", "Always", and "I loved you" are moved. according to the image on the film. Specifically, the subtitles "Actually ...", "I was hiding", and "my feelings." appear at the bottom of the screen, while the subtitles "Always" and "I loved you" are displayed at the top of the screen. The position of the rectangular area is moved so that the subtitles are away from the images when viewing the screen, considering the visibility of the film. A period of time during which subtitles appear at the bottom is Epoch 1, and a subsequent period of time during which subtitles appear at the top is Epoch 2. Each of Epochs 1 and 2 has a different area in which subtitles are rendered. The area in Epoch 1 is a Window 1 that is at the bottom of the screen, and the area in Epoch 2 is a Window 2 that is located in the upper part of the screen. Memory management is consecutive in each of Epochs 1 and 2 and, consequently, the rendering of the subtitles in Windows 1 and 2 is synchronous with the images.

Details about the Display Set (DS) are described below.

The dashed lines hk11 and hk12 in Figure 5 indicate which functional segment in the middle row belongs to which Epoch. A series of DS "Epoch Start", "Acquisition Point", and "Normal Case" constitute the Epoch in the top row. The "Start of Epoch", the "Acquisition Point", and the "Normal Case" are types of the DS, and an order between the "Acquisition Point" and the "Normal Case" does not matter and any one of them can come. first.

The Beginning of Epoch is a DS that has a "new visualization" display effect, indicating a beginning of a new Epoch. Because of this, the Epoch Start contains all the functional segments that are needed to display a new screen composition. The Epoch Start is provided at a position that is a target of an AVClip skip operation, such as a chapter in a movie.

ES 2 536 680 T3

The Acquisition Point is a DS that has a display effect of "regenerating display", and is identical in content that is used to render graphics with the Epoch Start which is a preceding DS. The Acquisition Point is not provided in a starting point of the Epoch, but contains all the functional segments that are needed to visualize the new composition of the screen. Therefore, it is possible to display the graphs without failure when a bypass operation is performed at the Acquisition Point. Consequently, with the Acquisition Point, it is possible to compose a screen in the middle of the Epoch.

The Acquisition Point is provided in a position that could be a target for the bypass operation. An example of such a position is a position that could be specified when performing a time search. Time search is an operation in response to a user's input of a time to start playing from a playback point that corresponds to the time specified by the user. The time is specified roughly, such as by 10 minutes or by 10 seconds, and accordingly, the points at which playback begins are given at an interval such as 10 minutes, or an interval of 10 seconds. By providing the Acquisition Point at the points where playback can start, it is possible to perform playback smoothly after time search.

The Normal Case is a DS that has a “display update” display effect, and contains only elements that are different from the preceding screen composition. Specifically, when the subtitles on a DSv are the same as the subtitles on a DSu but the screen is displayed differently on the DSv and DSu, the DSv is provided in order to include only the PCS and makes the DSv the Normal case. By this, it is not necessary to provide an ODS with the same content as the content of the ODS in the preceding DS, and a data size in the BD-ROM can be reduced. On the other hand, because the DS as the Normal Case contains only the difference, it is not possible to compose the screen using the Normal Case only.

Details of the Definition Segments (ODS, WDS, and PDS) are explained hereinafter. The Object Definition Segment (ODS) is a functional segment that defines the Graphics Object. An explanation of the Graphics Object is given first. A competitive advantage of the AVClip recorded on the BD-ROM is its resolution as high as Hi-Vision and therefore the resolution for the Graphics Object is set to 1920 x 1080 pixels. Due to the high resolution of 1920 x 1080 pixels, a specific character style for subtitles can be displayed clearly on the screen. As far as subtitle colors are concerned, a bit length of an index value for each pixel (Red Color Difference Cr, Blue Color Difference Cb, Luminance Y, and Transparency T) is 8 bits and therefore , you can choose any of 256 colors out of full color (16,777,216 colors) for subtitles. The subtitles that are obtained through the Graphics Object are rendered by placing texts on a transparent background.

The ODS syntax for defining the Graphics Object is shown in Figure 7A. The ODS is made of a segment_type that indicates that the segment is the ODS, a segment_length that indicates a data length of the ODS, an object_id that uniquely identifies the Graphics Object that corresponds to the ODS at the Epoch, a object_version_number that indicates a version of the ODS within the Epoch, a last_insequence_flag and an object_data_fragment that is a consecutive sequence of bytes that corresponds to a part or all of the Graphics Object.

The object_id is to uniquely identify the Graphics Object that corresponds to the ODS at the Epoch. The Epoch of the graphics stream contains more than one ODS that has the same ID. ODS that have the same ID also have the same width and height, and are assigned a common area in the Object Buffer. After one of the ODS that have the same ID is read into the common area, the read ODS is overwritten by a subsequent ODS that has the same ID. By overwriting the ODS that is read from the Object Buffer by the subsequent ODS having the same ID as the video stream progresses, the graphics by the ODS are updated accordingly. A size restriction that the width and height of the Graphics Object having the same ID should be the same applies only during one Epoch, and the Graphics Objects in different Epochs can have different sizes.

Here are some explanations about the last_sequence_flag and the object_data_fragment. In some cases, it is not possible to store the decompressed graphics that make up the subtitle in an ODS due to a payload restriction of the PES packet. In such cases, the graphics are divided into a series of consecutive fragments, and one fragment wraps around the object_data_fragment. When a Graphics Object is stored as more than one chunk, each chunk except one last chunk is the same size. The last chunk is less than or equal to the size of previous chunks. The ODS carrying the fragments appears in the same sequential order in the DS, with one end of the sequence indicated by the ODS having the last_sequence_flag. Although the syntax described above for the ODS is based on a premise that the fragments are stacked from the preceding PES, the fragments can be stacked in such a way that each PES contains a blank part.

Next, the Palette Definition Segment (PDS) is explained. The PDS is used to define a palette for a color conversion. Figure 7B shows the PDS syntax. The PDS is made of segment_type that indicates that the segment is the PDS, segment_length that indicates a data length of the PDS, palette_id that identifies in a way

ES 2 536 680 T3 unique the palette that is contained in the PDS, palette_version_number that indicates a version of the PDS within the Epoch, and palette_entry_id that specifies an input number of the palette. The palette_entry_id indicates the Red Color Difference (Cr_value), the Blue Color Difference (Cb_value), the Luminance (Y_value), and the Transparency (T_value).

Next, an explanation about the Window Definition Segment (WDS) is given hereinafter.

The WDS is used to define the rectangular area on the Graphics Plane. As described above, memory management is sequential only when removal and rendering are performed within a certain area on the Graphics Plane. The area on the Graphics Plane is defined by the WDS and is called "Window". Figure 8A illustrates the syntax of the WDS. As shown by the drawing, the WDS is made of segment_type that indicates that the segment is the WDS, segment_length that indicates a data length of the WDS, a window_id that uniquely identifies the Window in the Graphics Plane, a window_horizontal_position which specifies a horizontal direction of the upper left pixel of the Window on the Graphics Plane, a window_vertical_position that specifies a vertical direction of the upper left pixel of the Window on the Graphics Plane, a window_width that specifies a width of the Window on the Graphics Plane, and a window_height that specifies a height of the Window on the Graphics Plane.

The ranges of values ​​that window_horizontal_position, window_vertical_position, window_width, and window_height can take are explained hereinafter. A coordinate system for these values ​​is inside an area in the Graphics Plane, and whose size is indicated in a two-dimensional way by the window_height for a height and the window_width for a width.

The window_horizontal_position specifies the horizontal direction of the upper left pixel of the Window on the Graphics Plane, and is within a range of 0 to (window_width) - 1. Likewise, the window_vertical_position specifies the vertical direction of the upper left pixel of the Window on the Graphics Plane, and is within a range of 0 to (window_height) - 1.

The window_width specifies the width of the Window on the Graphics Plane. The specified width falls within a range of 1 to (video_width) - (window_horizontal_position). Also, the window_height specifies the height of the Window on the Graphics Plane, and the specified height is within a range of 1 to (video_height) - (window_vertical_position).

The position and size of the Window on the Graphics Plane for each Epoch is defined by the window_horizontal_position, the window_vertical_position, the window_width and the window_height. Consequently, it is possible to adjust the position and size of the Window at creation, such that the Window in an Epoch appears in the position that does not get in the way of the image when viewing the movie. By this, the visibility of the subtitles becomes higher. Since the WDS is defined for each Epoch, it is possible to adjust the position of the Window according to the image, even if the image changes over time. As a result, the quality of the film remains as high as in a case where subtitles are incorporated into the main body of the film.

The following explains the End of the Display Set Segment (END). The END provides an indication that a DS transmission is complete. The End is inserted into a stream immediately after a last ODS in a DS. The End is made of segment_type that indicates that the segment is the END and segment_length that indicates a data length of the END. The END does not include any other items that require additional explanation.

Next, an explanation about the Presentation Composition Segment (PCS) is given hereinafter.

The PCS is a functional segment that is used to compose an interactive display. Figure 8B illustrates the syntax of the PCS. As shown in the drawing, the PCS is made up of segment_type, segment_length, composition_number, composition_state, palette_update_flag, palette_id, and a 1-m window information.

The composition_number identifies the Graphics Update in the DS through values ​​in a range from 0 to 15. If the Graphics Update exists between the Epoch header and the PCS, the composition_number increases each time the Graphics Update takes place. .

The composition_state indicates the type of the DS in which the PCS is contained, Normal Case, Acquisition Point, or Epoch Start.

The palette_update_flag indicates that the PCS describes a Palette-only Display Update. Palette Only Display Refresh indicates that only the palette is updated from an immediately previous palette. The palette_update_flag field is set to “1”, if the Palette-only Display Update is performed.

ES 2 536 680 T3

The palette_id identifies the palette to be used in the Palette-only Display Update.

The window information 1 - m indicates how to control each Window in the DS to which the PCS belongs. A dashed line wd1 in Figure 8B is to detail an internal syntax for window information i. The window information i is made up of an object_id, a window_id, an object_cropped_flag, an object_horizontal_position, an object_vertical_position and a cropping_rectangle information 1 - n.

The object_id identifies the ODS in a Window that corresponds to the window information i.

The window_id identifies the Window to which the Graphics Object is assigned in the PCS. Up to two Graphics Objects can be assigned to a Window.

The object_cropped_flag is used to toggle between displaying and not displaying a Clipped Graphics Object in the Object Buffer. When the object_cropped_flag is set to "1", the clipped Graphics Object is displayed in the Object Buffer, and if it is set to "0", the Graphics Object is not displayed.

The object_horizontal_position specifies a horizontal direction of the upper left pixel of the Graphics Object on the Graphics Plane.

The object_vertical_position specifies a vertical direction of the upper left pixel of the Graphics Object on the Graphics Plane.

The cropping_rectangle information 1 - n is a few items that are used when the object_cropped_flag is set to "1". A dashed line wd2 is to detail an internal syntax for the cropping_rectangle information i. As shown by the dashed line wd2, the cropping_rectangle information i is made up of four fields, an object_cropping_horizontal_position, an object_cropping_vertical_position, an object_cropping_width, and an object_cropping_height.

The object_cropping_horizontal_position specifies a horizontal direction of an upper left corner of a cropping rectangle to be used when rendering the Graphics Object on the Graphics Plane. The clipping rectangle is a clipping pattern used to specify and clipping a part of the Graphics Object, and corresponds to Region in the ETSI EN 300 743 standard.

The object_cropping_vertical_position specifies a vertical direction of the upper left corner of the cropping rectangle to be used when rendering the Graphics Object on the Graphics Plane.

The object_cropping_width specifies a width of the cropping rectangle.

The object_cropping_height specifies a height of the cropping rectangle.

A specific example of the PCS is detailed below. In the example, the captions "Actually ...", "I was hiding," and "my feelings." as shown in figure 6 appear gradually by writing on the Graphics Plane 3 times as the image progresses. Figure 9 is an example of a description to obtain such a subtitle display. An Epoch in the drawing includes a DS1 (Start of Epoch), a DS2 (Normal Case), and a DS3 (Normal Case). The DS1 contains a WDS to specify the Window in which the subtitles are displayed, an ODS to specify the line "Actually ... I was hiding my feelings.", And a first PCS. DS2 contains a second PCS, and DS3 contains a third PCS.

Figures 10-12 illustrate examples of the WDS and PCS that are contained in the DS. Figure 10 shows an example of the PCS on the DS1.

In Figure 10, the window_horizontal_position and window_vertical_position of the WDS are indicated by an LP1, a position of the upper left pixel of the Window on the Graphics Plane. The window_width and the window_height indicate the width and height of the Window, respectively.

In Figure 10, the object_cropping_horizontal_position and object_cropping_vertical_position indicate a reference point ST1 of the cropping rectangle in the coordinate system in which an origin is the upper left pixel of the Graphics Object. The crop rectangle is an area that has the width from ST to object_cropping_width, and the height from ST to object_cropping_height (a rectangle shown by a heavy line box). The clipped Graphics Object is inside a rectangle shown by a dashed line box cpl, with a reference point in the coordinate system with an origin at the object_horizontal_position and the object_vertical_position (the upper left pixel of the Graphics Object) on the Graphics Plane. By this, the subtitle "Actually ..." is written in the Window on the Graphics Plane and then composed with the movie image and displayed on the screen.

ES 2 536 680 T3

Figure 11 shows an example of the PCS on the DS2. The WDS in DS2 is not explained, because the WDS in DS2 is the same as the WDS in DS1. A description of the clipping information in the DS2 is different from the description of the clipping information shown in Figure 10.

In Figure 11, the object_cropping_horizontal_position and object_cropping_vertical_position in the cropping information indicate a top left pixel of the caption "I was hiding" out of "Actually ... I was hiding my feelings." in the Intermediate Memory of Objects. The object_cropping_width and the object_cropping_height indicate a width and height of a rectangle that contains the caption "I was hiding." By doing this, the subtitle "Was hiding" is written in the Window on the Graphics Plane and then composed with the movie image and displayed on the screen.

Figure 12 shows an example of the PCS on the DS3. The WDS in the DS3 is not explained, because the WDS in the DS3 is the same as the WDS in the DS1. A description of the clipping information in the DS3 is different from the description of the clipping information shown in Figure 10.

In Figure 12, the object_cropping_horizontal_position and object_cropping_vertical_position in the cropping information indicate a top left pixel of the subtitle "my feelings." from "Actually ... I was hiding my feelings." in the Intermediate Memory of Objects. The object_cropping_width and the object_cropping_height indicate a width and a height of a rectangle that contains the subtitle "my feelings.". By this, the subtitle "my feelings." it is written to the Window on the Graphics Plane and then composed with the film image and displayed on the screen.

By describing the DS1, DS2, and DS3 as explained above, it is possible to achieve a display effect of the subtitles on the screen. It is also possible to achieve other types of effect, and description protocols for obtaining other effects are explained hereinafter.

First, a description protocol for an In / Out Cut effect is explained. Figure 13 shows an example of the description of the DS when performing an In / Out Cut, which is illustrated along a timing diagram.

In the drawing, x and y in Window (x, y, u, v) respectively indicate values ​​of window_vertical_position and window_horizontal_position, yu and v indicate respectively values ​​of window_width and window_height. Also in the drawing, a and b in the Clipping Rectangle (a, b, c, d) respectively indicate object_cropping_vertical_position and object_cropping_horizontal_position values, and c and d indicate object_cropping_width and object_cropping_height values, respectively. The DS11, DS12, and DS13 Display Sets are located at points t11, t12, and t13 on the playback timing diagram in the drawing.

The DS11 at point t11 ​​includes a PCS No. 0 in which the composition_state is "Start of Epoch" and the object_cropped_flag is "0" (no_cropping_rectangle_visible), a WDS No. 0 that has a statement for a Window in width 700 x a height 500 in (100, 100) in the Graphics Plane, a PDS No. 0, an ODS No. 0 indicating a subtitle “Credits:”, and an END.

The DS12 at point t12 includes a PCS No. 1 whose composition_state is "Normal Case" and which indicates that a graphics object crop operation is at a size of 600 x 400 from (0, 0) in Memory Object Intermediate (cropping_rectangle N ° 0 (0, 0, 600, 400)), and the location of the Clipped Graphics Object at coordinates (0, 0) on the Graphics Plane (in Window N ° 0 (0, 0)).

The DS13 at point t13 includes a PCS No. 2 whose composition_state is "Normal Case" and in which the object_cropped_flag is set to "0" in order to remove the clipped Graphics Object (no_cropping_rectangle_visible).

With the Display Sets that have been explained above, the subtitle "Credits:" is non-display in t11, it appears in t12, then it becomes non-display in t13 again, and the Cut In / Cut Out effect.

Second, a description protocol for a Fade In / Out effect is explained. Figure 14 shows an example of the description of the DS when Fade In / Fade Out, which is illustrated along a timing diagram. The DS21, DS22, DS23 and DS24 Display Sets are located at points t21, t22, t23 and t24 on the playback timing diagram in the drawing.

The DS21 at point t21 includes a PCS No. 0 whose composition_state is "Start of Epoch" and which indicates that the Graphics Object trim operation is at a size of 600 x 400 starting from (0, 0) in the Object Buffer (cropping_rectangle N ° 0 (0, 0, 600, 400)), and the location of the Clipped Graphics Object at coordinates (0, 0) on the Graphics Plane (in Window N ° 0 (0 , 0)), a WDS N ° 0 that has a sentence for a Window in a width 700 x a height 500 in (100, 100) in the Graphics Plane, a PDS N ° 0, an ODS N ° 0 that indicates a subtitle "End" , and an END.

ES 2 536 680 T3

The DS22 at point t22 includes a PCS No. 1 whose composition_state is “Normal Case”, and a PDS No. 1. PDS No. 1 indicates the same level of Cr and Cb as PDS No. 0, but a luminance indicated by PDS No. 1 is higher than luminance at PDS No. 0.

The DS23 at point t23 includes a PCS No. 2 whose composition_state is "Normal Case", a PDS No. 2, and an END. PDS No. 2 indicates the same level of Cr and Cb as PDS No. 1, but the luminance indicated by PDS No. 2 is lower than the luminance in PDS No. 1.

The DS24 at point t24 includes a PCS whose composition_state is "Normal Case" and the object_cropped_flag is "0" (no_cropping_rectangle_visible), and an END.

Each DS specifies a different PDS from a preceding DS, and consequently the luminance of the Graphics Object that is modeled with more than one PCS in an Epoch becomes gradually high, or low,. By this, it is possible to obtain the Fade In / Fade Out effect.

Next, a description protocol for a Displacement is explained. Figure 15 shows an example of the DS description when Shifting is performed, which is illustrated along a timing diagram. The DS31, DS32, DS33 and DS34 Display Sets are located at points t31, t32, t33 and t34 on the playback timing diagram in the drawing.

The DS31 at point t31 includes a PCS No. 0 whose composition_state is set to "Start of Epoch" and the object_cropped_flag is "0" (no_cropping_rectangle_visible), a WDS No. 0 that has a statement for a Window in width 700 x a height 500 in (100, 100) in the Graphics Plane, a PDS No. 0, an ODS No. 0 indicating a subtitle "Credits: Company", and an END.

The DS32 at point t32 includes a PCS N ° 1 whose composition_state is "Normal Case" and which indicates that the graphics object crop operation is at a size of 600 x 400 from (0, 0) in Memory Object Intermediate (cropping_rectangle N ° 0 (0, 0, 600, 400)), and the location of the Clipped Graphics Object at coordinates (0, 0) on the Graphics Plane (in Window N ° 0 (0, 0)). A 600 x 400 size area starting from (0, 0) in the Object Buffer includes a “Credits:” part of the “Credits: Company” subtitle that is displayed on two lines and therefore the part "Credits:" appears on the Graphics Plane.

The DS33 at point t33 includes a PCS N ° 2 whose composition_state is "Normal Case" and which indicates that the graphics object clipping operation is at a size of 600 x 400 from (0, 100) in Memory Object Intermediate (cropping_rectangle N ° 0 (0, 100, 600, 400)), and the location of the Clipped Graphics Object at coordinates (0, 0) on the Graphics Plane (in Window N ° 0 (0, 0)). The 600 x 400 size area starting from (0, 100) in the Object Buffer includes the "Credits:" part and a "Company" part of the subtitle "Credits: Company" which is displayed on two lines and, therefore, the “Credits:” and “Company” parts appear on two lines on the Graphics Plane.

The DS34 at point t34 includes a PCS N ° 3 whose composition_state is “Normal Case” and which indicates that the graphics object's crop operation is at a size of 600 x 400 from (0, 200) in Memory Object Intermediate (cropping_rectangle N ° 0 (0, 200, 600, 400)), and the location of the Clipped Graphics Object at coordinates (0, 0) on the Graphics Plane (in Window N ° 0 (0, 0)). The 600 x 400 size area starting from (0, 200) in the Object Buffer includes the “Company” part of the “Credits: Company” subtitle which is displayed on two lines and therefore the “ Company ”appears in the Graphics Plane. Using the PCS description above, it is possible to scroll down the subtitle by two lines.

Finally, a description protocol for an Entry / Exit Wipe effect is explained. Figure 16 shows an example of the description of the DS when an Entry / Exit Wipe is performed, which is illustrated along a timing diagram. The DS21, DS22, DS23 and DS24 Display Sets are located at points t21, t22, t23 and t24 on the playback timing diagram in the drawing.

The DS51 at point t51 includes a PCS No. 0 whose composition_state is set to "Start of Epoch" and the object_cropped_flag is "0" (no_cropping_rectangle_visible), a WDS No. 0 that has a statement for a Window in a width of 700 x a height 500 in (100, 100) in the Graphics Plane, a PDS No. 0, an ODS No. 0 indicating a subtitle "End", and an END.

The DS52 at point t52 includes a PCS N ° 1 whose composition_state is "Normal Case" and which indicates that the graphics object crop operation is at a size of 600 x 400 from (0, 0) in Memory Intermediate Object (cropping_rectangle N ° 0 (0, 0, 600, 400)), and the location of the Cropping Object at coordinates (0, 0) on the Graphics Plane (in Window N ° 0 (0, 0)). An area the size of 600 x 400 starting from (0, 0) in the Object Buffer includes the subtitle "End" and therefore the subtitle "End" appears on the Graphics Plane.

ES 2 536 680 T3

The DS53 at point t53 includes a PCS N ° 2 whose composition_state is "Normal Case" and which indicates that the graphics object crop operation is at a size of 400 x 400 from (200, 0) in Memory Object Intermediate (cropping_rectangle N ° 0 (200, 0, 400, 400)), and the location of the Clipped Graphics Object at coordinates (200, 0) on the Graphics Plane (in Window N ° 0 (200, 0)). By this, an area indicated by the coordinates (200, 0) and (400, 400) in the Window becomes a display area, and an area indicated by the coordinates (0, 0) and (199, 400) becomes a non-viewing area.

The DS54 at point t54 includes a PCS N ° 3 whose composition_state is "Normal Case" and which indicates that the graphics object crop operation is at a size of 200 x 400 from (400, 0) in Memory Object Intermediate (cropping_rectangle N ° 0 (400, 0, 200, 400)), and the location of the Clipped Graphics Object at coordinates (400, 0) on the Graphics Plane (in Window N ° 0 (400, 0)). By this, an area indicated by the coordinates (0, 0) and (399, 400) becomes the non-display area.

By this, as the non-display area becomes larger, the display area becomes smaller, and thus, the In / Out Wipe effect is obtained.

As described above, various effects such as Cut In / Out, Fade In / Out, Wipe In / Out, and Scroll can be obtained using corresponding scripts and therefore it is possible to make various arrangements in the rendering of the subtitles.

The restrictions to obtain the above effects are as follows. In order to obtain the Scroll effect, some operations for erasing and redrawing of the Window become necessary. Taking the example of figure 15, it is necessary to perform "a window erase" to eliminate the Graphics Object "Credits:" in the t32 of the Graphics Plane and then perform a "window redraw" to write a part a lower part of “Credits:” and a higher part of “Company” in the Graphics Plane during an interval between t32 and t33. Since the interval is the same as a video frame interval, a transfer rate between the Object Buffer and the Graphics Plane desirable for the Offset effect becomes an important point.

In the present case, a constraint on how big the Window can be is investigated. One Rc is the transfer rate between the Object Buffer and the Graphics Plane. In the present case, a worst case scenario is to perform both the Window erase and Window redraw at the rate Rc. In this case, each of the Window erasing and Window redrawing is required to be performed at a rate of half Rc (Rc / 2).

In order to make Window erase and Window redraw synchronize with a video frame, it is necessary that the equation be satisfied in the following.

Window size x Frame rate = Rc / 2

If the Frame Rate is 29.97, Rc is expressed by the equation hereinafter.

Rc = Window size x 2 x 29.97

In the subtitle rendering, the Window size represents at least 25% to 33% of the Graphics Plane. A total number of pixels in the Graphics Plane is 1920 x 1080. Assuming that an index bit length per pixel is 8 bits, a total capacity of the Graphics Plane is 2 Mbytes (= 1920 x 1080 x 8).

Taking the Window size to be 1/4 of the total capacity of the Graphics Plane, the Window size becomes 500 Kbytes (= 2 Mbytes / 4). By substituting this value in the above equation, Rc is calculated to be 256 Mbps (= 500 Kbytes x 2 x 29.97). If the rate for Window erasure and Window redraw can be half or a quarter of the frame rate, it is possible to double or quadruple the size of the Window even if the Rc is the same.

By keeping the Window size at 25% to 33% of the Graphics Plane and displaying the subtitles at the transfer rate of 256 Mbps, it is possible to keep the display in sync between the graphics and the movie image, no matter what. type of display effect to be obtained.

The position, size and area of ​​the Window are explained below. As explained above, the position and area of ​​the Window do not change in an Epoch. The position and size of the Window are adjusted to be the same during an Epoch because it is necessary to change a target writing direction of the Graphics Plane if the position and size change, and changing the direction results in a tare which reduces the transfer rate from the Object Buffer to the Graphics Plane.

A number of Graphics Objects per Window has a limitation. The limitation of the number is provided in order to reduce the overhead in the transfer of a decoded Graphics Object. In the present case, the tare is

ES 2 536 680 T3 generates when the direction of an edge of the Graphics Object is adjusted, and the greater the number of edges, the more tare is generated.

Figure 17 shows examples in comparison, an example where a Window has four Graphics Objects and another example where a Window has two Graphics Objects. The number of edges in the example with four Graphics Objects is twice the number of edges in the example with two Graphics Objects.

Without the limitation on the number of the Graphics Object, it becomes unknown how many tares could be generated in the transfer of the Graphics and therefore the load for the transfer increases and decreases drastically. On the other hand, when a maximum number of the Graphics Object in a Window is two, the transfer rate can be adjusted taking into account up to 4 tares. Consequently, it is easier to adjust the number of a minimum transfer rate.

Here is an explanation of how the DS that the PCS and ODS have is assigned to the AVClip timing diagram. The Epoch is a period of time in which a memory management is consecutive along the playback timing diagram. Since the Epoch is made up of more than one DS, it is important how to map the DS to the AVClip's playback timing diagram. The AVClip playback timing chart is a timing chart for specifying timings for the decoding and playback of each piece of image data that make up the video stream that is multiplexed on the AVClip. The decoding and playback timings in the playback timing diagram are expressed with an accuracy of 90 kHz. A DTS and a PTS attached to the PCS and the ODS in the DS indicate timings for synchronous control in the reproduction timing diagram. Assigning the Display Set to the playback timing diagram means to perform synchronous control using the DTS and the PTS that are attached to the PCS and the ODS.

First of all, how synchronous control is performed using the DTS and PTS attached to the ODS is explained hereinafter.

The DTS indicates, with the precision of 90 kHz, a time when the decoding of the ODS begins, and the PTS indicates a time when the decoding ends.

The decoding of the ODS does not finish at the same time, and it has a certain length of time. In response to a request to clearly indicate a starting point and an end point of a decoding duration, the DTS and PTS of the ODS respectively indicate the times when decoding begins and ends.

The value of the PTS indicates the deadline and, therefore, it is necessary that the decoding of the ODS has to be completed at the time indicated by the PTS and the decompressed Graphics Object is written in the Object Buffer in the device reproduction.

The decoding start time of any ODSj in a DSn is indicated by a DTS (DSn [ODS]) with the precision of 90 kHz. Adding a maximum length of the decoding duration to the DTS (DSn [ODS]) is the time at which the decoding of the ODSj ends.

When an ODSj size is “SIZE (DSn [ODSj])” and an ODS decoding rate is one “Rd”, the maximum time required to decode indicated per second is expressed in “SIZE (DSn [ODSj]) // Rd ". The symbol "//" indicates an operator for a division with rounding up after one decimal place.

By converting the maximum period of time into a number that is expressed with the precision of 90 kHz and adding the ODSj to the DTS, the time in which decoding ends (90 kHz) indicated by the PTS is calculated.

The PTS of the ODSj in the DSn is expressed in the following equation.

PTS (DSn [ODSj]) = DTS (DSn [ODSj]) + 90,000 x (SIZE (DSn [ODSj]) // Rd

Furthermore, a relationship between two subsequent ODSs, ODSj and ODSj + 1, needs to satisfy the following equation.

PTS (DSn [ODSj]) <DTS (DSn [ODSj + 1])

The following explains the DTS and PTS settings of the PCS.

It is necessary for the PCS to be loaded into the Object Buffer in the playback apparatus before the decoding start time (DTS (DSn [ODS1])) of a first ODS (ODS1) in the DSn, and before the time ( PTS (DSn [PDS1])) in which a first PDS (PDS1) in the DSn becomes effective. Consequently, the DTS needs to be adjusted in order to satisfy the following equations.

PTS (DSn [PCS]) <DTS (DSn [ODS1])

ES 2 536 680 T3

PTS (DSn [PCS]) <PTS (DSn [PDS1])

Furthermore, the PTS of the PCS in the DSn is expressed in the following equation.

PTS (DSn [PCS])> DTS (DSn [PCS]) + decode_duration (DSn)

The "decode_duration (DSn)" indicates a time duration to decode all the Graphics Objects that are used to update the PCS. The decoding duration is not a fixed value, but it does not vary according to a status of the reproducing apparatus and a device or software mounted on the reproducing apparatus. When the Object used to compose a screen of a DSn.PCSn is a DSn.PCSn.OBJ [j], the decode_duration (DSn) is affected by the time (i) required to clear the Window, the decode durations (ii) to decode a DSn.PCSn.OBJ, and the time (iii) necessary for writing the DSn.PCSn.OBJ. When Rd and Rc are set, the decode_duration (DSn) is always the same. Therefore, the PTS is calculated by calculating the lengths of these durations at creation.

The calculation of the decode_duration is performed on the basis of a program shown in Figure 18. Figures 19, 20A and 20B are flowcharts that schematically show algorithms of the program. An explanation about the calculation of the decode_duration is given hereinafter with reference to these drawings. In the flowchart shown in Fig. 19, first, a PLANEINITIALZE function is called (step S1 in Fig. 19). The PLANEINITIALZE function is used to call a function to calculate a period of time necessary to initialize the Graphics Plane to model the DS. In step S1 in Fig. 19, the function is called with some arguments DSn, DSn. PCS. OBJ [0], and decode_duration.

The following explains the PLANEINITIALZE function with reference to Fig. 20A. In the drawing, initialize_duration is a variable that indicates a return value of the PLANEINITIALZE function.

Step S2 in Fig. 20 is a yes statement to switch operations depending on whether the page_state in the PCS in the DSn indicates the Epoch Start or not. If the page_state indicates the Epoch Start (DSn.PCS.page_state == epoch_start, step S2 = Yes in FIG. 18), a time period required to clear the Graphics Plane is set to an initialize_duration (step S3).

When the transfer rate Rc between the Object Buffer and the Graphics Plane is 256,000,000 as described above, and the total size of the Graphics Plane is set to video_width * video_height, the time period needed to delete is "video_width * video_height // 256,000,000". When multiplied by 90,000 Hz in order to express with PTS time precision, the time period required to erase the Graphics Plane is "90,000 x video_width * video_height // 256,000,000". This time period is added to the initialize_duration.

If the page_state does not indicate the Epoch Start (step S2 = No), a period of time necessary to clear Window [i] defined by the WDS is added to the initialize_duration for all Windows (step S4). When the transfer rate Rc between the Object Buffer and the Graphics Plane is 256,000,000 as described above and a total size of the Window [i] belonging to the WDS is Σ SIZE (WDS. WIN [i]), the time period required to erase is “Σ SIZE (WDS. WIN [i]) // 256,000,000”. When multiplied by 90,000 Hz in order to express with the time precision of PTS, the period of time required to clear the Windows belonging to the WDS is “90,000 x Σ SIZE (WDS. WIN [i]) // 256,000. 000 ". This time period is added to the initialize_duration, and the initialize_duration is returned as a result. The above is the PLANEINITIALZE function. Step S5 in Figure 19 to switch operations depending on whether the number of Graphics Objects in the DSn is 2 or 1 (if (DSn.PCS.num_of_object == 2, if (DSn.PCS.num_of_object == 1 in Fig. 18), and if the number is 1 (step S5), a wait time for decoding the Graphics Object is added to the decode_duration (step S6). The time-out calculation is done by calling a WAIT function (decode_duration + = WAIT (DSn, DS.PCS.OBJ [0], decode_duration) in Figure 18). The function is called using arguments set to DSn, DSn.PCS.OBJ [0], decode_duration, and a return value is wait_duration.

Fig. 20B is a flow chart showing an operation of the WAIT function.

In the flowchart, the decode_duration of an invoker is set as a current_duration. An object_definition_ready_time is a variable set to the PTS of the DS Graphics Object.

A current_time is a variable set to a total value of the current_duration and the DTS of the PCS in the DSn. When the object_definition_ready_time is larger than the current_time (Yes to step S7, yes (current_time <object_definition_ready_time)), the wait_duration as the return value is set to be a difference between the object_definition_ready_time and the current_time (step S8, wait_duration + = object_definition_ready_time - current_time). The decode_duration adjusts to the period of time that the return value of the WAIT function added to the period of time required to redraw the Window, (90.000 * (SIZE (DSn.WDS. WIN [0])) // 256.000.000) .

ES 2 536 680 T3

The above explanation is for the case where the Graphics Object number is one. In step S5 in Fig. 5, it is judged whether the number of the Graphics Objects is two. If the number of Graphics Objects in the DSn is more than two (if (DSn.PCS.num_of_object == 2) in figure 18), the WAIT function is called using OBJ [0] in the PCS as an argument , and adding a return value to the decode_duration (step S10).

In a next step S11, it is evaluated whether the Window to which the OBJ [0] of the DSn belongs is the same as the Window to which the Graphics Object [1] belongs (if (DSn.OBj [0]. Window_id == DSn.PCS.OBJ [1] .window_id). If the Window is the same, call the WAIT function using OBJ [1] as an argument, and add a wait_duration return value to the decode_duration (step S12), and add the time needed to redraw the Window to which OBJ [0] (90,000 * (SIZE (DSn.WDS.OBJ [0] .window_id)) // 256,000,000) belong to the decode_duration (step S13).

If the Windows are evaluated to be different (step S11, "different"), the time required to redraw the Window to which OBJ [0] (90,000 * (SIZE (DSn.WDS.OBJ [0]) belongs is added. window_id)) // 256,000,000) to decode_duration (step S15), call the WAIT function using OBJ [1] as an argument, and add a wait_duration return value to decode_duration (step S16), and the time required to redraw the Window to which OBJ [1] (90,000 * (SIZE (DSn.WDS.OBJ [0] .window_id)) // 256,000,000) belongs to the decode_duration (step S17).

The decode_duration is calculated by the algorithm above. A specific way in which the OCS PTS is set is explained below.

Figure 21A illustrates a case where an ODS is included in a Window. Figures 21B and 21C are timing diagrams showing values ​​in a time order referred to in Figure 18. A bottom line "ODS Decoding" and a middle line "Graphics Plane Access" on each The diagram indicates two operations that are performed simultaneously during playback. The above algorithm is described assuming that these two operations are performed in parallel.

The Graphics Plane Access includes an erase period (1) and a write period (3). The erase period (1) indicates either a period of time necessary to erase the entirety of a Graphics Plane (90,000 x (Graphics Plane size // 256,000,000)), or a period of time necessary to erase all Windows in the Graphics Plane (Σ (90,000 x (Window size [i] // 256,000,000)).

The write period (3) indicates a period of time necessary to render the whole of a Window (90,000 x (Window size [i] // 256,000,000)).

Furthermore, a decoding period (2) indicates a time period between the DTS and the PTS of the ODS.

The lengths of the erase period (1), the decode period (2), and the write period (3) may vary depending on an interval to be erased, a size of ODS to be decoded, and a size of the Graphics Object to be written to the Graphics Plane. For convenience, a starting point of the decoding period (2) in the drawing is the same as a starting point of the erasing period (1).

Fig. 21B illustrates a case where the decoding period (2) is long, and the decode_duration is equal to a total of the decoding period (2) and the writing period (3).

Fig. 21C illustrates a case where the erase period (1) is long, and the decode_duration is equal to a total of the erase period (1) and the write period (3).

Figures 22A to 22C illustrate a case where two ODS are included in a Window. The decoding period (2) in both of Figures 22B and 22C indicates a total period of time required to decode two Graphics. Similarly, the write period (3) indicates a total period of time required to write two Graphics to the Graphics Plane.

Even though the number of ODS is two, it is possible to calculate the decode_duration in the same way as in the case of figure 21. When the decoding period (3) to decode the two ODS is long, the decode_duration is equal to a total of the decoding period (2) and the writing period (3) as shown in Fig. 22B.

When the erase period (1) is long, the decode_duration is equal to a total of the erase period (1) and the write period (3).

Figure 23A describes a case where each of two Windows includes an ODS. As in the previous cases, when the erase period (1) is longer than the decode period (3) to decode the two ODS, the decode_duration is equal to a total of the erase period (1) and the period decoding (2). However, when the erase period (1) is shorter than the decode period (3), it is possible to write to

ES 2 536 680 T3 a first Window before the end of the decoding period (2). Consequently, the decode_duration is not equal to either one of a total of the erase period (1) and the write period (3), or a total of the decode period (2) and the write period (3).

When a period of time necessary to decode a first ODS is a write period (31) and a period of time necessary to decode a second ODS is a write period (32), FIG. 23B illustrates a case where the period decoding period (2) is longer than a total of erase period (1) and write period (31). In this case, the decode_duration is equal to a total of the decoding period (2) and the writing period (32).

Fig. 23C illustrates a case in which a total of the erase period (1) and the write period (31) is longer than the decode period (2). In this case, the decode_duration is equal to a total of the erase period (1), the write period (31), and the write period (32).

The size of the Graphics Plane is known from a model of the reproduction apparatus in advance. Likewise, the size of the Window, and the size and number of the ODS are also known at creation. Consequently, it is possible to find which combination of time periods is equal to the decode_duration: the erase period (1) and the write period (3), the decode period (2) and the write period (3), the decoding period (2) and the writing period (32), or the erasing period (1), the writing period (3) and the writing period (32).

By adjusting the PTS of the ODS based on the calculation of the decode_duration explained above, it is possible to display the graphics synchronously with the image data with high precision. Such a synchronized display with high precision is made possible by defining the Window and limiting an area to redraw to the Window. Therefore, introducing a Window concept into a creation environment is of great importance.

The following is an explanation about the DTS and PTS settings of the WDS on the DSn. The DTS of the WDS can be adjusted in order to satisfy the equation hereinafter.

DTS (DSn [WDS])> DTS (DSn [PCS])

On the other hand, the OTS of the WDS in the DSn indicates a deadline to start writing in the Graphics Plane. Since it is sufficient to write in the Window in the Graphics Plane, the time to start writing in the Graphics Plane is determined by subtracting a length of time indicated by the PTS of the PCS from a necessary period of time. to write the WDS. When a total size of the WDS is Σ SIZE (WDS. WIN [i]), the time required to erase and redraw is “Σ SIZE (WDS. WIN [i]) // 256,000,000”. When expressed with a time precision of 90,000 kHz, the time is "90,000 x Σ SIZE (wDs. WIN [i]) // 256,000,000".

Consequently, it is possible to calculate the PTS of the WDS by the following equation.

PTS (DSn [WDS]) =

PTS (DSn [PCS]) - 90000 x Σ SIZE (WDS. WIN [i]) // 256,000,000

The PTS indicated in the WDS is the deadline, and it is possible to start writing to the Graphics Plane before the PTS. In other words, as shown in figure 23, once the ODS to be rendered in one of the Windows is decoded, the writing of the Graphics Object obtained by decoding can begin at this point.

As described above, it is possible to assign the Window to any point in time in the AVClip playback timing diagram using the DTS and PTS that are added to the WDS.

Explanations are given hereinafter about an example of DTS and PTS settings in a Display Set based on the settings, referring to a specific example illustrated in Figures 24-25. The example is about a case where subtitles are displayed by writing on the Graphics Plane four times, and an update is performed to display each of two subtitles "what is blu-ray." and "blu-ray is everywhere." Figure 24 illustrates some deviations in the update time in the example. Up to a point t1, “what” is displayed, and “what is” is displayed after t1 through t2, and then “what is blu-ray.” is displayed on a t3. After the entirety of a sentence from a first subtitle has appeared, a second subtitle "the blu-ray is everywhere." is displayed on a t4.

Figure 25A illustrates four Display Sets that are described in order to perform the update explained above. A DS1 includes a PCS1.2 to control an update on t1, a PDS1 for coloring, an ODS1 that corresponds to the subtitle "what is blu-ray.", And an END as a DS1 end code.

ES 2 536 680 T3

A DS2 includes a PCS1.2 to control an update at t2, and an END. A DS 3 includes a PCS1.3 to control an update on a t3 and an END. A DS 4 includes a PCS2 to control an update at t2, a PDS2 for a color conversion, an ODS2 that corresponds to the subtitle "Blu-ray is everywhere.", And an END.

Referring to a timing diagram in FIG. 25B, DTS and PTS settings are explained for each functional segment in the four Display Sets.

The playback timing diagram in the timing diagram is the same as the timing diagram in Fig. 24. In the timing diagram of Figure 25A, the PTS (PCS1.1), the PTS (PCS1.2), the PTS (PCS1.3) and the PTS (PCS2) are respectively set at a display point t1 to display "What", a display point t2 to display "what is", a display point t3 to display "what is blu-ray.", And a display point t4 to display "blu-ray is everywhere. ”. Each of the PTS is set as above, because it is necessary that the control such as the clipping described in each PCS is performed at the display point of each subtitle.

The PTS (ODS1) and the PTS (ODS2) are adjusted in order to indicate points that are calculated by subtracting the decode_duration from the points indicated by the PTS (PCS1.1) and the PTS (PCS2), respectively, due to to which the PTS (PCS) is required to adjust in order to satisfy the formula hereafter. PTS (DSn [PCS]) 2 DTS (DSn [PCS]) + decode_duration (DSn)

In Fig. 25B, the PTS (ODS2) is adjusted in order to indicate a point t5 that you eat before the point t4, and the PTS (ODS1) is adjusted in order to indicate a point t0 that you eat before the point t1.

The DTS (ODS1) and the DTS (ODS2) are adjusted to indicate points that are calculated by subtracting the decode_duration from the points indicated by the PTS (ODS1) and the PTS (ODS2), respectively, because the DTS (ODS) is required to be adjusted in order to satisfy the equation hereinafter. PTS (DS [ODSj]) = DTS (DSn [ODSj]) + 90,000 x (SIZE (DSn [ODSj]) // Rd)

In Fig. 25B, the PTS (ODS2) is adjusted in order to indicate the point t5 that you eat before the point t0, and the PTS (ODS1) is adjusted in order to indicate a point that you eat before the point t0. A relationship indicated by the DTS (ODS2) = PTS (ODS1) is satisfied in the present case.

By setting a PTS of an ODS immediately after a PTS of a preceding ODS to be displayed before, the playback apparatus performs an operation in which the ODS is acquired for memory in order to overwrite the preceding ODS and Therefore, the playback process may be performed using a small memory size. By obtaining such a reproduction process, the options for a memory size for a reproducing apparatus become wider.

The DTS of PCS1.1 is set to be DTS (PCS1.1) = DTS (ODS1), because the value for the DTS of PCS1.1 can be any point before the point indicated by the DTS (ODS1 ).

The PTS of ODS1, the DTS of ODS2, and the PTS of PCS1.2, PCS1.3, and PCS2 are adjusted at point t0, in order to satisfy a relationship indicated by the equation hereinafter.

PTS (ODS1) = DTS (ODS2) = PTS (PCS1.2) = PTS (PCS1.3) = PTS (PCS2)

This is because the value for the DTS of PCS1.2 and PCS1.3 can be any points before the point indicated by the PTS (PCS1.3), and the DTS of PCS2 can be any point before the point indicated by the DTS (PCS2).

As explained above, it is possible to perform the update of a next PCS as soon as the update of a previous PCS is completed, by acquiring more than one PCS at the same time.

It is sufficient that the DTS and PTS of PCS and the DTS and PTS of ODS satisfy the relationships indicated by the formulas above. Consequently, it becomes possible for the values ​​to be adjusted to be DTS (ODS2) = PTS (ODS1) or PTS (ODS1) = DTS (ODS2) = PTS (PCS1.2) = PTS (PCS1.3) = DTS ( PCS2). By such settings for the timestamps, it is possible to adjust the length of time of a period in which the load on decoding increases or more buffers are required. Such an adjustment expands the possibility of controls during playback, and is advantageous for those involved in the creation or manufacture of playback apparatus.

The data structures of the Display Sets (PCS, WDS, PDS, ODS) that have been explained above are an example of the class structure that is described in a programming language. Producers doing the build can get the data structures on the BD-ROM by describing the class structure according to the syntax provided in the Pre-Record Disc Format.

ES 2 536 680 T3

Blu-ray.

Next, a practical example of a reproduction apparatus according to the present invention is explained hereinafter. Fig. 26 illustrates an internal structure of the reproduction apparatus according to the present invention. The reproduction apparatus according to the present invention is industrially produced on the basis of the internal structure shown in the drawing. The reproduction apparatus according to the present invention is mainly structured by three parts: a system LSI, a drive device, and a microcomputer system, and it is possible to industrially produce the reproduction apparatus by assembling the three parts in a cabinet and a substrate of the apparatus. The system LSI is an integrated circuit in which various processing units are integrated to carry out a function of the reproduction apparatus. The playback apparatus that is manufactured in the above manner comprises a BD unit 1, a Read Buffer 2, a PID filter 3, the Transport Buffers 4a-4c, a peripheral circuit 4d, a Video Decoder 5, a Video Plane 6, an Audio Decoder 7, a Graphics Plane 8, a CLUT unit 9, an adder 10, a Graphics Decoder 12, a Coded Data Buffer 13, a peripheral circuit 13a, a Flow Graphics Processor 14, an Object Buffer 15, a Composition Buffer 16, and a Graphics Controller 17.

BD drive 1 performs BD-ROM load / read / eject, and accesses BD-ROM.

Read Buffer 2 is a FIFO memory for storing the TS packets that are read from the BD-ROM in a first-in-first, first-out order.

PID filter 3 filters more than one TS packet issued from Read Buffer 2. Filtering by PID filter 3 is to write the only TS packets that have a desired PID to Transport Buffers 4a - 4c. Buffering is not necessary for filtering by PID filter 3, and consequently TS packets input to PID filter 3 are written to Transport Buffers 4a-4c without delay.

Transport Buffers 4a - 4c are for storing TS packets emitted from PID filter 3 in first-in-first, first-out order. A rate at which TS packets from Transport Buffers 4a-4c are output is an Rx rate.

The peripheral circuit 4d is wired logic to convert the TS packets that are read from the Transport Buffers 4a-4c into functional segments. The functional segments obtained by the conversion are stored in the Coded Data Buffer 13.

The Video Decoder 5 decodes the more than one TS packets emitted from the PID filter 3 to give a decompressed image and writes in the Video Plane 6.

Video Plane 6 is a plane memory for a moving picture.

The Audio Decoder 7 decodes the TS packets emitted from the PID filter 3 and outputs decompressed image data.

Graphics Plane 8 is a map memory that has an area for one screen, and is capable of storing uncompressed graphics for one screen.

The CLUT unit 9 converts an index color of the decompressed Graphics that are stored in the Graphics Plane 8 on the basis of the values ​​for Y, Cr and Cb indicated by the PDS.

The adder 10 multiplies the decompressed Graphics to which the color conversion has been carried out by the CLUT unit 9 by the value of T (Transparency) indicated by the PDS, adds the decomposed image data that are stored in the Video Plane per pixel, then fetches and outputs the composite image.

The Graphics Decoder 12 decodes the Graphics Stream to get the decomposed graphics, and writes the decomposed graphics as the Graphics Object on the Graphics Plane 8. By decoding the Graphics Stream, the subtitles and menus appear on the screen. . Graphics Decoder 12 includes Coded Data Buffer 13, peripheral circuit 13a, Flow Graphics Processor 14, Object Buffer 15, Composition Buffer 16, and Graphics Controller 17.

The Coded Data Buffer 13 is a buffer in which the functional segment is stored together with the DTS and the PTS. The functional segment is obtained by removing a TS packet header and a PES packet header from each TS packet in the Transport Stream that is stored in Transport Buffer 4a - 4c and by disposing payloads sequentially. The PTS and the DTS between the TS packet header and the deleted PES packet header are stored after mapping between the

ES 2 536 680 T3 PES packages.

The peripheral circuit 13a is hardwired logic that achieves a transfer between the Coded Data Buffer 13 and the Flow Graphics Processor 14, and a transfer between the Coded Data Buffer 13 and the Composition Buffer 16. In the transfer operation, when a current time is a time indicated by the DTS of the ODS, the ODS is transferred from the Coded Data Buffer 13 to the Flow Graphics Processor 14. When the current time is a time indicated by DTS from PCS and PDS, PCS and PDS are transferred to Composition Buffer 16.

The Flow Graphics Processor 14 decodes the ODS, and writes the decompressed graphics of the index color obtained by decoding as the Graphics Object into the Object Buffer 15. The decoding by the Flow Graphics Processor 14 begins at the time of the DTS that corresponds to the ODS, and ends at the decoding end time indicated by the PTS that corresponds to the ODS. The Graphics Object decoding rate Rd is an output rate of the Streaming Graphics Processor 14.

The Object Buffer 15 is a buffer that corresponds to a pixel buffer in the ETSI EN 300 743 standard, and the Graphics Object obtained by decoding carried out by the Flow Graphics Processor 14 is provided. It is necessary that the Object Buffer 15 be adjusted to twice or four times as large as the Graphics Plane 8, because in the case that the Displacement effect is performed, it is necessary that the Object Buffer 15 store the Graphics Object that is twice or four times as large as the Graphics Plane.

The Composition Buffer 16 is a memory in which the PCS and PDS are arranged.

The Graphic Controller 17 decodes the PCS that is arranged in the Composition Buffer 16, and performs a control on the basis of the PCS. A timing to perform the control is based on the PTS that is attached to the PCS.

Next, recommended values ​​for the transfer rate and the buffer size are explained to structure the PID filter 3, the Transport Buffer 4a - 4c, the Graphics Plane 8, the CLUT unit 9, the Coded Data 13, and Graphic Controller 17. Figure 27 illustrates sizes of the write rates Rx, Rc, and Rd, Graphics Plane 8, Coded Data Buffer 13, Object Buffer 15, and Composition Buffer 16.

The transfer rate Rc between the Object Buffer 15 and the Graphics Plane 8 is the highest transfer rate in the playback apparatus of the present embodiment, and is calculated as 256 Mbps (= 500 Kbytes x 29.97 x 2) from window size and frame rate.

Unlike Rc, the transfer rate Rd (Pixel Decoding Rate) between Streaming Graphics Processor 14 and Object Buffer 15 need not be updated every cycle of video frames, and 1/2 or 1/4 of the Rc is sufficient for the Rd. Consequently, the Rd is either 128 Mbps or 64 Mbps.

The Transport Buffer Leak Rate Rx between the Transport Buffer 4a - 4c and the Encoded Data Buffer 13 is a transfer rate of the ODS in a compressed state. Consequently, the transfer rate Rd multiplied by the compression rate is sufficient for the Transport Buffer Leak Rate Rx. Since the compression rate of the ODS is 25%, 16 Mbps (= 64 Mbps x 25%) is sufficient.

The transfer rates and buffer sizes shown in the drawing are the minimum standard, and it is also possible to set them to higher rates and larger sizes.

In the reproduction apparatus structured above, each of the elements performs a decoding operation in a conduit structure.

FIG. 28 is a timing diagram illustrating pipeline processing by the playback apparatus. A 5^to row in the drawing is a Display Set on the BD-ROM, a 4^to row shows reading periods from PCS, WDS, pDs and ODS for the Coded Data Buffer 13. A 3rd row shows decoding periods of each ODS by the Flow Graphics Processor 14. A 1st row shows some operations performed by the Graphic Controller 17.

The DTS (decode starting time) attached to the ODS1 and ODS2 indicates t31 and t32 in the drawing, respectively. Because the decoding start time is set by the DTS, each ODS is required to be acquired from the Encoded Data Buffer 13. Consequently, the reading from ODS1 is completed before a decoding period dp1 in which ODS1 is decoded to Coded Data Buffer 13. Likewise, the reading of the ODS2 is completed before a decoding period dp2 in which the ODS2 is decoded to the Coded Data Buffer 13.

ES 2 536 680 T3

On the other hand, the PTS (decode ending time) attached to ODS1 and ODS2 indicates t32 and t33 in the drawing, respectively. Decoding of ODS1 by Streaming Graphics Processor 14 is completed by t32, and decoding of ODS2 is completed for a time indicated by t33. As explained above, the Flow Graphics Processor 14 reads the ODS for the Coded Data Buffer 13 at the time indicated by the DTS of the ODS, and decodes the ODS read for the Coded Data Buffer. 13 at the time that the PTS of the ODS indicates, and writes the decoded ODS in the Object Buffer 15.

A period cd1 at 1^to row in the drawing indicates a period required for the Graphics Controller 17 to erase the Graphics Plane. Likewise, a period td1 indicates a period necessary to write the Graphics Object obtained in the Object Buffer in the Graphics Plane 8. The PTS of the WDS indicates the deadline to start writing, and the PTS of the PCS indicates completion of writing and a sync for display. At the time indicated by the PTS of the PCS, the decompressed graphics to compose an interactive screen are obtained in Graphics Plane 8.

After the CLUT unit 9 performs the color conversion of the decompressed graphics and the adder 10 performs the composition of the decomposed graphics and a composite image that is stored in the Video Plane 6, a composite image is obtained.

In Graphics Decoder 12, Stream Graphics Processor 14 performs decoding continuously while Graphics Controller 17 performs Graphics Plane erasure 8. By above pipeline processing, it is possible to perform prompt display of the graphics.

In Fig. 28, a case is explained where the erasing of the Graphics Plane ends before completing the decoding of the ODS. FIG. 29 illustrates a timing diagram in a pipeline processing of a case in which the decoding of the ODS ends before the erasure of the Graphics Plane is completed. In this case, it is not possible to write to the Graphics Plane at an ODS decoding completion time. When the erasing of the Graphics Plane is completed, it becomes possible to write the graphics obtained by decoding to the Graphics Plane.

Next, how the control unit 20 and the Graphics Decoder 12 are implemented is explained hereinafter. The control unit 20 is implemented by writing a program that performs an operation shown in Fig. 30, and causing a general CPU to execute the program. The operation that is performed by the control unit 20 is explained by referring to Fig. 30.

Fig. 30 is a flow chart showing a process of a functional segment load operation. In the flowchart, SegmentK is a variable that indicates each of the Segments (PCS, WDS, PDS, and ODS) that is acquired in the AVClip playback. An ignore flag is a flag to determine if the SegmentK is ignored or loaded. The flowchart has a loop structure, in which first the ignore flag is initialized to 0, and then steps S21 - S24 and steps S27 - S31 are repeated for each SegmentK (step S25 and step S26).

Step S21 is to evaluate whether the SegmentK is the PCS, and whether the SegmentK is the PCS, evaluations are performed at the step S27 and the step S28.

Step S22 is to evaluate whether the ignore flag is 0. If the ignore flag is 0, the operation moves to step S23, and if the ignore flag is 1, the operation moves to step S24. If the ignore flag is 0 (Yes in step S22), the SegmentK is loaded into the Coded Data Buffer 13 in step S23.

If the ignore flag is 1 (No in step S22), the SegmentK is ignored in step S24. By this, the rest of all functional segments belonging to the DS are ignored because step S22 is No (step S24).

As explained above, whether the SegmentK is ignored or loaded is determined by the ignore flag. Steps S27-S31, S34 and S35 are steps for setting the ignore flag.

In step S27, it is evaluated whether the segment_type of the SegmentK is the Acquisition Point. If SegmentK is the Acquisition Point, the operation moves to step S28, and if SegmentK is either Epoch Start or Normal Case, then the operation moves to step S31.

In step S28, it is evaluated whether a preceding DS exists in any of the buffers in the Graphics Decoder 12 (the encoded data buffer 13, the stream graphics processor 14, the object buffer 15, and the composition buffer 16). The evaluation in step S28 is made when the evaluation in step S27 is Yes. A case where a preceding DS does not exist in the Graphics Decoder 12 indicates a case where the skip operation is performed. In this case, the display starts from the DS which is the Acquisition Point and, therefore, the operation moves to the

ES 2 536 680 T3 step S30 (Not in step S28). In step S30, the ignore flag is set to 0 and the operation moves to step S22.

A case where a preceding DS exists in the Graphics Decoder 12 indicates a case where normal playback is performed. In this case, the operation moves to step S29 (Yes in step S28). In step S29, the ignore flag is set to 1 and the operation moves to step S22.

In step S31, it is evaluated whether the segment_type of the PCS is the Normal Case. If the PCS is the Normal Case, the operation moves to step S34, and if the PCS is the Epoch Start, then the ignore flag is set to 0 in step S30.

In step S34, as in step S28, it is evaluated whether a preceding DS exists in any of the buffers in Graphics Decoder 12. If the preceding DS exists, the ignore flag is set to 0 (step S30) . If the preceding DS does not exist, it is not possible to obtain enough functional segments to compose an interactive screen and the ignore flag is set to 1 (step S35).

By setting the ignore flag as above, the functional segments constituting the Normal Case are ignored when the preceding DS does not exist in Graphics Decoder 12.

Taking an example of a case where the DS is multiplexed as shown in Fig. 31, a way in which the reading of the DS is performed is explained. In the example of Figure 31, three DSs are multiplexed with a moving picture. The segment_type of a DS1 is Epoch Start, the segment_type of a DS10 is Acquisition Point, and the segment_type of a DS20 is Normal Case.

Given this, in an AVClip in which the three DS and the moving image are multiplexed, a skip operation is performed on a pt10 image data as shown by an arrow am1, the DS10 is closest to a target. bypass and therefore DS10 is the DS described in the flow chart in Figure 30. Although the segment_type is evaluated to be the Acquisition Point in step S27, the ignore flag is set to 0 because there is no preceding DS in the Coded Data Buffer 13, and the DS10 is loaded into the Coded Data Buffer 13 of the reproduction apparatus as shown by an arrow md1 in FIG. 32. On the other hand, in a case where the skip target is after DS10 (am2 arrow in figure 31), DS20 will be ignored because DS20 is Normal Case Display Set and DS20 due to that a preceding DS does not exist in Coded Data Buffer 13 (an arrow md2 in FIG. 32).

Figure 33 illustrates the loading of the DS1, DS10, and DS20 in normal playback. The DS1 whose PCS segment_type is Epoch Start is loaded into the Coded Data Buffer 13 as it is (step S23). However, because the ignore flag of the DS10 whose PCS segment_type is the Acquisition Point is set to 1 (step S29), the functional segments that make up the DS10 are ignored and not loaded into the Data Buffer. Encoded 13 (an arrow rd2 in FIG. 34, and step S24). Furthermore, the DS20 is loaded into the Coded Data Buffer 13, because the segment_type of the DS20's PCS is the Normal Case (an arrow rd3 in figure 34).

Next, operations by the Graphical Controller 17 are explained. Figures 35-37 illustrate a flow chart showing the operations that are performed by the Graphical Controller 17.

Steps S41-S44 are steps for a main routine of the flow chart and wait for any of the events stipulated in steps S41-S44 to take place.

Step S41 is to evaluate whether a current playing time is a time indicated by the DTS of the PCS, and if the evaluation is Yes, then an operation is performed in steps S45-S53.

Step S45 is to evaluate whether the composition_state of the OCS is the epoch_start, and if it is evaluated to be the epoch_start, the entire Graphics Plane 8 is cleared in step S46. If it is evaluated to be one other than the epoch_start, the Window indicated by the window_horizontal_position, the window_vertical_position, the window_width and the window_height of the WDS is deleted.

Step S48 is a step that is performed after erasing that is performed in step S46 or step S47, and to evaluate whether the time indicated by the PTS of any ODSx has passed. Decoding of any ODSx could have already been completed by the time erasure ends, since erasing the entirety of a Graphics Plane 8 takes time. Therefore, in steps S48, it is judged whether the decoding of any ODSx is already completed at the time the erasure ends. If the evaluation is No, the operation returns to the main routine. If the time indicated by the PTS of any ODSx has already passed, an operation is performed in steps S49-S51. In step S49, it is evaluated whether the object_crop_flag is 0, and if the flag indicates 0, then the Graphics Object is set to "no display" (step S50).

ES 2 536 680 T3

If the flag is not 0 in step S49, then an object clipped on the basis of the object_cropping_horizontal_position, the object_cropping_vertical_position, the cropping_width and the cropping_height is written to the Window on the Graphics Plane 8 at the position indicated by the object_cropping_horizontal_position and the object_cropping_vertical_position (step S51). By the above operation, one or more Graphics Objects are rendered in the Window.

In step 52, it is evaluated whether the time that corresponds to a PTS of another ODSy has passed. When the ODSx is written to the Graphics Plane 8, if the decoding of the ODSy has already been completed, then the ODSy becomes ODSx (step S53), and the operation moves to step S49. By this, the operation from steps S49-S51 is also performed for another ODS.

Next, referring to Fig. 36, step S42 and steps S54-S59 are explained hereinafter.

In step 42, it is judged whether the current playback point is in the PTS of the WDS. If the evaluation is that the current reproduction point is in the PTS of the WDS, then it is evaluated whether the number of the Window is one or not in step S54. If the evaluation is two, the operation returns to the main routine. If the evaluation is one, loop processing of steps S55-S59 is performed. In loop processing, the operations in steps S55-S59 are performed for each of the two Graphics Objects that are displayed in the Window. In step S57, it is evaluated whether the object_crop_flag indicates 0. If it indicates 0, then the Graphics are not displayed (step S58).

If this does not indicate 0, then a cropped object based on the object_cropping_horizontal_position, the object_cropping_vertical_position, the cropping_width and the cropping_height is written to the Window on the Graphics Plane 8 at the position indicated by the object_cropping_horizontal_position and the object_cropping_vertical_position (the stage S59). By repeating the above operations, more than one Graphics Object is rendered in the Window.

In step S44, it is judged whether the current reproduction point is in the PTS of the PDS. If the evaluation is that the current reproduction point is in the PTS of the PDS, then it is evaluated whether the pallet_update_flag is one or not in step S60. If the evaluation is one, the PDS indicated by pallet_id is set in the CLUT unit (step S61). If the evaluation is 0, then step S61 is skipped.

After that, the CLUT unit performs the color conversion of the Graphics Object in the Graphics Plane 8 to be combined with the moving picture (step S62).

Next, referring to Fig. 37, step S43 and steps S64-S66 are explained hereinafter.

In step 43, it is judged whether the current reproduction point is in the PTS of the ODS. If the evaluation is that the current reproduction point is in the PTS of the ODS, then it is evaluated whether the number of the Window is two or not in step S63. If the evaluation is one, the operation returns to the main routine. If the evaluation is two, the operations are performed in steps S64-S66. In step S64, it is evaluated whether the object_crop_flag indicates 0. If it indicates 0, then the Graphics are not displayed (step S65).

If it does not indicate 0, then an object clipped on the basis of the object_cropping_horizontal_position, the object_cropping_vertical_position, the cropping_width and the cropping_height is written to the Window on the Graphics Plane 8 at the position indicated by the object_cropping_horizontal_position and the object_cropping_vertical_position (step S66 ). By repeating the above operations, the Graphics Object is rendered in each Window.

The above explanations are about the DTS and the PTS of the PCS, and the DTS and the PTS of the ODS that belong to the DSn. The DTS and PTS of the PDS, and the DTS and PTS of the END are not explained. First, the DTS and PTS of the PD belonging to the DSn are explained.

With regard to the PDS belonging to the DSn, it is sufficient if the PDS is available in the CLUT unit 9 through the PCS is loaded into the Composition Buffer 16 (DTS (DSn [PCS])) after decoding the starting point of a first ODS (DTS (DSn [ODS1])). Accordingly, a PTS value of each PDS (PDS1-PDSlast) in the DSn is required to be adjusted in order to satisfy the following relationships.

DTS (DSn [PCS]) <PTS (DSn [PDS1])

DTS (DSn [PDSj]) <PTS (DSn [PDSj + 1]) <PTS (DSn [PDSlast])

PTS (DSn [PDSlast]) <DTS (DSn [ODS1])

Note that the DTS of the PDS is not referenced during playback, the DTS of the ODS is set to the same value as the PTS of the PDS in order to satisfy the MPEG2 standard.

ES 2 536 680 T3

Next, an explanation is given about the roles of the DTS and the PTS in the pipeline processing of the playback apparatus when the DTS and the PDS are adjusted in order to satisfy the above relationships. Fig. 38 illustrates the conduit of the playback apparatus based on the PTS of the PDS. Figure 38 is based on Figure 26. A first row in Figure 38 indicates the ODS setting in the CLUT unit 9. Below the first row are the same as the first to fifth rows in figure 26. The adjustment of the PDS1 - PDSlast to the CLUT unit 9 is done after the transfer of the PCS and the WDS and before decoding. of the ODS1 and consequently the setting of the PDS1-PDSlast to the CLUT unit 9 is set before a point indicated by the DTS of the ODS1 as shown by the arrows up2 and up3.

As described above, the adjustment of the PDS is performed before the decoding of the ODS.

Next, an END PTS setting of the Display Set segment in the DSn is explained. The END belonging to the DSn indicates the end of the DSn and, consequently, it is necessary that the PTS of the END indicates the end time of decoding the ODS2. The end of decoding time is indicated by the PTS (PTS (DSn [ODSlast])) of the ODS2 (ODSlast) and therefore the PTS of the END is required to be set to a value that satisfies the equation hereinafter .

DTS (DSn [END]) = PTS (DSn [ODSlast])

In terms of a relationship between the DSn and the PCS belonging to the DSn + 1, the PCS in the DSn is loaded into the Composition Buffer 16 before a load time of the first ODS (ODS1) and therefore the PTS of the END should be after a load time of the PCS in the DSn and before a load time of the PCS that belongs to the DSn + 1. Consequently, the PTS of the END is required to satisfy a relationship hereafter.

DTS (DSn [PCS]) <PTS (DSn [END]) <DTS (DSn + 1 [PCS])

On the other hand, the loading time of the first ODS (ODS1) is before a loading time of a last PDS (PDSlast) and, therefore, the PTS of the END (PTS (DSn [END])) should be after of a PDS load time belonging to the DSn (PTS (DSn [PDSlast])). Consequently, the PTS of the END is required to satisfy a relationship hereafter.

PTS (DSn [PDSlast]) <PTS (DSn [END])

An explanation is given below about the importance of the PTS of the END in the conduit processing of the playback apparatus. Fig. 39 is a diagram describing the importance of END in the duct process of the reproduction apparatus. Figure 39 is based on Figure 26, and each row in Figure 39 is substantially the same as that of Figure 26, with the exception of that, a first row in Figure 3-9 indicates the content of the Composition 16. Furthermore, in figure 39, 2 Display Sets, DSn and DSn + 1 are illustrated. The ODSlast in the DSn is the last ODSn of A-ODSs and, consequently, the point indicated by the PTS of the END is before the DTS of the PCS in the DSn + 1.

Using the PTS of the END, it is possible to find out when the loading of the ODS into the DSn is completed during playback.

Note that although the DTS of the END is not referenced during playback, the DTS of the END is set to the same value as the PTS of the END in order to satisfy the MPEG2 standard.

As described above, a part of the Graphics Plane is specified as the Window for displaying the Graphics according to the present embodiment, and therefore the playback apparatus does not have to render the Graphics for the whole. of a Plane. The rendering apparatus can render the Graphics only for a predetermined Window size, such as 25% to 33% of the Graphics Plane. Because rendering of Graphics other than Graphics in Window is not necessary, the load for software on the playback apparatus is decreased.

Even in a worst case where the update of the Graphics is performed such as 1/4 of the Graphics Plane, it is possible to display the Graphics synchronously with the image by performing by the reproduction apparatus the writing on the Plane. Graphics at a predetermined transfer rate such as 256Mbps, and by adjusting the Window size to ensure image-synchronized display.

Therefore, it is possible to obtain a high-resolution subtitle display for various playback apparatus, because the synchronized display is easily ensured.

ES 2 536 680 T3

[Second embodiment]

In the first embodiment, the Window size is set to 1/4 of the entire Graphics Plane and the write rate Rc in the Graphics Plane is set to 256 Mbps, in order to update the Graphics for each video frame. Also, by adjusting the Refresh Rate to be 1/2 or 1/4 of the Video Frame Rate, it becomes possible to update a larger size of Graphics. However, when the update rate is 1/2 or 1/4 of the video frame rate, writing to the Graphics Plane takes 2 or 4 frames. When a Graphics Plane is provided, a process of writing Graphics during the 2 or 4 frames during which Graphics are written becomes visible to a user. In such a case, a display effect such as switching from a Graph to a larger Graph at a time may not be effectively achieved. Therefore, in a second embodiment, two Graphics Planes are provided. Fig. 40 illustrates an internal structure of a reproduction apparatus according to the second embodiment. The reproduction apparatus in Fig. 40 is new compared to the reproduction apparatus according to Figs. 24 and 25 in that the reproduction apparatus in Fig. 40 has two Graphics Planes (a Graphics Plane 81 and a Graphics 82 in the drawing), and the two Graphics Planes constitute a double buffer. Consequently, it is possible to write on one of the Graphics Planes while reading is done from the other of the Graphics Planes. Furthermore, a Graphics Controller 17 according to the second embodiment switches the Graphics Plane which is acquired at a point indicated by the PTS of the PCS.

Figure 41 schematically illustrates an acquisition and writing operation on the Graphics Planes that constitute the double buffer. An upper row indicates contents of Graphics Plane 81, and a lower row indicates contents of Graphics Plane 82. The contents of both of the Graphics Planes per frame are illustrated from a first frame to a fifth frame (from left to right) . A portion of Graphics Planes 81 and 82 for each frame that are surrounded by a thick line is an acquisition target. In the drawing, a one-sided mark is contained in Graphics Plane 81, and the one-sided mark is to be replaced by a mark of a sun found in Object Buffer 15. A mark size of a sun is 4 Mbytes, which is a maximum size of the Object Buffer 15.

To write the mark of a sun on the Graphics Plane 82 at the write rate on the Graphics Plane (Rc = 256 Mbps), it takes 4 frames until the writing is complete, and only 1/4 of the mark of a sol is written to Graphics Plane 82 during the first frame, 2/4 during the second frame, and 3/4 during the third frame. Because the Graphics Plane 81 is the target to be displayed on the screen, however, the process of writing a sun mark on the Graphics Plane is not visible to the user. In the fifth frame, when the display target switches to the Graphics Plane 82, the contents of the Graphics Plane 82 become visible to the user. Therefore, the switching from the one-sided mark to the one-sun mark has been completed.

As described above, according to the second embodiment, it is possible to switch the display on the screen to another graphic at the same time even when writing large graphics on the Graphics Plane for four frames and, therefore, it is useful with displays such as credits, a movie outline, or a warning, at the same time on the entirety of a screen.

[Third embodiment]

A third embodiment relates to a BD-ROM manufacturing process. Fig. 42 is a flow chart illustrating the manufacturing process of the BD-ROM according to the third embodiment.

The manufacture of the BD-ROM includes a material manufacturing step S201 for producing material and recording movies and sound, a creating step S202 for generating an application format using a creation apparatus, and a pressing step S203 for manufacturing a disc. BD-ROM master and press to finalize the BDROM.

The BD-ROM creation step includes steps S204-S209 as follows.

In step S204, the WDS is described in order to define the Window in which subtitles are displayed, and in step S205, a period of time during which the window is defined to appear in the same position therein. size, set as an Epoch, and the PCS for each Epoch is described.

After obtaining the OCS in the above manner, the Graphics as subtitle material are converted into the ODS, and the Display Set is obtained by combining the ODS with the PCS, the WDS and the PDS in step S206. Next, in step S207, each functional segment in the Display Set is divided into the PES packets, and the Graphics Stream is obtained by attaching the timestamp.

Finally, in step S208, the AVClip is generated by multiplexing the graphics stream with the video stream and the audio stream that are generated separately.

ES 2 536 680 T3

After obtaining the AVClip, the application format is completed by setting the AVClip to BD-ROM format.

[Other issues]

The above explanations do not illustrate all embodiments in accordance with the present invention. The present invention can also be obtained by modified examples shown hereinafter. The inventions described in the claims of the present application include the above embodiments as well as extensions or generalizations of the modified examples. Although the degree of extension and generalization is based on characteristics of technological levels of the related art at the time of application, the inventions according to the claims of the present application reflect the means to solve technical problems in the art. conventional and therefore the scope of the invention does not exceed the technological scope that those skilled in the art would recognize as means of solving technical problems in the conventional art. Therefore, the inventions according to the claims of the present application correspond substantially with the descriptions of the details of the invention.

(1) The BD-ROM is used in the explanations of all of the above embodiments. However, the characteristics of the present invention are found in the Graphics Stream that is recorded on a medium, and such characteristics are not dependent on physical properties of the BD-ROM. Any recording medium that is capable of storing the Graphics Stream can be achieved by the present invention. Examples of such a recording medium include optical discs such as a DVD-ROM, a DVD-RAM, a DVD-Rw, a DVD-R, a DVD + RW, a DVD + R, a CD-R, and a CD- RW, magneto-optical discs such as a PD and MO, semiconductor memory cards such as a Compact Flash card, a smart media, a Memory Stick memory, a multimedia card, and a PCM-CIA card, and magnetic discs such as a floppy disk, a SuperDisk, a Zip and a Clik! and removable hard drives such as an ORB, Jaz, SparQ, SyJet, EZFley, and Microdrive, plus integrated hard drives.

(2) The playback apparatus described in all of the above embodiments decodes the AVClip recorded on the BD-ROM and outputs the decoded AVClip to a TV. However, it is also possible to obtain the present invention by the playback apparatus including only a BD-ROM drive, and the TV provided with other items. In this case, the playback device and the TV can be connected by IEEE1394 to create a home network. Furthermore, although the playback apparatus in the embodiments is used by connecting to the TV, the playback apparatus may be a TV and playback apparatus all in one. Furthermore, the LSI (integrated circuit) only which forms an essential part of the processing in the reproducing apparatus of each embodiment can be implemented. Both such a reproduction apparatus and the LSI are described in the present specification and, therefore, the manufacture of a reproduction apparatus on the basis of the internal structure of the reproduction apparatus according to the first embodiment is an implementation of the present invention, no matter what working example it may entail. Furthermore, the transfer either as a gift or profit, the loan and the importation of the reproduction apparatus according to the present invention is also considered to be the implementation of the present invention. Offering such a transfer and loan to common users as a showcase display and brochure distribution is also considered to be the implementation of the present invention.

(3) The information processing that is executed by a program shown in the flowcharts is obtained using hardware resources and, consequently, the program whose processing is shown in each flowchart is stated only as an invention . Although all of the above embodiments describe the program according to the present invention as integrated in the reproduction apparatus, the program according to the first embodiment only can be implemented. Examples of program-only implementation include (i) producing the programs, (ii) transferring the programs as a gift or profit, (iii) lending the programs, (iv) importing the programs, (v) providing the general public with the programs through a line of interactive electronic communications, and (vi) offer the transfer and loan to common users as a showcase display and distribution of brochures.

(4) The time elements in the steps that are performed in a sequential order in each flow chart are essential features of the present invention, and it is obvious that the process shown in each of the flow charts discloses a method Of reproduction. Carrying out the processes that are illustrated by the flowcharts by carrying out the operation in each stage sequentially to obtain the object of the present invention and obtain the effects is the implementation of the recording method according to the present invention.

(5) It is desirable to add an extension header to each of the packets that make up the AVClip when writing to BD-ROM. The extension header is a 4-byte piece of data called TP_extra_header that includes arrival_time_stamp and copy_permission_indicator. TS packets that have the TP_extra_header (hereinafter referred to as TS packets with EX) are grouped by 32 packets and written in 3 sectors. A group that includes 32 TS packets with EX is 6144 bytes (= 32 x 192), which is the same size as a size of 3 sectors 6144 bytes (= 2048 x 3). The group of 32 TS Packets with EX that are stored in 3 sectors is called the Aligned Drive.

When the playback device is used in the home network connected by IEEE1394, the playback device

ES 2 536 680 T3 reproduction transmits the Aligned Unit in the following transmission procedure. A sender obtains a Tp_extra_header from each of the 32 TS Packets with EX that are included in the Aligned Unit, and emits the main body of the TS packets after decoding based on the DTCP standard. When TS packets are broadcast, isochronous packets are inserted between any two successive TS packets. The insertion points are positions based on the time indicated by arrival_time_stamp in TP_extra_header. Along with the output of the TS packets, the playback apparatus issues a DTCP_descriptor. The DTCP_descriptor indicates a setting for the copy permission. By describing the DTCP_descriptor to indicate that copying is prohibited, TS packets are not recorded by other devices when used in the home network connected via IEEE1394.

(6) The digital stream in the above embodiments is the AVClip. However, the digital stream can be a Video Object (VOB) in the DVD-Video standard or the DVD-Video Recording standard. VOB is a program stream based on the ISO / IEC13818-1 standard that is obtained by multiplexing the video stream and the audio stream. Furthermore, the video stream in the AVClip can also be based on the MPEG4 or WMV standard. Also, the audio stream can be based on Linear PCM, Dolby-AC3, MP3, MPEG-AAC or DTS.

(7) The film in the above embodiments can be obtained by encoding analog image signals that are transmitted by means of analog broadcast, or it can be a stream data constituted by a transport stream that is transmitted by means of digital broadcast.

It is also possible to obtain content by encoding analog or digital image signals that are recorded on a video tape. In addition, the contents can also be obtained by encoding analog or digital image signals that are loaded directly from a video camera. In addition, the contents can be a digital work delivered by a distribution server.

(8) The Graphics Object in the first and second embodiments is a scan grating data that is encoded on the basis of run-length limited encoding. Run-length limited encoding is adopted to compress and encode the Graphics Object because run-length limited encoding is the most appropriate for compressing and decompressing subtitles. Subtitles have the characteristics that a length in a horizontal direction becomes relatively long, and consequently a high compression rate is obtained through the use of run-length limited encoding. In addition, encoding limited by run length is preferable for making software to decode because the load on decompression is low. Furthermore, in order to share the structure of the decoding apparatus between the subtitles and the Graphics Object, the same compression / decompression method as for the subtitles is used for the Graphics Object. However, using run-length limited encoding is not an essential part of the present invention, and the Graphics Object may be PNG data. Also, the Graphics Object is not required to be the scan grid data and may be vector data. Also, the Graphics Object can be transparent graphics.

(9) A target for the display effect by the PCS may be the graphics for the subtitles which are selected on the basis of a language setting of the playback apparatus. Obtaining such a display has a high practical value, because it becomes possible to obtain an effect, which is obtained by the moving picture itself on the conventional DVD, by the subtitle graphics which are displayed according to the setting of language of the playback device.

(10) A target for the display effect by the PCS may be the graphics for the subtitles which are selected on the basis of a display setting of the playback apparatus. Specifically, Graphics for various display modes such as wide view, panoramic scan, and landscape format are recorded on the BD-ROM, and the playback apparatus selects any of the recorded settings based on the setting for the TV. to which the playback device is connected. In this case, the display effect on the basis of the PCS is realized for the subtitle graphics that are displayed according to the display setting, the subtitles appear more impressive and professional. Obtaining such a display has a high practical value, because it becomes possible to obtain an effect similar to the effect that is obtained in the motion picture itself on the conventional DVD, by means of the subtitle graphics that are being displayed according to the display setting of the playback device.

(11) In the first embodiment, the Window size is adjusted to be 25% of the entire Graphics Plane in order to adjust the write rate Rc in the Graphics Plane to the rate at which the Erasing the Graphics Plane and redrawing is done in a frame. However, the Rc can be adjusted such that erasing and redrawing are completed during a vertical retrace period. Since the vertical lag period is 25% of 1 / 29.93 seconds, the Rc is 1 Gbps. Adjusting the Rc in such a way has a high practical value, because it is possible to display the graphs more smoothly.

In addition, it is also possible to perform synchronous writing with a line scan, in addition to writing during the vertical retrace period. By this, it is possible to display graphics more smoothly even at Rc write rate is 256Mbps.

(12) In the above embodiments, the Graphics Plane is mounted on the reproduction apparatus. However, it is also possible to mount a line buffer for storing decompressed pixels for a line at the location of the Graphics Plane in the reproduction apparatus. Conversion to image signals is performed per line, and therefore conversion to image signals can be carried out with the line buffer only.

(13) In the above embodiment, the explanations are given by taking the text subtitles for the movie as the examples of the graphics. However, the charts may include such as a combination of devices,

ES 2 536 680 T3 characters, and colors that constitute a trademark, a national emblem, a national flag, a national emblem, a symbol and a great seal for supervision or certification used by the government of a country, a shield, a flag or emblem of an international body, or a mark of origin of a particular article.

(14) In the first embodiment, the Window for rendering the subtitles is defined either in an upper part of the screen, or in the lower part of the screen, assuming that the subtitles are written horizontally. However, the Window can be set to appear on either the left or right side of the screen in order to display subtitles on the left and right of the screen. In this way, it is possible to change the direction of the text and display subtitles vertically.

(15) The AVClip in the above embodiments constitutes the movie. However, the AVClip can also be used for karaoke. In this case, the PCS can realize the display effect such that the color of the subtitles changes along with a song.

BD unit reference numbers

Read Buffer Memory

PID filter

TB Buffer

Video Decoder

Video Plane

Audio decoder

Graphics plane unit of CLUT adder

Graphics Decoder

Coded Data Buffer

Flow Graphics Processor

Composition Buffer

Graphics Controller

200 reproduction apparatus

300 TV

400 remote controller

Industrial application susceptibility

A recording medium and a reproducing apparatus according to the present invention are capable of displaying subtitles with a display effect. Consequently, it is possible to add higher values ​​to the films supplied in the market, and activate markets for films and consumer products. Therefore, the recording medium and the reproducing apparatus according to the present invention have high industrial applicability in the industry, such as the motion picture industry and the consumer products industry.