Image decoding apparatus and image decoding method

The present invention relates to an image decoding apparatus and an image decoding method for performing image processing.

On the conventional decoding side, before analyzing the VOP header information, VOP that does not need to be analyzed (information that is dropped in the case of dropped frames of the image signal) and VOP that needs to be analyzed (dropped frames of the image signal) In that case, it was indistinguishable from information that is not dropped.

<p> Since the conventional device is configured as described above, the VOP start code, modulo time base, and VOP time increment included in each VOP header must be analyzed, which makes the processing cumbersome and accurate. There was a problem that it could lead to a decline.</p><p> In addition, when decoding and synthesizing encoded signals in units of objects such as the subject, background, and logo that make up the image, each object has a synthesis timing signal (absolute) required for decoding and synthesizing. Information representing the time) must be added. Since the image decoding device cannot synthesize each object without obtaining the information expressing the absolute time, it is impossible to regenerate the image. In short, when trying to create one image from a plurality of objects including an object that does not have information indicating absolute time, there is a problem that composition with an object that does not have information indicating absolute time becomes impossible. In addition, the modulo time-based bit length increases until the next GOV header is multiplexed, and if the optional GOV header is not multiplexed, the modulo time-based bit length increases. There was the issue of continuing to do so.</p><p> The present invention has been made to solve the above-mentioned problems, and provides an image decoding device and an image decoding method for preventing the generation of an unnecessary amount of information, which improves processing accuracy by simple processing. The purpose is.</p>

<p> The image decoding apparatus according to the present invention is composed of a plurality of VOPs and is encoded and included in the header information portion of a layer higher than the VOP layer, and VOP rate information indicating the number of VOPs to be displayed per unit time is VOP. It is provided with a VOP rate information decoding means for decoding when the rate is fixed.</p><p> The image decoding method according to the present invention includes a plurality of VOPs encoded in the header information portion of a layer higher than the VOP layer, and VOP rate information indicating the number of VOPs to be displayed per unit time is VOP. It is provided with a VOP rate information decoding step for decoding when the rate is fixed.</p>

<p> According to the present invention, the image restoration process can be performed more smoothly and accurately with a simple process using a small amount of coded transmission. It has the effect of preventing the generation of an unnecessary amount of information and realizing a system that obtains a reproduced image by synthesizing a plurality of objects with a simple configuration.</p>

Embodiment 1. In the present embodiment 1, the MPEG-4 video coding method disclosed in ISO / IEC JTC11 SC29 / WG11 / N1796 is applied based on the display speed information of the object which is an element of the present embodiment. A VOP encoder having a means for encoding and a means for adding display speed information used for each object and multiplexing the coded bit stream will be described.

MPEG-4 is a method that regards a moving image sequence as a collection of moving image objects that take an arbitrary shape in time / space, and encodes / decodes each moving image object as a unit. Figure 1 shows the video data structure in MPEG-4. In MPEG-4, the moving image object including the time axis is called a video object [Video Object (VO)], the component of VO is called a video object layer [Video Object Layer (VOL)], and the component of VOL is a group. It is called the Group of Video Object Plane (GOP), and it represents the state of each time of GOV, and the image data that is the unit of coding is the Video Object Plane [Video Object]. Plane (VOP)]. For example, VO corresponds to each speaker and background in the video conference scene, VOL is a unit with unique temporal and spatial resolutions such as those speakers and background, and VOP is each of those VOLs. Image data at time (= corresponding to a frame). GOV is a data structure that is a unit for editing or random access by collecting multiple VOPs, and does not necessarily have to be used for coding.

A specific example of VOP is shown in Fig. 2. In the figure, two VOPs (VOP1 is a person and VOP2 is a painting on the wall) are shown. Each VOP consists of texture data representing the color shading level and shape data representing the shape of the VOP. The texture data consists of an 8-bit luminance signal and color difference signal (size subsampled to 1/2 in the horizontal and vertical directions with respect to the luminance signal), and the shape data is 1 inside the VOP and 0 outside the VOP. It is binary matrix data that is the same as the image size of the luminance signal.

In the moving image representation by VOP, the conventional frame image is obtained by arranging a plurality of VOPs on the screen. However, if there is one VO in the moving image sequence, each VOP is synonymous with a frame. In this case, there is no shape data and only the texture data is encoded.

Hereinafter, the image coding apparatus according to the first embodiment will be described. This is based on an MPEG-4 video encoder, and since the MPEG-4 video decoder performs encoding in units of the above VOP, it is hereinafter referred to as a VOP encoder. Since the operation of the existing VOP encoder is disclosed in ISO / IEC JTC1 / SC29 / WG11 / N1796, etc., the explanation of the existing VOP encoder itself is avoided here, and the explanation of the VOP encoder including the elements of the first embodiment is described. Do.

FIG. 3 shows a configuration example of the VOP encoder according to the first embodiment. 110 is a coded VOP determination unit, 111 is a shape coding unit, 113 is a motion estimation unit, 115 is a motion compensation unit, and 118 is a texture code. The conversion unit, 122 is a memory, 124 is a header multiplexing unit, 126 is a video signal multiplexing unit, 128 is a subtractor, and 129 is an adder.

Next, the operation will be described. The coded VOP determination unit 110 determines the VOP to be coded in the input object image based on the VOP rate information 7 set according to the external setting and the coding status, and determines the VOP to be coded. It is output to the shape coding unit 111, the motion estimation unit 113, and the subtractor 128. Here, the VOP rate information 7 corresponds to the display speed information in the present invention, and how many VOPs included in units such as VOL and GOV are displayed per second (fixed rate). , Or information indicating whether the rate is variable.

A specific example of the operation of the coded VOP determination unit 110 is shown. The fixed rate indicates a case where the number of VOPs included in a unit such as VOL or GOV to be displayed per second is always constant in the VOL or GOV. For example, when the input object image is 30 images / second and the VOP rate information 7 is 15 images / second, the coded VOP determination unit 110 has one VOP to be encoded among the VOPs included in the input object image. Judges that it is every other, and outputs the VOP to be encoded every other sheet. This indicates a case where the information indicating how many images are displayed per second is fixed, and every other VOP to be encoded among the VOPs included in the input object image by the coded VOP determination unit 110. The reason why it can be determined is that the information indicating how many images are displayed per second is always at a constant interval (15 images / second). .. The variable rate refers to the case where the rate is not fixed, and the number of sheets to be displayed in a certain period (second unit) included in a unit such as VOL or GOP and the period included in the unit such as VOL or GOV. The case where it is different from the number of sheets to be displayed in (second unit) is shown.

The VOP specified as the coding target by the coding VOP determination unit 110 has shape data for each area of 16 pixels × 16 pixels called an alpha block, and texture data for each area of 16 pixels × 16 pixels called a macro block. Encode for each region of.

The shape coding unit 111 encodes the input alpha block and outputs the shape coding information 112 and the locally decoded shape information 109. The shape coding information 112 is sent to the video signal multiplexing unit 126, and the locally decoded shape information 109 is input to the motion estimation unit 113, the texture coding unit 115, and the texture coding unit 118. The motion estimation unit 113 inputs the reference data 123a in the memory 122, performs block matching in macroblock units, and obtains motion information 114. At this time, motion information is obtained by block matching targeting only the objects included in the macroblock based on the locally decoded shape information 109.

The motion compensation unit 115 inputs the reference data 123b at the position indicated by the motion information 114 in the memory 122, and creates a predicted image based on the locally decoded shape information 109. The predicted image 116 created by the motion estimation unit 115 is input to the subtractor 128 and the adder 129. The subtractor 128 calculates the difference between the predicted image 116 and the input macroblock, and creates the predicted error image 117. The texture coding unit 118 encodes the input prediction error image 117 by a predetermined method defined by MPEG-4 to obtain texture coding information 119 and a local decoding prediction error image 120. At this time, coding is performed only for the objects included in the block based on the locally decoded shape information 109. The texture coding information 119 is sent to the video signal multiplexing unit 126, and the local decoding prediction error image 120 is output to the adder 129.

The adder 129 adds the predicted image 116 and the local decoding prediction error image 120 to create a decoded image 121, and writes the decoded image 121 to the memory 122. Each header information is multiplexed in the header multiplexing unit 124, and the bit stream 125 in which each header information is multiplexed is input to the video signal multiplexing unit 126. The video signal multiplexing unit 126 multiplexes the shape coding information 112, the motion information 114, and the texture coding information 119 into the bit stream 125 in which each header information is multiplexed, and outputs the coded VOP bit stream.

FIG. 4 is a block diagram showing the configuration of the header multiplexing unit of FIG. In the figure, 1 is the VO header multiplexing unit, 2 is the VOL header multiplexing unit, 3 is the GOV header multiplexing selection unit, 4 is the GOV header multiplexing unit, 5 is the VOP header multiplexing unit, and 6 is the GOV multiplexing information. , 7 are VOP rate information.

Next, the operation will be described. The VO header multiplexing unit 1 creates a bit stream in which the VO header information is multiplexed, and outputs the created bit stream to the VOL header multiplexing unit 2. The VOL header multiplexing unit 2 multiplexes the input bit stream with the VOL header information, and outputs the multiplexed bit stream to the GOV header multiplexing selection unit 3.

The GOV header multiplexing selection unit 3 determines the output destination of the bit stream output from the VOL header multiplexing unit 2 based on the GOV multiplexing information 6 indicating whether or not to perform the GOV header multiplexing. If the GOV multiplexing information 6 indicates that the GOV header is not multiplexed, the VOP header multiplexing section 5 is contacted. If the GOV multiplexing information 6 indicates that the GOV header is multiplexed, the GOV header multiplexing is performed. Output the bit stream to part 4.

The GOV header multiplexing unit 4 multiplexes the VOP rate information 7 to the input bit stream, and outputs the bit stream to the VOP header multiplexing unit 5. Table 1 shows an example of the above VOP rate information 7, and shows an example of expressing four types of VOP rates. If the VOP rate is 30 sheets / sec, multiplex "01". If the VOP encoded immediately before and the VOP to be encoded are the same, the VOP rate information "00" is multiplexed and the subsequent VOP header information and VOP data information are not multiplexed. If the VOP rate is variable, the VOP rate information "11" is multiplexed. That is, the VOP rate information 7 indicates whether the VOP rate is fixed or variable, and also indicates the value of the rate when the VOP rate is fixed.

The VOP start code multiplexing unit 8 in the VOP header multiplexing unit 5 modulates the VOP start code on the input bit stream into a modulo time base (modulo _time_base) multiplexing unit 9 and VOP. Time increment (VOP_time_increment) Output to the multiplexing unit 10.

Here, as shown in FIG. 5, the modulo time base 13 is information indicating how many seconds have passed since the VOP is a certain reference time, and the VOP time increment 14 is the same as the figure. As shown in 5, it is information that fine-tunes the display time with an accuracy of 1/1000 second from the time determined by the modulo time base. That is, in MPEG-4, the display time of VOP can be specified with an accuracy of 1/1000 second.

The management time creation unit 12 in the VOP header multiplexing unit 5 creates a modulo time base 13 and a VOP time increment 14 based on the VOP rate information 7, and multiplexes the modulo time base 13 to the modulo time base. The VOP time increment 14 is output to the conversion unit 9 and the VOP time increment multiplexing unit 10 is output. However, to indicate that the VOP rate information 7 is variable, the modulo time base 13 and the VOP time increment 14 are set regardless of the VOP rate information 7.

The above-mentioned Modulo Time Base Multiplexing Unit 9 multiplexes the Modulo Time Base 13 to the bit stream output from the VOP Start Code Multiplexing Unit 8, and VOP Time Increment Multiplexes the multiplexed bit stream. Output to unit 10. This VOP time increment multiplexing unit 10 multiplexes the VOP time increment 14 output from the management time creation unit 12 to the bit stream output from the modular time base multiplexing unit 9, and the bits stream after multiplexing. Is output to the video information header multiplexing unit 11. The video information header multiplexing unit 11 multiplexes the video information header on the bit stream output from the VOP time increment multiplexing unit 10, and outputs the multiplexed bit stream to the video signal multiplexing unit 126.

As described above, according to the first embodiment, the GOV header is configured to multiplex the VOP rate information. Therefore, if only the VOP start code of each VOP header is analyzed on the decoder side, the VOP to be decoded can be obtained. It has the effect of creating a bitstream that allows you to determine if decoding is needed and to easily combine multiple objects.

As shown in FIG. 6, VOP rate information may be defined in units of VOL, and coding and multiplexing of VOP rate information may be performed. In this case, the VOP rate information 7 is determined in VOL units and is multiplexed by the VOL header multiplexing unit 2. Based on this, the modulo time base 13 and the VOP time increment 14 are determined.

As described above, in the first embodiment, in the image coding device that encodes the image in object units, the coding means that encodes the image based on the predetermined display speed information and the coding means are used for coding. An embodiment of a device provided with a multiplexing means for multiplexing and outputting the predetermined display speed information on a digitized image-encoded signal has been disclosed. Further, in the first embodiment, the multiplexing means discloses one embodiment in which the display speed information is multiplexed for each object.

Embodiment 2. In the present embodiment 2, another embodiment of the VOP encoder described in the first embodiment will be described. In the VOP encoder according to the second embodiment, the display speed information includes a 1-bit VOP rate flag indicating whether the display speed of the object is a fixed speed or a variable speed, and VOP rate information indicating the value of the display speed of the object. It is provided with a means for encoding the above and multiplexing it into a bit stream.

When the VOP rate flag indicates a variable speed, the VOP rate corresponds to variable in Table 1 described in the first embodiment, and when the VOP rate flag indicates a fixed speed, the first embodiment. In Table 1 described above, the VOP rate corresponds to 30 sheets / second or 15 sheets / second.<tables num="1"><img file="JP2004166311A_D0001.tif" /></tables>

FIG. 7 shows a configuration example of the VOP encoder according to the second embodiment of the present invention, where 1000 is a header multiplexing unit, 1001 is a VOP rate flag, and 1026 is a VOP rate. Since the VOP encoder according to the second embodiment differs only in the configuration operation of the header multiplexing unit 1000 corresponding to the header multiplexing unit 124 of the VOP encoder described in the first embodiment, only this portion will be described.

FIG. 8 is a block diagram showing a configuration of a header multiplexing unit 1000 of the VOP encoder unit according to the second embodiment of the present invention. In the figure, 1002 is a VOL header multiplexing section and 1003 is a VOP header multiplexing section.

Next, the operation will be described. The VO header multiplexing unit 1 creates a bit stream in which the VO header information is multiplexed, and outputs the created bit stream to the VOL header multiplexing unit 1002. The VOL header multiplexing unit 1002 multiplexes the input bit stream with the VOL header information, and outputs the multiplexed bit stream to the GOV header multiplexing selection unit 3. At this time, the VOP rate information and the VOP rate flag are also multiplexed.

Table 2 shows an example of multiplexing with a VOP rate of 1026. In this case, if the VOP rate 1026 is 2 sheets / sec, "000", if the VOP rate is 5 sheets / sec, "001", if the VOP rate is 25 sheets / sec, "010", VOP. When the rate is 30 sheets / sec, "011" is multiplexed, and when the VOP rate other than the above (for example, the VOP rate is 10 sheets / sec), "100" is multiplexed as VOP rate information.<tables num="2"><img file="JP2004166311A_D0002.tif" /></tables> The VOP rate information does not depend on the value of the VOP rate flag, which will be described later, and the propriety of multiplexing is independently determined. In addition, Table 3 may be used as another example of VOP rate multiplexing. In this case, if all VOPs in the VOL are exactly the same image, it is regarded as a still image and "101" is multiplexed as VOP rate information.<tables num="3"><img file="JP2004166311A_D0003.tif" /></tables>

Regarding the VOP rate flag, "1" is multiplexed when the VOP rate flag indicates a fixed speed, and "0" is multiplexed when the VOP rate flag indicates a variable speed. FIG. 9 shows an example of the bit stream output from the VOL header multiplexing unit 1002. The GOV header multiplexing selection unit 3 determines the output destination of the bit stream output from the VOL header multiplexing unit 1002 based on the GOV multiplexing information 6 indicating whether or not to perform GOV header multiplexing. If the GOV multiplexing information 6 indicates that the GOV header is not multiplexed, the VOP header multiplexing unit 1003 is contacted. If the GOV multiplexing information 6 indicates that the GOV header is multiplexed, the GOV header multiplexing is performed. Part 4 Outputs a header stream.

The GOV header multiplexing unit 4 multiplexes the GOV header information in the input bit stream, and outputs the multiplexed bit stream to the VOP header multiplexing unit 1003. FIG. 10 shows the details of the VOP header multiplexing unit 1003. In FIG. 10, 1004 is a management time creation unit.

Next, the operation will be described. The management time creation unit 1004 is based on the VOP rate 1026 when the input VOP rate flag 1001 indicates a fixed speed, and the timer inside the VOP encoder when the input VOP rate flag 1001 indicates a variable speed. Create a modular time base and VOP time increment based on. The created modulo time base is output to the modulo time base multiplexing unit 9, and the created VOP time increment is output to the VOP time increment multiplexing unit 10.

The VOP start code multiplexing unit 8 multiplexes the input bit stream with the VOP start code, and outputs the multiplexed bit stream to the modulo time-based multiplexing unit 9. The modulo time-based multiplexing unit 9 performs modulo time-based multiplexing on the input bit stream, and outputs the multiplexed bit stream to the VOP time increment multiplexing unit 10.

The VOP time increment multiplexing unit 10 multiplexes the input bit stream by VOP time increment, and outputs the multiplexed bit stream to the video information header multiplexing unit 11. The video information header multiplexing unit 11 multiplexes the video information header on the bit stream output from the VOP time increment multiplexing unit 10, and outputs the multiplexed bit stream to the video signal multiplexing unit 126.

As described above, according to the second embodiment, since the VOP rate flag and the VOP rate information are multiplexed in the VOL layer, the user desires to use the VOP rate flag and the VOP rate on the decoder side. Since it is possible to instantly identify the VOP to be decrypted, by analyzing only the VOP start code of each VOP header, it is possible to determine whether or not the VOP to be decrypted needs to be decrypted, and it is easy to determine multiple objects. It has the effect of creating a bitstream that can be combined. Even if only the VOP rate flag is multiplexed, it is possible to identify whether the speed is variable or fixed, so that the VOP to be decoded can be decoded.

As described above, in the second embodiment, in the image coding device that encodes an image in units of objects, a means for encoding a flag indicating whether the display speed of the object is a fixed speed or a variable speed, and A multiplexing means that multiplexes and outputs the above flag to an image coding signal encoded by the coding means, a coding means that encodes an image based on predetermined display speed information, and the coding means. An embodiment of a device provided with a multiplexing means for multiplexing and outputting the predetermined display speed information on a coded image-encoded signal has been disclosed.

Embodiment 3. In the third embodiment, an image decoding device for decoding and outputting the VOP rate information described in the first embodiment from the coded bit stream, that is, an MPEG-4 video decoder (hereinafter, VOP). A system in which a plurality of decoders (called decoders) are provided corresponding to each object and a plurality of decoded objects are combined to reproduce an image will be described.

First, the configuration and operation of the image decoding device (VOP decoder) in the third embodiment will be described. Since the operation of the existing VOP decoder is disclosed in ISO / IEC JTC1 / SC29 / WG11 / N1796, etc., the description of the existing VOP decoder itself is avoided here, and the description of the VOP decoder including the elements of the third embodiment is described. Do. The VOP decoder according to the third embodiment is a decoder capable of decoding the coded bit stream generated by the VOP encoder described in the first embodiment.

FIG. 11 shows an example of the internal configuration of the VOP decoder according to the third embodiment of the present invention. As shown in the first and second embodiments, the VOP decoder is composed of texture data and shape data, and this decoder has a function of restoring each data by inputting data obtained by compressing and encoding these data. It shall be. In the figure, 150 is a coded VOP bit stream, 151 is a header analysis unit, 152 is a bit stream in which header information is analyzed, 153 is a video signal analysis unit, 154 is a shape coding data, 155 is a shape decoding unit, and 156 is a shape decoding unit. Decoded shape data, 157 is texture coding data, 158 is motion information, 159 is motion compensation unit, 160 is predicted texture data, 161 is texture decoding unit, 162 is decoded texture data, 164 is memory, 165 is reference data. ..

Hereinafter, the operation will be described in detail based on the figure. The coded VOP bitstream 150 is input to the header analysis unit 151, and the header information is analyzed according to a predetermined syntax. The bit stream 152 whose header information has been analyzed by the header analysis unit 151 is input to the video signal analysis unit 153 and analyzed by the shape coding data 154, the texture coding data 157, and the motion information 158. The shape decoding unit 155 decodes the input shape coding data 154 and outputs the decoded shape data 156. The motion compensation unit 159 outputs the predicted texture data 160 from the reference data 165 in the memory 164 and the motion information 158 input from the video signal analysis unit 153. The texture decoding unit 161 restores the image data by a predetermined method defined by MPEG-4 based on the texture coding data 157 and the predicted texture data 160, and generates the decoded texture data 162. Since this decoded texture data 162 is used for the subsequent decoding of the VOP, it is written to the memory 164.

FIG. 12 shows the internal configuration of the header analysis unit 151, which is a feature of the third embodiment of the present invention. In the figure, 51 is a start code analysis unit, 52 is a VO header analysis unit, 53 is a VOL header analysis unit, 54 is a GOV header analysis unit, 58 is a VOP rate information, and 55 is a VOP header analysis unit. The header analysis unit 151 according to the third embodiment is characterized in that the GOV header analysis unit 54 decodes the VOP rate information 58 of the VOP included in the GOV from the bit stream and outputs it to the outside. How to use this VOP rate information 58 will be described later.

The start code analysis unit 51 analyzes the start code included in the input coded VOP bit stream 150. If the analyzed start code indicates VO, go to the VO header analysis unit 52. If the analyzed start code indicates VOL, go to the VOL header analysis unit 53. If the analyzed start code indicates GOV. If the analyzed start code indicates VOP, the bit stream is output to the GOV header analysis unit 54 to the VOP header analysis unit 55. After completing the analysis process of the VOP header analysis unit 55, the bit stream is output to the video signal analysis unit 153.

The VO header analysis unit 52 analyzes the VO header information from the input bit stream, and outputs the analyzed bit stream to the start code analysis unit 51. The VOL header analysis unit 53 analyzes the VOL header information from the input bit stream, and outputs the analyzed bit stream to the start code analysis unit 51. The GOV header analysis unit 54 analyzes the GOV header information from the input bit stream, and outputs the analyzed bit stream to the start code analysis unit 51. At this time, the VOP rate information 58 included in the GOV header information is decoded and output. The VOP header analysis unit 55 analyzes the VOP header information from the input bit stream, and outputs the analyzed bit stream to the video signal analysis unit 153 via the start code analysis unit 51.

According to the VOP decoder based on the above configuration and operation, the VOP rate information of the VOP included in the GOV unit can be output. Figure 13 shows a system that synthesizes multiple objects using this information. In the figure, 200 is a coded VOP bitstream a, 201 is a coded VOP bitstream b, 202 is a coded VOP bitstream c, 203a is a VOP decoder that decodes the coded VOP bitstream a200, and 203b is a coded VOP. The VOP decoder section that decodes the bitstream b201, 203c is the VOP decoder section that decodes the encoded VOP bitstream c202, 204 is the decoding object image a, 205 is the decoding object image b, 206 is the decoding object image c, and 207 is the VOP rate. Information a and 208 are VOP rate information b, 209 is VOP rate information c, 210 is a composition unit, and 211 is a decoded image. The decoded object image is defined as a combination of the decoded shape data 156 of each VOP and the corresponding decoded texture data 162, and the combined VOP in a unit (for example, GOV, VOL, etc.).

The encoded VOP bitstreams a200 to c202 are decoded by the corresponding VOP decoder units 203a to 203c, respectively, and the decoded VOP images a204 to c206 are generated. At this time, each VOP decoder unit decodes the corresponding VOP rate information a207 to c209 and outputs this to the composition unit 210. Based on the VOP rate information a207 to c209, the composition unit 210 determines which time image frame of the decoded image 211 is to be combined with each decoded VOP image, and maps the decoded VOP image to the image frame at the corresponding time. To do. For example, assume that the decoded image 211 is displayed at 30 images per second (which corresponds to the display speed of a normal television signal). Furthermore, the following situations are assumed. Decoded VOP image a204 is displayed at 5 images per second (that is, VOP rate information a207 represents 5 images / second). The decoded VOP image b205 is displayed at 10 images per second (that is, the VOP rate information b208 represents 10 images / second). Decoded VOP image c206 is displayed at 15 images per second (that is, VOP rate information c209 represents 15 images / second). In this case, all of the decoded VOP images a204 to c206 are mapped to the image frame at the beginning of each second of the decoded image 211, and the decoded VOP image a204 is mapped to every five image frames from the beginning of each second, and each second. The decoded VOP image b205 is mapped to every 10 image frames from the beginning of each second, and the decoded VOP image c206 is mapped to every 15 image frames from the beginning of each second. As a result, it is possible to display an image in which a plurality of image objects are combined into an image frame according to each display speed.

As described above, by using a VOP decoder that decodes a coded bit stream in which VOP rate information is encoded in the GOV layer, it is possible to realize a system that synthesizes a plurality of objects with a simple configuration to obtain a reproduced image. Is possible. The VOP rate information may be encoded on the image coding apparatus side in units of VOL. In this case, the image decoding apparatus side can decode the VOP rate information encoded in VOL as a unit, and can synthesize a plurality of simple objects as described above in VOL as a unit.

Further, in the third embodiment, the VOP decoder is used as a system for synthesizing a plurality of objects, but a configuration in which only one VOP decoder is used in a system for decoding and reproducing only one object is also possible. As described above, in the third embodiment, the display speed information decoding for decoding the display speed information from the coded bit stream in the image decoding device for decoding the coded bit stream in which the image is encoded for each object. An embodiment is disclosed that includes means and control means that controls reproduction processing of an image processed in object units based on display speed information decoded by the display speed information decoding means. Further, in the third embodiment, the display speed information decoding means discloses an embodiment in which the display speed information is decoded for each object.

Embodiment 4. In the present embodiment 4, another embodiment of the VOP decoder described in the third embodiment will be described. The VOP decoder according to the fourth embodiment has a function of identifying and decoding the VOP to be decoded based on the value of the VOP rate assumed by the decoder. Since the VOP decoder of the fourth embodiment differs only in the configuration operation of the header analysis unit 151 of the VOP decoder described in the second embodiment, only this member will be described.

FIG. 14 is a block diagram showing a configuration of a header analysis unit of the VOP decoder unit according to the fourth embodiment of the present invention, in which the VOP rate on the encoder side and the VOP rate on the decoding side do not match. In the figure, 59 is a decoding VOP selection unit, and outputs VOP selection information 62 by comparing the VOP rate 58 output from the GOV header analysis unit 54 with the VOP rate 61 assumed on the decoding side. Further, the VOP header analysis unit 55 has a counter unit 60 in addition to the time management information header analysis unit 56 and the video information header analysis unit 57.

Next, the operation will be described. The decoding VOP selection unit 59 uses VOP selection information 62 that indicates VOP information to be decoded based on a comparison between the VOP rate 58 analyzed by the GOV header analysis unit 54 and the VOP rate 61 assumed by the decoder side. Output to counter unit 60 of 55. The counter unit 60 determines based on the VOP selection information 62 whether or not to decode the VOP header information following the VOP start code included in the input bit stream.

Specifically, when the VOP rate 58 analyzed by the GOV header analysis unit 55 is 30 sheets / sec and the VOP rate assumed by the decoder side is 15 sheets / sec, it indicates that there is a VOP that analyzes every 1 VOP. The VOP selection information 62 is output to the counter unit 60 in the VOP header analysis unit 55. In the counter unit 60, first, every time the VOP header is input, the counter 60a counts.

Next, the determination device 60b determines whether or not it is necessary to analyze the input VOP based on the count number input from the counter 60a and the VOP rate selection information 62 input from the decorator VOP selection unit 59. When it is determined that it is necessary to analyze the input VOP, the input bit stream is output to the time management information header analysis unit 56. If it is determined that it is not necessary to analyze the input VOP, the input bit stream is output to the start code analysis unit 51.

A specific example is shown below. If the VOP rate selection information 62 is information that one VOP needs to be analyzed for three VOPs, in the judgment device 60b, the remainder obtained by dividing the count number input from the counter 60a by 3 is 0. If the remainder is 1 or 2 obtained by dividing the number of counts input from the counter 60a by 3, it is judged as a VOP that does not require analysis. In the fourth embodiment, the VOP decoder corresponding to the case where the GOV header contains the VOP rate information has been described, but as described in the second embodiment, the VOP rate information is included in the VOL header. You may. In that case, as shown in FIG. 15, the VOL header analysis unit 300 may be provided with a decoding function for VOP rate information 58. Further, the VOP decoder according to the fourth embodiment can be used in a system that synthesizes a plurality of objects or a system that decodes and reproduces only one object.

As described above, in the fourth embodiment, the control means is based on the display speed information of the object decoded by the display speed information decoding means and the display speed information of the object preset in the decoding device. Disclosed an embodiment of a device comprising a decoding time specifying means for specifying a time to be decoded in the object and a decoding means for decoding the object based on the decoding target time obtained by the decoding time specifying means. did.

Embodiment 5. In the present embodiment 5, another embodiment of the VOP decoder described in the third embodiment or the fourth embodiment will be described. The VOP decoder in the fifth embodiment sets a VOP rate flag indicating whether the display speed of the object is a fixed speed or a variable speed, VOP rate information indicating the display speed of the object, and time information set by the user from the outside. It shall have a function to identify and decode the VOP to be decoded based on the external setting display control information and the time code shown.

As shown in FIG. 16, the VOP decoder of the fifth embodiment differs only in the configuration operation of the header analysis unit 1005 corresponding to the header analysis unit 151 of the VOP decoder described in the third embodiment, and thus only this part will be described. To do. FIG. 17 is a block diagram showing a configuration of a header analysis unit 1005 of the VOP decoder unit according to this embodiment. In the figure, 1006 is a VOL header analysis unit, 1007 is a GOV header analysis unit, 1008 is a VOP header analysis unit, 1009 is an external setting display control information, 1010 is a VOP rate flag, and 1011 is a time code. The external setting display control information 1009 may be information indicating an absolute time, or may be VOP selection information indicating how many VOPs one VOP needs to be decoded. ..

Next, the operation will be described. The start code analysis unit 51 analyzes the start code included in the input coded VOP bit stream. If the analyzed start code indicates VO, go to the VO header analysis unit 52. If the analyzed start code indicates VOL, go to the VOL header analysis unit 1006. If the analyzed start code indicates GOV. If the analyzed start code indicates VOP, the VOP header analysis unit 1008 heavy stream is output to the GOV header analysis unit 1007. After completing the analysis process of the VOP header analysis unit 1008, the bit stream is output to the video signal analysis unit 153.

Next, the above-mentioned VO header analysis unit 52 analyzes the VO header from the input bit stream, and outputs the analyzed bit stream to the start code analysis unit 51. In addition, the VOL header analysis unit 1006 analyzes the VOL header, VOP rate information 58, and VOP rate flag 1010 from the input bit stream, outputs the analyzed bit stream to the start code analysis unit 51, and VOP. The rate information 58 is output to the composition unit 210 and the VOP header analysis unit 1008, and the VOP rate flag 1010 is output to the VOP header analysis unit 1008.

The GOV header analysis unit 1007 analyzes the GOV header from the input bitstream, outputs the analyzed bitstream to the start code analysis unit 51, and the time code 1011 included in the analyzed GOV header. Is output to the VOP header analysis unit 1008.

FIG. 18 is a diagram showing details of the VOP header analysis unit 1008. Reference numeral 1012 is a decoding VOP determination unit (1), which has a counter unit 1012a and a determination device 1012b. 1013 is a modulo time-based analysis unit, 1014 is a VOP time increment analysis unit, 1015 is a decoding VOP determination unit (2), and 1016 is a decoding VOP determination method selection unit.

Next, the operation will be described. The decoding VOP determination method selection unit 1016 selects the output destination of the input bit stream based on the VOP rate flag 1010. When the VOP rate flag 1010 indicates a fixed speed, the decoding VOP determination unit (1) 1012 is used as the output destination, and when the VOP rate flag 1010 indicates a variable speed, the modulo time-based analysis unit 1013 is used as the output destination.

First, a case where the VOP rate flag 1010 indicates a fixed speed will be described. The counter unit 1012a in the decoding VOP determination unit (1) 1012 increments the count number each time the VOP start code is detected by the start code analysis unit 51 and a bitstream is input to the VOP header analysis unit 1006, and the count number is counted. And output the bitstream to the judge 1012b. Next, the determination device 1012b determines whether or not it is necessary to decode the VOP to be decoded. Regarding the operation of the judgment device 1012b, the case where the external setting display control information 1009 is given in absolute time is the first case, and the case where the external setting display control information 1009 is given in the VOP selection information is the second case. explain.

(First case) The absolute time of the VOP to be decoded is calculated based on the count number input from the counter unit 1012a, the VOP rate information 58, and the time code 1011. For example, when the count number is 4, the VOP rate information indicates 2 sheets / sec, and the absolute time is 0h10m0sec0msec, the absolute time of the VOP to be decoded is calculated as 0h10m02sec0msec. If the calculated absolute time of the VOP to be decoded and the external setting display control information 1009 are equal, it is determined that decoding is necessary.

On the other hand, if they are not equal, the absolute time of the VOP to be decoded next is calculated. This is to compare the absolute time of the VOP to be decoded next with the absolute time of the VOP currently to be decoded, and to decode the VOP with an absolute time closer to the external setting display control information 1009. To make it. Next, the absolute time of the VOP to be decoded is calculated from the already calculated absolute time of the VOP currently to be decoded and the VOP rate information 58. If this calculated value does not exceed or is equal to the external setting display control information 1009, it is determined that the VOP to be decoded next is to be decoded, and the VOP currently to be decoded is not decoded. If the calculated value exceeds the external setting display control information 1009, Decrypt the VOP currently targeted for decoding Decrypt the VOP currently targeted for decoding (= VOP currently targeted for decoding is decoded No) Decryption of VOP with an absolute time close to the external setting display control information 1009, that is, the difference from the external setting display control information 1009 may be selected.

(Second case) This is a case where the display speed is controlled on the VOP decoder side. For example, the user can determine the display speed, or the optimum display speed can be specified according to the CPU resource. Become.

Next, the operation will be described. Suppose that the VOP selection information is information that one VOP needs to be decoded for three VOPs. In this case, the determination device 1012b determines that the VOP needs to be decoded when the remainder obtained by dividing the count number input from the counter unit 1012a by 3 becomes 0, and determines the count number input from the counter unit 1012a. If the remainder divided by 3 is 1 or 2, it is judged as a VOP that does not need to be decrypted.

In both the first case and the second case, when it is determined that the VOP to be decoded needs to be decoded, it is determined that it is not necessary to decode the bitstream to the modulo time-based analysis unit 1013. In this case, the input bit stream is output to the start code analysis unit 51. The modulo time-based analysis unit 1013 performs modulo time-based analysis and outputs the VOP time increment analysis unit 1014 hebitstream. The VOP time increment analysis unit 1014 analyzes the VOP time increment and outputs the video information header analysis unit 57 heavy stream. The video information header analysis unit 57 analyzes the video information header and outputs the start code analysis unit 51 heavy stream.

Next, a case where the VOP rate flag 1010 indicates a variable speed will be described. The modulo time-based analysis unit 1013 performs modulo time-based analysis and outputs the VOP time increment analysis unit 1014 hebitstream. The VOP time increment analysis unit 1014 analyzes the VOP time increment and outputs the decoding VOP determination unit (2) 1015 hebit stream.

The decoding VOP determination unit (2) 1015 is based on the modulo time base analyzed by the modulo time base analysis unit 1013, the VOP time increment analyzed by the VOP time increment analysis unit 1014, and the time code 1011. , The absolute time of the VOP to be decoded is created, and it is determined whether or not it is necessary to decode the VOP to be decoded based on the created absolute time and the external setting display control information 1009. If it is determined that decoding is necessary, the bitstream is output to the video information header analysis unit 57, and if it is determined that decoding is not necessary, the bitstream is output to the start code analysis unit 51. The video information header analysis unit 57 analyzes the video information header and outputs the start code analysis unit 51 heavy stream.

As described above, according to the fifth embodiment, the VOL layer is configured to be able to analyze the bit stream in which the VOP rate flag and the VOP rate information are encoded. Therefore, if the VOP rate flag and the VOP rate are used, The user can instantly identify the desired VOP, and by analyzing only the VOP start code included in each VOP header information, it is possible to determine whether or not the VOP to be decrypted needs to be decrypted. It has the effect of easily synthesizing multiple objects. If all the VOPs included in the coded VOP bitstream input to the VOP decoder are intra-coded, there is also an effect that the VOP desired by the user can be instantly identified and displayed. ..

As described above, in the fifth embodiment, when the display speed identification information decoded by the display speed information decoding means indicates a fixed speed , the control means indicates a variable speed based on the display speed information. An embodiment is disclosed in which the display time of the image at each time is specified based on the display time information multiplexed for each image at each time and the reproduction is characterized.

Embodiment 6. In the present embodiment 6, another embodiment of the VOP decoder described in the fifth embodiment will be described. The VOP decoder according to the sixth embodiment shows a VOP rate flag indicating whether the display speed of the object is a fixed speed or a variable speed, a VOP rate indicating the display speed of the object, and time information set by the user from the outside. It shall have a function to identify and decode the VOP to be decoded based on the external setting display control information and the time code.

FIG. 19 is a diagram showing a header analysis unit according to the sixth embodiment of the present invention. Since the VOP decoder of the sixth embodiment differs only in the configuration operation of the VOL header analysis unit 1006 and the VOP header analysis unit 1008 in the header analysis unit described in the fifth embodiment, only this member will be described. The VOL header analysis unit 1017 analyzes the VOL header, VOP rate information, and VOP rate flag from the input bit stream, sends the analyzed bit stream to the start code analysis unit 51, and analyzes the VOP rate flag 1010 in the VOP header. When outputting to unit 1018 and indicating that the analyzed VOP rate information is some fixed rate value (for example, the VOP rate indicated by the VOP rate information "100" in Table 2), the VOP rate information 58 is analyzed by the VOP header. When the analyzed VOP rate information indicates a certain unique value to part 1018 (for example, the VOP rate indicated by the VOP rate information "000", "001", "010", "011" in Table 2), the VOP rate information 58 is displayed. Output to VOP header analysis unit 1018 and composition unit 210.

FIG. 20 is a diagram showing details of the VOP header analysis unit 1018. Reference numeral 1025 is a decoding VOP determination method selection unit, and 1019 is a decoding VOP determination unit (3), which includes a counter unit 1019a, a count number determination unit 1019b, and a determination device 1019c. 1020 is a time information holding unit, 1021 is a VOP rate information calculation unit, 1022 is a VOP rate information holding unit, 1023 is a modular time-based analysis unit, and 1024 is a VOP time increment analysis unit.

The decoding VOP determination method selection unit 1025 selects the output destination of the input bit stream based on the input VOP rate flag 1010 and the VOP rate information 58. Specifically, when the VOP rate flag 1010 indicates a fixed speed and the VOP rate information 58 indicates some fixed rate value, the decoding VOP determination unit (3) 1019 is set as the output destination. Further, when the VOP rate flag 1010 indicates a variable speed, the operation is as described in the fifth embodiment, and thus the description thereof will be omitted. If the VOP rate flag 1010 indicates a fixed speed and the VOP rate information 58 indicates an eigenvalue, the decoding VOP determination unit (1) outputs a 1012 heavy stream. In this case, the operation after the decoding VOP determination unit (1) 1012 is the same as that described in the fifth embodiment, and thus the description thereof will be omitted. Therefore, the case where the VOP rate flag 1010 indicates a fixed speed and the VOP rate information 58 indicates some fixed rate value will be described below.

The counter unit 1019a in the decoding VOP determination unit (3) 1019 increments the count number each time the VOP start code is detected by the start code analysis unit 51 and a bitstream is input to the VOP header analysis unit 1018, and the count number is counted. And the bitstream are output to the count number determination unit 1019b. When the count number indicates the first VOP or the second VOP, the count number determination unit 1019b outputs the bit stream and the count number to the modulo time base analysis unit 1023, and in cases other than the above, the count number determination unit 1019b outputs the bit stream and the count number. Output the bitstream and count number to the judge 1019c.

The modular time-based analysis unit 1023 analyzes the modular time base, and if the input count number indicates the first VOP, it is sent to the time information holding unit 1020, and the input count number is 2. When indicating the first VOP, the VOP rate information calculation unit 1021 hemodulo time base is output, and the bit stream and the count number are output to the VOP time increment analysis unit 1024. The VOP time increment analysis unit 1024 analyzes the VOP time increment, and if the input count number indicates the first VOP, it is sent to the time information holding unit 1020, and the input count number is the second VOP. Is displayed, the VOP time increment is output to the VOP rate information calculation unit 1021, and the bit stream is output to the video information header analysis unit 57. The video information header analysis unit 57 analyzes the video information header and outputs the start code analysis unit 51 heavy stream.

The time information holding unit 1020 holds the input modulo time base and the VOP time increment. When the modular time base and VOP time increment for the second VOP are input, the VOP rate information calculation unit 1021 is the same as the modular time base for the first VOP from the time information holding unit 1020. The VOP time increment related to the first VOP is input, the VOP rate information is calculated based on these, and the VOP rate information is output to the VOP rate information holding unit 1022. A specific example of the calculation of VOP rate information by the VOP rate information calculation unit 1021 when the VOP time increment is expressed with 6-bit precision is shown below.

The modular time base for the first VOP is "10", the VOP time increment for the first VOP is "000000" (that is, the time information for the first VOP is 1.0 second), and the second VOP. If the modular time base for the second VOP is "10" and the VOP time increment for the second VOP is "100000" (that is, the time information for the second VOP is 1.5 seconds), the difference between the two time information is 0.5. It will be seconds. This means that there is one VOP to be decoded every 0.5 seconds, that is, the VOP rate is 2 sheets / second (using Table 2, the VOP rate information is "1111"). Even if the VOP rate information 58 is not multiplexed, as long as the VOP rate flag 1010 is multiplexed, it can be determined that the rate is fixed, so that the above operation is possible.

The VOP rate information holding unit 1022 holds the input VOP rate information and outputs the VOP rate information to the composition unit 210. Regarding the operation of the judgment device 1019c, the case where the external setting display control information 1009 is given in absolute time is explained below as the first case, and the case where the external setting display control information 1009 is given in the VOP rate is explained below as the second case. To do.

(First case) Judgment device 1019c calculates the absolute time of the doubled target VOP based on the count number input from the count number determination unit 1019b and the VOP rate information output from the VOP rate information holding unit 1022. .. If the calculated absolute time of the VOP to be decoded and the external setting display control information 1009 are equal, it is determined that decoding is necessary.

On the other hand, if they are not equal, the absolute time of the VOP to be decoded next is calculated. This is to compare the absolute time of the VOP to be decoded next with the absolute time of the VOP currently to be decoded, and to decode the VOP with an absolute time closer to the external setting display control information 1009. To make it. Next, the absolute time of the VOP to be decoded is calculated from the already calculated absolute time of the VOP currently to be decoded and the VOP rate information 58. If this calculated value does not exceed or is equal to the external setting display control information 1009, it is determined that the VOP to be decoded next is to be decoded, and the VOP currently to be decoded is not decoded. If the calculated value exceeds the external setting display control information 1009, Decrypt the VOP currently targeted for decoding Decrypt the VOP currently targeted for decoding (= VOP currently targeted for decoding is decoded No) Decryption of VOP with an absolute time close to the external setting display control information 1009, that is, the difference from the external setting display control information 1009 may be selected.

(Second case) In the judgment device 1019c, the VOP rate given by the external setting display control information 1009 is 2 sheets / sec, and the VOP rate indicated by the VOP rate information output from the VOP rate information holding unit 1022 is 4 sheets / sec. In the case of seconds, the VOP selection information indicating how many VOPs one VOP needs to be decoded is the information that one in two VOPs needs to be decoded. In this case, the determination device 1019c determines that the VOP in which the remainder obtained by dividing the count number input by the count number determination unit 1019b by 2 is 0 is a VOP that needs to be decoded, and inputs the count number determination unit 1019b. If the remainder of dividing the number of counts to be performed by 2 is 1, it is judged as a VOP that does not need to be decrypted.

In both the first case and the second case, when it is determined that the VOP to be decoded needs to be decoded, it is determined that it is not necessary to decode the bitstream to the modulo time-based analysis unit 1013. In this case, the input bit stream is output to the start code analysis unit 51. Modulo time-based analysis unit 1013 performs modular time-based analysis and outputs VOP time increment analysis unit 1014 heavy stream, VOP time increment analysis unit 1014 analyzes VOP time increment, and video information header Analysis unit 57 outputs a heavy stream, video information header analysis unit 57 analyzes the video information header and outputs a start code analysis unit 51 heavy stream.

As described above, according to the sixth embodiment, when the bit stream in which the VOP rate flag and the VOP rate information are encoded in the VOL layer can be analyzed and the VOP rate flag indicates a fixed speed, the first sheet is used. Since the VOP rate information is calculated from the absolute time of the VOP and the second VOP, the VOP rate flag and the VOP rate can be used to instantly identify the VOP desired by the user. By analyzing the VOP start code included in each VOP header information for the fixed VOP rate of, it is possible to determine whether or not the VOP to be decrypted needs to be decoded, and to easily synthesize multiple objects. There is an effect that can be done. If all the VOPs included in the coded VOP bitstream input to the VOP decoder are intra-coded, there is also an effect that the VOP desired by the user can be instantly identified and displayed. ..

As described above, in the sixth embodiment, the control means is a value in which the display speed information decoded by the display speed information decoding means indicates fixed and the fixed speed is not represented by the display speed information. In some cases, an embodiment of the one characterized in that the display time of the image at each time is specified and the reproduction is controlled based on the display time information multiplexed for each image at each time is disclosed.

Embodiment 7. In the present embodiment 7, another embodiment of the VOP encoder described in the first embodiment will be described. The VOP encoder according to the seventh embodiment has a function of adding a time code that defines the absolute display time of each VOP included in the VOL in units of VOL. Here, the time code is the time information disclosed in the IEC standard publication 461 for time and control codes for video tape recorders, and is an image of each time (a frame in MPEG-2) that constitutes a moving image. , VOP in MPEG-4, etc.) is information that defines the display time with the accuracy of hours, minutes, and seconds. For example, when editing on a frame-by-frame basis with a commercial video editing device, by adding this information to each frame, the desired frame can be accessed simply by specifying the time code value. Have.

Since the VOP encoder of the seventh embodiment differs only in the configuration operation of the header multiplexing unit 124 of the VOP encoder described in the first embodiment, only this member will be described. FIG. 21 is a block diagram showing a configuration of a header multiplexing unit of the VOP encoder unit according to the seventh embodiment of the present invention, and the same parts as those of the first embodiment shown in FIG. 4 are designated by the same reference numerals and described in duplicate. Is omitted.

Next, the operation will be described. The bitstream in which the VO header information is multiplexed in the VO header multiplexing unit 1 is input to the VOL header multiplexing unit 2. The VOL header multiplexing unit 2 outputs a bit stream in which the input bit stream is multiplexed with the VOL header information and the time code 18 which is the basis of time management, to the GOV header multiplexing selection unit 3. The GOV header multiplexing selection unit 3 determines the output destination of the bit stream output from the VOL header multiplexing unit 2 based on the GOV multiplexing information 6 indicating whether or not to perform the GOV header multiplexing. If the GOV multiplexing information 6 indicates that the GOV header is not multiplexed, the VOP header multiplexing section 5 is contacted. If the GOV multiplexing information 6 indicates that the GOV header is multiplexed, the GOV header multiplexing is performed. Output the bit stream to part 4. In this case, the GOV header multiplexing unit 4 multiplexes the GOV header information in the bit stream output from the GOV header multiplexing selection unit 3 and outputs it to the VOP header multiplexing unit 5. The VOP header multiplexing unit 5 outputs a bit stream obtained by multiplexing the VOP start code, the time management information header, and the video information header to the input bit stream to the video signal multiplexing unit 126 (see FIG. 3). The operations after the video signal multiplexing unit 126 are the same as those described above.

As described above, according to the seventh embodiment, since the time code is multiplexed in the VOL header that is always encoded by MPEG-4, it is possible to create a screen composed of a plurality of objects based on the time code. Bitstream can be configured. Further, when the editing operation is performed while decoding the coded bit stream according to the seventh embodiment on a commercial video object unit editing device or the like, random access to the VOP at any time of the object is always freely available. It has the effect of being possible. From such an effect, the degree of freedom of video composition can be increased. In the seventh embodiment, the encoder that adds the time code in the unit of VOL has been described, but the time code information may be added in the unit of VOP. In this case, as shown in FIG. 22, a time code 18 that defines the absolute display time of each VOP may be input to the VOP header multiplexing unit 301 to be configured to multiplex the time code 18. Further, in the seventh embodiment, an example involving coding of the VOP rate information is shown, but of course, the multiplexing of the time code is independent of the VOP rate information, and even when the VOP rate information is not encoded. A similar effect can be obtained.

As described above, in the seventh embodiment, in the image coding device that encodes the image for each object, the information expressing the absolute time for the object is multiplexed with the encoded image signal for each object. An embodiment of an object provided with absolute time multiplexing means has been disclosed.

Embodiment 8. In the present embodiment 8, a system provided with a plurality of VOP decoders that decode and output a time code from a VOL header in a coded bit stream, and synthesizes a plurality of decoded objects to reproduce an image. explain. First, the configuration and operation of the VOP decoder according to the eighth embodiment will be described. FIG. 23 shows the internal configuration of the VOP decoder according to the eighth embodiment. Since this decoder differs only in the header analysis unit 302 from the configuration operation of the VOP decoder described in the second embodiment, only this member will be described below. The header analysis unit 302 has a function of decoding and outputting the time code in the VOL header.

FIG. 24 shows the internal configuration of the header analysis unit 302. In the figure, 303 is a VOL header analysis unit. The start code analysis unit 51 analyzes the start code included in the input coded VOP bit stream 150. If the analyzed start code indicates VO, go to the VO header analysis unit 52. If the analyzed start code indicates VOL, go to the VOL header analysis unit 303. If the analyzed start code indicates GOV. If the analyzed start code indicates VOP, the bit stream is output to the GOV header analysis unit 54 to the VOP header analysis unit 55. After completing the analysis process of the VOP header analysis unit 55, the bit stream is output to the video signal analysis unit 153.

The VO header analysis unit 52 analyzes the VO header information from the input bit stream, and outputs the analyzed bit stream to the start code analysis unit 51. The VOL header analysis unit 303 analyzes the VOL header information from the input bit stream, and outputs the analyzed bit stream to the start code analysis unit 51. At this time, the time code 64 included in the VOL header information is decoded and output. The GOV header analysis unit 54 analyzes the GOV header information from the input bit stream, and outputs the analyzed bit stream to the start code analysis unit 51. The VOP header analysis unit 55 analyzes the VOP header information from the input bit stream, and outputs the analyzed bit stream to the video signal analysis unit 153 via the start code analysis unit 51.

According to the VOP decoder based on the above configuration and operation, the absolute display time of the VOP included in the VOL unit can be output. Figure 25 shows a system that synthesizes multiple objects using this information. In the figure, 400 is a coded VOP bitstream a, 401 is a coded VOP bitstream b, 402 is a coded VOP bitstream c, 403a is a VOP decoder that decodes the coded VOP bitstream a400, and 403b is a coded VOP. The VOP decoder section that decodes the bitstream b401, 403c is the VOP decoder section that decodes the encoded VOP bitstream c402, 404 is the decoding object image a, 405 is the decoding object image b, 406 is the decoding object image c, and 407 is the time code. a and 408 are the time code b, 409 is the time code c, 410 is the composition part, and 411 is the decoded image. The decoded object image refers to the decoded shape data 156 of each VOP and the corresponding decoded texture data 162, which are summarized in a unit (for example, GOV, VOL, etc.) for summarizing the VOPs. The coded VOP bitstream a400 to coded VOP bitstream c402 are decoded by the corresponding VOP decoder units 403a to 403c, respectively, and the decoded object images a404 to c406 are generated. At this time, each VOP decoder unit decodes the corresponding time codes a407 to c409 and outputs this to the composition unit 410. Based on the same time codes a407 to c409, the composition unit 410 determines which time image frame of the decoded image 411 the VOP of each time of each decoded object image is combined with, and corresponds to the corresponding time. Map to the image frame of. For example, assume the following situation. -The composition unit has a time code generation function and determines the absolute display time of each image frame to be combined. -It is assumed that 01:00:00 is decoded as the time code of the first VOP of the decoded object image a404. Here, 01:00:00 represents (hours) :( minutes) :( seconds). -It is assumed that 01:00:10 is decoded as the time code of the first VOP of the decoded object image b405. -It is assumed that 01:01:00 is decoded as the time code of the first VOP of the decoded object image c406.

Here, assuming that the time code of the first image frame of the decoded image 411 defined by the composition unit 410 is 01:00:00, the decoded object image a404 is mapped from the first frame of the decoded image 411, and the decoded object image The b405 can be mapped from 10 seconds after the first frame of the decoded image 411, and the decoded object image c406 can be mapped from 1 minute after the first frame of the decoded image 411 and displayed on the screen. As a result, it is possible to display a video composed of a plurality of video objects in an image frame according to an absolute time as a reference.

As described above, by using the VOP decoder that decodes the coded bit stream in which the time code is encoded in the VOL layer, it is possible to realize a system that synthesizes multiple objects with a simple configuration to obtain a reproduced image. Is. As shown in FIG. 26, the time code may be encoded on the image coding apparatus side in units of VOP. In this case, on the image coding apparatus side, it is possible to decode the time code encoded in units of VOL and synthesize a plurality of simple objects as described above for each VOP.

Further, as shown in FIG. 27, a VOP decoder in which a coded bit stream in which VOP rate information is multiplexed is input together with a time code in the VOL header can be considered. In this case, the absolute display time of the VOP at the beginning of the VOL is determined by the time code, and then the absolute display time of each VOP can be easily known from the VOP rate information, so a multiple object composition system can be configured more easily. can do. Further, in the eighth embodiment, the VOP decoder is used as a system for synthesizing a plurality of objects, but a configuration in which only one VOP decoder is used in a system for decoding and reproducing only one object is also possible. ..

As described above, in the eighth embodiment, in the image decoding device that decodes the coded bit stream in which the image is encoded for each object, the information expressing the absolute time for the object is analyzed for each object. An embodiment of an absolute time analysis means and one that reproduces an image processed in object units based on information expressing an absolute time analyzed by the absolute time analysis means has been disclosed.

Embodiment 9. In Embodiment 9, the representation of the modulo time base (corresponding to the first time information) and the VOP time increment (corresponding to the second time information) currently used in MPEG-4. The expression method which improved the modulo time-based coding method and the VOP encoder which realizes it will be described.

Prior to that, the expression method of the modulo time base 20 in MPEG-4 will be explained first. As described in the first embodiment, the modulo time-based value is information indicating how many seconds after the reference time of the VOP is displayed as shown in FIG. It is expressed by the number of bits of the value "1". Specify the end of data by adding the value "0". That is, it becomes "111110" after 5 seconds. In this representation method, if the reference time does not change at all, the amount of information on the modulo time base becomes infinitely large. Currently, in MPEG-4, this reference time is specified by the time code multiplexed in the GOV header, but since GOV is an option, the GOV header must be encoded as the MPEG-4 specification. There is no. This means that unless the GOV header appears, the modulo timebase value can be infinitely long. Embodiment 9 implements an encoder that avoids such problems in encoding modulo time-based data.

In the ninth embodiment, only the configuration operation of the header multiplexing unit 124 of the VOP encoder described so far is changed, so only this member will be described. FIG. 28 shows the internal configuration of the header multiplexing unit 124 according to the ninth embodiment of the present invention. 500 is the VOP header multiplexing unit, 19 is the bit length arithmetic unit, 20 is the modulo time base, 21 is the shifted modulo time base, 22 is the information bit indicating the number of repetitions, and 501 is the modulo time base multiplexing. It is a chemical department.

Next, the operation will be described. The bitstream in which the VO header information is multiplexed in the VO header multiplexing unit 1 is input to the VOL header multiplexing unit 2. This VOL header multiplexing unit 2 multiplexes the VOL header information in the input bit stream, and outputs the multiplexed bit stream to the GOV header multiplexing selection unit 3.

The GOV header multiplexing selection unit 3 determines the output destination of the bit stream output from the VOL header multiplexing unit 2 based on the GOV multiplexing information 6 indicating whether or not to perform the GOV header multiplexing. If the GOV multiplexing information 6 indicates that the GOV header is not multiplexed, the VOP header multiplexing section 5 is sent. If the GOV multiplexing information 6 indicates that the GOV header is multiplexed, the GOV multiplexing section is used. Output the bit stream to 4. In this case, the GOV header multiplexing unit 4 multiplexes the GOV header information in the bit stream output from the GOV header multiplexing selection unit 3 and outputs it to the VOP header multiplexing unit 5.

The VOP start code multiplexing unit 8 in the VOP header multiplexing unit 500 multiplexes the input bit stream with the VOP start code, and outputs the multiplexed bit stream to the modular time-based multiplexing unit 501. To do. The bit length calculation unit 19 in the VOP header multiplexing unit 500 compares the bit length of the modulo time base 20 with a preset threshold value that takes a positive value, and bits of the modulo time base 20. If the length is longer, the modulo time base 20 is repeatedly left-shifted by the length of the threshold until the bit length of the modulo time base 20 falls below the above threshold, and the resulting bit string is the shift. Outputs the modulo time base 21 and the information bit 22 indicating the number of repeated shifts. The information bit 22 indicating the number of repeated shifts may be expressed in binary notation in which the number of repeated shifts is expressed by a predetermined fixed number of bits, or may be expressed in variable bit length notation in which the number of repeated shifts is expressed by a variable length code. Good.

A specific example of the operation in the bit length calculation unit 19 is shown below. When the above threshold value is set to 4, if the modulo time base 20 is "1111111110", the number of repeated shifts is 2, and the shifted modulo time base 21 is "10". The information bit 22 indicating the number of repeated shifts is "10" when expressed by a fixed length of 2 bits. The modular time base multiplexing unit 501 in the VOP header multiplexing unit 500 shifts the shifted modular time base 21 and the information bit 22 indicating the number of repeated shifts to the bit stream output from the VOP start code multiplexing unit 8. The bitstream in which the above is multiplexed is output to the VOP time increment multiplexing unit 10. The VOP time increment multiplexing unit 10 outputs a bit stream obtained by multiplexing the VOP time increment to the bit stream output from the modular time base multiplexing unit 501 to the video information header multiplexing unit 11. The video information header multiplexing unit 11 outputs a bit stream obtained by multiplexing the video information header to the bit stream output from the VOP time increment multiplexing unit 10 to the video signal multiplexing unit 26.

As described above, according to the ninth embodiment, the modular time base is represented by two types of information bits (a shifted modular time base and an information bit indicating the number of repeated shifts), and in MPEG-4. Since it is configured to multiplex the above two types of information bits instead of encoding with the currently defined modular time-based representation as it is, it is possible to suppress the amount of information generated compared to the representation method in MPEG-4. There is an effect.

As described above, in the ninth embodiment, in the image coding device that encodes an image for each object, the time from the reference time to the display time is used as information that defines the display time of the image at each time for each object. The first time information that defines the first time information, the second time information that defines the display time with finer precision than the time specified by the first time information, and the time information that encodes the image corresponding to each time. A coding means is provided, and the time information coding means expresses the first time information by converting it into a bit length, and the bit length of the first time information is longer than a predetermined set value. , An embodiment in which the bit shift for the set value is repeated until the value becomes shorter than the set value, the number of times the bit shift is executed is counted, and the number of times the bit shift is executed and the bit string obtained from the result of the repeated bit shift are encoded. Was disclosed.

Embodiment 10. In the present embodiment 10, the modulo time-based information multiplexed into the encoded bitstream by the modulo time-based multiplexing unit 501 described in the ninth embodiment is restored, and this is restored. A VOP decoder that defines the display time of each VOP based on the VOP time increment and the VOP time increment will be described.

In the tenth embodiment, only the configuration operation of the header analysis unit 151 of the VOP decoder described above is changed, and only this member will be described. FIG. 29 shows the internal configuration of the header analysis unit 151 according to the tenth embodiment of the present invention. 502 is the VOP header analysis unit, 65 is the modulo time base analysis unit, 66 is the VOP time increment analysis unit, 67 is the modulo time base calculation unit, 69 is the shifted modulo time base, and 70 is the number of repeated shifts. It is an information bit indicating.

Next, the operation will be described. The start code analysis unit 51 analyzes the start code from the code VOP bit stream in which the input shifted modular time base 69 and the information bit 70 indicating the number of repeated shifts are multiplexed, and the analyzed start code is VO. If it is included in the header, go to the VO header analysis unit 52. If the analyzed start code is included in the VOL header, go to the VOL header analysis unit 53. If the analyzed start code is included in the GOV header. Go to GOV header analysis unit 54, if the analyzed start code is included in the VOP header, go to VOP header analysis unit 55, if the analyzed start code is included in the VOP data information, go to video signal analysis unit 153 (Fig.) 11) Output the bit stream 152 to. The operations after the video signal analysis unit 153 are the same as those described above. The modular time base analysis unit 65 in the VOP header analysis unit 502 analyzes the shifted modular time base 69 and the information bit 70 indicating the number of repeated shifts from the bit stream output from the start code analysis unit 51. , The shifted modular time base 69 and the information bit 70 indicating the number of repeated shifts are output to the modular time base calculation unit 67, and the bit stream is output to the VOP time increment analysis unit 66.

The modulo time base calculation unit 67 calculates the modulo time base from the input shifted modulo time base 69 and the information bit 70 indicating the number of repeated shifts, and outputs the modulo time base to the composition unit 210. Specifically, the modulo time base value is restored by reversing the procedure shown in the ninth embodiment. Shift the threshold value that takes a preset positive value by 4 (this must be set on the decoder side exactly the same as the same threshold value shown in the encoder example of the ninth embodiment). When the modulo time base 69 is "10" and the information bit 70 indicating the number of repeated shifts is "10", "1111111110", which is the high-order bit of "10" plus "11111111", is the modulo time base. It becomes the restored value. The obtained modulo time-based restoration value is used together with the VOP time increment information for the purpose of defining the display time of the VOP.

The VOP time increment analysis unit 66 analyzes the VOP time increment on the bit stream output from the modular time base analysis unit 65, and outputs the analyzed bit stream to the video information header analysis unit 57. The video information header analysis unit 57 analyzes the video information header in the bit stream output from the VOP time increment analysis unit 66, and outputs the analyzed bit stream to the video signal analysis unit 153.

As described above, according to the tenth embodiment, two types of information bits (a shifted modular time base and an information bit indicating the number of repetitions) are used to calculate the modular time base. Therefore, MPEG. There is an effect that it becomes possible to analyze the bit stream described in the twelfth embodiment in which the amount of information generated is suppressed as compared with the coded expression specified in -4.

As described above, in the tenth embodiment, in the image display device that decodes the bit stream in which the image is encoded for each object, the reference time is used as the information that defines the display time of the image at each time for each object. The first time information that defines the time from to the display time and the second time information that defines the display time with finer precision from the time specified by the first time information are displayed as images corresponding to each time. The time information decoding means is provided with a time information decoding means for decoding and a decoding / synthesizing means for decoding an input coded image signal for each object and synthesizing these decoded image signals, and the time information decoding means is the first time. By decoding the number of times the bit shift is performed and the bit string obtained from the result of the repeated bit shift as the coded data of the information, and adding a code having a predetermined set value length to the bit string for the number of times the bit shift is performed. One of those that decodes the first time information and synthesizes the decoded image signal based on the first time information and the second time information decoded by the time information decoding means. The examples have been disclosed.

Embodiment 11. In the present embodiment 11, another representation method that improves the modulo time-based coding method in the representation of the modulo time base and VOP time increment currently used in MPEG-4. And the VOP encoder that realizes it will be described.

In the eleventh embodiment, only the configuration operation of the header multiplexing unit 124 of the VOP encoder described so far is changed, so only this member will be described. FIG. 30 shows the internal configuration of the header multiplexing unit 124 in the eleventh embodiment. 503 is the VOP header multiplexing unit, 23 is the modulo time base holding unit, 24 is the differential modulo time base creating unit, 25 is the differential modulo time base multiplexing unit, and 26 is the differential modulo time base. is there. The VOP start code multiplexing unit 8 in the VOP header multiplexing unit 503 multiplexes the VOP start code to the input bit stream, and converts the multiplexed bit stream to the differential modular time-based multiplexing unit 25. Output. The modulo time base holding unit 23 in the VOP header multiplexing unit 503 holds the value of the modulo time base of the VOP encoded immediately before, and the modulo time base of the VOP encoded immediately before. Is output, then the modulo timebase of the VOP to be encoded is written. The difference modulo time base creation unit 24 in the VOP header multiplexing unit 503 is the modulo time base of the VOP encoded immediately before and the VOP to be encoded, which are input from the modulo time base holding unit 23. The difference bit string from the modulo time base is calculated, the difference modulo time base 26 is obtained based on the number of "1" bits included in the calculated difference bit string, and the difference modulo time base multiplexing unit 25 is used. Output.

Here, a specific example of differential modulo time base generation is shown. Difference when the modulo time base of the VOP encoded immediately before is "11110" (decimal display: 30) and the modulo time base of the VOP to be encoded is "111110" (decimal display: 62). The bit string is "100000" (decimal number display: 32). Next, the number of "1" bits included in the difference bit string "100000" obtained by the previous calculation is one. When the difference modulo time base is calculated using the conversion table as shown in Table 4, the difference modulo time base corresponding to one "1" bit is "10", so "10". Is output as a differential modulo time base. The conversion table in Table 4 is an example, and other conversion tables may be defined and used.

Another possible representation of the differential modulo time base is to simply compare only the bit lengths. For example, in the above example, the VOP encoded immediately before has a modulo time-based bit length of 5, and the VOP to be encoded has a modulo time-based bit length of 6, so the difference is 1. The value is obtained. This can be substituted for the "number of" 1 "bits contained in the difference bit string" in the conversion table shown in Table 4 to express the difference modulo time base. The differential modular time base multiplexing section 25 in the VOP header multiplexing section 503 multiplexes the differential modular time base 26 to the input bit stream, and VOP time increments the multiplexed bit stream. Output to the conversion unit 10. The VOP time increment multiplexing unit 10 in the VOP header multiplexing unit 503 performs VOP time increment multiplexing on the bit stream output from the differential modular time base multiplexing unit 25, and obtains the multiplexed bit stream. Output to the video information header multiplexing unit 11.

As described above, according to the eleventh embodiment, the modulo time base is expressed by the differential modulo time base, and instead of encoding with the expression of the modulo time base currently defined by MPEG-4 as it is. Since the differential modulo time base is configured to be multiplexed, the amount of information generated can be suppressed as compared with the expression method in MPEG-4.

As described above, in the eleventh embodiment, in the image coding apparatus that encodes the image for each object, the information that defines the display time of the image at each time for each object is from the reference time to the display time. Time to encode the first time information that defines the time, the second time information that defines the display time with finer precision than the time specified by the first time information, and the image corresponding to each time. An information coding means is provided, and the time information coding means includes a first time information holding means for holding the first time information encoded in the image at the immediately preceding time, and a first time information holding means for the coded image. The difference bit string between the time information and the first time information of the image at the immediately preceding time obtained from the first time information holding means is obtained, and the difference bit string is encoded as the first time information of the coded image. One example of the thing was disclosed.

Embodiment 12. In the present embodiment 12, the VOP is based on the information of the differential modulo time base multiplexed into the coded bit stream by the differential modulo time base multiplexing unit 25 described in the eleventh embodiment. A VOP decoder that restores the modulo timebase value of and defines the display time of each VOP based on this will be described.

In the twelfth embodiment, only the configuration operation of the header analysis unit 151 of the VOP decoder described so far is changed, and only this member will be described. FIG. 31 shows the internal configuration of the header analysis unit 151 according to the twelfth embodiment of the present invention. 504 is the VOP header analysis unit, 71 is the difference modulo time base analysis unit, 72 is the modulo time base creation unit, 73 is the VOP time increment analysis unit, 74 is the modulo time base holding unit, and 75 is the difference modulo. It is time-based. The difference modulo time base analysis unit 71 in the VOP header analysis unit 504 analyzes the difference modulo time base 75 from the bit stream output from the start code analysis unit 51, and analyzes the difference modulo time base 75. The base 75 is output to the modulo time base creation unit 72, and the analyzed bitstream is output to the VOP time increment analysis unit 73. In the modulo time base creation unit 72 in the VOP header analysis unit 504, the modulo time base 75 of the VOP analyzed immediately before is analyzed based on the conversion table shown in Table 4 from the analyzed differential modulo time base 75. Find the number of "1" bits included in the difference bit string between the base and the modulo time base of the VOP to be analyzed, and analyze the number of "1" bits found and just before the modulo time base holder 74 obtains it. A modulo time base is created based on the modulo time base of the VOP, and the created modulo time base is output to the modulo time base holding unit 74.

A concrete example of creating a modulo timebase is shown. It is assumed that the analyzed differential modulo time base is "10" and the modulo time base analyzed immediately before and held in the modulo time base holder is "11110". When the number of "1" bits included in the difference bit string between the modulo time base of the VOP analyzed immediately before and the modulo time base of the VOP to be analyzed is calculated using the conversion table shown in Table 4, the difference modulo -It can be seen that the number of "1" bits included in the difference bit string corresponding to the time base "10" is one. Next, one "1" bit is added to the most significant bit of the VOP modulo time base "11110" analyzed immediately before to obtain the modulo time base "111110". The conversion table in Table 4 is an example, and other conversion tables may be defined and used. The obtained modulo time-based restoration value is used together with the VOP time increment information for the purpose of defining the display time of the VOP.<tables num="4"><img file="JP2004166311A_D0004.tif" /></tables>

Further, as described in the eighth embodiment, "the number of" 1 "bits included in the difference bit string between the modulo time base of the VOP analyzed immediately before and the modulo time base of the VOP to be analyzed" is described as "the number of" 1 "bits". Even if the bitstream is encoded as "the difference between the modulo time-based bit length of the VOP analyzed immediately before and the modulo time-based bit length of the VOP to be analyzed", as shown in Table 4. This can be done by changing the interpretation of the conversion table. The modulo time base holding unit 74 in the VOP header analysis unit 504 holds the modulo time base of the VOP analyzed immediately before, and analyzes after outputting the modulo time base of the VOP analyzed immediately before. The modulo timebase of the target VOP is entered. The VOP time increment analysis unit 73 in the VOP header analysis unit 504 analyzes the VOP time increment from the bit stream output from the differential modular time base analysis unit 71, and analyzes the analyzed bit stream in the video information header analysis unit. Output to 57.

As described above, according to the twelfth embodiment, the modulo time base can be calculated using the differential modulo time base expressed with a small amount of information. Therefore, the reference code specified in MPEG-4. There is an effect that it becomes possible to analyze the bit stream described in the eighth embodiment in which the amount of information generated is suppressed as compared with the conversion expression.

As described above, in the twelfth embodiment, in the image decoding apparatus that decodes the bit stream in which the image is encoded for each object, the information that defines the display time of the image at each time in the image series is used as the information. The first time information that defines the time from the reference time to the display time and the second time information that defines the display time with finer accuracy from the time specified by the first time information correspond to each time. A time information decoding means for decoding the image to be output and a decoding / synthesizing means for decoding the input coded image signal in object units and synthesizing these decoded image signals are provided, and the time information decoding means decodes immediately before. The first time information of the image to be decoded is retained, and the bit string decoded as the first time information of the image to be decoded is the first of the images decoded immediately before being obtained from the first time information holding means. The time information is added to decode the first time information of the image to be decoded, and the decoding / synthesizing means is based on the first time information and the second time information decoded by the time information decoding means. An example of synthesizing a decoded image signal has been disclosed.

Embodiment 13. In the above-described first to twelfth embodiments, the image coding device multiplexes the display speed information into the image coding signal, and the image coding device image-codes the information expressing the absolute time. Although the point of multiplexing into a signal has been disclosed, one image coding device may multiplex the display speed information and the information expressing the absolute time into the image-encoded signal. Regarding the configuration and operation, the respective image coding devices described in the above-described first to 12th embodiments may be arranged in parallel or in series.

On the other hand, the same applies to the image decoding apparatus side. Briefly, in the above-described first to twelfth embodiments, the image decoding device decodes the display speed information, and based on the decoded display speed information, reproduces the image processed in object units. We have disclosed the points to be performed and the point that the image decoding device decodes the information expressing the absolute time and reproduces the image processed in object units based on the information expressing the decoded absolute time. , One image decoding device may perform reproduction processing of the image processed in the object unit based on the display speed information and the information expressing the absolute time. Regarding the configuration and operation, the display speed information decoding unit and the information decoding unit expressing the absolute time of each of the image decoding devices described in the above-described first to 12th embodiments are arranged in parallel or in series. Based on the information decoded by each decoding unit, the image processed in the object unit may be reproduced. With the above configuration, the image restoration process and the composition process can be performed more smoothly and accurately.

Embodiment 14. In the above-described first to thirteenth embodiments, the image coding device multiplexes the display speed information into the image coding signal, and the image coding device has the first time information and the second time. Although we have disclosed the point of encoding and multiplexing information and images, even if one image encoding device encodes and multiplexes display speed information, first time information, second time information, and images. Good. Regarding the configuration and operation, the respective image coding devices described in the above-described first to thirteenth embodiments may be arranged in parallel or in series.

On the other hand, the same applies to the image decoding apparatus side. Briefly, in the above-described first to thirteenth embodiments, the image decoding apparatus decodes the display speed information, and based on the decoded display speed information, reproduces the image processed for each object. The point and the image decoding device decode the first time information, the second time information, and the image, and based on the decoded first time information, the second time information, and the image, the image Although the point of performing the reproduction process has been disclosed, one image decoding device may perform the image reproduction process based on the display speed information, the decoded first time information, and the second time information. .. Regarding the configuration and operation, the display speed information decoding unit and the time information decoding unit of each of the image decoding devices described in the above-described first to 13th embodiments are arranged in parallel or in series, and the respective decoding units ( Based on the information decoded by the means), the image processed in the object unit may be reproduced. With the above configuration, the image restoration process can be performed more smoothly and accurately with a small amount of coded transmission.

Embodiment 15. In the above-described first to fourteenth embodiments, the image coding device has information expressing an absolute time, a point of multiplexing into an image coding signal, and the image coding device has the first time information and the first time information. We have disclosed the point of encoding and multiplexing the time information and the image of 2, but the information that one image encoding device expresses the absolute time, and the first time information, the second time information, and the image. May be encoded and multiplexed. Regarding the configuration and operation, the respective image coding devices described in the above-described first to 14th embodiments may be arranged in parallel or in series.

On the other hand, the same applies to the image decoding apparatus side. Briefly, in the above-described first to 14th embodiments, the image decoding apparatus decodes the information representing the absolute time, and is processed in object units based on the information expressing the decoded absolute time. The point of performing the reproduction processing of the image, and the image decoding device decodes the first time information, the second time information, and the image, and the decoded first time information, the second time information, and the image. We have disclosed the point that the image reproduction processing is performed based on the above, but based on the information that one image decoding device expresses the absolute time, and the decoded first time information and the second time information. The image may be reproduced. Regarding the configuration and operation, the information decoding unit expressing the absolute time of each of the image decoding devices described in the above-described first to 14th embodiments and the time information decoding means are arranged in parallel or in series, respectively. Based on the information decoded by the decoding unit (means) of the above, the image processed in the object unit may be reproduced. With the above configuration, image composition processing can be performed smoothly and accurately with a small amount of coded transmission.

<figref num="1">It is a figure which shows the video data structure in MPEG-4.</figref><figref num="2">It is a figure which shows the specific example of VOP.</figref><figref num="3">It is a block diagram which shows the VOP encoder part by Embodiment 1 of this invention.</figref><figref num="4">It is a block diagram which shows an example of the structure of the header multiplexing part of the VOP encoder part by Embodiment 1 of this invention.</figref><figref num="5">It is a figure explaining a modulo time base and VOP time increment.</figref><figref num="6">It is a block diagram which shows an example of the structure of the header multiplexing part of the VOP encoder part by Embodiment 1 of this invention.</figref><figref num="7">It is a block diagram which shows the VOP encoder part by Embodiment 2 of this invention.</figref><figref num="8">It is a block diagram which shows an example of the structure of the header multiplexing part of the VOP encoder part by Embodiment 2 of this invention.</figref><figref num="9">It is a figure which shows an example of a bit stream.</figref><figref num="10">It is a block diagram which shows an example of the structure of the VOP header multiplexing part of the header multiplexing part by Embodiment 2 of this invention.</figref><figref num="11">It is a block diagram which shows the internal structure of the VOP decoder part by Embodiment 3 of this invention.</figref><figref num="12">It is a block diagram which shows an example of the structure of the header analysis part of the VOP decoder part by Embodiment 3 of this invention.</figref><figref num="13">It is a block diagram which shows the system which synthesizes a plurality of objects by Embodiment 3 of this invention.</figref><figref num="14">It is a block diagram which shows an example of the structure of the header analysis part of the VOP decoder part by Embodiment 4 of this invention.</figref><figref num="15">It is a block diagram which shows an example of the structure of the header analysis part of the VOP decoder part by Embodiment 4 of this invention.</figref><figref num="16">It is a block diagram which shows the internal structure of the VOP decoder part by Embodiment 5 of this invention.</figref><figref num="17">It is a block diagram which shows an example of the structure of the header analysis part of the VOP decoder part by Embodiment 5 of this invention.</figref><figref num="18">It is a block diagram which shows an example of the structure of the VOP header analysis part of the VOP decoder part according to Embodiment 5 of this invention.</figref><figref num="19">It is a block diagram which shows an example of the structure of the header analysis part of the VOP decoder part by Embodiment 6 of this invention.</figref><figref num="20">It is a block diagram which shows an example of the structure of the VOP header analysis part of the VOP decoder part according to Embodiment 6 of this invention.</figref><figref num="21">It is a block diagram which shows an example of the structure of the header multiplexing part of the VOP encoder part by Embodiment 7 of this invention.</figref><figref num="22">It is a block diagram which shows an example of the structure of the header multiplexing part of the VOP encoder part by Embodiment 7 of this invention.</figref><figref num="23">It is a block diagram which shows an example of the internal structure of the VOP decoder part by Embodiment 8 of this invention.</figref><figref num="24">It is a block diagram which shows an example of the structure of the header analysis part of the VOP decoder part by Embodiment 8 of this invention.</figref><figref num="25">It is a block diagram which shows the system which synthesizes a plurality of objects by Embodiment 8 of this invention.</figref><figref num="26">It is a block diagram which shows an example of the structure of the header analysis part of the VOP decoder part by Embodiment 8 of this invention.</figref><figref num="27">It is a block diagram which shows an example of the internal structure of the VOP decoder part by Embodiment 8 of this invention.</figref><figref num="28">It is a block diagram which shows an example of the structure of the header multiplexing part of the VOP encoder part by Embodiment 9 of this invention.</figref><figref num="29">It is a block diagram which shows an example of the structure of the header analysis part of the VOP decoder part by Embodiment 10 of this invention.</figref><figref num="30">It is a block diagram which shows an example of the structure of the header multiplexing part of the VOP encoder part by Embodiment 11 of this invention.</figref><figref num="31">It is a block diagram which shows an example of the structure of the header analysis part of the VOP decoder part by Embodiment 12 of this invention.</figref>

Code description

200,201,202 Encoded VOP bitstream a, b, c, 203a, 203b, 203c VOP decoder section.