Multiplexing method preventing overflow of audio decoder buffer

This application is a continuation of U.S. application Ser. No. 10/677,228, filed Oct. 3, 2003, U.S. Pat. No. 7,457,525 claiming priority of Japanese Application No. 2003-038467, filed Feb. 17, 2003, the entire contents of each of which are hereby incorporated by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to system multiplexing for forming one stream of digitally compressed image and audio data and, more specifically, to a system multiplexing apparatus preventing an overflow of a buffer in an audio decoder.

2. Description of the Background Art

Recently, along with performance improvement of semiconductor devices, methods of digitally compressing images and sounds have been vigorously studied, and international standard specifications for digital compression of images and sounds, including MPEG (Moving Picture Experts Group) specification, have been determined. In compliance therewith, video recording and reproducing apparatuses for digital TV broadcast, DVD (Digital Versatile Disc) and the like utilizing digital compression are becoming popular.

When moving images and sounds are to be handled simultaneously, a process referred to as system multiplexing is necessary, by which digitally compressed image and audio data are turned into one stream. In the system multiplexing process, image data and audio data are divided into small units, and the data are arranged in order considering amounts of data and time points of display and reproduction, to form one collective stream.

Japanese Patent Laying-Open No. 8-98160 discloses a related technique. A data multiplexing apparatus disclosed in Japanese Patent Laying-Open No. 8-98160 includes a plurality of encoders encoding input digital sounds and moving images and outputting the result as an encoded stream, multiplexes a plurality of such encoded streams to enable simultaneous reproduction, and the apparatus includes control means for performing virtual decoding buffer simulation for each of the encoded streams and outputting a multiplex request signal based on an amount of occupation by the data in the buffer, and multiplexing means for multiplexing each of the encoded streams based on the multiplex request signal.

In the data multiplexing apparatus disclosed in Japanese Patent Laying-Open No. 8-98160 mentioned above, virtual decoding buffer simulation is performed for each encoded stream and multiplexing is done based on the amount of data occupation in the buffer. When audio completing of a VOBU (video object unit) boundary is performed, however, an overflow may occur undesirably in a buffer of an audio decoder, as will be described later.

SUMMARY OF THE INVENTION

An object of the present invention is to provide a system multiplexing apparatus capable of reducing loss in multiplexing, while preventing an overflow in an audio buffer.

According to an aspect, the present invention provides a system multiplexing apparatus for outputting video and audio packs multiplexed with each other, including a calculating unit calculating a time zone to be set in a video object unit in accordance with audio bit rate, a comparing unit comparing a time point of multiplexing an audio pack with the time zone calculated by the calculating unit, and a setting unit setting whether the audio pack is to be completed or not, in accordance with the result of comparison by the comparing unit.

The comparing unit compares the time point of multiplexing the audio pack and the time zone calculated by the calculating unit, and the setting unit sets whether the audio pack is to be completed or not in accordance with the result of comparison by the comparing unit. Therefore, the completing process takes place before the VOBU boundary, a complete PCK is not generated immediately after the VOBU boundary, and therefore, generation of a buffer overflow can be prevented.

The foregoing and other objects, features, aspects and advantages of the present invention will become more apparent from the following detailed description of the present invention when taken in conjunction with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

<figref idref="DRAWINGS">FIG. 1</figref> is a block diagram representing schematic configurations of a recorder and a reproducer, including the system multiplexing apparatus.

<figref idref="DRAWINGS">FIG. 2</figref> shows an example of a PS configuration.

<figref idref="DRAWINGS">FIGS. 3A to 3C</figref> represent PCK configuration in compliance with DVD specification.

<figref idref="DRAWINGS">FIG. 4</figref> shows transition of audio buffer storage amount of a decoder.

<figref idref="DRAWINGS">FIG. 5</figref> is a flow chart representing process steps performed by a system multiplexing unit <b>5</b>.

<figref idref="DRAWINGS">FIG. 6</figref> is a flow chart representing details of step S<b>13</b> shown in <figref idref="DRAWINGS">FIG. 5</figref>.

<figref idref="DRAWINGS">FIG. 7</figref> represents relation of image recording mode, sound recording mode, bit rate, frame length and PCK period.

<figref idref="DRAWINGS">FIG. 8</figref> represents buffer transition when a virtual buffer control method is applied to a mode in which one audio frame is long.

<figref idref="DRAWINGS">FIGS. 9A to 9D</figref> illustrate an example in which an overflow occurs in an audio buffer.

<figref idref="DRAWINGS">FIG. 10</figref> shows an exemplary solution to the problem illustrated in <figref idref="DRAWINGS">FIGS. 9A to 9D</figref>.

<figref idref="DRAWINGS">FIG. 11</figref> is a flow chart representing a process for recognizing in advance the VOBU boundary.

<figref idref="DRAWINGS">FIGS. 12A and 12B</figref> illustrate an example of calculation of video complete time point.

<figref idref="DRAWINGS">FIGS. 13A to 13H</figref> represent <b>8</b> variations generated dependent on whether the last V_PCK time point of each picture exceeds a picture period time or not.

<figref idref="DRAWINGS">FIG. 14</figref> represents a functional configuration of the system multiplexing apparatus in accordance with the first embodiment of the present invention.

<figref idref="DRAWINGS">FIG. 15</figref> is a flow chart representing process steps performed by the system multiplexing apparatus in accordance with the first embodiment of the present invention.

<figref idref="DRAWINGS">FIGS. 16A to 16D</figref> illustrate a complete zone applied in the system multiplexing apparatus in accordance with the first embodiment of the present invention.

<figref idref="DRAWINGS">FIG. 17</figref> is a schematic illustration of a complete zone applied in the system multiplexing apparatus in accordance with the first embodiment of the present invention.

<figref idref="DRAWINGS">FIG. 18</figref> illustrates a method of calculating a threshold value.

<figref idref="DRAWINGS">FIG. 19</figref> illustrates a method of calculating a threshold value when the audio buffer has the capacity of 4.2 Kbytes and A_ES included in A_PCK is 1.8 Kbytes.

<figref idref="DRAWINGS">FIG. 20</figref> illustrates audio buffer transition and an A_PCK configuration in the system multiplexing apparatus in accordance with the first embodiment of the present invention.

<figref idref="DRAWINGS">FIG. 21</figref> illustrates audio buffer transition and an A_PCK configuration in the system multiplexing apparatus in accordance with the first embodiment of the present invention, with multiplexing loss reduced.

<figref idref="DRAWINGS">FIG. 22</figref> illustrates audio buffer transition when VOBU completing process is forced immediately after an output of an ordinary PCK.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

<figref idref="DRAWINGS">FIG. 1</figref> is a block diagram representing a schematic configuration of a recorder (encoder) and a reproducer (decoder), including a system multiplexing apparatus (hereinafter also referred to as a system multiplexing unit).

The recorder includes a video encoder <b>1</b> encoding image data, an audio encoder <b>2</b> encoding audio data, a video buffer <b>3</b> temporally storing V-ES (Video-Elementary Stream) output from video encoder <b>1</b>, an audio buffer <b>4</b> temporally storing A-ES (Audio-Elementary Stream) output from audio encoder <b>2</b>, and a system multiplexing unit <b>5</b> multiplexing V-ES and A-ES output from video buffer <b>3</b> and audio buffer <b>4</b> and outputting the result as a PS (Program Stream).

The reproducer includes a system separating unit <b>6</b> separating the PS into V-ES and A-ES, a video buffer <b>7</b> temporally storing V-ES separated by system separating unit <b>6</b>, an audio buffer <b>8</b> temporally storing A-ES separated by system separating unit <b>6</b>, a video decoder <b>9</b> decoding V-ES output from video buffer <b>7</b>, and an audio decoder <b>10</b> decoding A-ES output from audio buffer <b>8</b>.

<figref idref="DRAWINGS">FIG. 2</figref> illustrates an exemplary configuration of PS. PS consists of VOBUs. Each VOBU includes Audio, Video and RDI (Real time Data Information) PCKs, beginning with RDI_PCK and ending with completed V_PCK (Vc) and A_PCK (Ac).

According to DVD-Video Recording specification (hereinafter referred to as DVD specification), a completing process for completing images and sounds in a frame unit is necessary for each VOBU (GOP (Group Of Pictures): a unit of about 15 pictures). The completing process is a process to fill ES shortage of a PCK of 2 KB (2048 bytes) with a stuff or padding.

<figref idref="DRAWINGS">FIGS. 3A to 3C</figref> represent a PCK configuration in compliance with the DVD specification. <figref idref="DRAWINGS">FIG. 3A</figref> represents an ordinary PCK configuration that includes a Pack Header of 14 bytes, a Packet Header of 14 bytes, and ES of 2020 bytes.

<figref idref="DRAWINGS">FIG. 3B</figref> represents a VOBU complete PCK with a stuff inserted, which includes Pack Header of 14 bytes, Packet Header of 14 bytes, Stuff of 1 to 7 bytes and ES of an arbitrary length.

<figref idref="DRAWINGS">FIG. 3C</figref> represents a configuration of a VOBU complete PCK with a padding, which includes Pack Header of 14 bytes, Packet Header of 14 bytes, ES of an arbitrary length, and Padding Packet of at least 8 bytes.

In video encoding in accordance with the DVD specification or the like that involves VBR (Variable Bit Rate) control, the amount of codes for one picture is not constant, and a VOBU boundary appears at any time point dependent on the video code amount.

Further, different conditions for system multiplexing are defined as specifications for different applications, as can be seen from the fact that specification of DVD-Video for reproduction only differs from the specification of DVD-VR for both recording and reproduction. According to the DVD specification, the size of an audio buffer of a decoder is defined to be 4 KB.

<figref idref="DRAWINGS">FIG. 4</figref> shows a transition of the amount of storage in the audio buffer of the decoder. When a PCK is output from system separating unit <b>6</b>, data is input to audio buffer <b>8</b> (amount of storage increases), and at a decode time point for every frame period, data is output to audio decoder <b>10</b> (amount of storage decreases). To realize control (audio buffer control) that prevents any overflow/underflow while inputting PCK of 2 KB into audio buffer <b>8</b> of 4 KB, the amount of storage is changed with 2 KB being the center.

For this purpose, virtual buffer remaining capacity is calculated assuming that data is drawn out from audio buffer <b>8</b> at a constant rate, and when the virtual buffer remaining capacity reaches 1 KB, the next PCK is output from system separating unit <b>6</b>, as shown in <figref idref="DRAWINGS">FIG. 4</figref>. This method will be referred to as virtual buffer control method in the following.

<figref idref="DRAWINGS">FIG. 5</figref> is a flow chart representing process steps performed by system multiplexing unit <b>5</b>. System multiplexing unit <b>5</b> confirms whether V_ESs sufficient to form one PCK exist in video buffer <b>3</b> or not (S<b>11</b>). When there are sufficient V_ESs in video buffer <b>3</b>, system multiplexing unit <b>5</b> determines which one of the PCK configurations shown in <figref idref="DRAWINGS">FIGS. 3A to 3C</figref> is to be employed (S<b>12</b>).

Thereafter, system multiplexing unit <b>5</b> determines whether VOBU boundary is recognized and a complete PCK is formed, in the video PCK configuration, and determines whether an audio PCK should also be completed or not (S<b>13</b>). Details of this step will be described later.

Next, as in the case of V_ES, system multiplexing unit <b>5</b> confirms whether A_ESs sufficient to form one PCK exist in audio buffer <b>4</b> or not (S<b>14</b>). When there are sufficient A_ESs in audio buffer <b>4</b>, system multiplexing unit <b>5</b> determines which one of the PCK configurations shown in <figref idref="DRAWINGS">FIGS. 3A to 3C</figref> is to be employed (S<b>15</b>).

Then, system multiplexing unit <b>5</b> determines whether a video PCK or an audio PCK is to be output (S<b>16</b>). Finally, system multiplexing unit <b>5</b> outputs the selected video PCK or audio PCK (S<b>17</b>).

<figref idref="DRAWINGS">FIG. 6</figref> is a flow chart representing the details of the process step S<b>13</b> shown in <figref idref="DRAWINGS">FIG. 5</figref>. First, system multiplexing unit <b>5</b> determines whether the video PCK configuration is a complete PCK or not (S<b>21</b>). When the video PCK is a complete PCK (S<b>21</b>, YES), the next audio PCK is determined to be a complete PCK, and an audio complete flag is set to 1 (S<b>22</b>). When the video PCK is not a complete PCK (S<b>21</b>, NO), the next audio PCK is determined not to be a complete PCK, and the audio complete flag is set to 0 (S<b>23</b>). In the following, this method will be referred to as VOBU boundary forced complete method.

<figref idref="DRAWINGS">FIG. 7</figref> represents relation between each of image recording mode, sound recording mode, bit rate, frame length and PCK period in AC-3. By way of example, when the image recording mode is “high quality” and the sound recording mode is 11, the bit rate is 384 kbps, the frame length is 1536 bytes, and the PCK period is 3788 SCR.

<figref idref="DRAWINGS">FIG. 8</figref> shows buffer transition when the virtual buffer control method is applied to a mode in which one audio frame is long, for example the high quality mode (with bit rate of 384 kbps and frame length of 1536 bytes) of <figref idref="DRAWINGS">FIG. 7</figref>. The decoder controls the amount of storage with the buffer storage amount of 2 KB regarded as the center. In this example, however, the amplitude to the upper and lower sides becomes large, resulting in undesirably small margin for the upper and lower limits of audio buffer <b>8</b>.

In such a virtual buffer control method, when the audio frame length is long and the completing process at the VOBU boundary that is unpredictable occurs while the buffer has large amount of storage, an overflow is likely.

<figref idref="DRAWINGS">FIGS. 9A to 9D</figref> illustrate an occurrence of an overflow in the audio buffer. <figref idref="DRAWINGS">FIG. 9A</figref> represents a V_ES having the picture period of 3003 SCR. <figref idref="DRAWINGS">FIG. 9B</figref> represents an A_ES having the audio frame length of 1536 bytes and the frame period of 2880 SCR. <figref idref="DRAWINGS">FIG. 9C</figref> represents a PS having an audio PCK period of 3840 SCR. <figref idref="DRAWINGS">FIG. 9D</figref> represents buffer transition of audio buffer <b>8</b> of the decoder.

As can be seen from <figref idref="DRAWINGS">FIG. 9C</figref>, when a VOBU boundary occurs, a complete PCK is generated. Therefore, occurrence of the VOBU when the amount of storage in the audio buffer is large may cause a buffer overflow.

<figref idref="DRAWINGS">FIG. 10</figref> represents an exemplary solution to the problem described with reference to <figref idref="DRAWINGS">FIGS. 9A to 9D</figref>. As shown in <figref idref="DRAWINGS">FIG. 10</figref>, when all audio PCKs are completed, it becomes unnecessary to generate any new complete PCK at an occurrence of the VOBU boundary, and therefore, a buffer overflow can be prevented. It is noted, however, that the stuff or padding inserted to the complete PCK is meaningless data, and therefore, this solution leads to a problem of lower efficiency of multiplexing.

As a method of preventing buffer overflow without degrading efficiency of multiplexing, a method has been known in which the VOBU boundary is recognized in advance in an audio complete determining process. <figref idref="DRAWINGS">FIG. 11</figref> is a flow chart representing the process for recognizing the VOBU boundary in advance. This process corresponds to step S<b>13</b> of <figref idref="DRAWINGS">FIG. 5</figref>. Further, <figref idref="DRAWINGS">FIGS. 12A and 12B</figref> illustrate examples of calculation of the video complete time point. In the following, the process for recognizing in advance the VOBU boundary shown in <figref idref="DRAWINGS">FIG. 11</figref> will be described.

First, system multiplexing unit <b>5</b> calculates, from the code amount of Video-ES of a picture (picture (n−2)) preceding the last picture of the VOBU by two frames, how many V_PCKs are included therein, and calculates the last V_PCK time point of the picture (n−2) (S<b>31</b>).

Thereafter, in the similar manner, the last V_PCK time point of picture (n−1) is calculated (S<b>32</b>), and the last V_PCK time point of picture (n) is calculated (S<b>33</b>). Then, system multiplexing unit <b>5</b> tentatively holds the last V_PCK time point of picture (n) as the video complete time point (S<b>34</b>).

Referring to <figref idref="DRAWINGS">FIG. 12A</figref>, when the video complete time point of picture (n) is earlier than the last time point of picture (n) (picture period time), the last time point of picture (n) is set as the video complete time point. When the video complete time point of picture (n) exceeds the last time point of picture (n), the time point is set as the video complete time point.

In the examples shown in <figref idref="DRAWINGS">FIGS. 12A and 12B</figref>, the last V_PCK time points of pictures (n−2) and (n−1) exceed the picture period time. When the video code amount (number of PCKs) of one picture is smaller than the number of PCKs that can be transmitted within one picture time, it is necessary to set the picture period time as the last V_PCK time point of the picture, and when the number of PCKs of one picture is larger than the number of PCKs that can be transmitted within one picture time, it is necessary to set the time point calculated from the number of PCKs as the last V_PCK time point of the picture.

Here, the time by which the last V_PCK time point exceeds the picture period time will be the start time point of the next picture. Therefore, dependent on whether the last V_PCK time point of respective pictures exceed the picture period time or not, as determined by the code amounts of pictures (n−3), (n−2) and (n−1), there would be eight variations.

<figref idref="DRAWINGS">FIGS. 13A to 13H</figref> illustrate the eight variations generated dependent on whether the last V_PCK time point of respective pictures exceeds the picture period time or not. For example, <figref idref="DRAWINGS">FIG. 13A</figref> shows an example in which the last V_PCK time points of pictures (n−2), (n−1) and (n) exceed the picture period time, respectively. <figref idref="DRAWINGS">FIG. 13C</figref> shows an example in which the last V_PCK time point of picture (n−2) exceeds the picture period time, the last V_PCK time point of picture (n−1) is within the picture period time, and the last V_PCK time point of picture (n) exceeds the picture period time.

As can be seen from the foregoing, in order to calculate the video complete time point, many situations must be taken into consideration, and hence a complicated process is necessary to recognize in advance the VOBU boundary.

In the following, a system multiplexing apparatus that improves efficiency of multiplexing while suppressing increase in time necessary for the audio completing process will be described.

First Embodiment

Schematic configuration of the encoder and decoder including the system multiplexing apparatus in accordance with the first embodiment of the present invention is the same as that of <figref idref="DRAWINGS">FIG. 1</figref>. Therefore, overlapping descriptions of the configuration and functions will not be repeated.

<figref idref="DRAWINGS">FIG. 14</figref> represents a functional configuration of the system multiplexing apparatus in accordance with the first embodiment of the present invention. The system multiplexing apparatus includes a time zone start time point calculating unit <b>11</b> calculating a start time point of a time zone having a possible completing (hereinafter also referred to as a complete zone), a time zone comparing unit <b>12</b> comparing a time point at which an audio PCK is to be multiplexed with the time zone start time point calculated by time zone start time point calculating unit <b>11</b>, and a flag setting unit <b>13</b> setting an audio complete flag in accordance with the result of comparison by the time zone comparing unit <b>12</b>. The functional configuration shown in <figref idref="DRAWINGS">FIG. 14</figref> may be implemented by a CPU (Central Processing Unit) executing a prescribed program, or it may be implemented by hardware.

<figref idref="DRAWINGS">FIG. 15</figref> is a flow chart representing the process steps performed by the system multiplexing apparatus in accordance with the first embodiment of the present invention. The system multiplexing apparatus of the present embodiment is conceived noticing that A_PCK is always output at least once in one period of A_PCK.

First, time zone start time point calculating unit <b>11</b> calculates the complete zone from the PCK period that is derived with reference to the audio bit rate (S<b>41</b>). Thereafter, time zone comparing unit <b>12</b> compares the time point at which the audio PCK is to be multiplexed (hereinafter referred to as the SCR time point) with the time zone start time point, and determines whether the time point at which the audio PCK is to be multiplexed is within the time zone (S<b>42</b>).

When the time point at which the audio PCK is to be multiplexed is within the time zone (S<b>42</b>, YES), flag setting unit <b>13</b> sets the audio complete flag to 1, and generates a complete PCK (S<b>43</b>). When the time point at which the audio PCK is to be multiplexed is not within the time zone (S<b>42</b>, NO), flag setting unit <b>13</b> sets the audio complete flag to 0, and generates an ordinary A_PCK (S<b>44</b>). It is noted that the start point of the complete zone is the GOP boundary time point.

The audio bit rate is determined, for example, by a user setting the image recording (sound recording) mode. Though it also depends on the product specification of the DVD recorder, the sound recording mode shown in <figref idref="DRAWINGS">FIG. 7</figref> itself may be set, or the sound recording mode may be selected by switching between the high quality (short time)/standard/long time mode.

The complete zone start time point is calculated, backward from the GOP boundary time point, by using a time zone in which at least one A_PCK surely exists, that is, by using the audio PCK period. The SCR time point per 1 byte will be given by the following. <br />SCR time point per 1 byte=(8 bits/bit rate×90 KHz)  (1)

When the bit rate is 384 kbps, the SCR time point per 1 byte will be given by the following, in SCR time point equivalent. <br />SCR time point per 1 byte=8 bits/384 kbps×90 KHz=720/384 SCR=1.875 SCR  (2)

It is assumed that the ES amount in one PCK corresponds to the PCK length of 2048 bytes with PS header and PES header subtracted. When the ES amount is assumed to be 2020 bytes as shown in <figref idref="DRAWINGS">FIG. 3A</figref>, the period of one PCK will be <br />Period of 1PCK=1.875SCR/byte×2020 bytes=3787.5SCR  (3)

Therefore, the complete zone is rounded up to 3788 SCR.

<figref idref="DRAWINGS">FIGS. 16A to 16D</figref> illustrate the complete zone applied in the system multiplexing apparatus in accordance with the first embodiment. <figref idref="DRAWINGS">FIGS. 16A and 16B</figref> are similar to <figref idref="DRAWINGS">FIGS. 9A and 9B</figref>, and therefore, detailed description thereof is not repeated. As can be seen from <figref idref="DRAWINGS">FIG. 16C</figref>, the complete zone is from a time point preceding by one A_PCK period from the GOP boundary time point until the VOBU boundary time point. In the example of <figref idref="DRAWINGS">FIG. 16C</figref>, two complete A_PCKs are generated.

<figref idref="DRAWINGS">FIG. 16D</figref> represents transition of storage amount of the audio buffer. As shown in <figref idref="DRAWINGS">FIG. 16D</figref>, as the completing process takes place before the VOBU boundary, a complete A_PCK is not generated immediately after the VOBU boundary, and therefore, generation of a buffer overflow can be prevented.

<figref idref="DRAWINGS">FIG. 17</figref> is a schematic illustration of the complete zone applied in the system multiplexing apparatus in accordance with the first embodiment of the present invention. It can be seen that two complete PCKs are generated in the complete zone.

As described above, in the system multiplexing apparatus in accordance with the first embodiment of the present invention, a complete zone using the GOP boundary time point as a start point is calculated, and when a time point at which an audio PCK is to be multiplexed is within the complete zone, the audio PCK is completed. Therefore, it becomes possible to improve efficiency of multiplexing by reducing multiplexing loss caused by insertion of a padding or stuff, while preventing generation of an overflow of the audio buffer.

Second Embodiment

In the first embodiment of the present invention, complete zones are provided for all the bit rates, and dependent on whether the time point at which an audio PCK is to be multiplexed is within a complete zone or not, an A_PCK is completed. When the code length is so short as to cause no overflow of the audio buffer with the use of the virtual buffer control method only, the above described approach rather lowers the efficiency of multiplexing.

In the second embodiment, whether the time zone complete method in accordance with the first embodiment is to be applied or not is determined in accordance with audio frame length. The configuration of the encoder including the system multiplexing apparatus and the functional configuration of the system multiplexing apparatus in accordance with the present embodiment are the same as the configuration of the encoder shown in <figref idref="DRAWINGS">FIG. 1</figref> and the functional configuration of the system multiplexing apparatus shown in <figref idref="DRAWINGS">FIG. 14</figref> in accordance with the first embodiment. Therefore, overlapping description of the configurations and functions will not be repeated.

When the audio frame length is equal to or longer than a threshold value, the completing process is performed using the above described time zone complete method. When the audio frame length is shorter than the threshold value, the completing process is performed using the VOBU boundary forced complete method.

When the audio buffer has the capacity of 4 Kbytes and the remaining capacity of the buffer is to be controlled using 2 Kbytes as a central value as in a system multiplexing apparatus in compliance with the DVD specification, the threshold value of the audio frame length is set to 1024 bytes.

<figref idref="DRAWINGS">FIG. 18</figref> is an illustration of the method of calculating the threshold value described above. At the center of the storage amount of the 4Kbyte audio buffer, a PCK of 2 Kbytes is placed, and a difference between the upper end thereof and the upper limit of the buffer (1024 byte) is used as the threshold value. The reason for this is as follows. The storage amount of the audio buffer changes with 2 Kbytes being the center, and therefore, even when a VOBU boundary appears immediately after an output of A_PCK of 2 Kbytes and the completing process takes place, what results is simply a storage of data corresponding to the audio frame length in the audio buffer.

<figref idref="DRAWINGS">FIG. 19</figref> is an illustration of the method of calculating the threshold value when the audio buffer has the capacity of 4.2 Kbytes and an A_ES included in an A_PCK is 1.8 Kbytes. At the center (2.1 Kbytes) of the storage amount of the 4.2 Kbytes audio buffer, a PCK of 1.8 Kbytes is placed, and a difference between the upper end thereof and the upper limit of the buffer (1.2 Kbytes) is used as the threshold value.

As described above, in the system multiplexing apparatus in accordance with the present embodiment, the time zone complete method and the VOBU boundary forced complete method are switched dependent on whether the audio frame length is not shorter than the threshold value or not. Therefore, lowering of the efficiency of multiplexing experienced when the audio frame length is short can be avoided.

Third Embodiment

In the first embodiment, the process for calculating the complete zone in accordance with the bit rate is necessary, and in the second embodiment, a process for switching the methods by comparing the audio frame length with a threshold value is necessary. Therefore, in either method, process time becomes longer.

In the third embodiment of the present invention, the worst (longest) complete zone employed in a product is set in advance, so as to improve speed of processing. The configuration of the encoder including the system multiplexing apparatus and the functional configuration of the system multiplexing apparatus in accordance with the present embodiment are the same as the configuration of the encoder shown in <figref idref="DRAWINGS">FIG. 1</figref> and the functional configuration of the system multiplexing apparatus shown in <figref idref="DRAWINGS">FIG. 14</figref> in accordance with the first embodiment. Therefore, overlapping description of the configurations and functions will not be repeated.

By way of example, assume that the product bit rate of up to 192 Kbps is supported. In that case, the time zone start time point is 7575 SCR earlier than the GOP boundary. This value is set in advance in the system multiplexing apparatus, and the completing process is performed using this value regardless of the bit rate. Though AC-3 only is shown in <figref idref="DRAWINGS">FIG. 7</figref>, the present invention is also applicable to other Audio type such as MPEG-1Audio.

As described above, in the system multiplexing apparatus in accordance with the present embodiment, the completing process is performed using a time zone start time point set in advance, and therefore, speed of processing of the system multiplexing apparatus can be improved.

Fourth Embodiment

In the first embodiment, when the time point is within the complete time zone, the completing process is performed even if the actual storage amount of the buffer is small and the completing process is unnecessary, causing multiplexing loss. In the present embodiment, whether the completing process is necessary or not is determined to reduce the multiplexing loss. The configuration of the encoder including the system multiplexing apparatus in accordance with the present embodiment is the same as the configuration of the encoder in accordance with the first embodiment shown in <figref idref="DRAWINGS">FIG. 1</figref>. The functional configuration of the system multiplexing apparatus of the present embodiment differs from the functional configuration of the system multiplexing apparatus shown in <figref idref="DRAWINGS">FIG. 14</figref> only in that the flag setting unit additionally has a function of calculating the remaining capacity of the audio buffer. Therefore, overlapping description of the configurations and functions will not be repeated. In the present embodiment, the flag setting unit will be denoted by the reference character <b>13</b>′.

<figref idref="DRAWINGS">FIG. 20</figref> illustrates transition of the audio buffer of the system multiplexing apparatus and the configuration of A_PCK in accordance with the first embodiment of the present invention. As can be seen from <figref idref="DRAWINGS">FIG. 20</figref>, four A_PCKs, of which time point to multiplex the audio PCK is within the complete time zone, will be completed.

<figref idref="DRAWINGS">FIG. 21</figref> illustrates transition of the audio buffer of the system multiplexing apparatus and the configuration of A_PCK in accordance with the first embodiment of the present invention, with the multiplexing loss reduced. As can be seen from <figref idref="DRAWINGS">FIG. 21</figref>, even when the first two A_PCKs within the time zone are not completed, no overflow occurs in the audio buffer.

In the system multiplexing apparatus in accordance with the fourth embodiment of the present invention, flag setting unit <b>13</b>′ calculates increase in the amount of storage of the buffer caused by the ESs in the PCKs output from the start of multiplexing and the amount of data drawn out from the audio buffer at the PTS (decode time point), and calculates the amount of storage of the audio buffer.

When it is determined by time zone comparing unit <b>12</b> that the SCR time point is within the time zone, flag setting unit <b>13</b>′ refers to the actual remaining capacity of the audio buffer, and determines whether a buffer overflow can be avoided even when an uncompleted, ordinary PCK is output and thereafter a complete PCK is continuously output. When it is determined that an overflow can be avoided, flag setting unit <b>13</b>′ sets the audio complete flag to 0, and outputs an ordinary A_PCK.

When it is determined that an overflow will occur, flag setting unit <b>13</b>′ sets the audio complete flag to 1, and outputs a complete A_PCK.

<figref idref="DRAWINGS">FIG. 22</figref> represents audio buffer transition when the VOBU completing process is forced immediately after an output of an ordinary PCK. Here, whether the A_PCK is to be completed or not is determined by actually calculating the remaining capacity of the buffer, and therefore, an overflow does not occur in the audio buffer even when the VOBU completing process is forced immediately after the output of an ordinary PCK.

As described above, in the system multiplexing apparatus in accordance with the present embodiment, whether an A_PCK is to be completed or not is determined by calculating remaining capacity of the audio buffer. Therefore, generation of an unnecessary completed PCK can be avoided and the efficiency of multiplexing can be improved.

Although the present invention has been described and illustrated in detail, it is clearly understood that the same is by way of illustration and example only and is not to be taken by way of limitation, the spirit and scope of the present invention being limited only by the terms of the appended claims.