Image processing apparatus and method, and program
Abstract
Problem to be solved.To provide an image processing apparatus, an image processing method and a program capable of improving the efficiency of using a buffer in accordance with a buffer model. An image processing device 1 includes an encoder IP device 11, a control device 12, and an additional information addition device 13. The encoder IP device 11 encodes the moving image data. The additional information addition device 13 adds additional data to the output data of the encoder IP device 11. The control device 12 monitors the amount of output data to be output by the encoder IP device 11, and when the additional information addition device 13 adds the additional data to the output data, the capacity of the output data including the additional data is equal to or less than the specified value. The output of the encoder IP device 11 is controlled so as to be. [Selection diagram] Fig. 1

Term
Projected expiry 10 December 2028.
- Priority and filed
- Published
- Today
- Projected expiry
9 claims: 4 independent, 5 dependent
- 1動画像データをエンコードするエンコード処理部と、 上記エンコード処理部の出力データに付加データを付加する付加部と、 上記エンコード処理部が出力すべき上記出力データの容量を監視し、上記付加部が上記出力データに上記付加データを付加したときに、当該付加データを含む当該出力データの容量が規定値以下となるように、上記エンコード処理部の出力を制御する制御部と を有する画像処理装置。
- 2上記エンコード処理部は、 上記出力すべき上記出力データを格納するバッファを有し、 上記制御部は、 上記エンコード処理部が出力した上記バッファの状態に関するバッファ情報と、上記付加データの容量とに基づいて、上記バッファの容量が上記規定値以下となるように上記バッファを制御する 請求項1記載の画像処理装置。
- 3上記バッファ情報は、 上記規定値を規定する仮想バッファの容量に関する情報を含み、 上記制御部は、 上記バッファ情報が含む上記仮想バッファの容量と、上記付加データの容量との差分に基づいて、上記エンコード処理部が上記動画像データをエンコードする際のビットレートを設定する 請求項2記載の画像処理装置。
- 4上記エンコード処理部が行うエンコードに関する初期設定情報が格納された記憶部を有し、 上記エンコード処理部は、 上記動画像データをフレーム単位でエンコードし、 上記制御部は、 上記エンコード処理部が初めのフレームに対してエンコードを行う場合には、上記記憶部に格納された初期設定情報を参照して上記設定すべきビットレートを設定し、 上記エンコード処理部が以後のフレームに対してエンコードを行う場合には、上記バッファの容量とフレームレートとの積を算出し、当該算出結果に基づいて上記設定すべきビットレートを設定する 請求項3記載の画像処理装置。
- 5上記制御部は、 上記エンコード処理部が設定しているビットレートと、上記算出結果としての算出ビットレートとの大きさを比較し、 上記ビットレートが上記算出ビットレート以下の場合には、設定すべきビットレートを当該ビットレートに設定し、 上記ビットレートが上記算出ビットレート以上の場合には、設定すべきビットレートを当該算出ビットレートに設定する 請求項4記載の画像処理装置。
- 6上記エンコード処理装置は、 上記バッファの容量が上記規定値を超えた場合には、エンコード対象のフレームに対するエンコードを停止し、 上記制御部は、 上記エンコード処理部のエンコードが停止した旨の停止信号を取得した場合には、上記設定すべきビットレートを設定されているビットレートよりも下げる 請求項5記載の画像処理装置。
- 7上記制御部は、 上記エンコード処理部による上記出力データの出力後、上記付加データの容量の変更を検知した場合には、当該付加データの容量の増減量を算出し、上記エンコード処理部が次のフレームに対するエンコードを行うときに、当該増減量に基づいて上記設定すべきビットレートを設定する 請求項6記載の画像処理装置。
- 8動画像データをエンコードする第1のステップと、 上記エンコードしたデータの容量を監視する第2のステップと、 上記エンコードしたデータに付加データを付加したときに、当該付加データを含む当該エンコードしたデータの容量が規定値以下となるように、出力すべき当該エンコードしたデータの容量を制御する第3のステップと、 上記エンコードしたデータに上記付加データを付加する第4のステップと を有する画像処理方法。
- 9動画像データをエンコードする第1の処理と、 上記エンコードしたデータの容量を監視する第2の処理と、 上記エンコードしたデータに付加データを付加したときに、当該付加データを含む当該エンコードしたデータの容量が規定値以下となるように、出力すべき当該エンコードしたデータの容量を制御する第3の処理と、 上記エンコードしたデータに上記付加データを付加する第4の処理と をコンピュータに実行させるプログラム。
Independent claims9
76 paragraphs, as filed
The present invention relates to, for example, an image processing apparatus, an image processing method, and a program capable of encoding moving image data.
An image processing device that encodes (encodes) moving image data is well known (see, for example, Patent Documents 1 to 3). In such an image processing device, additional data such as header information is added to the encoded data to generate a final elementary stream. Hereinafter, an outline of a general image processing apparatus will be described.
8 and 9 are block diagrams showing a configuration example of a general image processing apparatus. The encoder IP device 31 of the image processing device 3 illustrated in FIG. 8 acquires size information S32 indicating the size (data capacity) of the additional information S31 for each frame. Then, the rate control unit 312 controls the bit rate based on this so that the VBV buffer does not collapse. At this time, the output buffer fixed information S34 is controlled so that the buffer size (VBA buffer size) of the VBV buffer is maximized.
After that, the encoder unit 311 encodes the input signal S33 based on the control of the initial information S35 and the rate control unit 312 used for the encoding process. The additional information addition device 313 adds the additional information S31 to this encoded data. As a result, the encoding process is completed and the elementary stream S36 is generated.
In the image processing device 3 illustrated in FIG. 8, additional information S31 is added to the encoded data inside the encoder IP device 31. That is, the encoder IP device outputs the encoded data to which the additional information S31 is added.
On the other hand, in the image processing device 3a illustrated in FIG. 9, the additional information addition device 313 that adds the additional information S31 is arranged outside the encoder IP device 31. In this device, the additional information S31 is not added to the encoded data S37 output by the encoder IP device 31. Therefore, after the encoding process by the encoder IP device 31, the additional information addition device 313 adds the additional information S31 to the encoding data S37.<patcit num="1"><text>WO2004 / 071085</text></patcit><patcit num="2"><text>Japanese Patent Application Laid-Open No. 2003-299091</text></patcit><patcit num="3"><text>Japanese Unexamined Patent Publication No. 11-205266</text></patcit>
<p> FIG. 10 is a diagram showing the relationship between the VBV buffer size by the general image processing apparatus shown in FIGS. 8 and 9 and the output buffer size. The broken line a indicates the VBV buffer size. The broken line b indicates the encoded data S37 to which the additional information S31 is not added. The broken line c indicates the elementary stream S36 to which the additional information S31 is added.</p><p> In the buffer model, it is necessary to control the amount of bits input to the output buffer (not shown) so that the VBV buffer does not collapse. As shown in FIG. 10, the encoding process for the one-frame image is performed during the period Δt. In such devices 3 and 3a, as shown by the broken lines b and c in FIG. 10, the bit amounts of the encoded data including the additional information S31 and the encoded data not including the additional information S31 are equal.</p><p> In general image processing devices 3 and 3a, the size of the output buffer cannot be grasped for each encoding process. Therefore, for example, the output buffer fixed information S34 needs to give the maximum size in advance so that the output buffer size does not exceed the VBV buffer size, resulting in poor use efficiency of the output buffer.</p><p> An object of the present invention is to provide an image processing apparatus, an image processing method, and a program that conform to a buffer model and can improve the efficiency of using a buffer.</p>
<p> The image processing apparatus of the present invention has an encoding processing unit that encodes moving image data, an additional unit that adds additional data to the output data of the encoding processing unit, and a capacity of the output data that the encoding processing unit should output. A control unit that monitors and controls the output of the encoding processing unit so that when the additional unit adds the additional data to the output data, the capacity of the output data including the additional data is equal to or less than the specified value. And have.</p><p> The image processing method of the present invention includes a first step of encoding moving image data, a second step of monitoring the capacity of the encoded data, and when additional data is added to the encoded data. The third step of controlling the capacity of the encoded data to be output so that the capacity of the encoded data including the data is equal to or less than the specified value, and the fourth step of adding the additional data to the encoded data. Has steps and.</p><p> The program of the present invention performs the first process of encoding the moving image data, the second process of monitoring the capacity of the encoded data, and when the additional data is added to the encoded data, the additional data is added. The third process of controlling the capacity of the encoded data to be output so that the capacity of the encoded data including the data is equal to or less than the specified value, and the fourth process of adding the additional data to the encoded data. Let the computer run.</p><p> According to the present invention, the encoding processing unit encodes moving image data. At this time, the control unit monitors the amount of output data to be output by the encoding processing unit. Then, the control unit controls the output of the encoding processing unit so that the capacity of the output data including the additional data becomes equal to or less than the specified value when the additional unit adds the additional data to the output data. After that, the addition unit adds the addition data to the output data of the encoding processing unit.</p>
<p> According to the present invention, it is possible to improve the buffer usage efficiency in accordance with the buffer model.</p>
Hereinafter, embodiments of the present invention will be described in association with the drawings. The explanation will be given in the following order. 1. Embodiment 1.1. Configuration example of image processing device 1 1.2. Operation example of image processing device 1 1.3. Relationship between output buffer size and VBV buffer size 1.4. Exception handling of image processing device 1
<1. Embodiment> [1.1. Configuration example of image processing device 1] FIG. 1 is a block diagram showing a configuration example of an image processing device according to an embodiment of the present invention. The image processing device 1 illustrated in FIG. 1 includes an encoder IP device 11, a control device 12, an additional information addition device 13, and an initial setting storage device 14. The encoder IP device 11 includes an encoder unit 111, a rate control unit 112, a VBV buffer 113, and an output buffer 114. The control device 12 has a bit rate unit 121 and a buffer unit 122.
The encoder IP device 11 corresponds to the encoding control unit of the present invention. The control device 12 corresponds to the control unit of the present invention. The additional information addition device 13 corresponds to the additional part of the present invention. The initial setting storage device 14 corresponds to the storage unit of the present invention. The VBV buffer 113 corresponds to the virtual buffer of the present invention. The output buffer 114 corresponds to the buffer of the present invention.
The image processing device 1 has an encoding function capable of encoding (encoding or compressing coding) the input signal S0 and outputting the elementary stream S2 on an IP (Internet Protocol) network. This input signal S0 is moving image data to be encoded. It consists of multiple frames and is encoded on a frame-by-frame basis. Specifically, the image processing device 1 controls the buffer size (buffer capacity) of the output buffer 114 so as to conform to the buffer model by controlling the bit rate. Hereinafter, the buffer size of the output buffer 114 is simply referred to as "output buffer size". Then, the image processing device 1 adds additional information (additional data) S8 to the output data S1 of the encoder IP device 11 to generate an elementary stream S2 conforming to a standard of a desired encoding format. The additional information S8 is, for example, information related to header information and user data.
The image processing device 1 supports various types of encoding formats such as MPEG-2 (Moving Picture Experts Group 2) and MPEG-4. The encoding format can be suitably selected according to conditions such as the data capacity and image quality after the encoding process. The above-mentioned MPEG-2 and MPEG-4 are standardized by organizations such as ITU-T (International Telecommunication Union Telecommunication Standardization Sector).
The encoder IP device 11 encodes the input signal S0 in a desired encoding format and outputs this as output data S1. Specifically, the encoder IP device 11 has the initial information S3 input from the initial setting storage device 14, the output buffer information S4 input from the control device 12, and the bit rate information S5 input from the device 12. Based on this, the input signal S0 is encoded on a frame-by-frame basis. The process performed by the encoder IP device 11 is called an encoding process.
This initial information S3 is composed of, for example, information on the encoding format, profile, level, bit rate, frame rate, and delay amount. A profile refers to a set of tools (elemental technologies) required to decode the elementary stream S2 with a decoder (not shown). This is defined by the encoding format standard. The levels differ depending on the standard of the encoding format, but are classified into, for example, level 1 to level 5 according to the resolution of the image.
After the encoding process is completed, the encoder IP device 11 outputs the encoded data of one frame to the additional information addition device 13 as output data S1. At this time, the encoder IP device 11 outputs the output result information S6 to the control device 12. This output result information S6 is composed of information indicating whether or not the output of the output data S1 from the encoder IP device 11 to the additional information addition device 13 was successful, and information regarding the output buffer size of the encoder IP device 11. ..
The encoder unit 111 encodes the input signal S0 based on the rate information S10 input from the rate control unit 112 and the initial information S3. At the time of encoding, an encoding process according to a desired encoding format is performed. After the encoding process, the encoder unit 111 outputs the result of the encoding process to the rate control unit 112 as output result information S9. This output result information S9 is the same as the output result information S6.
The rate control unit 112 controls the output buffer size by controlling the bit rate. At this time, the rate control unit 112 generates rate information S10 based on the output result information S9, the output buffer information S4, and the bit rate information S5, and outputs the rate information S10 to the encoder unit 111. This rate information S10 is information on the quantization coefficient and bit rate at the time of encoding one frame.
The VBV buffer 113 is a virtual buffer provided for controlling the buffer state on the decoder side when the decoder side (not shown) receives the elementary stream S2. Its buffer size is defined by the buffer model and is set by profile and level. Hereinafter, the buffer size of the VBV buffer 113 is simply referred to as "VBV buffer size". If this VBV buffer 113 fails, the decoding process on the decoder side may not be performed normally. Therefore, it is necessary to control the VBV buffer size so that failures such as overflow and underflow do not occur. Therefore, the amount of bits input to this buffer, that is, the bit rate, is controlled by the rate control unit 112 so that the output data S1 to which the additional information S8 is added does not exceed the VBV buffer size.
The output buffer 114 is a buffer for storing the output data S1 after the encoding process before outputting it to the additional information addition device 13.
The control device 12 controls the encoder IP device 11 based on the size information S7, the initial information S3 read from the initial setting storage device 14, and the output result information S6 input from the encoder IP device 11. Additional information S8 is added to the output data S1 of the encoder IP device 11 by the additional information addition device 13. The control device 12 monitors the output buffer 114, calculates the output buffer size so that the output data S1 including the additional information S8 does not exceed the VBV buffer size, and controls the bit rate to control the VBV buffer size. Control. At this time, the control device 12 generates the output buffer information S4 and the bit rate information S5, and outputs these to the encoder IP device 11. The output buffer information S4 is information on the output buffer size at the time of encoding one frame. Bit rate information S5 is information related to the bit rate at the time of encoding one frame.
The details of the control device 12 will be described. The bit rate unit 121 controls the bit rate during encoding processing by the encoder IP device 11 based on the output buffer information S11 input from the buffer unit 122. The output buffer information S11 is the same as the output buffer information S4.
The buffer unit 122 calculates the output buffer size based on the output result information S6, and outputs this calculation result to the bit rate unit 121 as the output buffer information S11.
The additional information addition device 13 acquires the size information S7 in advance, and acquires the size (data capacity) of the additional information S8 to be added. The size information S7 indicates the size of the additional information S8. After that, the additional information addition device 13 adds the additional information S8 to the output data S1 as the encoding result of one frame output by the encoder IP device 11. Then, the additional information addition device 13 outputs the output data S1 to which the additional information S8 is added as the elementary stream S2.
The initial setting storage device 14 stores the initial information S3. This initial information S3 is appropriately read by the encoder IP device 11 and the control device 12.
[1.2. Operation example of image processing device 1] An operation example of the image processing device 1 will be described. FIG. 2 is a sequence diagram showing an operation example of the image processing apparatus according to the embodiment of the present invention. FIG. 3 is a flowchart showing a detailed operation example of the control device according to the embodiment of the present invention.
The image processing device 1 (1) generates the elementary stream S2 for the first time (N = 0th frame), and (2) generates the elementary stream S2 for the N = Kth frame. And, the processing is different. First, the case of (1) N = 0th frame will be described, and then the case of (2) N = Kth frame will be described.
[(1) Generation of elementary stream S2 for N = 0th frame] As shown in FIG. 2, the additional information addition device 13 includes the size information S7 of this frame and the additional information at the timing when the input signal S0, that is, the N = 0th frame is input to the encoder IP device 11. Get S8 and (step ST1).
After that, the control device 12 performs the information generation process described below (step ST2). Specifically, as illustrated in FIG. 3, the control device 12 reads the initial information S3 from the initial setting storage device 14 when the size information S7 for the N = 0th frame is input. Then, the buffer unit 122 of the control device 12 calculates the output buffer size for this frame using the following equation (step ST21).
(Number 1) BUF = VBV-HS ... (Equation 1)
In (Equation 1), BUF indicates the output buffer size, VBV indicates the VBV buffer size, and HS indicates the additional information size. As shown in (Equation 1), the buffer unit 122 of the control device 12 calculates the output buffer size BUF at the current stage by subtracting the additional information size HS shown in the size information S7 from the VBV buffer size VBV. .. Then, the buffer unit 122 outputs this calculation result to the bit rate unit 121 as output buffer information S11.
After that, the bit rate unit 121 receives the output buffer information S11 from the buffer unit 122 and acquires the output buffer size. Then, the bit rate unit 121 sets the bit rate to be set based on the following equation (step ST22).
(Number 2) BR = TBR ... (Equation 2)
In (Equation 2), BR indicates the bit rate and TBR indicates the target bit rate. The target bit rate TBR shown on the right side of (Equation 2) is the bit rate at the time of encoding processing for the previous frame set by the rate control unit 112. When the elementary stream S2 is generated for the first time, the bit rate unit 121 sets the bit rate to be set to the target bit rate TBR. However, in the encoding process for the N = 0th frame, the bit rate for the previous frame is not set, so the rate control unit 112 sets the bit rate based on the initial information S3.
After that, the control device 12 uses the output buffer information S11 as the output buffer information S4, the bit rate calculated by the bit rate unit 121 as the bit rate information S5, and outputs these to the encoder IP device 11 (step ST23).
After that, the encoder IP device 11 reads the initial information S3 from the initial setting storage device 14. As shown in FIG. 2, when the output buffer information S4 and the bit rate information S5 are input from the control device 12, the encoder IP device 11 refers to the input signal S0, that is, N = 0th frame. And perform the encoding process (step ST3).
Specifically, the rate control unit 112 generates rate information S10 based on the output result information S9, the output buffer information S4, and the bit rate information S5 in order to control the bit rate, and the rate information unit 111 generates the rate information S10. Output to.
Then, the encoder unit 111 encodes the input signal S0 of one frame based on the rate information S10 input from the rate control unit 112 and the initial information S3. After the encoding process, the encoder unit 111 outputs the encoded data of one frame as the output data S1 to the additional information addition device 13. At the same time, the encoder unit 111 outputs the output result information S6 as the encoding result to the control device 12, and outputs the same output result information S9 to the rate control unit 112.
After that, the additional information addition device 13 adds the additional information S8 to the output data S1 as the encoding result of one frame input from the encoder IP device 11. Then, the additional information addition device 13 outputs the output data S1 to which the additional information S8 is added as the elementary stream S2.
[(2) Generation of elementary stream S2 for the kth frame] After that, the image processing device 1 sequentially performs encoding processing on the N = 1,2, ... th frame to generate the elementary stream S2. However, in the N = kth frame, the data up to the N = (K-1)th frame is stored in the output buffer 114, so the output buffer size and bit rate calculation method are the same as those described above. different. Hereinafter, the points different from the case of (1) N = 0th frame will be described.
Specifically, in step ST21, the buffer unit 122 of the control device 12 calculates the output buffer size for the N = kth frame using the following equation.
(Number 3) BUR = VBV-SB-HS ... (Equation 3)
In (Equation 3), SB indicates the conserved quantity of the output buffer 114 until the previous information generation processing (k-1), which is calculated by the following equation.
<maths num="4"><img file="JP2010141479A_D0001.tif" /></maths>
In (Equation 4), Dylay indicates the amount of delay in the encoding process, and FR indicates the frame rate. B (n) indicates the bit rate for the nth frame. S (n) indicates the output size of the elementary stream S2 for the nth frame. As shown in (Equation 3), the buffer unit 122 of the control device 12 includes the conserved quantity SB of the output buffer 114 from the VBV buffer size VBV to the previous (k-1) information generation process, and the additional information size HS. Is subtracted. As a result, the output buffer size BUR at the current stage is calculated.
In the next step ST22, the bit rate unit 121 calculates the bit rate to be set based on the following equation.
(Number 5) BR = BUR × FR ... (Equation 5)
In the N = kth frame, the bit rate unit 121 calculates the bit rate BR shown in (Equation 5) by the product of the output buffer size BUR and the frame rate FR. This bit rate is called the calculated bit rate BR. Further, the bit rate currently set by the rate control unit 112 is called the current bit rate BRα.
The bit rate unit 121 compares the current bit rate BRα with the calculated bit rate BR. When the current bit rate BRα is smaller than the calculated bit rate BR (BRα BR), the bit rate unit 121 sets the bit rate to be set to the current bit rate BRα. On the contrary, when the current bit rate BRα is larger than the calculated bit rate BR (BRα> BR), the bit rate unit 121 sets the bit rate to be set in the calculated bit rate BR.
[1.3. Relationship between output buffer size and VBV buffer size] According to the image processing apparatus 1 described above, the relationship between the output buffer size and the VBV buffer size is shown as follows.
FIG. 4 is a diagram showing the relationship between the output buffer size and the VBV buffer size according to the embodiment of the present invention. The vertical axis of FIG. 4 indicates the output buffer size, and the horizontal axis indicates time. The dashed line a indicates the VBV buffer size. The solid line b shows the output data S1 by the encoder IP device 11. The broken line c indicates the elementary stream S2 by the additional information addition device 13.
As shown by the broken line a in FIG. 4, the VBV buffer size is fixed by the buffer model. The encoding process for one frame is performed during the period of Δt shown in FIG. When the encoding process for one frame starts, the output buffer size increases linearly as the amount of bits stored in the output buffer 114 increases.
As shown in the solid line b in FIG. 4, the encoder IP device 11 outputs the output data S1 as the encoded data to the additional information addition device 13. Then, the additional information addition device 13 adds the additional information S8 to the output data S1. As a result, the elementary stream S2 is illustrated by the broken line c in FIG. During the encoding process, the control device 12 determines the bit rate indicated by the slope of the solid line b and the broken line c so that the elementary stream S2 does not exceed the VBV buffer size when the additional information S8 is added to the output data S1. To control. Therefore, it is possible to generate an elementary stream containing additional information conforming to the encoding format standard without breaking the VBV buffer 113.
[1.4. Exception handling of image processing device 1] By the way, the encoder IP device 11 may fail in the encoding process for the N = Kth frame. Hereinafter, exception handling of the image processing apparatus 1 in such a case will be described.
FIG. 5 is a diagram for explaining an operation example of the control device when the encoding process according to the embodiment of the present invention fails. The broken line d in FIG. 5 shows the output data S1 when the encoding process by the encoder IP device 11 fails. The other symbols and parameters shown in FIG. 5 are the same as those in FIG.
In the period Δt, the encoder IP device 11 may fail in the encoding process, and the capacity of the output data S1 may be larger than that in other periods, as shown by the broken line d in FIG. In this case, if the additional information S8 is added to the output data S1, the output buffer size may exceed the VBV buffer size.
In this case, the encoder IP device 11 stops (skips) the encoding process for this frame. Then, the encoder IP device 11 outputs the output result information (stop signal) S6 indicating that the encoding has failed to the control device 12. When the control device 12 receives the output result information S6, the bit rate unit 121 sets a bit rate lower than the bit rate calculated based on (Equation 2) or (Equation 5) as shown in FIG. 5c. To do. After the period Δt, the encoding process for the N = (K + 1) th frame is restarted. In this way, since the control device 12 controls the encoder IP device 11, as shown by the broken line c in FIG. 5, the final encoding data does not exceed the VBV buffer size, and the encoding process conforming to the buffer model is performed. It can be carried out.
The size information S7 and the additional information S8 may be changed after the encoder IP device 11 outputs the output data S1 to the additional information addition device 13 during the encoding process for the N = Kth frame.
In this case, the control device 12 calculates the increase / decrease amount of the additional information S8 based on the size information S7. This increase / decrease amount is calculated by the following equation.
(Number 6) HS (K + 1) = HS (K) + ΔHS (K) ... (Equation 6)
In (Equation 6), HS (K + 1) indicates the additional information size in the (K + 1) th frame. HS (K) indicates the additional information size in the Kth frame. ΔHS (K) indicates the increase / decrease in the additional information in the Kth frame. As shown in (Equation 6), the control device 12 sets the additional information size in the (K + 1) th frame to HS (K + 1) by adding the increase / decrease ΔHS of the additional information to the additional information size HS (K). Calculate K + 1).
After that, the bit rate unit 121 of the control device 12 takes into consideration the amount of increase / decrease calculated by (Equation 6) at the time of encoding the (K + 1) th frame, and (Equation 3) and (Equation 5). ) To calculate the bit rate. Specifically, the HS (K + 1) shown in (Equation 6) may be substituted for the additional information size HS shown in (Equation 3). The control device 12 controls the encoder IP device 11 based on the calculation result.
Hereinafter, the effect of the image processing device 1 according to the embodiment of the present invention will be described by comparing the image processing device 1 according to the embodiment of the present invention with a general image processing device.
FIG. 6 is a block diagram showing a configuration example of a general image processing device. FIG. 7 is a diagram showing the relationship between the output buffer size and the VBV buffer size in the image processing apparatus of FIG.
The general image processing device 2 shown in FIG. 6 includes an encoder IP device 11, an additional information addition device 13, and an initial setting storage device 14. A device having the same reference numerals as those shown in FIG. 1 has the same functions as the device of the image processing device 1.
However, as shown in FIG. 6, the image processing device 2 is not equipped with the control device 12 which is a feature of the embodiment of the present invention. Therefore, the image processing device 2 cannot monitor the output buffer size, and it is necessary to fix the output buffer size in advance. The output buffer size is fixed to the maximum size by, for example, the output buffer fixing information S12 input to the encoder IP device 11.
As a result, the bit rate cannot be controlled for each frame encoding process, and the final encoded data exceeds the VBV buffer size indicated by the dashed line a, as shown by the dashed line c in FIG. There is. In this case, the VBV buffer 113 fails, and the elementary stream conforming to the buffer model cannot be generated.
In addition to this, if the encoding process fails or if the size information S7 and additional information S8 are changed after the encoding process, the output buffer size cannot be controlled according to the situation. It is not possible to deal with such a situation.
On the other hand, in the image processing device 1 equipped with the control device 12, since the control device 12 monitors the output buffer size and controls the bit rate, it is possible to generate an elementary stream conforming to the buffer model. In addition to this, since the control device 12 controls the bit rate, it is not necessary to fix the output buffer size, and the size can be minimized. In the above situation, by performing exception handling, the elementary stream can be generated without breaking the VBV buffer 113.
As described above, according to the embodiment, the additional information addition device 13 can generate an elementary stream including additional information conforming to the encoding format standard. Since the additional information is added after the encoding process, the encoder IP device 11 can be suitably configured. Therefore, the elementary stream can be decoded by decoders having various configurations regardless of the configuration of the encoder IP device 11.
The method described in detail above can also be formed as a program according to the above procedure and configured to be executed by a computer having a CPU or the like. Such a program can be accessed by a recording medium such as a semiconductor memory, a magnetic disk, an optical disk, a floppy (registered trademark) disk, or a computer in which the recording medium is set, and can be configured to execute the above program.
The image processing apparatus according to the embodiment can be equipped with a decoding function for decoding the elementary stream S2 in addition to the encoding function. The functions of the image processing apparatus according to the embodiment are processed by software, but some or all of them can also be processed by hardware. The image processing apparatus and image processing method according to the present invention can be applied to various electronic devices that handle moving image data, such as video cameras, surveillance cameras, televisions, personal computers, and various video recording / playback devices. ..
<figref num="1">FIG. 1 is a block diagram showing a configuration example of an image processing device according to an embodiment of the present invention.</figref><figref num="2">FIG. 2 is a sequence diagram showing an operation example of the image processing apparatus according to the embodiment of the present invention.</figref><figref num="3">FIG. 3 is a flowchart showing a detailed operation example of the control device according to the embodiment of the present invention.</figref><figref num="4">FIG. 4 is a diagram showing the relationship between the output buffer size and the VBV buffer size according to the embodiment of the present invention.</figref><figref num="5">FIG. 5 is a diagram for explaining an operation example of the control device when the encoding process according to the embodiment of the present invention fails.</figref><figref num="6">FIG. 6 is a block diagram showing a configuration example of a general image processing device.</figref><figref num="7">FIG. 7 is a diagram showing the relationship between the output buffer size and the VBV buffer size in the image processing apparatus of FIG.</figref><figref num="8">FIG. 8 is a block diagram showing a configuration example of a general image processing device.</figref><figref num="9">FIG. 9 is a block diagram showing a configuration example of a general image processing device.</figref><figref num="10">FIG. 10 is a diagram showing the relationship between the VBV buffer size by the general image processing apparatus shown in FIGS. 8 and 9 and the output buffer size.</figref>
Code description
1 ... Image processing device, 11 ... Encoder IP device, 12 ... Control device, 13 ... Additional information addition device, 14 ... Initial setting storage device, 111 ... Encoder section, 112. .. Rate control section, 113 ... VBV buffer, 114 ... output buffer, 121 ... bit rate section, 122 ... buffer section
12 sheets
Sheet 1 Sheet 2 Sheet 3 Sheet 4 Sheet 5 Sheet 6 Sheet 7 Sheet 8 Sheet 9 Sheet 10 Sheet 11 Sheet 12
Every citation, both ways
| Document | Relation | Office | Cited during |
|---|---|---|---|
| JP2017504282A | Cited by | Japan | Search report |
| JP2017504282A | Cited by | Japan | Search report |
| US10074382B2 | Cited by | United States of America | Applicant |
| JP2001186532A | Cites | Japan | Examiner |
| JP2001211452A | Cites | Japan | Search report |
| JP2001211452A | Cites | Japan | Examiner |
| JPH10108179A | Cites | Japan | Examiner |
| JPH10190745A | Cites | Japan | Search report |
| JPH10190745A | Cites | Japan | Examiner |
2 members in 1 office
Priority claims2
| Document | Office | Kind | Date |
|---|---|---|---|
| 2008314254 | Japan | A | |
| JP20080314254 | – | – | – |
Members2
| Document | Office | Kind | |
|---|---|---|---|
| JP2010141479AThis record | Japan | A | |
| JP5206382B2 | Japan | B2 |
10 legal events, as the office reported them to INPADOC
Over the term
Point at a mark for the eventEvents
| Event | Code | |
|---|---|---|
| Cancellation because of no payment of annual feesLAPS | LAPS | |
| Renewal fee payment (event date is renewal date of database)FPAY | FPAY | |
| Renewal fee payment (event date is renewal date of database)FPAY | FPAY | |
| First payment of annual fees (during grant procedure)JAPANESE INTERMEDIATE CODE: A61A61 | A61 | |
| Written decision to grant a patent or to grant a registration (utility model)JAPANESE INTERMEDIATE CODE: A01A01 | A01 | |
| Decision of grant or rejection writtenTRDD | TRDD | |
| Written amendmentJAPANESE INTERMEDIATE CODE: A523A521 | A521 | |
| Notification of reasons for refusalJAPANESE INTERMEDIATE CODE: A131A131 | A131 | |
| Report on retrievalJAPANESE INTERMEDIATE CODE: A971007A977 | A977 | |
| Written request for application examinationJAPANESE INTERMEDIATE CODE: A621A621 | A621 |
Numbers
- Publication
- 2010141479
- Publication, DOCDB
- 2010141479
- Publication, EPODOC
- JP2010141479
- Application
- 314254
- Application, DOCDB
- 2008314254
- Application, EPODOC
- JP20080314254
Titles2
- Japanese
- 画像処理装置、画像処理方法およびプログラム
- English
- Image processing equipment, image processing methods and programs
Classification
- IPC, 9
- H04N19 00
- G11B20 10
- H04N19 115
- H04N19 134
- H04N19 15
- H04N19 172
- H04N19 196
- H04N19 423
- H04N7 26