Image coding apparatus, image coding method, integrated circuit, and camera

The present invention relates to an image coding apparatus and an image coding method for compressing and coding an image and recording it on a storage medium such as an optical disk, a magnetic disk, or a flash memory, and in particular, an MPEG4 AVC / H.264 compression coding method. The present invention relates to an image coding apparatus, an image coding method, an image coding integrated circuit, and a camera that perform compression coding by means of.

With the development of digital video technology, the technology of compressing and coding digital video data is developing in response to the increase in the amount of data. Its development has emerged as a compression coding technology specialized for video data, making the best use of the characteristics of video data.

In addition, with the improvement of the processing capacity of information processing devices such as computers, complicated calculations in compression coding technology have become possible, and the compression rate of video data is increasing significantly. Specifically, the compression coding technology adopted in satellite and terrestrial digital high-definition broadcasting is a compression coding method called MPEG2 (Moving Picture Experts Group 2). For example, in satellite digital high-definition broadcasting, video data is compressed to about 1/30 by MPEG2.

MPEG4 AVC / H.264 (hereinafter referred to as H.264), which is one of the standardized video compression coding methods following MPEG2, is said to achieve a compression rate about twice that of MPEG2. There is. H.264 is also adopted as a video compression coding method for Blu-ray, which is one standard for optical discs, and AVCHD (Advanced Video Codec High Definition), which is a standard for recording high-definition video with a video camera. It is expected to be used in a wide range of fields.

However, as disclosed in Non-Patent Document 1, H.264 implements many compression coding technologies and realizes a high compression rate by combining them, so the amount of calculation is also compared with MPEG2. Has increased significantly.

Generally, in the compression coding of a moving image, the amount of information is compressed by reducing the redundancy in the temporal direction and the spatial direction. Specifically, in inter-screen prediction coding for the purpose of reducing temporal redundancy, motion detection (motion vector detection) and prediction image generation (motion) are performed in block units by referring to the front or rear picture. Compensation) is performed, and the difference value between the generated predicted image and the image to be encoded is encoded.

In addition, weighting prediction is one of the compression coding techniques for compressing moving images. In motion compensation for performing this weighted prediction, the predicted pixel value p is set to w × pred + by multiplying the pixel value pred of the reference picture by the weighted value w and adding the offset value o. o . On the other hand, in motion compensation without weighting prediction, the predicted pixel value p becomes the pixel value pred of the reference picture.

As described above, in the weighting prediction, the brightness of the image can be predicted by using the weighting value w and the offset value o. That is, it is possible to generate a highly accurate predicted image by predicting a change in the brightness of an image and determining a weighting value w and an offset value o based on the predicted change in brightness. it can. Therefore, it is possible to improve the coding efficiency of a faded image or a live image having many flashes.

There are two modes of this weighted prediction, Implicit mode and Explicit mode. When motion compensation using the Implicit mode is selected, the above-mentioned weighting value w and offset value o are obtained by a calculation method determined by the standard. When motion compensation using the Explicit mode is selected, the encoder that performs compression coding describes the above-mentioned weighting value w and offset value o in the stream header of the coded stream and encodes the standard. It is possible to specify the weighting value "w" and the offset value "o" within the range of the values allowed in.

For example, Patent Document 1 describes a technique for determining whether or not a moving image to be encoded is a fading moving image, and determining whether or not to perform weighting prediction. Specifically, by detecting a change in the brightness value of the image, it is determined whether or not the moving image to be encoded is a fading moving image.

As a result, it is possible to switch between applying and not applying the weighted prediction, so that the coding efficiency can be increased more effectively.

<p><patcit num="1"><text>Japanese Unexamined Patent Publication No. 2007-306619</text></patcit></p>

<p><nplcit num="1"><text>ITU-T Recommendation H.264</text></nplcit></p>

<p> However, the technique described in Patent Document 1 has a problem that the image quality and the coding efficiency may be lowered. Specifically, it is as follows.</p><p> In the technique described in Patent Document 1, it is determined whether or not the moving image to be encoded is a fading moving image by detecting a change in the brightness value. However, although the moving image that is fading has a change in the brightness value, the moving image in which the brightness value is changing is not always faded.</p><p> Further, the weighted prediction contributes to the improvement of the coding efficiency when the luminance value is changed as a whole such as when fading, but in other cases, the image quality and the coding efficiency are lowered. There is a risk of letting you. Therefore, as in the technique described in Patent Document 1, it is not enough to detect a change in the luminance value, and there are cases where the image quality and the coding efficiency are lowered.</p><p> It should be noted that the coding efficiency was good by providing two types of motion compensation methods, motion compensation without weighted prediction and motion compensation with weighted prediction, and performing both types of motion compensation for each macroblock. A common method is conceivable, in which the motion compensation method is selected and the coding is performed. However, in this method, since the amount of processing required for coding increases, problems such as an increase in circuit scale and power consumption occur.</p><p> Therefore, the present invention has been made to solve such a problem, and it is possible to improve image quality and coding efficiency while reducing the amount of calculation, speeding up, and power consumption in the image compression coding process. It is an object of the present invention to provide an image coding apparatus and an image coding method capable of performing the same.</p>

<p> In order to achieve the above object, the image coding device according to the present invention is an image coding device that compresses and encodes moving image data including a coded image, and includes a storage unit that stores a reference image and the above-mentioned storage unit. Based on the reference image stored in the storage unit, motion compensation with weighted prediction, which is a process of adding a predetermined offset value to the pixel value of the reference image, or motion compensation without the weighted prediction is performed. , A predictive image generation unit that generates a predicted image of the coded target image, a coding unit that encodes a difference between the predicted image generated by the predicted image generation unit and the coded target image, and the coding. The prediction image generation unit includes a determination unit for determining whether or not the target image is an image included in a series of picture groups showing a movement of a predetermined movement amount or more in the moving image data, and the prediction image generation unit determines the determination. When the unit determines that the coded image is an image included in the picture group, the predicted image is generated by performing motion compensation without the weighted prediction, and the coded image is the coded image. When it is determined that the image is not included in the picture group, the predicted image is generated by performing motion compensation accompanied by the weighted prediction.</p><p> As a result, it is determined whether or not the image to be encoded is an image included in a series of picture groups showing a movement of a predetermined amount or more, and it is selected whether or not to perform weighting prediction based on the determination result. Therefore, the image quality and the coding efficiency can be improved. In addition, the amount of calculation required can be reduced as compared with the case where both types of motion compensation, that is, motion compensation without weighted prediction and motion compensation with weighted prediction, are performed. Further, since the presence or absence of motion is determined, the accuracy of prediction can be further improved and the coding efficiency can be improved as compared with the case of detecting a change in the luminance value.</p><p> Further, the determination unit acquires imaging information indicating the operation of the imaging device when the coding target image is captured, and based on the acquired imaging information, the coding target image is included in the picture group. It may be determined whether or not it is.</p><p> As a result, the presence or absence of movement can be determined based on the operation of the imaging device, so that the accuracy of prediction can be further improved and the coding efficiency can be improved.</p><p> Further, the image pickup information indicates whether or not the image pickup device is in a zoomed, panned or tilted state, and the determination unit is in a state in which the acquired image pickup information is zoomed, panned or tilted. When indicating that, it may be determined that the coded image is an image included in the picture group.</p><p> As a result, the imaging device encodes the image captured during zooming, panning, or tilting without using weighting prediction, so that the coding efficiency can be improved. This is because the weighted prediction is preferably used when the brightness value fluctuates over the entire screen such as a faded image or an image with a lot of flashes, and conversely, it is used for an image with a lot of movement to reduce the coding efficiency. This is because it is unfavorable.</p><p> Further, the imaging information further indicates whether or not the imaging device is in a state of changing the aperture, and the determination unit indicates that the acquired imaging information is not in a state of changing the aperture. In some cases, or when indicating that the aperture is being changed and the image is zoomed, panned, or tilted, the image to be encoded is an image included in the picture group. You may judge.</p><p> As a result, the amount of calculation can be reduced by utilizing the change in the aperture.</p><p> Further, the image coding device further includes a parameter setting unit for setting the offset value to a predetermined value for each one or more reference images referenced by the coded target image, and the moving image data can be obtained. It is moving image data of an interrace method, and when the determination unit determines that the coded image is an image included in the picture group, the parameter setting unit has an inverse parity with the coded image. The offset value of the related reference image is set to 0, and when the determination unit determines that the coded image is an image included in the picture group, the prediction image generation unit determines the parameter. The predicted image may be generated by adding the offset value set by the setting unit and the reference image.</p><p> As a result, when the moving image data is interlaced image data, even if it is determined that the image to be encoded is a moving image, the offset value o is obtained when the reference image is inverse parity. By setting "to 0", weighted prediction can be prevented. As a result, it is possible to more accurately determine whether or not the image to be coded is a moving image, and it is possible to improve the image quality and the coding efficiency.</p><p> Further, when the feature amount calculated based on the coded target image and the past coded target image is equal to or more than a predetermined threshold value, the determination unit includes the coded target image in the picture group. It may be determined that the image is an image, and when the feature amount is smaller than the threshold value, it may be determined that the coded target image is an image not included in the picture group.</p><p> As a result, it is determined whether or not the weighting prediction is performed based on the moving image data including the image to be encoded, so that it is possible to more accurately determine whether or not the picture to be encoded is a moving image. The image quality and coding efficiency can be improved.</p><p> Further, the determination unit determines that the coded image is an image included in the picture group when the feature amount calculated based on the coded image already encoded is equal to or greater than a predetermined threshold value. If the feature amount is smaller than the threshold value, it may be determined that the coded image is not included in the picture group.</p><p> As a result, it is determined whether or not to perform weighting prediction based on the already encoded image to be encoded, so that it is not necessary to calculate a new feature amount, and the picture to be encoded can be obtained with a low processing amount. It is possible to more accurately determine whether or not the image is moving. Therefore, the image quality and the coding efficiency can be improved.</p><p> Further, the image coding device compresses and encodes the moving image data based on the MPEG4 AVC / H.264 compression coding method, and the image coding device further sets the offset value in Implicit mode or Explicit mode. It may be provided with a parameter setting unit set in.</p><p> Further, in the Explicit mode, the parameter setting unit sets the offset value based on the amount of change between the average value of the brightness values of the coded image and the average value of the brightness values of the reference image. May be good.</p><p> The present invention can also be realized as a camera, and the camera according to the present invention is a camera that compresses and encodes moving image data including a coded target image obtained by capturing a subject image. An optical system that forms the subject image by forming an image of light from the subject, an imaging element that acquires the moving image data by imaging the subject image, and imaging information indicating the operation of the camera. Weighted prediction, which is a process of adding a predetermined offset value to the pixel value of the reference image based on the imaging information generation unit that generates the image, the storage unit that stores the reference image, and the reference image stored in the storage unit. A prediction image generation unit that generates a prediction image of the coded target image by performing motion compensation with or without the weighted prediction, a prediction image generated by the prediction image generation unit, and the above. A series of pictures showing the movement of the coded target image by a predetermined amount of movement or more based on the coding unit that encodes the difference from the coded target image and the imaging information generated by the imaging information generation unit. When the determination unit determines whether or not the image is included in the group and the prediction image generation unit determines that the image to be encoded is an image included in the picture group, the determination unit determines. The predicted image is generated by performing motion compensation without the weighted prediction, and when it is determined that the coded target image is not an image included in the picture group, motion compensation accompanied by the weighted prediction is performed. This generates the predicted image.</p><p> It should be noted that the present invention can be realized not only as an image coding device and a camera, but also as a method in which a processing unit constituting the image coding device is used as a step.</p><p> In addition, some or all of the components constituting each of the above image coding devices and cameras may be composed of one system LSI (Large Scale Integration). A system LSI is an ultra-multifunctional LSI manufactured by integrating a plurality of components on a single chip. Specifically, it is configured to include a microprocessor, ROM, RAM (Random Access Memory), and the like. Computer system.</p>

<p> According to the present invention, it is possible to improve image quality and coding efficiency while reducing the amount of calculation, speeding up, and power consumption in the image compression coding process.</p>

<figref num="1">It is a block diagram which shows an example of the structure of the image coding apparatus which concerns on Embodiment 1. FIG.</figref><figref num="2">It is a block diagram which shows an example of the structure of the image coding part in the image coding apparatus which concerns on Embodiment 1. FIG.</figref><figref num="3">It is a flowchart which shows an example of the processing executed by the motion compensation part in the image coding part of the image coding apparatus which concerns on Embodiment 1. FIG.</figref><figref num="4">It is a flowchart which shows an example of the processing executed by the weighting prediction determination unit in the image coding apparatus which concerns on Embodiment 1. FIG.</figref><figref num="5">It is a flowchart which shows another example of the processing executed by the weighting prediction determination unit in the image coding apparatus which concerns on Embodiment 1. FIG.</figref><figref num="6">It is a block diagram which shows an example of the structure of the image coding apparatus which concerns on Embodiment 2. FIG.</figref><figref num="7">It is a flowchart which shows an example of the processing executed by the weighting prediction determination unit in the image coding apparatus which concerns on Embodiment 2. FIG.</figref><figref num="8">It is a block diagram which shows an example of the structure of the image coding apparatus which concerns on Embodiment 3.</figref><figref num="9">It is a flowchart which shows an example of the processing executed by the weighting prediction determination unit in the image coding apparatus which concerns on Embodiment 3. FIG.</figref><figref num="10">It is a flowchart which shows an example of the processing executed by the weighting prediction determination unit in the image coding apparatus when the input image data is the image data of the interlace system.</figref><figref num="11">It is a block diagram which shows an example of the structure of the image coding apparatus in the modification of the embodiment which concerns on this invention.</figref><figref num="12">It is a schematic diagram which shows an example of the camera in the modification of the embodiment which concerns on this invention.</figref><figref num="13">It is a block diagram which shows an example of the structure of the camera in the modification of the embodiment which concerns on this invention.</figref>

Hereinafter, the image reproduction apparatus and the image reproduction method according to the present invention will be described with reference to the drawings based on the embodiments.

(Embodiment 1) The image coding apparatus according to the first embodiment includes a determination unit for determining whether or not the image to be coded is an image included in a series of picture groups showing a predetermined amount of movement or more, and an image to be coded. Is determined to be an image included in the picture group, a predicted image is generated by performing motion compensation without weighted prediction, and it is determined that the image to be encoded is not an image included in the picture group. In this case, it is characterized by including a prediction image generation unit that generates a prediction image by performing motion compensation accompanied by weighted prediction.

FIG. 1 is a block diagram showing an example of the configuration of the image coding device 100 according to the first embodiment. As shown in FIG. 1, input moving image data including a plurality of pictures and imaging information which is additional information of the input moving image data are input to the image coding apparatus 100. The image coding device 100 encodes the input input video data by the H.264 compression coding method, and outputs the encoded input video data as an output stream.

In the coding by the H.264 compression coding method, one picture is divided into one or a plurality of slices, and the slice is used as a processing unit. In the coding by the H.264 compression coding method in the first embodiment, it is assumed that one picture is one slice. This also applies to the second and third embodiments described later.

As shown in FIG. 1, the image coding device 100 includes a weighting prediction determination unit 101, a weighting parameter determination unit 102, and an image coding unit 103.

The weighted prediction determination unit 101 acquires imaging information as additional information of the input moving image data, and generates weighted prediction flag information based on the acquired imaging information. The generated weighted prediction flag information is output to the image coding unit 103.

The imaging information is information indicating the operation of an imaging device such as a camera that has acquired input moving image data by photographing. The operation of the image pickup apparatus includes the movement of the image pickup apparatus itself such as panning and tilting, and the operation inside the imaging apparatus such as changing the zoom and aperture. That is, specifically, the imaging information includes information indicating whether or not the imaging device is in the zooming state, information indicating whether or not the imaging device is in the panning state, and tilting. Information indicating whether or not it is in a state. The imaging information is additional information regarding the input moving image data input to the image coding apparatus 100 together with the input moving image data.

The weighting parameter determination unit 102 determines the coding parameters required for coding using the weighting prediction based on the pixel information of the input moving image data. The determined parameter information is output to the image coding unit 103 as a weighting parameter. The weighting prediction is a process of adding a predetermined offset value o to the pixel value of the reference image. Further, the offset value o may be added after multiplying the pixel value of the reference image by the weighting value w.

The coding parameters are, for example, the above-mentioned weighting value w and offset value o. The weighting value w is a value that is multiplied by the pixel value of the reference picture as described above, and the offset value o is a value that is added to the reference picture. For example, in H.264, since the coded target picture can refer to a plurality of reference pictures, the weighting parameter determination unit 102 sets the offset value o for each reference picture referenced by the coded target picture. To do.

The image coding unit 103 H.264 compression code of the input moving image data including the picture to be coded according to the weighting prediction flag information generated by the weighting prediction determination unit 101 and the weighting parameter determined by the weighting parameter determination unit 102. It is compressed and coded by the coding method. As will be described later, the image coding unit 103 encodes the difference between the predicted image and the image to be coded.

Next, an example of a detailed configuration of the image coding unit 103 will be described with reference to FIG. Note that FIG. 2 is a block diagram showing an example of a detailed configuration of the image coding unit 103 in the image coding device 100 according to the first embodiment. The image coding unit 103 compresses and encodes the coded target picture included in the input moving image data for each block.

As shown in FIG. 2, the image coding unit 103 includes an input image data memory 201, a reference image data memory 202, an in-plane prediction unit 203, a motion vector detection unit 204, a motion compensation unit 205, a prediction mode determination unit 206, and a difference. It includes a calculation unit 207, an orthogonal conversion unit 208, a quantization unit 209, an inverse quantization unit 210, an inverse orthogonal conversion unit 211, an addition unit 212, and an entropy coding unit 213.

The input image data memory 201 is a memory for storing input moving image data. The information held by the input image data memory 201 is referred to by the in-plane prediction unit 203, the motion vector detection unit 204, the prediction mode determination unit 206, and the difference calculation unit 207.

The reference image data memory 202 is an example of a storage unit such as a memory for storing a reference image, and stores a locally decoded image generated by the addition unit 212 as a reference image.

The in-plane prediction unit 203 predicts the in-plane prediction by performing the in-plane prediction from the locally decoded image (reference image) stored in the reference image data memory 202 using the encoded pixels in the same screen. Generate an image. The generated prediction image is output to the prediction mode determination unit 206.

The motion vector detection unit 204 searches for a locally decoded image (reference image) stored in the reference image data memory 202, detects an image area closest to the input image, and determines a motion vector indicating the detected position. .. Further, the motion vector detection unit 204 determines the size of the coded block having the smallest error and the motion vector at that size, and provides information indicating the determined size and motion vector to the motion compensation unit 205 and the entropy coding unit 213. Send to.

The motion vector detection unit 204 compares the coded block (macro block) with the block at an arbitrary position of the reference picture, and determines the position of the most similar block as the motion vector. The comparison error between the coded block and the reference block is generally used to determine whether or not they are similar, and in particular, the absolute difference (SAD) is often used. Further, since the amount of calculation becomes enormous when the reference block is searched in the entire reference picture, the search range (search range) in the reference picture is narrowed down and the narrowed down range is searched.

Based on the reference image stored in the reference image data memory 202, the motion compensation unit 205 performs motion compensation with weighted prediction or motion compensation without weighted prediction to obtain a predicted image of the picture to be encoded. This is an example of a predicted image generation unit to be generated. For example, the motion compensation unit 205 uses the motion vector included in the information received from the motion vector detection unit 204 to obtain the optimum image for the predicted image from the locally decoded image (reference image) stored in the reference image data memory 202. A region is taken out, a predicted image of inter-plane prediction is generated, and the generated predicted image is output to the prediction mode determination unit 206. The specific operation performed by the motion compensation unit 205 will be described in detail later.

The prediction mode determination unit 206 determines the prediction mode, and based on the determination result, is generated by the prediction image generated by the in-plane prediction from the in-plane prediction unit 203 and the inter-plane prediction from the motion compensation unit 205. Select one of the predicted images. The selected predicted image is output to the difference calculation unit 207 and the addition unit 212. As a method of determining the prediction mode performed by the prediction mode determination unit 206, for example, for interplane prediction and in-plane prediction, the sum of the absolute values of the differences between the input image and the predicted image of each pixel is calculated and calculated. The one with the smaller sum of absolute values is determined to be the prediction mode.

The difference calculation unit 207 reads out the input image data to be encoded from the input image data memory 201, and calculates the pixel difference value between the read input image and the predicted image output from the prediction mode determination unit 206. The calculated pixel difference value is output to the orthogonal conversion unit 208.

The orthogonal conversion unit 208 converts the pixel difference value input from the difference calculation unit 207 into a frequency coefficient, and outputs the converted frequency coefficient to the quantization unit 209. For example, the orthogonal transform unit 208 performs an orthogonal transform process such as DCT (Discrete Cosine Transform) on the pixel difference value.

The quantization unit 209 quantizes the frequency coefficient input from the orthogonal conversion unit 208, and outputs the quantized value, that is, the quantization value to the entropy coding unit 213 and the inverse quantization unit 210 as encoded data.

The inverse quantization unit 210 dequantizes the quantization value input from the quantization unit 209 and restores it to a frequency coefficient. The restored frequency coefficient is output to the inverse orthogonal converter 211.

The inverse orthogonal conversion unit 211 restores the pixel difference value by performing inverse frequency conversion on the frequency coefficient input from the inverse quantization unit 210. The restored pixel difference value is output to the addition unit 212.

The addition unit 212 generates a locally decoded image by adding the pixel difference value input from the inverse orthogonal conversion unit 211 and the prediction image output from the prediction mode determination unit 206. The generated locally decoded image is stored in the reference image data memory 202 as a reference image.

Here, the locally decoded image (reference image) stored in the reference image data memory 202 is basically the same image as the input image stored in the input image data memory 201. However, after the orthogonal conversion unit 208 and the quantization unit 209 perform processing such as orthogonal conversion and quantization once, the inverse quantization unit 210 and the inverse orthogonal conversion unit 211 perform processing such as inverse quantization and inverse orthogonal conversion. Therefore, the locally decoded image has a distortion component such as quantization distortion.

The entropy encoding unit 213 entropy-encodes the quantization value input from the quantization unit 209 and the motion vector input from the motion vector detection unit 204, and outputs the encoded data as an output stream.

Next, with reference to FIGS. 1 and 2, the flow of processing executed by the motion compensation unit 205 in the image coding unit 103 of the image coding device 100 according to the first embodiment will be described with reference to FIG. Note that FIG. 3 is a flowchart showing an example of the flow of processing executed by the motion compensation unit 205 in the image coding unit 103 of the image coding device 100 according to the first embodiment. It is also a flowchart showing an example of the flow of processing executed by the motion compensation unit in the image coding unit of the image coding apparatus according to the second and third embodiments described later.

As shown in FIG. 3, the motion compensation unit 205 corresponds to the position indicated by the motion vector received from the motion vector detection unit 204 from the locally decoded image (reference image) stored in the reference image data memory 202. Read the image area (hereinafter referred to as reference image block data) (S101).

Next, the motion compensation unit 205 generates interpolated image data by performing interpolation processing with 1/4 pixel accuracy from the reference image block data and the motion vector received from the motion vector detection unit 204 (S102). Interpolation processing with 1/4 pixel accuracy is determined by the standard, and interpolation processing using a FIR (Finite Impulse Response) filter is performed according to the value of the motion vector.

In H.264, if the motion vector indicates the position with one pixel accuracy, the filtering process is not performed. If the motion vector indicates a position with 1/2 pixel accuracy, interpolation processing is performed using a 6-tap filter. If the motion vector indicates a position with 1/4 pixel accuracy, interpolation processing is performed using a 6-tap filter and a 2-tap filter.

Next, the motion compensation unit 205 determines whether the weighting flag information received from the weighting prediction determination unit 101 is ON or OFF (S103). When it is determined that the weighting flag information is OFF (OFF in S103), the motion compensation unit 205 outputs the generated interpolated image data as the predicted image data to the prediction mode determination unit 206 (S105).

When it is determined that the weighting flag information is ON (ON in S103), the motion compensation unit 205 performs weighting prediction on the interpolated image data based on the weighting parameter determined by the weighting parameter determination unit 102. By doing so, the predicted image data is generated (S104).

As the weighting parameter when weighting prediction is performed on the interpolated image data, the weighting parameter determined by the weighting parameter determination unit 102 is used when weighting prediction is performed using the Explicit mode. When weighting prediction is performed using the Implicit mode, the weighting parameters are determined according to the procedure determined by the standard. In H.264, when weighting prediction is performed using Implicit mode, the weighting value "w" is obtained from the time distance between the input image data to be encoded and the reference image data, and the offset value "o" is obtained. Set to 0.

Next, an example of the processing executed by the image coding apparatus 100 including the image coding unit 103 configured as described above will be described.

First, the input moving image data is input to the weighting parameter determination unit 102 and the image coding unit 103, and the imaging information is input to the weighting prediction determination unit 101. The input moving image data includes a plurality of pictures and is stored in the input image data memory 201 of the image coding unit 103. One frame (picture) of a moving image is composed of, for example, 1920 pixels × 1080 pixels.

Further, as described above, the imaging information is information indicating the operation of the camera that captured the input image data. For example, was the camera in a state of panning, tilting, or zooming when the input image data was captured? This is additional information related to the input image data, such as information indicating whether or not the data is displayed. The imaging information is, for example, information output from a sensor (eg, an acceleration sensor) provided in the camera, a control unit, or the like.

The weighted prediction determination unit 101 encodes by determining whether or not the coded target picture included in the input moving image data is an image included in a series of picture groups showing a motion equal to or greater than a predetermined motion amount. It is determined whether or not to use the weighted prediction when encoding the target picture. In the following, "the image to be coded is an image included in a series of picture groups showing a movement of a predetermined amount of movement or more" is also referred to as "the picture to be coded is an image with motion". Describe.

When the weighted prediction determination unit 101 determines that the image to be encoded is a moving image, the weighting prediction determination unit 101 determines that the weighting prediction is not used, and sets the weighting prediction flag information to OFF. On the contrary, when it is determined that the picture to be encoded is not a moving image, it is determined that the weighted prediction is used, and the weighted prediction flag information is set to ON.

Specifically, the weighted prediction determination unit 101 acquires imaging information, and determines whether or not to use weighting prediction for coding the coded target picture from the acquired imaging information. Then, the weighted prediction flag information is set to ON or OFF according to the determination result, and the set weighted prediction flag information is output to the image coding unit 103.

Next, an example of the process executed by the weighted prediction determination unit 101 will be described with reference to FIG. Note that FIG. 4 is a flowchart showing an example of the processing executed by the weighting prediction determination unit 101 in the image coding apparatus 100 according to the first embodiment.

As shown in FIG. 4, the weighting prediction determination unit 101 determines whether or not the camera is in the pan, tilt, or zoom state when the image to be encoded is captured (S201). When it is determined that the camera is panning, tilting, or zooming (Yes in S201), the weighting prediction determination unit 101 sets the weighting prediction flag information to OFF, and sets the set weighting prediction flag information as an image code. Output to the conversion unit 103 (S202).

When it is determined that the camera is not in the pan, tilt, or zoom state (No in S101), the weighting prediction determination unit 101 sets the weighting prediction flag information to ON, and sets the set weighting prediction flag information as an image code. Output to the conversion unit 103 (S203).

The weighting parameter determination unit 102 determines the coding parameters required for coding using the weighting prediction based on the pixel information of the picture to be coded, and uses the determined coding parameters as weighting parameters in the image coding unit 103. Output. As a method of determining the weighting parameter, for example, the weighting value w is set to 1, and the difference value between the average value of the brightness values of the encoded target picture and the average value of the brightness values of the reference image is set to the offset value o. There is a method of setting, but the method is not limited to this, and other methods may be used. The weighting parameter is determined, for example, in picture units.

The image coding unit 103 performs motion vector detection, motion compensation, in-plane prediction, and orthogonality based on the weighting prediction flag information generated by the weighting prediction determination unit 101 and the weighting parameter determined by the weighting parameter determination unit 102. Performs a series of coding processes such as conversion, quantization, and entropy coding. Specifically, the motion compensation unit 205 generates a prediction image by executing only motion compensation or weighting prediction and motion compensation processing based on the weighted prediction flag information and the weighted parameter.

At this time, when weighted prediction is performed using the Explicit mode, the motion compensation unit 205 performs weighted prediction using the Explicit mode if the weighted prediction flag information is ON, and uses it as a parameter required for encoding the weighted prediction. The weighting parameter determined by the weighting parameter determination unit 102 is used. At this time, if the weighted prediction flag information is OFF, motion compensation without weighting prediction is performed.

When performing weighted prediction using Implicit mode, the motion compensation unit 205 performs weighted prediction using Implicit mode if the weighted prediction flag information is ON, and weighted prediction if the weighted prediction flag information is OFF. Do not perform motion compensation.

In the image coding device 100 according to the first embodiment, the image coding unit 103 compresses and encodes the input moving image data according to the H.264 compression coding method. Further, this also applies to the second and third embodiments described later.

Weighted prediction can improve the coding efficiency of an image with a change in brightness of the entire screen such as a faded scene, but on the other hand, it may lower the coding efficiency in a scene with a large movement. Therefore, the image to be encoded is a moving image by using the imaging information indicating whether or not the camera is in the pan, tilt, or zoom state when the picture to be encoded is taken. It can be determined whether or not. Thereby, the coding efficiency can be improved.

As described above, the weighted prediction flag information is determined based on the imaging information, and when the weighted prediction flag information is ON, the coded picture is encoded by using the weighted prediction, and when the weighted prediction flag information is OFF. , The input image data to be encoded is encoded without using weighted prediction. That is, when it is determined that the image to be encoded is a moving image, the predicted image is generated by performing motion compensation without weighting prediction, and it is determined that the image to be encoded is not a moving image. In that case, a predicted image is generated by performing motion compensation accompanied by weighted prediction. As a result, the prediction accuracy of motion compensation can be improved, so that the coding efficiency can be improved.

Further, if only the change in the brightness value is detected as in the technique described in Patent Document 1, it is determined that the change in the brightness value is large even in the image having a large movement, and the weighted prediction is performed for the image having a large movement. It ends up. In this case, the prediction accuracy is deteriorated, so that the image quality and the coding efficiency are deteriorated. Therefore, it is possible to improve the coding efficiency by not performing weighting prediction on an image having a large motion.

On the other hand, the image coding device 100 according to the first embodiment determines whether or not the picture to be coded is a moving image based on the imaging information, not the change in the brightness value, and encodes the picture. If the target picture is a moving image, weighting prediction is not performed. As a result, according to the image coding apparatus according to the first embodiment, the coding efficiency can be improved.

In addition, it is equipped with two types of motion compensation methods, motion compensation without weighted prediction and motion compensation with weighted prediction, and compared to the common-sense method of selecting the optimum method for each macroblock, the circuit scale and calculation The amount can be reduced.

From the above, according to the image coding apparatus 100 according to the first embodiment, the image quality and the coding efficiency are improved while reducing the calculation amount of the image coding method, increasing the speed, and reducing the power consumption. It becomes possible.

When setting the weighted prediction flag information to ON or OFF, not only whether or not the image has motion, but also the change in the brightness value of the image may be used. For example, the imaging information may include information indicating whether or not the camera is in a state of changing the aperture. FIG. 5 is a flowchart showing another example of the process executed by the weighted prediction determination unit 101 in the image coding apparatus 100 according to the first embodiment.

As shown in FIG. 5, the weighting prediction determination unit 101 determines whether or not the camera is in a state of changing the aperture when the image to be encoded is captured (S301). When it is determined that the camera is not in the state where the aperture is changed (No in S301), the weighting prediction determination unit 101 sets the weighting prediction flag information to OFF (S303). This is because when the aperture does not change, it means that the brightness value of the picture to be encoded does not change significantly, and the effect of weighting prediction cannot be expected sufficiently.

When it is determined that the camera is changing the aperture (Yes in S301), the weighting prediction determination unit 101 is in a state where the camera is panning, tilting, or zooming when the image to be encoded is taken. Determine if it exists (S302). When it is determined that the camera is panning, tilting, or zooming (Yes in S301), the weighting prediction determination unit 101 sets the weighting prediction flag information to OFF, and sets the set weighting prediction flag information as an image code. Output to the conversion unit 103 (S303).

When it is determined that the camera is not in the pan, tilt, or zoom state (No in S302), the weighting prediction determination unit 101 sets the weighting prediction flag information to ON, and sets the set weighting prediction flag information as an image code. Output to the conversion unit 103 (S304).

As described above, even when the aperture of the camera is changed, it is possible to select whether or not to perform weighting prediction by determining the presence or absence of movement.

As a result, the amount of processing is smaller when determining the change in the aperture than when determining the presence or absence of movement, so that the processing amount can be reduced as compared with the case where only the presence or absence of movement is determined.

(Embodiment 2) In the image coding apparatus according to the second embodiment, when the feature amount calculated based on the coded target image and the past coded target image is equal to or more than a predetermined threshold value, the coded target image moves. It is characterized in that it is determined that the image is a certain image, and when the feature amount is smaller than the threshold value, it is determined that the coded target image is not a moving image. As described above, the image coding apparatus according to the second embodiment determines whether or not there is movement from the image to be encoded, and selects whether or not to perform weighting prediction based on the determination result.

FIG. 6 is a block diagram showing an example of the configuration of the image coding device 300 according to the second embodiment.

As shown in FIG. 6, the image coding device 300 includes a weighting prediction determination unit 301, a weighting parameter determination unit 102, and an image coding unit 103. Since the weighting parameter determination unit 102 and the image coding unit 103 are the same as those in the first embodiment, they are designated by the same reference numerals, and the description thereof will be omitted below.

The weighted prediction determination unit 301 determines whether or not the feature amount calculated based on the coded target picture included in the input input moving image data and the past coded target picture is equal to or greater than a predetermined threshold value. To do. When it is determined that the feature amount is equal to or greater than the threshold value, it is determined that the picture to be encoded is a moving image, and the weighting prediction flag information is set to OFF. When it is determined that the feature amount is smaller than the threshold value, it is determined that the image to be encoded is not a moving image, and the weighted prediction flag information is set to ON. In this way, the weighted prediction determination unit 301 generates weighted prediction flag information using the pixel value information of the input moving image data, and outputs the generated weighted prediction flag information to the image coding unit 103.

The feature amount is a value obtained from the picture to be encoded, and is a value indicating the movement of the image. For example, the feature amount is a motion vector feature amount (hereinafter, referred to as a preprocessed motion vector feature amount) indicating the movement between the coded object picture and the past coded object picture.

Next, an example of the processing executed by the weighting prediction determination unit 301 in the image coding apparatus 300 configured as described above will be described.

First, the input moving image data including the picture to be coded is input to the weighting prediction determination unit 301 and the weighting parameter determination unit 102 together with the image coding unit 103. The weighted prediction determination unit 301 determines whether or not to use the weighted prediction for coding the coded target picture from the input pixel information of the coded target picture by a method described later. If it is determined that weighted prediction is used to encode the picture to be encoded, the weighted prediction flag information is set to ON, otherwise the weighted prediction flag information is set to OFF, and the set weighted prediction flag information is set to the image. Output to the coding unit 103.

Next, an example of the process executed by the weighted prediction determination unit 301 will be described with reference to FIG. 7. Note that FIG. 7 is a flowchart showing an example of the processing executed by the weighting prediction determination unit 301 in the image coding apparatus 300 according to the second embodiment.

As shown in FIG. 7, the weighted prediction determination unit 301 calculates the preprocessed motion vector feature amount using the coded target picture included in the input moving image data and the past coded target picture (S401). Then, it is determined whether or not the calculated preprocessing motion vector feature amount is equal to or greater than a predetermined threshold value (S402).

When it is determined that the preprocessing motion vector feature amount is equal to or higher than a predetermined threshold value (Yes in S402), the weighting prediction determination unit 301 sets the weighting prediction flag information to OFF and displays the set weighting prediction flag information as an image. Output to the encoding unit 103 (S403). When it is determined that the preprocessing motion vector feature amount is smaller than the predetermined threshold value (No in S402), the weighting prediction determination unit 301 sets the weighting prediction flag information to ON and displays the set weighting prediction flag information as an image. Output to the encoding unit 103 (S404).

Here, the preprocessed motion vector feature amount is, for example, the X component and the Y component of each vector calculated from the current input image (encoded picture) and the past input image (past encoded picture). The sum of the absolute values of and the magnitude of the motion vector such as the Euclidean norm that represents the distance between two points can be considered.

Further, the past coded target picture may be a past coded target picture corresponding to a reference picture used when encoding the current coded target picture. Alternatively, it may be a past coded picture input immediately before the current coded picture. However, the present invention is not limited to these, and if, for example, the desired past coded target picture as described above cannot be used due to circuit design reasons, input is performed several frames before the current coded target picture. Other past encoded pictures as described above may be substituted.

Further, the motion vector between the current coded image and the past coded picture is, for example, the present generated by reducing the current coded picture and the past coded picture, respectively. It is conceivable that the representative values of the motion vectors obtained by performing the motion vector detection in the predetermined search range for each of the reduced image of the above and the reduced image of the past. The representative value referred to here may be, for example, an average value of motion vectors, but is not limited to this, and the representative value may be determined by using statistical processing or the like.

As described above, the image coding apparatus 300 according to the second embodiment determines whether or not to perform weighting prediction based on the input moving image data including the picture to be coded. As a result, it is possible to more accurately determine whether or not the image to be encoded is a moving image, so that the image quality and the coding efficiency can be improved.

(Embodiment 3) In the image coding apparatus according to the third embodiment, when the feature amount calculated based on the already coded coded image is equal to or greater than a predetermined threshold value, the current coded image is a moving image. When the feature amount is smaller than the threshold value, it is determined that the current coded image is not a moving image. As described above, the image coding apparatus according to the third embodiment selects whether or not to perform weighting prediction based on the determination result by determining the presence or absence of motion from the already encoded image to be encoded. To do.

FIG. 8 is a block diagram showing an example of the configuration of the image coding device 400 according to the third embodiment.

As shown in FIG. 8, the image coding device 400 includes a weighting prediction determination unit 401, a weighting parameter determination unit 102, and an image coding unit 403. Since the weighting parameter determination unit 102 is the same as that of the first embodiment, the same reference numerals are given, and the description thereof will be omitted below.

The weighted prediction determination unit 401 determines whether or not the feature amount calculated based on the already encoded picture to be encoded is equal to or greater than a predetermined threshold value. When it is determined that the feature amount is equal to or greater than the threshold value, it is determined that the picture to be encoded is a moving image, and the weighting prediction flag information is set to OFF. When it is determined that the feature amount is smaller than the threshold value, it is determined that the image to be encoded is not a moving image, and the weighted prediction flag information is set to ON. As described above, the weighting prediction determination unit 401 generates the weighting prediction flag information by using the image coding information of the picture encoded before the current input image data in the image coding unit 403, and the generated weighting. The prediction flag information is output to the image coding unit 403.

The image coding unit 403 performs the same processing as the image coding unit 103 described in the first embodiment, and further outputs the image coding information, which is the information used for the compression coding, to the weighting prediction determination unit 401.

Next, an example of the processing executed by the image coding apparatus 400 configured as described above will be described.

First, the input moving image data including the picture to be encoded is input to the weighting parameter determination unit 102 and the image coding unit 403. Similar to the first and second embodiments, the image coding unit 403 sets the image data of the picture to be coded according to the weighting prediction flag information output by the weighting prediction determination unit 401 and the weighting parameter output by the weighting parameter determination unit 102. Is compressed and coded by the H.264 compression coding method. The compressed coded image data is output as an output stream. Further, the image coding unit 403 outputs the image coding information, which is the information used for the compression coding, to the weighting prediction determination unit 401.

The weighted prediction determination unit 401 receives the image coding information output from the image coding unit 403, and uses the received image coding information to determine whether or not to use the weighted prediction for coding the picture to be coded. Judgment is made by the method described later. The weighted prediction determination unit 401 sets the weighted prediction flag information to ON when it is determined that the weighted prediction is used for encoding the picture to be encoded, and sets it to OFF otherwise, and sets the weighted prediction flag information. Is output to the image coding unit 403.

In the above description, the image coding information is transmitted from the image coding unit 403 to the weighting prediction determination unit 401, but the present invention is not limited to this, and an output stream including the image coding information may be transmitted. Absent. However, in this case, the weighted prediction determination unit 401 needs to have a function of extracting image coding information from the output stream input from the image coding unit 403.

Next, an example of the process executed by the weighted prediction determination unit 401 will be described with reference to FIG. Note that FIG. 9 is a flowchart showing an example of the processing executed by the weighting prediction determination unit 401 in the image coding apparatus 400 according to the third embodiment.

As shown in FIG. 9, the weighting prediction determination unit 401 first calculates the coded motion vector feature amount from the image coding information of the picture encoded before the current coded target picture in the image coding unit 403. (S501). Next, it is determined whether or not the calculated coded motion vector feature amount is equal to or greater than a predetermined threshold value (S502).

When the weighted prediction determination unit 401 determines that the coded motion vector feature amount is equal to or greater than a predetermined threshold value (Yes in S502), the weighted prediction flag information is set to OFF, and the set weighted prediction flag information is displayed as an image. Output to the encoding unit 403 (S503). When it is determined that the coded motion vector feature amount is smaller than the predetermined threshold value (No in S502), the weighted prediction flag information is set to ON, and the set weighted prediction flag information is output to the image coding unit 403. (S504).

Here, as the image coding information of the picture encoded and output before the current coded target picture in the image coding unit 403, for example, a motion vector encoded by each macroblock can be considered. The present invention is not limited to this, and parameters other than the motion vector may be used.

Further, as the coded motion vector feature amount, for example, the average value of the motion vectors belonging to the same picture can be considered, but the present invention is not limited to this, and the average value of the absolute values of the motion vectors belonging to the same picture may be used. Alternatively, the variance of the motion vectors belonging to the same picture may be used.

As described above, the image coding apparatus 400 according to the third embodiment determines whether or not to perform weighting prediction based on the already encoded picture to be coded. Specifically, since the image coding information used when encoding the past coded target picture is used, it is not necessary to newly calculate the feature amount as in the second embodiment, and the processing amount is low. , It is possible to more accurately determine whether or not the picture to be encoded is a moving image. Thereby, the image quality and the coding efficiency can be improved.

As described above, in the image coding devices 100, 300, or 400 according to the first to third embodiments, the weighted prediction determination unit 101, 301, or 401 uses the imaging information and the input moving image, which are additional information of the input moving image data. Weighted prediction flag information based on either pixel information of image data or image coding information of a picture compressed and encoded before the current coded picture generated by the image coding unit 403. Is generated, and the generated weighted prediction flag information is output to the image coding unit 103 or 403.

Further, the weighting parameter determination unit 102 determines the weighting parameter and outputs it to the image coding unit 103 or 403. Further, in the image coding unit 103 or 403, the input moving image is based on the weighting prediction flag information output from the weighting prediction determination unit 101, 301 or 401 and the weighting parameter output from the weighting parameter determination unit 102. Compress and encode the data and output the stream.

Since the image coding devices 100, 300, or 400 according to the first to third embodiments are configured as described above, the prediction accuracy of motion compensation can be improved, so that the coding efficiency can be improved. ..

Although the image coding apparatus and the image coding method according to the present invention have been described above based on the embodiments, the present invention is not limited to these embodiments. As long as the gist of the present invention is not deviated, various modifications that can be conceived by those skilled in the art are applied to the embodiment, and a form constructed by combining components in different embodiments is also included in the scope of the present invention. ..

For example, as a method of determining whether or not to use weighted prediction for the compression coding process of input moving image data, three different methods are shown in the first to third embodiments, but by combining a plurality of these methods. , It may be determined whether or not the weighted prediction is used for the compression coding process of the input image data.

Further, in the first embodiment, as the imaging information, the information indicating whether or not the camera that captured the input moving image data is in the pan, tilt, or zoom state, and the aperture of the camera are changed. The information indicating whether or not the state is in the state is given as an example, but the present invention is not limited to this. For example, it may be information indicating whether or not the sensitivity of the camera is changing, or information indicating other physical movements of the camera.

Further, in the first to third embodiments, the case where SAD is used as the error between the target block and the reference block is given as an example in the motion vector detection, but the SAD calculation formula Σ | pixel of the target block-reference. The pixel of the block | may be modified and the pixel of the target block-the pixel of the reference block-offset value | may be used as an error. In this case, if the weighted prediction flag information is ON, the weighting parameter is used as the offset value, and if the weighted prediction flag information is OFF, the offset value is set to 0.

Further, in the first to third embodiments, the case where H.264 is used as the compression coding method is given as an example, but the present invention is not limited to this, and a compression coding method having the same function as the weighted prediction is used. You may use it.

When the input moving image data is interlaced moving image data, the weighting parameter determination unit 102 may determine the weighting parameter based on the parity relationship between the reference picture and the encoded picture. It should be noted that when both the reference picture and the picture to be encoded are top fields or bottom fields, it is same parity, and when one is a top field and the other is a bottom field, it is reverse parity.

Specifically, the weighting parameter determination unit 102 sets the offset value o of the reference picture, which has the inverse parity to the coded picture, to 0, and sets the offset value o of the reference picture, which has the same parity as the coded picture, to Determine o based on the Ecplicit mode.

FIG. 10 is a flowchart showing an example of processing executed by the weighting parameter determination unit 102 in the image coding apparatus 100 when the input moving image data is interlaced image data.

As shown in FIG. 10, the weighting prediction determination unit 101 determines whether or not the camera is in the pan, tilt, or zoom state when the image to be encoded is captured (S601). When it is determined that the camera is panning, tilting, or zooming (Yes in S601), the weighting prediction determination unit 101 sets the weighting prediction flag information to OFF, and sets the set weighting prediction flag information as an image code. Output to the conversion unit 103 (S602).

When it is determined that the camera is not in the pan, tilt, or zoom state (No in S601), the weighting prediction determination unit 101 sets the weighting prediction flag information to ON, and sets the set weighting prediction flag information as an image code. Output to the conversion unit 103 (S603).

The weighting parameter determination unit 102 determines whether the reference pictures have the same parity or the inverse parity for each of the one or more reference pictures referenced by the coded picture (S604). When the reference pictures have the same parity (same in S604), the weighting parameter determination unit 102 sets the offset value o based on the Explicit mode (S605). When the reference picture is inverse parity (inverse in S605), the weighting parameter determination unit 102 sets the offset value o to 0 (S606).

As described above, when the input moving image data is the interlaced image data, even if the weighted prediction flag information is set to ON based on the imaging information, the offset value By setting o to 0, weighted prediction can be prevented.

This is because selecting a picture with inverse parity as a reference picture generally means that the picture to be encoded is a moving image. That is, even if it is determined from the imaging information that there is no movement, it is possible to more accurately determine whether or not the picture to be encoded is a moving image by determining the parity relationship. it can. Thereby, the image quality and the coding efficiency can be improved.

Further, the present invention can be provided not only as an image coding device including each processing unit according to the first to third embodiments, but also image coding in which each processing unit included in the image coding device is set as each step. It is also possible to provide a method, an image coding integrated circuit including each processing unit included in the image coding apparatus, and an image coding program capable of realizing the image coding method.

In addition, such an image coding program is used for communication with recording media such as CD-ROM (Compact Disc-Read Only Memory), DVD-ROM (Digital Versatile Disc-ROM), BD (Blu-ray Disc), and the Internet. It can be distributed via a network.

Further, for example, the present invention may be realized as an image coding integrated circuit including a part or all of the components included in the image coding devices 100, 300 or 400 of the first to third embodiments. FIG. 11 is a block diagram showing an example of the configuration of the image coding device 500 in the modified examples of the first to third embodiments according to the present invention.

As shown in FIG. 11, the image coding unit 103 is mounted as the LSI 501 in the image coding device 500. The weighting prediction determination unit 101 and the weighting parameter determination unit 102 may also be implemented as the LSI 501.

Further, the image-coded integrated circuit can be realized as an LSI which is a typical integrated circuit. In this case, the LSI may be composed of one chip or a plurality of chips. For example, a functional block other than the memory may be configured by a 1-chip LSI. The LSI may be referred to as an IC, a system LSI, a super LSI or an ultra LSI depending on the degree of integration.

In addition, the method of making an integrated circuit is not limited to the LSI, but may be realized by a dedicated circuit or a general-purpose processor, an FPGA (Field Programmable Gate Array) that can be programmed after the LSI is manufactured, or an inside of the LSI. A reconfigurable processor that can reconfigure circuit cell connections and settings may be used.

Furthermore, if an integrated circuit technology that replaces the LSI appears due to advances in semiconductor technology or other technologies derived from it, it is naturally possible to integrate functional blocks using that technology. For example, adaptation of biotechnology is considered to be a possible possibility.

Further, when the integrated circuit is formed, only the unit that stores the data among the functional blocks may not be incorporated into the one-chip configuration, but may be a separate configuration.

Further, the present invention can also be realized as a camera 600 as shown in FIG. Note that FIG. 12 is a schematic view showing an example of the camera 600 in the modified examples of the first to third embodiments according to the present invention. Hereinafter, an example of the configuration of the camera 600 will be described with reference to FIG. FIG. 13 is a block diagram showing an example of the configuration of the camera 600 in the modified examples of the first to third embodiments according to the present invention.

As shown in FIG. 13, the camera 600 is a camera that encodes moving image data obtained by imaging a subject image and records it on a recording medium, and includes an image coding device 100 and an imaging unit 610. Since the image coding apparatus 100 is the same as that shown in the first embodiment, the same reference numerals are given, and the description thereof will be omitted below.

The image pickup unit 610 is a processing unit that captures a subject image, and includes an optical system 611, an image sensor 612, and an image pickup information generation unit 613.

The optical system 611 is an optical lens or the like that forms a subject image by forming an image of light from the subject on the image sensor 612. The optical system 611 can change the zoom and the aperture by controlling, for example, a control unit (not shown) included in the camera 600.

The image sensor 612 is an image sensor that captures a subject image, and outputs the moving image data obtained by the imaging to the weighting parameter determination unit 102 and the image coding unit 103 as input moving image data. The sensitivity of the image sensor 612 can be changed, for example, by controlling a control unit (not shown) or the like.

The image pickup information generation unit 613 generates image pickup information indicating the operation of the camera 600, and outputs the generated image pickup information to the weighting prediction determination unit 101. Specifically, the imaging information generation unit 613 determines the movement of the camera 600 itself from the acceleration sensor (not shown) provided in the camera 600, and determines whether or not the camera 600 is in a panned or tilted state. judge. Then, the determination result is output to the weighting prediction determination unit 101 as imaging information. The acceleration sensor may be, for example, a sensor used for the camera shake correction function of the camera 600.

Further, the image pickup information generation unit 613 determines whether the zoom, aperture, sensitivity, or the like has changed in the optical system 611 and the image sensor 612. Then, the determination result is included in the imaging information and output to the weighted prediction determination unit 101. The imaging information generation unit 613 may receive information indicating whether or not zooming is in progress from the control unit (not shown).

With the above configuration, the camera 600 can encode the moving image data obtained by imaging with excellent image quality and coding efficiency and record it on a recording medium.

As described above, the image coding device and the camera according to the present invention include the pixel information of the input moving image data, the imaging information of the camera that captured the input moving image data, and the picture encoded before the picture to be encoded. At least one of the image coding information of the above is used to determine whether or not to use weighted prediction for coding the input moving image data. Since motion compensation is performed based on the determination result, the coding efficiency of the input moving image data can be improved.

In addition, it is equipped with two types of motion compensation methods, motion compensation that does not perform weighted prediction and motion compensation that performs weighted prediction, and compared to the common-sense method of selecting the optimum method for each macroblock, the circuit scale and calculation The amount can be reduced. Therefore, it is possible to improve the image quality and the coding efficiency while reducing the calculation amount of the image coding apparatus, increasing the speed, and reducing the power consumption.

The image coding apparatus and image coding method according to the present invention have an effect that it is possible to realize video coding by a compression coding method such as H.264 with a smaller circuit scale and low power consumption, for example. , Video cameras, personal computers, HDD (Hard Disk Drive) recorders, DVD recorders, mobile phones with cameras, etc.

100, 300, 400, 500 image encoder 101, 301, 401 Weighted prediction judgment unit 102 Weighted parameter determination unit 103, 403 Image coding unit 201 Input image data memory 202 Reference image data memory 203 In-plane prediction section 204 Motion vector detector 205 Motion Compensation Department 206 Prediction mode judgment unit 207 Differential calculation unit 208 Orthogonal converter 209 Quantization section 210 Inverse quantization unit 211 Inverse orthogonal converter 212 Addition part 213 Entropy encoding unit 501 LSI 600 camera 610 Imaging unit 611 optics 612 Image sensor 613 Imaging information generator