Picture decoding method

ES 2 379 337 T3

DESCRIPTION

Image encoding procedure

Technical field

The present invention relates to an image coding method.

Previous technique

With the advancement of multimedia applications, in recent years the handling of information from all types of media such as, for example, audio, video and text, in an integrated way has become more and more frequent. By doing so, it is possible to handle the media in its entirety by digitizing all the media. However, since digitized images have a huge amount of data, information compression techniques are a basic necessity for their storage and transmission. Furthermore, in order to interoperate with compressed image data, standardization of compression techniques is also important. Standards for image compression techniques include H.261 and H.263 recommended by the ITU-T (International Telecommunication Union Telecommunication Standardization Sector), and MPEG (Moving Picture Experts Group) -1, MPEG-2 and MPEG -4 of ISO (International Organization for Standardization).

Figure 1 is a block diagram showing a structure of a conventional image coding apparatus 100. It should be noted that, in the present invention, the unit consisting of an image sheet is called an image. In an interlaced image signal, an image means a field or a frame, and in a progressive image signal, an image means a frame.

The image coding apparatus 100 includes a difference calculator 101, an image coding unit 102, a variable-length coding unit 103, an image decoding unit 104, an adder 105, an inter-pixel filter 106, a image memory 107, an inter-image prediction unit 108, and an inter-image prediction calculation unit 109. Difference calculator 101 subtracts a predictive image input to one less input terminal from an input image input to one more input terminal to obtain the differential image. Image encoding unit 102 encodes the input differential image. For example, the image coding unit 102 encodes the input data by performing frequency transformation of said data using DCT or something similar and quantizing the frequency data as a result of the transformation. The variable length encoding unit 103 performs the variable length encoding of the encoded differential image and the predictive parameters from the inter-image prediction computing unit 109, adds the relevant data such as, for example, a header describing the relevant information to the resulting encoded data, it formats it into an output encoded bit stream and sends it out of the image encoding apparatus 100. The image decoding unit 104 decodes the differential image encoded by the image encoding unit 102 by performing reverse coding processing by an image encoding unit 102 for the differential image. For example, after performing the inverse quantization of the encoded differential image, the image decoding unit 104 performs the inverse frequency transformation, such as the reverse DCT, to decode the difference between the input image and the predictive image. The adder 105 adds the decoded differential image and the predictive image to decode the input image. The inter-pixel filter 106 performs filtering, such as suppression of coding noise, on the high-frequency components of the decoded input image. In the image memory 107 the image data of at least one image among the images decoded by the adder 105 is stored as a reference image. The inter-image prediction unit 108 reads a predictive image in the reference image from the image memory 107 based on the prediction result obtained by the inter-image prediction computing unit 109. The inter-image prediction computing unit 109 gets a predictive parameter PredParam which is the amount of change in motion of the input image from the reference image.

More specifically, the image data Img is input to the image coding apparatus 100 from the outside. The image data Img is input to the plus input terminal of the difference calculator 101. The difference calculator 101 calculates the difference between the pixel values ​​of this image data Img and the predictive image data Pred input to the terminal of input minus to get the result as differential image data Res. This Predictive Predictive Image data is obtained as follows. First, the reference image data Ref, which is an image of an already encoded image and which is encoded and decoded once to be the image of a single image, is stored in the

ES 2 379 337 T3 image memory 107. Next, from this reference image data Ref, the data representing an image corresponding to each block of the image data Img input is extracted based on the predictive parameter PredParam . The data representing this image from each block is Pred predictive image data. The image coding apparatus 100 stores several coded image sheets as reference image data Ref for prediction in the image memory 107, and the inter-image prediction unit 108 generates predictive image data Pred from the data of Ref reference images stored in image memory 107. The inter-image prediction computing unit 109 obtains PredParam predictive parameter data that is used for prediction (for example, the motion vector information that is used in the MPEG image coding method and the like) from the data. input images Img and reference image data Ref. Note that the pixel values ​​of a predictive image must be 0 in the case of inter-image encoding.

Image coding unit 102 encodes Res differential image data and sends it as CodedRes encoded differential image data. Image decoding unit 104 decodes the CodedRes encoded differential image data and sends it as ReconRes decoded differential image data in order to use it as a reference image for inter-image prediction. The pixel values ​​indicated by this ReconRes decoded differential image data and the pixel values ​​indicated by the predictive image data Pred are added by adder 105 and output as Recon decoded image data. The inter-pixel filter 106 performs the filtering operation processing of the Recon decoded image data and stores it as FilteredImg filtered decoded image data in the image memory 107.

The inter-pixel filter 106 has the effect of reducing the coding noise of the Recon decoded image data and improving the prediction efficiency if the image is used as a reference image. As an example of the inter-pixel filter 106, there is an H.261 loop filter that is recommended by ITU-T. FilteredImg filtered decoded image data resulting from the inter-pixel filtering operation performed by the inter-pixel filter 106 is stored in the image memory 107 and is used as a reference image when encoding the following images. Variable length encoding unit 103 performs variable length encoding of CodedRes encoded differential image data and PredParam predictive parameter data and joins them into a single Bitstream encoded data to send the result out of image encoding apparatus 100 Figure 2 is a block diagram showing the structure of a conventional image decoding apparatus 200. The image decoding apparatus 200 includes a variable-length decoding unit 201, an image decoding unit 202, an adder 203, an inter-pixel filter 204, an image memory 205, and an inter-image prediction unit 206. The Bitstream encoded data is input to the image decoding apparatus 200 from the outside. The variable length decoding unit 201 performs variable length decoding of the input Bitstream encoded data and separates it into CodedRes encoded differential image data and PredParam predictive parameter data. The image decoding unit 202 decodes the CodedRes encoded differential image data and sends it as ReconRes decoded differential image data. It should be noted that an image referenced by an image sent as ReconRes decoded differential image data, that is, an image corresponding to the reference image data Ref in the image coding apparatus 100, it has already been decoded and stored in image memory 107. Therefore, the inter-image prediction unit 206 generates predictive image data Pred from the reference image data Ref based on the predictive parameter data PredParam. The adder 203 adds the predictive image data Pred and the decoded differential image data ReconRes to output the result as decoded image data Recon. The inter-pixel filter 204 performs the inter-pixel filtering operation of the Recon decoded image data and outputs the result as FilteredImg filtered decoded image data outside the image decoding apparatus 200. By outside the image decoding apparatus 200 is meant a display device such as a television. Also, FilteredImg filtered decoded image data is stored in image memory 205 and will be referred to as Ref reference image data in subsequent images.

However, if the case of the transmission of moving images through a mobile phone or similar device is taken into account, it is desirable to minimize the power consumption of the mobile device in general in order that the available hours per charge are larger, so it is undesirable for the apparatus to have a high processing capacity that requires a greater amount of energy to process the images. Therefore, when using a mobile device you can only use an operating unit with a capacity of

ES 2 379 337 T3 low image processing, sometimes it happens that only a necessary inter-pixel filter can be used with a reduced processing load. On the other hand, it is sometimes necessary to use a high-performance inter-pixel filter for some applications that need to transmit high-quality moving images with high coding efficiency, even if an operating unit with a higher processing load is required. . If an encoding procedure is capable of responding to such requirements flexibly, it is useful in the growing range of applications for an image decoding procedure.

Japanese patent JP06 / 038197A relates to the encoding and decoding of image signals. A filter driver is disclosed in this document. The filter driver calculates an absolute sum of image data and determines whether the sum is greater or less than a threshold value. According to the decision, a control signal is sent to switch a loop filter for a prediction image between different conditions.

Japanese patent abstract JP-03-136586 discloses an apparatus for effectively removing noise, which is generated in a prediction coding loop between frames by applying an on / off control to a band boundary filter so that a band boundary is applied to an interframe prediction decoding signal. The band limit applies when there are many picture elements with a larger difference in amplitude (distortion) between an input signal and a decoding signal.

US Patent No. 6,249,610-B1 discloses an image coding apparatus, in which an image reproducing section is arranged in the last part of the image coding apparatus. In addition, noise is detected in a reproduced image and an appropriate filter is selected to correct the reproduced image. Image coding and related filtering are implemented in the form of a computer procedure using recording media.

European patent application 0-603-878-A2 discloses an interframe coding system that efficiently and adaptively removes higher frequency components contained in an image signal with an adaptive filter provided in a coding loop.

The present invention has been conceived in view of these conventional problems, and aims to provide an image encoding apparatus that uses an inter-pixel filter that selectively relies on various situations to generate encoded data, and an image decoding apparatus that decodes encoded data.

Description of the invention

A method for encoding image data in accordance with the present invention is set forth in claim 1.

According to the present invention, image coding at a low bit rate (high compression rate) is obtained to achieve high image quality, and particularly, the significant effect of image quality improvement can be achieved. in image encoding processing using software with limited hardware resources and therefore it can be said that the practical value of the present invention is extremely high in these days when the use of computers and information communication technology has become widespread.

Brief description of the figures

Figure 1 is a block diagram showing the structure of a conventional image coding apparatus.

Figure 2 is a block diagram showing the structure of a conventional image decoding apparatus.

Figure 3 is a block diagram showing the structure of an image coding apparatus according to a first embodiment of the present invention.

Figure 4 is a block diagram showing the structure of an image coding apparatus that switches an inter-pixel filter for each chunk of an input image.

ES 2 379 337 T3

Figure 5A is a diagram showing the flow structure of Bitstream encoded data that is sent from the image encoding apparatus of the present invention. Figure 5B is a diagram showing the flow structure of Bitstream encoded data that is sent when the image encoding apparatus of the present invention switches an inter-pixel filter in the unit of a chunk.

Figure 6 is a block diagram showing the structure of an image decoding apparatus that decodes Bitstream1 encoded data generated by the image encoding apparatus according to the first embodiment.

FIG. 7 is a block diagram showing the structure of an image decoding apparatus that uses a built-in inter-pixel filter instead of a specified inter-pixel filter if it is not included in the image decoding apparatus.

Figure 8 is a block diagram showing the structure of an image coding apparatus according to a second embodiment of the present invention.

Figure 9 is a diagram showing the details of the operation by means of an unblocking filter which is an example of the inter-pixel filter, as shown in Figure 8. Figure 9A is a diagram showing the pixel values in the vicinity of the boundary between ant blocks is of the filtering. Figure 9B is a diagram showing pixel values ​​in the vicinity of the inter-block boundary after filtering.

Figure 10 is a flow chart showing the flow of filtering processing by means of an inter-pixel filter.

Fig. 11 is a block diagram showing the structure of an image decoding apparatus that can select whether or not inter-pixel filter processing is to be performed.

Figure 12 is a block diagram showing the structure of an image decoding apparatus that is equipped with an inter-pixel filter that can be further selected at an output stage.

Fig. 13 is a block diagram showing the structure of an image decoding apparatus that can select a filter between pixels depending on the image type of each image.

Figure 14 is a block diagram showing the functional structure of an image coding apparatus according to a third embodiment of the present invention.

Figure 15 is a block diagram showing the detailed functional structure of a priority determination unit, as shown in Figure 14.

Figure 16 is a block diagram showing the detailed functional structure of a filter processing control unit, as shown in Figure 14.

Figure 17 is a diagram showing the reference relationships between images stored in an image memory, as shown in Figure 14.

Figure 18 is a flow chart showing switch activation processing performed by a switch change processing unit, as shown in Figure 16.

Figure 19 is a block diagram showing the functional structure of an image decoding apparatus according to a fourth embodiment of the present invention.

Fig. 20 is an explanatory diagram in the case of implementation by a computer system using a floppy disk storing the image encoding procedure or the image decoding procedure from the aforementioned first embodiment to the fourth embodiment. Figure 20A shows an example of a physical format of a floppy disk that is a main body of the recording medium. Figure 20B shows the front view of the appearance of the floppy disk, the cross-sectional view of the floppy disk and the floppy disk, and Figure 20C shows the structure for recording and reproducing the above program on the floppy disk FD.

Figure 21 is a block diagram showing the overall configuration of a power delivery system.

ES 2 379 337 T3 content that provides content distribution services.

Figure 22 is a diagram showing an example of the appearance of a mobile phone.

Figure 23 is a block diagram showing the structure of the mobile phone.

Figure 24 is a diagram explaining devices that perform encoding processing or decoding processing, as shown in the aforementioned embodiments, and a system using these devices.

Best mode of carrying out the invention

[Embodiment 1]

Specific embodiments of the present invention are explained below with reference to the figures. FIG. 3 is a block diagram showing the structure of an image coding apparatus 300 according to the first embodiment of the present invention. In this figure, the same reference numerals are assigned to the same constituent elements and data as those of the conventional image coding apparatus 100 shown in Figure 1 and explanations thereon will be omitted as they have already been given. . Similarly, in the following images, the same reference numbers are assigned to the constituent elements and data that have already been explained, and explanations thereon are omitted. The image coding apparatus 300 consists of the difference calculator 101, the image coding unit 102, a variable-length coding unit 305, the image decoding unit 104, the adder 105, the image memory 107, the inter-image prediction unit 108, inter-image prediction calculation unit 109, a switch 301, a switch 302, an inter-pixel filter A 303, and an inter-pixel filter B 304. The switch 301 and the switch 302 are switches that are respectively connected to a terminal 1 or a terminal 2 selectively depending on a value of the filter type information FilterType1. The switch 301 is located between the output terminal of the adder 105 and the input terminals of the inter-pixel filter A 303 and the inter-pixel filter B 304. And the switch 302 is located between the input terminal of the image memory 107 and the output terminals of the inter-pixel filter A 303 and the inter-pixel filter B 304. The inter-pixel filter A 303 and the inter-pixel filter B 304 are Unblocking filters to smooth out high-frequency noise in the vicinity of the inter-block boundary to remove block distortion, for example, and their smoothing levels are different. Also, their operation processing loads for smoothing are different according to their smoothing levels. Note that the constituent elements shown in this figure, such as switch 301 and switch 302, can be implemented as hardware or software. The same applies to other figures.

Variable-length encoding unit 305 performs variable-length encoding such as Huffman encoding of the entered filter type information FilterType1, CodedRes encoded differential image data, and PredParam predictive parameter data respectively, and the joins into a single encoded data Bitstream1 to send said encoded data out of the image encoding apparatus 300.

The operation of the image coding apparatus 300 structured in the above manner, particularly a part of a new structure, will be explained in more detail by comparing it with the conventional image coding apparatus 100. First, the filter type information FilterType1 is input to image coding apparatus 300 from outside. Here, by input from outside is meant, for example, input by a user via an interface such as a keyboard from outside of a still image or data encoding apparatus for an apparatus, and a value that determines the apparatus depending on a bit rate (compression rate) or an image size. FilterType1 filter type information is input to switch 301 and switch 302. Switch 301 and switch 302 switch the connection to terminal 1 or terminal 2, depending on the value of this filter type information FilterType1. For example, if the value of the filter type information FilterType1 is 1, both switch 301 and switch 302 are connected to terminal 1. In this case, filtering by inter-pixel filter A 303 is applied to the data of Recon decoded images sent from adder 105. If the value of the filter type information FilTerType1 is 2, the switch 301 and the switch 302 are switched to the side of the terminal 2, and the filtering by means of the inter-pixel filter B 304 is applied to the sent Recon decoded image data from adder 105. FilteredImg1 filtered decoded image data that is filtered by means of the inter-pixel filter A 303 or the inter-pixel filter B 304 is stored in the image memory 107 and used as a reference image for the predictive encoding of the following images. Also, the filter type information FilterType1 that specifies

ES 2 379 337 T3 an inter-pixel filter is input to the variable length encoding unit 305 together with the CodedRes encoded differential image data and PredParam predictive parameter data of the same image to perform variable length encoding of these data respectively. The result of the variable-length encoding of the FilterType1 filter type information is stored in the Bitstream 1 encoded data by associating it with the results of the variable-length encoding of this CodedRes encoded differential image data and PredParam predictive parameter data, and are recorded on a recording medium or transmitted to an image decoding apparatus. In other words, the type of inter-pixel filter applied to the Recon decoded image data of each image is notified to the image decoding apparatus by the filter type information FilterType1 stored in the Bitstream1 encoded data. Therefore, since the image decoding apparatus that decodes the Bitstream1 decoded data can specify the inter-pixel filter that is applied to the Recon decoded image data of each image in the image encoding apparatus 300, it can use the same filter for the decoded image of each image. Note that although the value of the FilterType1 filter type information is 1 or 2 in the previous explanation, it is a value defined for explanatory reasons only, and other values ​​can be used if these values ​​can discriminate a plurality of filters.

As described above, the image coding apparatus 300, including a plurality of inter-pixel filters with different predictive performance and processing load than inter-pixel filters, can use them by switching them based on the filter type information FilterType1 introduced from the outside. Here are the benefits of using inter-pixel filters with different predictive performance and processing load by switching them. First of all, for explanation, we are going to assume that the inter-pixel filter A 303 has a lower processing load than the inter-pixel filter B 304, while the inter-pixel filter B 304 has a noise suppression effect greater than the inter-pixel filter A 303 to improve predictive coding efficiency. As the image decoding apparatus that decodes the encoded data sent from the image encoding apparatus 300 of the present invention, it is assumed that there are two types of image decoding apparatus: an image decoding apparatus A that includes only the filter inter-pixel A 303, and an image decoding apparatus B, including both inter-pixel filter A 303 and inter-pixel filter B 304. The first image decoding apparatus A, which requires less processing load, is suitable for an apparatus with less processing power. The second image decoding apparatus B is suitable for an apparatus with a higher processing capacity. The second image decoding apparatus B can also decode the encoded data obtained using the inter-pixel filter A 303 or the inter-pixel filter B 304, and is compatible with the image decoding apparatus A. In such a case, the encoding apparatus Image decoder 300 can function as an image encoding apparatus that supports these two types of image decoding apparatus. In other words, by structuring the image coding apparatus 300 to select the inter-pixel filter with the appropriate predictive performance and processing load depending on the processing power of the target image decoding apparatus, The encoding method using the same inter-pixel filter as applied in the image encoding apparatus 300 can be applied to a wide variety of devices (for decoding the Bitstream1 encoded data).

Furthermore, the inter-pixel filter can be switched as a function of the processing capacity of the image coding apparatus 300, not only for generating the encoded data depending on the processing capacity of the image decoding apparatus, but also for other uses. For example, if the size of an image and the speed of an image to be encoded are large, the processing load required for all encoding processing would be large. Therefore, the inter-pixel filter B 304 with a required high throughput is used when the image size and speed of the image to be encoded are fixed or lower values, and the inter-pixel filter A 303 with a lower processing power required is used when the image size and the speed of the image to be encoded are fixed values ​​or higher, in order to remove the processing load required for all encoding processing. Or, when the image coding is performed by a time-sharing system, in which an apparatus executes a plurality of procedures by time-sharing, there is a possibility that the processing load assignable to the image coding changes dynamically. under the influence of other procedures. Therefore, when the processing load assignable to image encoding is a fixed value or greater, the inter-pixel filter B 304 with high throughput is used, while when the processing load assignable to image encoding is a fixed value or less, the inter-pixel filter A 303 can be used with a lower processing load than the inter-pixel filter B 304.

As for the timing of the inter-pixel filter switching, if a plurality of inter-pixel filters are provided suitable for images of specific properties, they can be switched from image to image.

ES 2 379 337 T3 depending on the property of each image. For example, if edge information is important such as characters, an inter-pixel filter is used that maintains the edges well. They can be switched by automatic determination using image processing techniques such as edge detection and character detection, or they can be explicitly selected by a user between an inter-pixel filter suitable for nature images, an inter-pixel filter suitable for characters, or an inter-pixel filter suitable for edges. If a plurality of filters can be switched between pixels in this way, a suitable filter can be selected for the properties of the images, and thus the predictive efficiency can be further improved. In other words, it is also effective to switch filters to improve image quality, not switch them depending on the processing load. Therefore, in the present embodiment, the switching of filters as a function of the processing load has been explained, but they can also be switched to improve the image quality.

Also, the inter-pixel filter switching unit is not limited to going from image to image, but inter-pixel filters can be switched in the unit of an image area smaller than an image, such as a slice, a macro block and an MPEG block, or in the unit of an area that includes at least one pixel, because the properties of the images may vary in a part of the image.

Figure 4 is a block diagram showing the structure of an image coding apparatus 400 that switches filters between pixels for each chunk of an input image. The image coding apparatus 400 is an image coding apparatus that filters a decoded image by switching the filters between pixels in the MPEG chunk unit, and consists of the difference calculator 101, the image coding unit 102, the image decoding unit 104, the adder 105, the image memory 107, the inter-image prediction unit 108, the inter-image prediction calculation unit 109, a switch 403, a switch 404, the inter-pixel filter A 303, the inter-pixel filter B 304, the variable length encoding unit 305, a filter switch position determining unit 401, and a switch 402. The filter switch position determining unit 401 detects chunks of an input image in the image data Img input from the outside and sends them to the SetFType filter switch control data of the switch 402 to send a pulse, for For example, at each switch of the detected chunks. The switch 402 is a switch that is disconnected between terminals while the control data from the SetFType filter switch is not being sent, and carries the FilterType1 filter type information from outside to be routed to switch 403 and switch 404 for a short time. instant while the SetFType filter switch control data is being sent. The switch 403 and the switch 404 respectively connect their terminals 1 or terminals 2 depending on the value of the filter type information FilterType1 that is instantaneously input on each switching of the data chunks of the input image Img and maintain the state of Connection. In other words, while the switch 402 is off, the inter-pixel filters are not switched. As a result, a new inter-pixel filter is selected according to the filter type information FilterType1 at each switching of the chunks of the input image data Img, and therefore, the inter-pixel filters can be prevented from being used. commute in the middle of the slice.

As explained above, according to the image encoding apparatus of the present invention, it is possible to create encoded data using an inter-pixel filter that depends on the processing capacity of an image decoding apparatus that reproduces the encoded data sent. from the image coding apparatus of the present invention. Furthermore, the inter-pixel filter can be selected according to the processing capacity of the image coding apparatus.

It should be noted that although the image coding apparatus of the present embodiment has two inter-pixel filters, it may have three or more inter-pixel filters. The three or more inter-pixel filters are selected and used in the same way as in the present embodiment, and the filter type information indicating the type of inter-pixel filter used can be included in the encoded data.

Also, filters can be switched to improve image quality, not switched depending on processing load.

It should be noted that in the image coding apparatus 400, the control data of the SetFType filter switch is a pulse waveform indicating the value 1 at each switch of the detected chunks and 0 during the other period that It is not the time of the switching, but the present invention is not limited to such a case, and it can be a rectangular wave that is reversed at each switching of the chunks, for example, or any other waveform. In addition, switch 402 is a switch that is disconnected between the terminals while the control data of the SetFType filter switch is in the same phase, but the present

The invention is not limited to such a case, and the filter switch position determination unit 401 can send the control data of the SetFType filter switch indicating the value to turn off the switch in positions other than the position of the filter switch. Also, in the image coding apparatus 400, the inter-pixel filters are switched piece by piece of the image data Img, but the inter-pixel filters can be switched picture-to-picture, or they can be switched in the unit of a block, a macro block or a fixed number of pixels.

Figure 5A is a diagram showing the structure of the stream of Bitstream encoded data exiting the image encoding apparatus of the present invention. Figure 5B is a diagram showing the flow structure of Bitstream encoded data output when the image encoding apparatus of the present invention switches filters between pixels in the unit of a chunk. The function of the encoded data of the present invention is that the Bitstream encoded data includes information of the FilterType filter type that specifies a filter among a plurality of filters between pixels. Thanks to this stream structure, the image decoding apparatus of the present invention that decodes Bitstream encoded data can use the same type of inter-pixel filter that is used for encoding, checking the information of the FilterType filter type included in the data. Bitstream encoded.

In the Bitstream encoded data shown in Figure 5A, the value of the FilterType filter type information indicating the inter-pixel filter used to filter each image is described in (a diagonally shaded area, for example, of) the 901 header that is attached to all Bitstream encoded data. This Bitstream encoded data corresponds to the Bitstream1 encoded data that is sent from the image encoding apparatus 300 shown in Figure 3. Furthermore, in the Bitstream encoded data shown in Figure 5B, the value of the FilterType filter type information indicating the inter-pixel filter used to filter that chunk is described in (a diagonally shaded area, for example, of ) the header of the chunk 902 that is included in each chunk. This Bitstream encoded data corresponds to the Bitstream1 encoded data that is sent from the image encoding apparatus 400 shown in Figure 4. As already described, by storing the FilterType filter type information in header 901 at the beginning of the Bitstream encoded data or in header 902 of the chunk at the beginning of each chunk, which is not only the basic recording and transmission unit data, but also the unit of correction and modification of errors, the image decoding apparatus may specify the type of filtering of a chunk before decoding the chunk upon receiving Bitstream encoded data from the header 901 or chunk.

It should be noted that here it has already been explained that in the unit of a slice a filter is switched between pixels, but not only can it be switched in the unit of a slice, but also in the unit of an image area smaller than a chunk (the unit can be an area that includes one or more pixels, such as a macro block or an MPEG block). In addition, you can switch image to image, which is the unit of an image area larger than a slice. In this case, the value of the filter type information FilterType1 corresponding to each image can be described not only in (a diagonally shaded area, for example, of) the header 901, as shown in Figure 5A, but also in an image header provided for each encoded image data, for example. Furthermore, when a filtering procedure is toggled in the unit of a macro block or a block, the value of the filter type information FilterType1 of each macro block or block can be chunked together and described in the chunk header.

Also, to transmit the encoded data in the form of a packet or the like, the header and the data can be transmitted separately. In this case, the header and data are not included in a bit stream, as shown in Figure 5. However, in the case of packet transmission, the header paired with the data is transmitted in another packet, even if the packets are transmitted slightly out of sequence, and therefore the concept is the same as in the case of bit stream explained in Figure 5, even though they are not transmitted as a bit stream.

As explained above, by defining the value of the filter type information FilterType1 input into the image coding apparatus of the present invention to select an inter-pixel filter depending on the processing capacity of the image decoding apparatus, It is possible to create coded data as a function of the processing capacity of the image decoding apparatus which reproduces the coded data sent by the image coding apparatus of the present invention. Furthermore, an inter-pixel filter can be selected depending on the processing capacity of the image coding apparatus.

Also, a filter can be switched to improve image quality, not depending on the load of

ES 2 379 337 T3 processing.

FIG. 6 is a block diagram showing the structure of an image decoding apparatus 1000 that decodes Bitstream1 encoded data generated by the image encoding apparatus according to the first embodiment. The image decoding apparatus 1000 is a decoding apparatus that switches a filter between pixels from image to image or piece by piece according to the information of the filter type FilterType1 which is described in the header of the input encoded data Bitstream1 and decodes the data encoded in the Bitstream1 encoded data, and consists of the variable length decoding unit 201, the image decoding unit 202, the adder 203, the image memory 205, the inter-image prediction unit 206, a switch 1001, a switch 1002, an inter-pixel filter A 1003, and an inter-pixel filter B 1004.

Bitstream1 encoded data is input to image decoding apparatus 1000 from outside. This Bitstream1 encoded data, for example, is the data encoded by the image encoding apparatus 300 or the image encoding apparatus 400 of the first embodiment. The variable length decoding unit 201 performs variable length decoding of the Bitstream1 input encoded data, splits it into CodedRes encoded differential image data, PredParam predictive parameter data, and FilterType1 filter type information, and sends the image data coded differential CodedRes to image decoding unit 202, predictive parameter data PredParam to inter-image prediction unit 206, and filter type information FilterType1 to switch 1001 and switch 1002, respectively. When the value 1 is entered as the filter type information FilterType1, the switch 1001 and the switch 1002 change the connection to the terminal 1 side and apply filtering by the inter-pixel filter A 1003 to the Recon decoded image data. When the value 2 is entered as FilterType 1 filter type information, the switch 1001 and the switch 1002 change the connection to the terminal 2 side and apply the filtering by means of the inter-pixel filter B 1004 to the Recon decoded image data. Regardless of whether an inter-pixel filtering operation is performed by means of the inter-pixel filter, the filtered decoded image data FilteredImg1 is stored in the image memory 205 and sent out of the image decoding apparatus 1000, for example, to a display device or something similar.

As explained above, according to the image decoding apparatus 1000 of the present invention, it is possible to decode the Bitstream1 encoded data, including the FilterType1 filter type information that specifies the type of inter-pixel filter in the header. .

It should be noted that although the image decoding apparatus of the present embodiment includes two inter-pixel filters, it may include three or more inter-pixel filters. In this case, in the same way as in the present embodiment, three or more inter-pixel filters can be selected and used according to the filter type information in the Bitstream encoded data.

It should be noted that, as shown in the first embodiment, when the filter type is switched in the unit of an image or in the unit of an image area smaller than one image, the filter between pixels is switched at the point in time when the filter type is changed.

Figure 7 is a block diagram showing the structure of an image decoding apparatus 1100 that uses a built-in inter-pixel filter instead of a specified inter-pixel filter if such a filter is not built-in. The image decoding apparatus 1100 is characterized in that it uses any of the inter-pixel filters built into it if the inter-pixel filter selected by the filter type information included in the encoded data is not built into it. This image decoding apparatus 1100 includes the variable length decoding unit 201, the image decoding unit 202, the adder 203, the image memory 205, the inter-image prediction unit 206, the switch 1001, the switch 1002 , the inter-pixel filter A 1003, the inter-pixel filter B 1004, and an information conversion unit of the filter type 1101.

For example, it is assumed that the image decoding apparatus 1100 includes only two types of the inter-pixel filter A 1003 and the inter-pixel filter B 1004 indicated by the value 1 and the value2 of the filter type information FilterType1. Bitstream3 encoded data is input to image decoding apparatus 1100 from outside. Variable-length decoding unit 201 performs variable-length decoding of the Bitstream3 input encoded data, splits it into CodedRes encoded differential image data, PredParam predictive parameter data, and FilterType3 filter type information, and sends the data

ES 2 379 337 T3 of coded differential images CodedRes to the image decoding unit 202, the predictive parameter data PredParam to the inter-image prediction unit 206 and the filter type information FilterType3 to the information conversion unit of the type filter 1101, respectively. When the value of the filter type information FilterType3 is the value 3 indicating an inter-pixel filter not incorporated in the image decoding apparatus 1100, the filter type information conversion unit 1101 converts the value 3 of the filter type information FilterType3 into the value 2 that indicates the inter-pixel filter whose smoothing level is closest to that of the specified inter-pixel filter from among the inter-pixel filters built into image decoding apparatus 1100, for example, and sends it as filter type information FilterType4.

By performing this conversion processing, decoding processing to reconstruct an image similar to the original decoded image is possible, although the image quality is degraded to some extent because a inter-pixel filter different from the one used for encoding, so there is sufficient availability as a simple image decoding function. When the value 1 is input as the filter type information FilterType4, the switch 1001 and the switch 1002 change the connection to the terminal 1 side, and apply filtering by the inter-pixel filter A 1003 to the Recon decoded image data. When the value 2 is input as the filter type information FilterType4, the switch 1001 and the switch 1002 change the connection to the terminal 2 side and apply the filtering by means of the inter-pixel filter B 1004 to the Recon decoded image data. FilteredImg3 filtered and decoded image data that is the result of inter-pixel filter processing is sent to a display apparatus or the like outside of image decoding apparatus 1100.

As explained above, according to the image decoding apparatus 1100, even if the filter type information FilterType3 specifying an inter-pixel filter that is not incorporated in the image decoding apparatus 1100 is included in the Bitstream3 input encoded data, can be decoded using a built-in inter-pixel filter instead. Therefore, Bitstream encoded data can be decoded without substantially degrading the image quality.

It should be noted that if the image decoding apparatus 1100 has an inter-pixel filter (including the case where there is no inter-pixel filtering operation), the decoding can be performed using that inter-pixel filter forcibly. .

It should be noted that the image decoding apparatus of the present embodiment is equipped with two inter-pixel filters (the case of no inter-pixel filtering operation is counted as one filter), but the equipped image decoding apparatus with three or more inter-pixel filters you can perform the same processing. In other words, the storage processing of the Recon decoded image data in the image memory 205 as is, without performing the inter-pixel filtering operation, can be included as one of the inter-pixel filter processing.

It should be noted that, as shown in the first embodiment, if the filter type is switched in the unit of an image or in the unit of an image area smaller than an image, a filter is switched between pixels. at the point of change of the filter type.

The operation of the filter between pixels 303, 304, 1003 and 1004 will be explained in more detail using Figure 9 and Figure 10. Figure 9 is a diagram showing the details of the operation of an unblocking filter which is an example of a filter between pixels. Figure 9A is a diagram showing pixel values ​​in the vicinity of the inter-block boundary prior to filtering. Figure 9B is a diagram showing pixel values ​​in the vicinity of the inter-block boundary after filtering. Figure 10 is a flow chart showing a filtering processing flow by means of an inter-pixel filter. Figure 9A shows pixel values ​​of respective pixels 601-608 on a horizontal scan line. Pixels 601-604 are pixels from a 610 block, but pixels 605-608 are pixels from a 611 block adjacent to a 610 block. The pixel values ​​of pixels 601-604 are respectively p3, p2, p1 and p0, and the pixel values ​​of pixels 605-608 are respectively q0, q1, q2 and q3. In an image encoding apparatus, processing such as inter-image prediction, image encoding, variable-length encoding, and image decoding are generally performed in a one-block (or macro-block) unit. Therefore, the encoding noise is apt to appear at the higher frequencies between pixels, such as pixel 604 and pixel 605, across the boundary between adjacent blocks (or macro blocks), such as block 610 and block 611. For example, there is a tendency that the difference between the p0 value of the pixel 604 and the pixel value q0 of the pixel 605 is apt to increase due to the influence of the coding noise. Therefore, an inter-pixel filter is a filter that can be determined using a plurality of parameters, such as a

ES 2 379 337 T3 filter corresponding to parameters α and β to determine the filter, and the pixel values ​​of a group of pixels through the block boundary are filtered by means of this filter between pixels.

As shown in Figure 10, an inter-pixel filter first calculates the absolute value of the difference in pixel values ​​(p0-q0) between pixel 604 and pixel 605 across the boundary, and determines whether the absolute value calculated is less than or not the value of the parameter α (P701). As a result of the decision, if the absolute value of the difference (p0-q0) between the pixel values ​​is the value of the parameter α or greater, the inter-pixel filter does not perform unblocking filtering processing for the rendered pixel value. by Recon decoded picture data (P704). On the other hand, as a result of the decision of step P701, if the absolute value of the difference (p0-q0) between the adjacent pixel values ​​across the block boundary is less than the value of parameter a, the inter-pixel filter calculates also the absolute value of the difference (p1-q0) between the values ​​of pixel 604 and pixel 603, and determines whether or not the calculated absolute value is less than the value of parameter β (P702). Here, pixel 604 and pixel 603 are adjacent pixels of a block 610. As a result of the decision, if the absolute value of the difference (p1-p0) between the pixel values ​​is the value of the parameter β or greater, the Inter-pixel filter 503 does not perform unblocking filtering processing for the pixel value represented by the Recon decoded image data (P704). Also, as a result of the decision, if the absolute value of the difference (p1-q0) between the pixel values ​​is less than the value of the parameter β, it also calculates the absolute value of the difference (q1-q0) between the pixel 605 and pixel 606, and determines whether or not the calculated absolute value is less than the value of parameter β (P703). Here, pixel 605 and pixel 606 are adjacent pixels of a block 611. As a result of the decision, if the absolute value of the difference (q1-q0) of the pixel values ​​is the value of the parameter β or greater, the inter-pixel filter does not perform unblocking filtering processing for the rendered pixel value. by Recon decoded picture data (P704). On the other hand, if the absolute value of the difference (q1-q0) between the pixel values ​​is less than the value of the parameter β, the inter-pixel filter 503 performs the filtering of the Recon decoded image data to remove noise from encoding and processing ends. The inter-pixel filter repeats the above-mentioned processing for each pixel array in the direction of the horizontal scan line and the direction of the vertical scan line respectively through the boundaries of the block. In this way, by performing unblocking filter processing when any one of the value differences of three adjacent pixel sets is less than a set value, block distortion is eliminated.

It should be noted that in the unblocking filter processing of step P704 above, the smoothing filtering (filtering to remove the high frequency components) is performed for the pixels that are in the vicinity of the boundary. For example, a new pixel value P0 of pixel 604 can be generated by performing smoothing, using a low-pass filter to suppress the high-frequency components of the pixel value p0 of pixel 604, the pixel value q0 of pixel 605, the pixel value p1 of pixel 603 and the pixel value q1 of pixel 606.

[Embodiment 2]

FIG. 8 is a block diagram showing the structure of an image coding apparatus 500 according to the second embodiment. The image coding apparatus 500 is different from the image coding apparatus 300 in that, since it has inter-pixel processing, the former can select whether or not to store the Recon decoded image data in the image memory 107 as is. reference image data Ref. The image coding apparatus 500 consists of the difference calculator 101, the image coding unit 102, the image decoding unit 104, the adder 105, the image memory 107, the inter-image prediction unit 108, the image prediction computing device 109, a switch 501, a switch 502, an inter-pixel filter 503, a lookup table memory unit 504, and a variable length encoding unit 505.

When the value of the filter type information FilterType2 is 0, both switch 501 and switch 502 change the connection to the side of terminal 0 to store the Recon decoded image data that is sent from adder 105 as-is. image memory 107. When the value of the filter type information FilterType2 is 1, both the switch 501 and the switch 502 change the connection to the side of the terminal 1 to carry the Recon decoded image data that is sent from the adder 105 to the inter-pixel filter 503 . The inter-pixel filter 503 is a filter that is used to filter pixel values ​​and, for example, an unblocking filter to suppress the coding noise in the higher frequency components at the boundary of the blocks. FilteredImg2 filtered decoded image data, which comes from the inter-pixel filtering operation performed by inter-pixel filter 503, is stored in image memory 107. Variable-length encoding unit 505 performs variable-length encoding of this filter type information

ES 2 379 337 T3

FilterType2, CodedRes differential encoded image data, and PredParam predictive parameter data, joins them into Bitstream2 encoded data, as shown in Figure 5A, and sends them out of image encoding apparatus 500.

It should be noted that the image coding apparatus 500 has an inter-pixel filter, but may have two or more inter-pixel filters. You can select any of the two or more inter-pixel filters or no inter-pixel filter operations to use, and include the filter type information that indicates the type of inter-pixel filter used (including no inter-pixel filter operations) in the data. encoded. Furthermore, the image encoding apparatus 500 may bypass the lookup table memory unit 504 and instead have the function of the lookup table memory unit 504 in the inter-pixel filter 503.

As explained above, in the image encoding apparatus 500 of the present invention, it is possible to create encoded data using an inter-pixel filter that depends on the processing capacity of the image decoding apparatus to reproduce the Bitstream2 encoded data. which are sent from the image coding apparatus 500. In addition, an inter-pixel filter can be selected depending on the processing capacity of the image coding apparatus 500. Also, the filter type can be switched in the unit of an image or in the unit of an image area smaller than an image.

FIG. 11 is a block diagram showing the structure of an image decoding apparatus 1200 according to the second embodiment which can select whether or not inter-pixel filter processing is to be performed. The image decoding apparatus 1200 differs from the image decoding apparatus 1000 of Figure 6 in that the former does not perform inter-pixel operation as inter-pixel processing, but can choose to store the Recon decoded image data as is. image memory 205 as reference image data Ref. The image decoding apparatus 1200 consists of the variable-length image encoding unit 201, the image decoding unit 202, the adder 203, the image memory 205, the inter-image prediction unit 206, a switch 1201, a switch 1202 and an inter-pixel filter 1203.

The Bitstream2 encoded data in the header of which is included the information of the filter type FilterType2 that indicates the inter-pixel filter applied to the encoding, for example, the encoded data of Figure 9A encoding the image encoding apparatus 500 of Figure 8 , are input to the image decoding apparatus 1200. The filter type information FilterType2 includes the value indicating no inter-pixel filtering operation as a filter type. Variable-length decoding unit 201 performs variable-length decoding of input encoded data Bitstream2 and divides it into encoded differential image data CodedRes, predictive parameter data PredParam, and filter type information FilterType2. The divided CodedRes encoded differential image data, PredParam predictive parameter data, and FilterType2 filter type information are respectively sent to image decoding unit 202, inter-image prediction unit 206 and switch 1201 and switch 1202 .

When the value 0 is input as filter type information FilterType2, switch 1201 and switch 1202 change the connection to the side of terminal 0 and the Recon decoded picture data sent from adder 203 is stored as is in memory. of images 205. When the value 1 is entered as the FilterType2 filter type information, the switch 1201 and the switch 1202 change the connection at the terminal 1 side to apply filter filtering between pixels 1203 to the Recon decoded image data.

Also, if it is determined from the filter type information that an inter-pixel filter is not used for a decoded image to be a reference image, the decoded image is not filtered by the inter-pixel filter to store that image. in the image memory as a reference image, but the inter-pixel filter can only be used to send it to the outside of an image decoding apparatus. FIG. 12 is a block diagram showing the structure of an image decoding apparatus 1300 including an image output unit equipped with selectable inter-pixel filters. As explained above, the image decoding apparatus 1300 is an image decoding apparatus which, when the information of the filter type FilterType2 indicates that the decoded image coming out of the adder 203 is not filtered by means of a filter between pixels, it does not filter the decoded image stored in the image memory, but it performs the filtering of the decoded image that is sent to the outside using an inter-pixel filter that is on the output side, and consists of the variable-length image decoding unit 201, the image decoding unit 202, the adder 203, image memory 205, inter-image prediction unit 206, switch 1201, switch 1202, filter between

ES 2 379 337 T3 pixels 1203, a switch 1301, a switch 1302 and an inter-pixel filter 1303.

When the value of the FilterType2 filter type information is 1, the switch 1201, the switch 1202, the switch 1301 and the switch 1302 change the connection to the terminal 1 side. In this case, the switch 1201 and the switch 1202 connect the output of adder 203, inter-pixel filter 1203, and image memory 205, and switch 1302 breaks connection with the output of switch 1202 and inter-pixel filter 1303 to short circuit to switch 1301. Therefore, the inter-pixel filter 1203 performs the filtering operation for the Recon decoded image data and sends the FilterImg3 filtered decoded image data. The filtered decoded image data FilterImg3 is sent as an output image OutImg to a display apparatus or the like outside of the image decoding apparatus as is, that is, without being further filtered through the inter-pixel filter 1303. When the value of the FilterType2 filter type information is 0, the switch 1201, the switch 1202, the switch 1301 and the switch 1302 change the connection to the terminal 0 side. In this case, the switch 1201 breaks the connection with the output of adder 203 and inter-pixel filter 1203 to short-circuit with switch 1202. On the other hand, the switch 1302 connects the output of the switch 1202, the inter-pixel filter 1303, and the external output terminal of the switch 1301. Therefore, the Recon decoded image data that is sent from the adder 203 is not derived from the inter-pixel filtering operation by means of the inter-pixel filter 1203, but are stored as is in the image memory 205 as a reference image. The Recon decoded image data that is taken from the output side of the switch 1202, that is, the FilteredImg3 filtered decoded image data that is not actually filtered is derived from the inter-pixel filtering operation by the inter-pixel filter 1303, and sent as an output image OutImg to a display apparatus or the like outside the image decoding apparatus 1300.

Note that the inter-pixel filter 1203 and the inter-pixel filter 1303 are described here as different constituent elements for explanatory reasons, but an inter-pixel filter can be used for implementation (there is no problem to use an inter-pixel filter because two filters between pixels don't work at the same time). In addition, the inter-pixel filter 1203 and the inter-pixel filter 1303 may be the existing inter-pixel filter 106 or the inter-pixel filter 503 that includes the lookup table memory unit 504 shown in Figure 8. Also, it may be inter-pixel filter 503 including lookup table memory unit 504 for storing a plurality of parameter tables 620. However, in this case, you also have to enter the FilterType2 filter type information in the inter-pixel filter 503.

As explained above, according to the image decoding apparatus 1300, even if the filtering is not performed on a decoded image to be a reference image, the filtering by means of a filter between pixels can be performed for the decoded image, so that a display apparatus for viewing an output image OutImg which is sent from the image decoding apparatus 1300 can display moving images with higher quality. This is particularly effective for devices with sufficient processing power when filtering is not performed on a decoded image that is to be a reference image.

It should be noted that, as shown in the first embodiment, when the type of inter-pixel filter that indicates the information of the filter type FilterType2 is switched in the unit of an image or in the unit of an image area smaller than one image, the inter-pixel filter is toggled when the filter type is changed.

FIG. 13 is a block diagram showing the structure of an image decoding apparatus 1400 capable of selecting the filter between pixels 1203 depending on an image type of each image. The image decoding apparatus 1400 is an image decoding apparatus that decodes information about whether a decoded image is used as a reference image or not, such as encoded data including an image type of each image and the like data, and consists of the image decoding unit 202, the adder, 203, the image memory 205, the inter-image prediction unit 206, the inter-pixel filter 1203, a variable length decoding unit 1401, a switch 1402, a switch 1403, and an image type information conversion unit 1404.

Variable length decoding unit 1401 performs variable length decoding of Bitstream4 encoded data input from outside, and divides it into PType image type information, CodedRes encoded differential image data, and PredParam predictive parameter data. The PType split image type information, the CodedRes encoded differential image data, and the PredParam predictive parameter data are respectively sent to the image type information conversion unit 1404, the image decoding unit 202 and the recording unit. prediction between images 206. The information

ES 2 379 337 T3 of the image type Ptype is the information that indicates whether a current image is used as a reference image or not. For example, according to international standards, MPEG-1 and 2, the information called image type is included in the encoded data for each image, and an image called B image is not used as a reference image. Therefore, this image type included in the encoded data can be used as the PType image type information of the present embodiment. Even if filtering by an inter-pixel filter is not performed for an image that is not used as a reference image, it does not have a significant influence on the decoding of other images.

Here, the image decoding apparatus 1400 does not perform inter-pixel filtering if the current image is not used as a reference image. For example, when the processing capacity of the image decoding apparatus 1400 is too low to perform the decoding in time for the playing time, the processing load on the image decoding apparatus 1400 can be reduced by not performing the decoding. filtering by means of an inter-pixel filter for images that are not used as reference images. It will be explained using the block diagram in Figure 13. First, if the image type information Ptype entered in the image type information conversion unit 1404 indicates a different image than image B, that is, if it indicates that the current image is used as a reference image , both switch 1402 and switch 1403 change the connection to terminal 1. Therefore, the image decoding apparatus 1400 performs the inter-pixel filtering operation for the Recon decoded image data using the inter-pixel filter 1203, and stores the result of the operation in the image memory 205 as decoded image data. FilteredImg5 filters and sends them to a display apparatus or the like outside of the image decoding apparatus 1400. On the other hand, if the Ptype image type information indicates that the current image is a B image, that is, it indicates that the current image is not used as a reference image, switch 1402 and switch 1403 change the connection to the terminal 0 and the decoded image data that is sent from the adder 105 is sent directly to the outside without using the inter-pixel filter 1203.

As mentioned above, since the image decoding apparatus 1400 omits the filtering by means of the inter-pixel filter 1203 for an image B that is practically not referenced by other images, the processing load required to decode the data Bitstream encodes can be reduced without too much influence on the decoding of other images. Furthermore, since the image decoding apparatus 1400 selects an inter-pixel filter that depends on an image type of the data encoded in this way, the burden of filtering processing can be reduced even for the encoded data outputting an apparatus. of conventional image coding, with its header information such as, for example, an image header that does not include the selection information of a filter between pixels, because filtering processing is skipped for images that are not referenced.

It should be noted that it is not necessary to store images that are not referenced in the image memory 205 of Figure 13, for example, regardless of whether the filtering processing is performed for the images or not. Therefore, it is necessary to store in the image memory 205 only the data obtained by performing the filtering processing of the referenced images.

It should be noted that, in a strict sense, an image B does not imply an image that is not referenced, but rather an image coding procedure can be devised in which reference is made to a B image. Therefore, if you do not select an inter-pixel filter depending on an image type, but decide whether or not the image is actually referenced, more effective processing can be performed even when referring to a B image. However, even when referring to an image B, a filter can be switched between pixels depending on an image type to simplify implementation.

Furthermore, it is not switched regardless of whether the inter-pixel filtering is to be performed or not, but rather two filters can be switched, the inter-pixel filter 1003 and the inter-pixel filter 1004 depending on an image type or if it is done. reference or not to an image, as shown in Figure 6 or Figure 7.

Also, an example of an image decoding apparatus that switches a filter between pixels depending on an image type and whether or not an image is referenced has been explained, but an image coding apparatus can also perform this switching. in the same way.

[Embodiment 3]

Figure 14 is a block diagram showing the functional structure of an encoding apparatus for

ES 2 379 337 T3 images 1500 according to the third embodiment of the present invention. The image encoding apparatus 1500 is realized by means of a computer apparatus equipped with a CPU, a memory, a hard disk in which a program for encoding images and other programs is installed, and has, since it works for that, a operations console unit 1505, a preprocessing unit 1510, a subtraction unit 1512, an orthogonal transformation unit 1513, a quantization unit 1514, a variable-length coding unit 1517, a post-processing unit 1520, an inverse quantization unit 1521, an inverse orthogonal transformation unit 1522, an addition unit 1524, a switching unit 1530, an inter-pixel filter 1540, an image memory 1541, a calculation unit of motion 1542, a motion compensation unit 1543, a prioritization unit 1550, and a filter processing control unit 1560.

The operations console unit 1505 accepts an input operation from the operator. The preprocessing unit 1510 is equipped with a format conversion unit for converting a format of an input image signal into a space resolution designated by operation on the operation console unit 1505, an image reordering unit for reorder the images according to the types of images and other criteria, and send the images or similar items in sequence.

It must be taken into account that there are the following types of images: an I-image (intra-image: intra-coded image) that is created in the inter-image coding mode; a P-picture (predictive picture: predictive coded picture) that is created in the inter-picture coding mode and refers only to one picture; and a B-picture (bipredictive picture: plural predictive picture) that can also refer to a backward picture, and at the time of calculating motion in inter-picture encoding mode, the number of decoded pictures stored in memory is restricted of images 1541, which can be referenced at the same time by means of the motion calculation unit 1542.

Furthermore, when encoding an image, there is a mode to encode it using three types of images (also referred to as IPB encoding mode hereinafter) and a mode to encode it using only two types of images, an I image and a P image. Just as there is a way to encode it using only two types of images, an I image and a P image, there is also a way to encode a P image that has the possibility of being referenced and a P image that does not have no possibility of being referenced to it (hereinafter also referred to as first IP encoding mode) and a mode to encode a P-picture in the base layer of layered encoding, a P-picture that has the possibility of being referenced and a P-picture that has no possibility of being referenced in the enhancement layer (hereinafter referred to as the second IP encoding mode). In layer coding, the images are classified into two groups, a base layer and an enhancement layer, and the base layer is a group of images that can be reproduced by themselves and the enhancement layer is a group of images that they need the baseline image group for encoding and decoding. Layer encoding is characterized in that since the number of bits only for the base layer is small, but the number of bits for the base layer and the enhancement layer is large, and the number of images is large, it is possible to Easily achieve two kinds of uses by recording and transmitting the baseline in all cases, and recording and transmitting the enhancement layer only when necessary to achieve high image quality.

In the case of the first IP encoding mode, the information about the possibility or not the possibility is added to an image and the information about the possibility or not the possibility is also added to an image type. Furthermore, in the case of the second IP encryption mode, the information about the base, the possibility or the non-possibility is added to an image and the information about the base, the possibility or the non-possibility is also added to an image type .

The subtraction unit 1512 sends the image exiting the preprocessing unit 1510 as it is in the inter-image encoding mode, and calculates a motion compensation error (residual image) which is a differential value between the image and the image. motion compensation image that is output from motion compensation unit 1543 in inter-image encoding mode.

The orthogonal transformation unit 1513 outputs frequency components in the frequency domain that are obtained by performing the orthogonal transformation, such as the discrete cosine transformation, for the image in the inter-image coding mode and the motion compensation error in the inter-image encoding mode, which are output from the subtraction unit 1512, respectively. The quantization unit 1514 sends a quantized value by quantizing the frequency components that are sent from the orthogonal transform unit 1513. The variable-length coding unit 1517 sends a signal

Encoded ES 2 379 337 T3 for which another information compression is performed using a variable length code (Huffman code) that assigns a code length to the quantized value that is sent from the quantization unit 1514 depending on the frequency of occurrence. The post-processing unit 1520 is equipped with a buffer for temporarily memorizing the encoded signal or the like, a speed control unit for controlling a quantization interval in the quantization unit 1514, and other elements, and transforms the aforementioned motion vector , the type of image or the like, and the encoded signal sent from the variable encoding unit 1517 in an encoded signal such as a bit stream, and sends it.

The inverse quantization unit 1521 decodes the frequency components by inverse quantizing the quantized value generated by the quantization unit 1514. The inverse orthogonal transformation unit 1522 decodes the image in the inter-image coding mode and the motion compensation error (residual image) which is the differential value of pixels in the inter-image coding mode, by performing the inverse orthogonal transformation of the frequency components decoded by the inverse quantization unit 1521. The addition unit 1524 decodes the image by sending the image (decoded image) that is decoded by the inverse orthogonal transformation unit 1522 as it is in the inter-image encoding mode, and by adding the residual image that is decoded by means of the inverse orthogonal transformation unit 1522 and the motion compensation image generated by the motion compensation unit 1543 in the inter-image encoding mode.

The 1530 switch unit consists of a pair of 1531 and 1532 switches to switch the switching state synchronously under the on / off control of the 1560 filter processing control unit switch for each image, and incorporates the filter between 1540 pixels in a loop, or skip it from a loop, that is, make the filter between 1540 pixels skip its processing. The inter-pixel filter 1540 performs low-pass spatial filter processing for the decoded image that is sent from the addition unit 1524 on a block-by-block basis when the switches 1531 and 1532 are turned on, so as to output the decoded image without distortion of blocks or anything similar. For example, it calculates an average value between a pixel and the neighboring pixels and, if the difference between the pixel and a neighboring pixel is in a predetermined range, executes the processing of replacing each pixel around the block boundary with the average value calculated.

A plurality of decoded images output from the switching unit 1530 are stored in the image memory 1541. Therefore, it is possible to monitor the decoded image under the same conditions as the image decoding apparatus that decodes the encoded signal being send from post-processing unit 1520 or use the decoded image as a reference image in inter-image encoding mode. It should be borne in mind that in the first IP encoding mode and in the second IP encoding mode, a decoded P image is always stored in the image memory 1541 to which the information indicating the possibility of reference is added, and it is not necessary to store in the image memory 1541 an image P to which the information indicating that there is no possibility of reference is added. Therefore, the possibility or not possibility information means that the image is stored in the image memory 1541 or not.

In the inter-image encoding mode, the motion calculation unit 1542 searches for a reference image whose difference from the image sent from the preprocessing unit 1510 is the smallest of the decoded images stored in memory. images 1541, and sends a motion vector that is the momentum of a differential pixel. It should be noted that, when the motion vector is sent, a type of block prediction is sent that indicates whether the reference image is a forward image, a backward image, or the average value between both images. The motion compensation unit 1543 performs the operation indicating the motion vector and block prediction type, and generates a motion compensation image. The priority determining unit 1550 sends the priority of an image depending on the type of image and the base layer or the enhancement layer. The filter processing control unit 1560 controls the on and off of the switches 1531 and 1532 depending on the priority sent from the priority determining unit 1550 or the operating speed of the CPU.

Figure 15 is a block diagram showing the detailed functional structure of the priority setting unit 1550 shown in Figure 14. As shown in this figure, the priority setting unit 1550 sends the priority of an image depending on the type of image and the base layer or the enhancement layer, and consists of three tables 1551 ~ 1553, a selector 1554 and a determination processing unit 1555, as shown in Figure 15. It must be taken into account that in the case of a P image in the second IP encoding mode, the information that indicates base, possibility of reference to

ES 2 379 337 T3 it or no possibility of reference to it is added to your image type.

Table 1551 is a table that is selected when the IPB encoding mode is designated by operation in the operations console unit 1505 and in which the types of images are associated with the priority of the images, and the Priorities are set to 0 for I images, 1 for P images and 2 for B images. Bear in mind that priorities are defined so that they are lower the higher the number.

Table 1552 is a table that is selected when the first IP encryption mode is designated by operation in the operations console unit 1505 and in which the types of images are associated with their priority, and Priorities are set to 0 for I images, 1 for P images (which have the possibility of being referenced) and 2 for P images (which have no possibility of being referenced ).

Table 1553 is a table that is selected when the second IP encoding mode is designated (base, reference possibility, and no reference possibility), and the priorities are set to 0 for I pictures, 1 for P pictures ( base), at 2 for P images (which have the possibility of being referenced) and at 3 for P images (which have no possibility of being referenced).

Selector 1554 selects any of tables 1551 ~ 1553 based on the encryption mode (the IPB encryption mode or the first IP encryption mode) designated for the Operations Console unit 1505. The determining processing unit 1555 determines the priority depending on the type of image and the base layer or the enhancement layer that is sent from the preprocessing unit 1510 in relation to the table selected by the selector 1554, and sends the determined priority. Specifically, when the iPb encoding mode is designated, the selector 1554 selects the table 1551 and the determining processing unit 1555 sends the priority associated with an I-image, a P-image, or a B-image each time the data type is sent. image from preprocessing unit 1510. In addition, when the first IP encoding mode is designated, the selector 1554 selects the table 1552, and the determining processing unit 1555 sends the priority based on the type of image and the data added to an image P (possibility or not possibility ). Also, when the second IP encoding mode is designated, the selector 1554 selects the table 1553, and the determining processing unit 1555 sends the priority based on the type of image and the data added to an image P (base, possibility or no possibility).

Figure 16 is a block diagram showing the detailed functional structure of the filter processing control unit 1560 shown in Figure 14. As shown in this figure, the filter processing control unit 1560 controls the on and off of the switches 1531 and 1532 according to the priority that is sent from the priority determination unit 1550 and the speed of operation. of the CPU, and consists of three tables 1561 ~ 1563, a selector 1564 and a switch change processing unit 1565 shown in Figure 16. Table 1561 is a table that is selected when the IPB encoding mode is designated and indicates combinations of CPU operating speeds and priorities to perform filter processing, and is configured to turn on for priority 0 ~ 2 when CPU running speed is less than 70%, which only turns on for priority 0 and 1 when CPU running speed is 70% or higher and less than 80%, and that it only turns on for priority 0 when the CPU operating speed is 80% or higher.

Table 1562 is a table that is selected when the first IP encryption mode is designated and indicates combinations of CPU operating speeds and priorities to perform filter processing, and is configured to turn on for priorities 0 ~ 2 when CPU running speed is less than 70%, it only turns on for priority 0 and 1 when CPU running speed is 70% or higher and less than 80%, and that it lights only for priority 0 when the CPU running speed is 80% or higher.

Table 1563 is a table that is selected when the second IP encryption mode is designated and indicates combinations of the priorities and CPU running speeds to perform filter processing, and is set to turn on for priorities 0 ~ 3 when the CPU running speed is less than 70%, it only turns on for priorities 0, 1 and 2 when the CPU running speed is 70% or higher and less than 80%, and that it turns on only for priorities 0 and 1 when the CPU operating speed is 80% or higher.

ES 2 379 337 T3

Selector 1564 selects any of tables 1561 ~ 1563 based on the encryption mode (the IPB encryption mode, the first IP encryption mode, or the second IP encryption mode) designated by the operations console unit 1505. The switch change processing unit 1565 sends the on or off signal to control the switches 1531 and 1532 of the switch unit 1530 based on the priority sent from the priority determining unit 1550 and the obtained CPU running speed. for each image, referring to the table selected by selector 1564.

Specifically, when the IPB encoding mode is designated, the selector 1564 selects the table 1561, and the switch shift processing unit 1565 sends a power signal for the I-picture, P-picture, and B-picture if the speed CPU performance is less than 70%. Also, if the CPU operating speed is 70% or more and less than 80%, the 1565 switch shift processing unit sends a power signal only for an I image and a P image. Also, if the speed CPU performance is 80% or higher, the 1565 switch shift processing unit sends the power-on signal for an I image only.

In addition, when the first IP encoding mode is designated, selector 1564 selects table 1562 and switch change processing unit 1565 sends a power signal for image I, image P (possibility), and image P ( no possibility) if the CPU running speed is less than 70%. Also, if the CPU operating speed is 70% or more and less than 80%, the switch shift processing unit 1565 sends a power signal only for an I-image and a P-image (possibility). Also, if the CPU speed is 80% or higher, the switch shift processing unit 1565 sends a power-on signal only for an I image.

In addition, when the second IP encoding mode is designated, the selector 1564 selects the table 1563, and the switch change processing unit 1565 sends a power signal for the I image, P (base) image, P image (possibility) and the image P (not possibility). Also, if the CPU operating speed is 70% or more and less than 80%, the switch shift processing unit 1565 sends a power signal only for an I-image, a P-image (base), and a P-image (base). P (possibility). Also, if the CPU speed is 80% or higher, the switch shift processing unit 1565 sends a power-on signal only for an I image and a P (base) image.

Next, the operation of the image coding apparatus 1500 as structured above will be explained.

In the inter-image encoding mode to encode an image as an I-image, an image sent from the preprocessing unit 1510 is compressed and encoded into a frequency component by means of an orthogonal transformation performed by the orthogonal transformation unit 1513, and it is compressed and encoded into a quantized value by means of quantization performed by quantization unit 1514. This quantized value is compressed and encoded into a variable length by means of the variable length encoding performed by the variable length encoding unit 1517, it is converted into an encoded signal of a bit stream of an image I by means of the unit post-processing 1520, and is stored on a memory medium such as a hard disk.

On the other hand, the quantized value that is sent from the quantization unit 1514 is decoded into a frequency component by means of the inverse quantization performed by the inverse quantization unit 1521, and is decoded into an image by means of the orthogonal transformation inverse that performs the inverse orthogonal transformation unit 1522. When switches 1531 and 1532 are turned on under the control of filter processing control unit 1560, this decoded image is stored in image memory 1541 after being filter processed to remove block distortion by the filter between pixels 1540, and when switches 1531 and 1532 are turned off, it is stored in image memory 1541, without being filter processed.

In addition, in the inter-image encoding mode for encoding an image as a P-image and a B-image, a motion vector is generated by the motion computing unit 1542, a motion compensation image (predictive image ) by means of the motion compensation unit 1543 and a motion compensation error (differential image) is generated by means of the subtraction unit 1512. It must be taken into account that the motion estimation unit 1542 looks for a predictive image whose difference with the image sent from the preprocessing unit 1510 is the smallest of the

Decoded images stored in image memory 1541 as one or a plurality of forward or backward reference images.

Figure 17 is a diagram showing the reference relationships between the images stored in the image memory 1541. In particular, Figure 17A is a diagram showing the reference images for predictions in the IPB procedure, Figure 17B is a diagram showing reference images for predictions in the first IP procedure, and Figure 17C is a diagram showing reference images for predictions in the second IP procedure. It must be taken into account that under each image of each procedure, the priority (priority level) associated with the image is indicated.

To predict a P image in the case of the IPB procedure of Figure 17A, a forward I image and a P image can be referred to. To predict a B image, a forward I image or a forward image can be referred to. P, and an I image backwards and temporarily closer or a P image.

Note that to predict a B image in H.26L, a B image, as well as an I image and a P image, can be referred to as a forward image. In the mode that uses this image B as the reference image, the information about there is a possibility or there is no possibility is added to the image B, and the information that there is a possibility or there is no possibility is also added to the image type . And in this mode, the decoded B-picture to which the information that the possibility exists to refer is always stored in the picture memory 1541, and the decoded B-picture to which the information refers Since there is no possibility of reference to it, it does not need to be stored in image memory 1541.

To predict a P-picture (reference possibility) in the first IP procedure of Figure 17B, a forward I-picture and a P-picture (reference possibility) can be referenced. To predict a P-image (no possibility of reference), a forward I-image or a P-image (reference possibility) can be referenced.

To predict a P (base) image in the second IP procedure of Figure 17C, a forward I image and a P (base) image can be referred to. To predict a P (reference possibility) image, a forward I image and a P (base) image can be referenced. To predict a P-image (without reference possibility), a plurality of forward I-images, P-images (base) or P-images (reference possibility) can be referred to.

Note that for the sake of explanation, the case where the IPB encoding mode is designated will be explained.

Under said restriction, the motion estimation unit 1542 sends as a motion vector the amount of movement of the differential pixel between the searched reference image and the image that is sent from the preprocessing unit 1510, and also sends a prediction type of blocks indicating whether a reference image is a forward image, a backward image, or a mean value of predictive images. Also, the motion compensation unit 1543 performs, for the differential pixel, the operation that indicates the motion vector and the type of block prediction that is sent from the motion estimation unit 1542 to generate a motion compensation image. . And the subtraction unit 1512 generates a motion compensation error (differential image) by subtracting the motion compensation image generated by the motion compensation unit 1543 from the image that is output from the pre-processing unit 1510.

The motion compensation error (differential image) that is sent from the subtraction unit 1512 is compressed and encoded into a frequency component by means of the orthogonal transformation performed by the orthogonal transformation unit 1513, and compressed and encoded into a value quantized by means of the quantization performed by the quantization unit 1514. This quantized value is compressed and encoded into a variable length by means of the variable length encoding performed by the variable length encoding unit 1517, it is converted into a signal encoded in a bit stream of a P-image or a B-image together with the motion vector and others by means of the post-processing unit 1520, and is stored on a memory medium such as a hard disk.

On the other hand, the quantized value of a P-picture or a B-picture with a possibility of being referenced is sent from the quantization unit 1514 and decoded into a frequency component by means of

ES 2 379 337 T3 of the inverse quantization performed by the inverse quantization unit 1521, and is decoded into a motion compensation error (differential image) by means of the inverse orthogonal transformation performed by the inverse orthogonal transformation unit 1522. A then, the adding unit 1524 adds the motion compensation error (differential image) and the motion compensation image, and thus is decoded into one image. This decoded image is stored in image memory 1541 after it has been filter processed to remove block distortion by inter-pixel filter 1540 when switches 1531 and 1532 are turned on under the control of the image processing control unit. filters 1560, and when switches 1531 and 1532 are turned off, it is stored in image memory 1541 without being processed by filter.

Here, the on-off control of switches 1531 and 1532 by means of filter processing control unit 1560 will be explained in more detail.

FIG. 18 is a flow chart showing the switch activation processing performed by the switch change processing unit 1565 in the filter processing control unit 1560.

On the other hand, the determination processing unit 1555 of the priority determination unit 1550 determines the priority of each image that is sent from the preprocessing unit 1510 depending on the type of image with reference to the table 1551 selected by the selector 1554 , and sends the determined priority. Specifically, when the IPB encoding mode is designated, selector 1554 selects table 1551, and determining processing unit 1555 sends priority 0 for an I image, priority 1 for a P image, and priority 2 for an image. B each time the image type is sent from preprocessing unit 1510.

For each image encoding, the switch change processing unit 1565 of the filter processing control unit 1560 obtains the image priority and the CPU running speed included in this image encoding apparatus 1500 (P21 ) and determines the referenced entry in the table (table 1561 in the example in Figure 16) (P22).

Specifically, if the CPU operating speed is less than 70%, it determines that the input to be referenced is the first line, if the CPU operating speed is 70% or greater and less than 80% , determines that the input to be referenced is the second line, and if the CPU running speed is 80% or higher, it determines that the input to be referenced is the third line.

After determining the input to be referenced, the switch change processing unit 1565 reads the right column of the input (P23) and decides whether the priority defined for the picture type of the decoded picture is included. or not in the right column (P24). If listed in the right column (Yes in P24), the switch change processing unit 1565 sends a power signal to switches 1531 and 1532 (P25). Therefore, the filter processing is performed for the decoded image and the filter processed decoded image is stored in the image memory 1541.

On the contrary, if it is not included in the right column (Not in P24), the switch change processing unit 1565 sends an off signal to the switches 1531 and 1532 (P26). Therefore, filter processing of the decoded image is skipped and the decoded image is stored in image memory 1541 without being filtered.

Such control is performed for each image, and decoded images that have been filter processed and that have not been filter processed are stored in image memory 1541 in sequence. In this way, in image coding, inter-pixel filtering is not always performed to remove noise or similar elements, but inter-pixel filtering is done selectively when necessary, so if inter-pixel filtering only it is performed for images that have a lot of influence on the image quality, for example, even in an image decoding apparatus with a low throughput, It is possible to maintain the image quality of the important images that are stored in the image memory, reduce the accumulation of block distortion in the decoded images that have been stored in the image memory, improve the efficiency of the prediction made by the motion compensation unit, and reduce image quality deterioration more than with MPEG technique, and thus the great effect of image quality improvement can be achieved with low bit rate.

ES 2 379 337 T3

More specifically, when performing inter-pixel filtering, priority is given to an image that has a large influence on other images, i.e. an intracoded image, a forward predictive encoded image, a base layer image or a similar image, so that the effect of improving image quality, such as denoising, can be achieved by filtering between pixels more effectively even with the same increase in processing load.

In addition, the filter processing on and off can be controlled so that the processing capacity of the image encoding apparatus is fully utilized, thus effectively utilizing the CPU and thus encoding to obtain images from higher quality can be achieved even with the same hardware resources.

[Embodiment 4]

Next, an image decoding apparatus according to an embodiment of the present invention will be explained. FIG. 19 is a block diagram showing the functional structure of an image decoding apparatus 1600 according to the fourth embodiment of the present invention.

This image decoding apparatus 1600 is an apparatus for decoding the encoded signal that has been encoded by the image encoding apparatus 1500 shown in Figure 14, and is performed by means of a computer apparatus equipped with a CPU, a memory, a hard disk (HD) in which a program for decoding images or the like is installed and, as functions, includes a preprocessing unit 1610, a decoding unit of variable length 1617, an inverse quantization unit 1621, an inverse orthogonal transformation unit 1622, an addition unit 1624, a switching unit 1630, an inter-pixel filter 1640, a post-processing unit 1670, an image memory 1641, a compensation unit of movement 1643, a priority determining unit 1650, and a filter processing control unit 1660.

The preprocessing unit 1610 is equipped with a buffer or the like for storing a temporarily encoded signal, and divides it into the image type, the motion vector, and the encoded signal of an image itself included in the encoded signal to send them. It should be noted that when the image encoded signal is in the first IP encoding mode, the possibility or non-possibility information is added to the image and the possibility or non-possibility information is also added to the image type. Furthermore, when in the second IP encoding mode, the base information, possibility and non-possibility is added to the image, and the base information, possibility and non-possibility is also added to the image type.

Variable-length decoding unit 1617 outputs a variable-length quantized value by decoding (Huffman decoding) the encoded signal that is sent from preprocessing unit 1610. Inverse quantization unit 1621 decodes the frequency component by means of quantization inverse of the quantized value that is sent from variable length decoding unit 1617. The inverse orthogonal transformation unit 1622 decodes the image in the inter-image coding mode and the motion compensation error (residual image) which is a pixel differential value in the inter-image coding mode by means of the orthogonal transformation of the component. decoded by inverse quantization unit 1621.

The addition unit 1624 sends the image decoded by the inverse orthogonal transformation unit 1622 as it is in the inter-image encoding mode, and decodes the image by adding the motion compensation error (residual image) decoded by the transformation unit. inverse orthogonal 1622 and the motion compensation image generated by the motion compensation unit 1643 in the inter-image encoding mode. The 1630 Switch Unit consists of a pair of 1631 and 1632 switches to switch their switching state synchronously under the on and off control of the 1660 Filter Processing Control Unit switch for each image, and incorporates the filter between 1640 pixels in a loop, or skips it from the loop, that is, causes the filter between 1640 pixels to skip its processing.

The inter-pixel filter 1640 performs low-pass spatial filter processing for the decoded image that is sent from the adding unit 1624 by blocks when the switches 1631 and 1632 are turned on to generate the decoded image without block distortion or the like. For example, it calculates an average value between a pixel and the neighboring pixels, and if the difference between the pixel and the neighboring pixels is in a predetermined range, it executes replacement processing for each pixel around the block boundary with the value

ES 2 379 337 T3 calculated mean.

The 1670 post-processing unit is equipped with a format conversion unit to convert a format to a predetermined spatial resolution, an order restore unit to restore the order of images that have been rearranged depending on their image types to the original order. of the same, and similar elements, and sends the decoded image to a monitor or similar device. Image memory 1641 stores a plurality of decoded images, with a possibility of being referenced, which are sent from switching unit 1630. Motion compensation unit 1643 performs the operation indicated by motion vector and block prediction type that is sent from preprocessing unit 1610 for decoded images stored in image memory 1641 to generate motion compensation images. . The priority determining unit 1650 has the same structure as the priority determining unit 1550 shown in Figure 15 and sends the priority of an image depending on the type of image that is sent from the preprocessing unit 1610 and the layer. base or enhancement layer. The filter processing control unit 1660 has the same structure as the filter processing control unit 1560 shown in Figure 16 and controls the on and off of the switches 1631 and 1632 of the switch unit depending on the priority that is sent from the priority determining unit 1650 and the CPU running speed obtained by monitoring.

Next, the operation of the image decoding apparatus 1600 structured as above will be explained. It should be noted that, in order to explain it at the same time as the image coding apparatus 1500, the case in which the IPB coding mode is specified will be explained.

In the inter-picture decoding mode for decoding an encoded signal of an I-picture into a picture, the encoded signal that is sent from the preprocessing unit 1610 is decoded to a quantized value by means of variable-length decoding performed by the unit. 1617 variable length decode, it is expanded and decoded into a frequency component by inverse quantization performed by inverse quantization unit 1621, and decoded into an image (decoded image) by orthogonal transformation performed by orthogonal transformation unit 1622. When switches 1631 and 1632 are turned on under the control of filter processing control unit 1660, this decoded image is stored in image memory 1641 after it has been filter processed to remove block distortion by means of the filter. between 1640 pixels, and the original order of the images is restored in the 1670 post-processing unit, the image format is converted, and then sent to a monitor or similar device. In contrast, when switches 1631 and 1632 are turned off, the decoded image is stored in image memory 1641 without being processed by filter, and the order of the image is restored to the original in post-processing unit 1670, the format of which is converted and then sent to a monitor or similar device.

Furthermore, in the inter-picture decoding mode for decoding encoded signals of a P-picture and a B-picture into pictures, the encoded signal that is sent from the preprocessing unit 1610 is decoded to a quantized value by means of variable length decoding. performed by variable length decoding unit 1617, is expanded and decoded into a frequency component by means of the inverse orthogonal transformation performed by the inverse quantization unit 1621, and is decoded into a motion compensation error (differential image) by means of the inverse orthogonal transformation performed by the unit inverse orthogonal transformation 1622.

On the other hand, the motion compensation image (predictive image) is generated by the motion compensation unit 1643. It should be noted that the motion compensation unit 1643 performs the operation indicated by the motion vector and the type of block prediction that is sent from the preprocessing unit 1610 for the differential pixel in the reference image being read. from image memory 1641 to generate a motion compensation image.

And the addition unit 1624 adds the motion compensation error (differential image) and the motion compensation image to decode the result into one image. When switches 1631 and 1632 are turned on under the control of filter processing control unit 1660, after this decoded image is filter processed to remove block distortion by means of inter-pixel filter 1640, and then restored the order of the image in the post-processing unit 1670, the format of the image is converted, and the image is sent to a monitor or similar device, and the decoded picture with the possibility of being referenced is stored in the picture memory 1641. On the other hand, when the

ES 2 379 337 T3 switches 1631 and 1632, filter processing is not performed, but the order of the image is restored to the original order in the post-processing unit 1670, the format of the same is converted, and the image is sent a monitor or similar device, and the decoded image with the possibility of being referenced is stored in image memory 1641. Here, as is the case with switches 1531 and 1532 of image coding apparatus 1500, the on and off of switches 1631 and 1632 are controlled by means of filter processing control unit 1660.

More specifically, the switch change processing unit of the filter processing control unit 1660 obtains the priority of an image and the CPU running speed that is included in this image decoding apparatus 1600 for each image encoding. , determines the entry referenced in the IPB encoding mode table, reads the right column of the entry, and decides whether or not the priority defined for the image type of the decoded image is included in the right column. If included in the right column, the switch change processing unit of the filter processing control unit 1660 sends a power signal to the switches 1631 and 1632. Therefore, the filter processing is performed for the decoded image, and the filter processed decoded image is stored in the image memory 1641. On the other hand, if it is not included in the right column, the change processing unit The filter processing control unit 1660's switchgear sends an off signal to switches 1631 and 1632. Therefore, the filter processing for the decoded image is skipped, and the decoded image is stored in the image memory 1641 without being filtered.

Such control is performed for each image, and decoded images that have been filter processed and unfiltered processed are stored in image memory 1641 in sequence.

In this way, in image decoding, inter-pixel filtering is not always performed to remove noise or similar elements, but inter-pixel filtering is done selectively when necessary, so if inter-pixel filtering is performed only for images that have a great influence on the image quality, for example, even in image decoding devices with low processing power, It is possible to maintain the image quality of the important images that are stored in the image memory, reduce the accumulation of block distortion in the decoded images that have been stored in the image memory, improve the prediction efficiency by means of the motion compensation unit, and reduce the deterioration of the image quality more than with the MPEG technique and thus The great effect of improving image quality can be achieved with a low bit rate.

More specifically, when performing inter-pixel filtering, priority is given to an image that has a large influence on other images, i.e. an intracoded image, a forward predictive encoded image, a base layer image, or a similar image, whereby the effect of image quality improvement such as noise removal can be achieved by filtering between pixels more efficiently with the same increase in processing load.

In addition, the on and off of the filter processing can be controlled so that the processing power of the image decoding apparatus is fully utilized to use the CPU efficiently, and thus, the image decoding of the image can be achieved. more quality even with the same hardware resources.

It should be noted that the present invention can be realized not only as an image coding apparatus or an image decoding apparatus, but also as an image coding method or an image decoding method for feature units. functions that are included in these devices as steps, or as a program that has a computer that executes those steps. And, it goes without saying that the program can be distributed using a recording medium such as a CD-ROM or a transmission medium such as the Internet.

[Embodiment 5]

Also, if a program is recorded to achieve the structure of the image encoding procedure or the image decoding procedure as shown in each of the above-mentioned embodiments on a recording medium such as a floppy disk, it is possible to perform processing as shown in each of the above embodiments easily on a separate computer system.

ES 2 379 337 T3

Figure 20 is an illustration showing the case where the processing is performed in a computer system using a floppy disk in which the image encoding procedure or the image decoding procedure of the aforementioned first and second embodiments is stored. previously.

Figure 20B shows the front view and the cross-sectional view of the appearance of a floppy disk and the floppy disk, and Figure 20A shows an example of a physical format of a floppy disk as a recording medium itself. In the casing F there is a flexible disk FD, a plurality of tracks Tr are formed on the surface of the disk concentrically in the direction of the radius from the periphery, and each track is divided into 16 sectors Se in the angular direction. Therefore, as for the floppy disk in which the above-mentioned program is stored, the image encoding procedure as a program is recorded in the area allocated for it on the floppy disk FD.

Figure 20C shows the structure for recording and reproducing the program on and from the floppy disk FD. To record the program on the floppy disk FD, the computer system Cs writes the image encoding procedure or the picture decoding procedure as a program on the floppy disk FD through a floppy disk drive. To build the above image encoding method in the computer system by means of the program recorded on the floppy disk, the program is read from the floppy disk through the floppy disk drive and transferred to the computer system.

Note that the above explanation has been made assuming that a recording medium is a floppy disk, but the same processing can be performed using an optical disk. Also, the recording medium is not limited to these, but to any other medium such as IC card and ROM cassette which can be used in the same way if a program can be recorded on them.

Figures 21 ~ 24 are illustrations of the devices for performing the encoding or decoding processing shown in the above-mentioned embodiments, and the system using them.

Figure 21 is a block diagram showing the general configuration of a content delivery system ex100 to achieve a content delivery service. The area for providing the communication service is divided into cells of the desired size and the base stations ex107 ~ ex110, which are fixed wireless stations, are placed in their respective cells. This ex100 content delivery system connects to ex111 equipment, ex112 PDA (personal digital assistant), ex113 camera and ex114 mobile phone via internet ex101, internet service provider ex102 and phone network ex104, for instance. However, the content delivery system ex100 is not limited to the combination shown in Figure 21, and can be connected to a combination of any of them. Also, it can be connected directly to the ex104 telephone network, not through the ex107 ~ ex110 base stations which are fixed wireless stations.

The ex113 camera is a device like a digital video camera that is capable of taking moving images. The mobile phone can be any mobile phone of a PDC (Personal Digital Communications) system, a CDMA (Code Division Multiple Access) system, a W-CDMA (Wideband Code Division Multiple Access) system, or a GSM system. (Global System for Mobile Communications), a PHS (Personal Telephone System) and similar systems.

Additionally, an ex103 stream server connects to the ex113 camera via the ex109 base station and the ex104 telephone network, allowing for live distribution or the like using the ex113 camera based on the encoded data that is transmitted from the user. . The camera ex113 or the server to transmit the data can encode the data taken by the camera. Furthermore, the moving image data taken by a camera ex116 can be transmitted to a streaming server ex103 through the equipment ex111. The ex116 camera is a device such as a digital camera capable of taking still and moving images. In this case, the camera ex116 or the equipment ex111 can decode the moving image data. An LSI ex117 included in the ex111 kit or the ex116 camera performs the encoding processing. It should be noted that the software to encode and decode images can be integrated into any type of storage medium (for example, a CD-ROM, a floppy disk and a hard disk) which is a recording medium that can be read by the ex111 equipment or similar device. Also, the mobile phone equipped with the ex115 camera can transmit the moving image data. This moving picture data is the data encoded by the LSI that is included in the mobile phone ex115.

Fig. 22 is a diagram showing an example of the mobile phone ex115. The mobile phone ex115 has a

ES 2 379 337 T3 antenna ex201 to send and receive radio waves between the base station ex110, a camera unit ex203 such as, for example, a CCD camera capable of capturing video and still images, a display unit ex202 such as, for example , a liquid crystal screen to view the data obtained by decoding the video recorded by the camera unit ex203, the video received by the antenna ex201 or similar elements, a main body ex204 including a set of operating keys, a voice output unit ex208 as a speaker for voice output, a voice input unit ex205 as a microphone for voice input, a storage bracket ex207 to store encoded and decoded data such as still or moving image data taken by the camera or still or moving images of received e-mails, and a slot unit ex206 for connecting the storage medium ex207 to the mobile phone ex115. The ex207 storage media includes a flash memory element, a type of EEPROM (electrically erasable programmable read-only memory) that is electrically erasable and rewritable non-volatile memory, in a plastic casing like an SD card. .

In this ex100 content delivery system, content (eg live music video) taken by users using camera ex113, camera ex116 or the like is encoded in the same way as in the previous embodiments and transmitted to the stream server ex103, while the stream server ex103 performs the distribution of the content data stream to the clients upon request. Clients include equipment ex111, PDA ex112, camera ex113, mobile phone ex114, etc., which are capable of decoding the aforementioned encoded data. The content delivery system ex100 is a system in which clients can receive and reproduce the encoded data in this way and, furthermore, can receive, decode and reproduce the data in real time for personal retransmission.

Also, the mobile phone ex115 will be explained with reference to Figure 23. In the mobile phone ex115, a main control unit ex311 for general control of each unit of the display unit ex202 and the main body ex204 is connected to a power circuit unit ex310, an operation input control unit ex304 , an image coding unit ex312, a camera interface unit ex303, an LCD (liquid crystal display) control unit ex302, an image decoding unit ex309, a multiplexing / demultiplexing unit ex308, a recording / reproducing unit ex307, a modem circuit unit ex306 and a voice processing unit ex305 connected to each other via a synchronous bus ex313. When a key is pressed at one end of the call or a key is turned on by user operation, the power circuit unit ex310 supplies the respective units with power from a battery pack to activate the mobile phone equipped with digital camera ex115 to readiness status. In the mobile phone ex115, under the control of the main control unit ex311, which includes a CPU, ROM, and RAM, the voice processing unit ex305 converts the voice signals received by the voice input unit ex205 into speech mode. conversation in digital voice data, the modem circuit unit ex306 performs spread spectrum processing of the digital voice data, and the sending / receiving circuit unit ex301 performs the digital-analog conversion and the frequency transformation of the data in order to be able to transmit them through the antenna ex201. In addition, in the mobile phone ex115, the data received by the antenna ex201 in talk mode is amplified and the frequency transformation and analog-digital conversion is performed, the modem circuit unit ex306 performs the reverse spread spectrum processing of the data, and the voice processing unit ex305 converts it to analog voice data, so that the result is sent through the voice output unit ex208. Also, when an email message is transmitted in data communication mode, the email text data entered using the operation keys on the main body ex204 is sent to the main control unit ex311 through the unit. control of entry of operations ex304. In the main control unit ex311, after the modem circuit unit ex306 performs the spread spectrum processing of the text data and the send / receive circuit unit performs the digital-to-analog conversion and frequency transformation of said data, the result is transmitted to the base station ex110 through the antenna ex201.

When image data is transmitted in the data communication mode, the image data taken by the camera unit ex203 is provided to the image coding unit ex312 through the camera interface unit ex303. When image data is not transmitted, image data taken by camera unit ex203 is also displayed directly on display unit 202 through camera interface unit ex303 and LCD control unit ex302 .

The image encoding unit ex312 compresses and encodes the image data provided from the camera unit ex203 by means of the encoding method as shown in the above-mentioned embodiments in order to transform it into encoded image data, and the send to a unit

ES 2 379 337 T3 multiplexing / demultiplexing ex308. At this time, the mobile phone ex115 sends the voices received by means of the voice input unit ex205 during the taking of images by means of the camera unit ex203 to the multiplexing / demultiplexing unit ex308 as digital voice data via of the voice processing unit ex305.

The multiplexing / demultiplexing unit ex308 multiplexes the encoded image data provided from the image encoding unit ex312 and the voice data provided by the voice processing unit ex305 by means of a predetermined procedure, the modem circuit unit ex306 performs spread spectrum processing of the resulting multiplexed data, and the sending / receiving circuit unit ex301 performs the digital-analog conversion and frequency transformation of the result for transmission through the antenna ex201.

Regarding the reception of the data of a moving picture file that is linked to a website or the like in the data communication mode, the modem circuit unit ex306 performs the reverse spread spectrum processing of the received data. from base station ex110 through antenna ex201, and sends the resulting multiplexed data to multiplexing / demultiplexing unit ex308.

In order to decode the multiplexed data received through the antenna ex201, the multiplexing / demultiplexing unit ex308 demultiplexes the multiplexed data into coded picture data and voice data, and provides the coded picture data to the decoding unit of images ex309 and voice data to the voice processing unit respectively via the synchronous bus ex313.

Next, the image decoding unit ex309 decodes the encoded image data by means of the decoding method paired with the encoding method shown in the above-mentioned embodiments in order to generate reproduced moving image data, and provides these data to the display unit ex202 through the LCD control unit ex302, and in this way, The motion picture data contained in the motion picture file linked to a website, for example, is displayed. At the same time, the voice processing unit ex305 converts the voice data into analog voice data and provides this data to the voice output unit ex208, thereby reproducing the voice data included in an audio file. moving images linked to a website, for example.

The present invention is not limited to the aforementioned system. Lately, news about satellite or terrestrial digital broadcasting systems has been known and, at least the image coding procedure or the image decoding procedure of the aforementioned embodiments can be incorporated into said digital broadcasting system, as shown in Figure 24. More specifically, an encoded bit stream of video information is transmitted from a broadcast station ex409 to (or communicated with) a broadcast satellite ex410 by means of radio waves. Upon reception, the broadcast satellite ex410 transmits radio waves for broadcasting, a domestic anther ex406 with a satellite broadcast reception function receives the radio waves, and an apparatus such as a television set (receiver) ex401 or an apparatus that Integrated Encoder and Decoder (STB) ex407 decodes the encoded bit stream for playback. The image decoding apparatus shown in the above-mentioned embodiments can be implemented in the playback apparatus ex403 to read an encoded bit stream recorded on a storage medium ex402 which is a recording medium such as a CD. or DVD, and decodes it. In this case, the reproduced video signals are displayed on an ex404 monitor. It is also conceived to implement the apparatus for decoding images of the apparatus that integrates an encoder and decoder ex407 connected to a cable ex405 for a cable television or the antenna ex406 for broadcasting by land and / or satellite so that they are reproduced in a monitor ex408 from tv ex401. The image decoding apparatus can be incorporated in the television set, but not in the apparatus that integrates encoder and decoder. Or, an ex412 car having an ex411 antenna can receive signals from the ex410 satellite, the ex107 base station or the like to reproduce moving images on a display device such as an ex413 car navigation device or a similar device in the car. car ex412.

Also, the image encoding apparatus shown in the above embodiments can encode image signals for recording on a recording medium. As a concrete example, there is an ex420 recorder, such as a DVD recorder, to record image signals on a DVD ex421 disc and a disc recorder to record them on a hard disk. They can also be recorded on an ex422 SD card. If the ex420 recorder includes the image decoding apparatus, as shown in the above-mentioned embodiments, the image signals recorded on the ex421 DVD disc or ex422 SD card can be played back for viewing on

ES 2 379 337 T3 the monitor ex408.

It should be borne in mind that, although the structure of the car navigation device ex413 is the same as that of the mobile phone ex115 shown in Figure 23, for example, the structure can be conceived without the camera unit ex203, the unit camera interface interface ex303 and image coding unit ex312, outside the units, as shown in Figure 23. The same applies to computer ex111, television (receiver) ex401 and others.

In addition, three types of implementations can be conceived for a terminal such as the mobile phone ex114 mentioned above; a sending / receiving terminal equipped with an encoder and a decoder, a sending terminal equipped with only one encoder, and a receiving terminal equipped with only one decoder.

As described above, it is possible to make any type of apparatus and system as shown in the present embodiment by implementing the encoding and decoding procedure as shown in the present description.

Industrial application

The image coding method according to the present invention is useful as an image coding apparatus that is included in a personal computer, a PDA and a mobile phone with a communication function.