Method and system for modifying a digital image taking into account its noise

Field concerned, problem posed

The present invention relates to a method and system for modifying a digital image taking into account its noise.

We know in the state of the art the American patent N ° <patcit id="pcit0001" dnum="US5694484A"><text>US 5,694,484</text></patcit>, which relates to a system and method for automatically processing image data to provide images of optimum quality.

The invention relates to a method for calculating a transformed image from a digital image and formatted information relating to faults in an appliance chain. The device chain includes image capture devices and / or image rendering devices. The device chain includes at least one device. The method includes the step of automatically determining characteristic data from the formatted information and / or the digital image. The characteristic data are hereinafter referred to as the noise characteristic data.

It results from the combination of technical features that the transformed image does not present any visible or annoying defect, in particular defects linked to noise, for its subsequent use.

Noise estimation according to the image

Preferably, according to the invention, the method further comprises, for determining the characteristic data of the noise, the following steps:<ul id="ul0001" list-style="dash" compact="compact"><li>the step of selecting analysis zones on the digital image, in particular as a function of the devices in the device chain and / or of the formatted information,</li><li>the step of calculating local variations in luminance over the analysis zones,</li><li>the step of deducing the characteristic noise data as a function of a statistical calculation of the occurrence of local variations over all of the analysis zones.</li></ul>

Noise estimation from the image Luminance variation histogram

Preferably, according to the invention, the method further comprises, for deducing the characteristic data of the noise, the following steps:<ul id="ul0002" list-style="dash" compact="compact"><li>the step of constructing a histogram of the occurrences of local variations in luminance,</li><li>the step of selecting on the histogram at least part of the part situated before the first local maximum, including this one.</li></ul>

It results from the combination of technical features that we obtain local variations in luminance related to noise.

Noise estimation from image Noise as a function of luminance

Preferably, according to the invention, the method further comprises, for selecting on the digital image of the analysis zones, the step of classifying the analysis zones according to their average luminance, so as to obtain classes. The method further comprises:<ul id="ul0003" list-style="dash" compact="compact"><li>the step of deducing the noise characteristic data for the analysis zones belonging to the same class,</li><li>the step of iterating the previous step for each of the classes.</li></ul>

It results from the combination of technical features that we obtain the characteristic data of the noise as a function of the luminance.

Formatted information including characteristic noise data

Preferably, according to the invention the formatted information includes the characteristic data of the noise.

Clipping - Problem raised

Preferably, according to the invention the method further comprises the step of implementing a transformation algorithm to produce an intermediate digital image. The algorithm has the advantage of making desired modifications to the digital image but has the disadvantage of increasing the noise of the intermediate digital image.

Clipping - Solution

Preferably, according to the invention, for calculating a transformed image from the intermediate digital image obtained from the digital image, the method further comprises the step of implementing a function whose purpose is to modify the luminance of the digital image and having at least as arguments:<ul id="ul0004" list-style="dash" compact="compact"><li>the luminance of a point in the intermediate digital image,</li><li>the luminances of an area around the corresponding point of the digital image,</li><li>noise characteristic data.</li></ul>

It results from the combination of technical features that a transformed image is obtained, presenting the desired characteristics and a controlled noise level.

Preferably, according to the invention, the intermediate digital image is composed of the digital image.

Blur correction

Preferably, according to the invention, the method is more particularly intended for calculating a transformed image corrected for all or part of the blur. The method further comprises the following steps:<ul id="ul0005" list-style="dash" compact="compact"><li>the step of selecting in the digital image areas of images to be corrected,</li><li>the step of constructing, for each image zone to be corrected thus selected, an enhancement profile from the formatted information and the characteristic data of the noise,</li><li>the step of correcting each image zone to be corrected thus selected as a function of the enhancement profile, so as to obtain a transformed image zone,</li><li>the step of combining the transformed image areas so as to obtain the transformed image of the digital image.</li></ul>

It results from the combination of technical features that we obtain a blurred transformed image.

Calculation of the enhancement profile

Preferably, according to the invention, the formatted information makes it possible to determine, for each image area to be corrected, an image representation and a reference representation in a base relating to the image area to be corrected.

The method is such that, to build an enhancement profile from the formatted information and the noise, it further comprises the following steps:<ul id="ul0006" list-style="dash" compact="compact"><li>the step of determining, if necessary taking into account the noise, from the image representation and the reference representation a profile,</li><li>the step of determining a configured operator allowing to pass from the image representation to the profile.</li></ul>

The set of values ​​of the parameters of the configured operator makes up the enhancement profile PR.

Correction of blurring from the enhancement profile

Preferably, according to the invention, the method further comprises, for correcting each image area to be corrected as a function of the enhancement profile, the following steps:<ul id="ul0007" list-style="dash" compact="compact"><li>the step of representing at least in part the image area to be corrected in the database,</li><li>the step of applying the operator configured to the representation obtained at the end of the previous step, so as to obtain a corrected representation of the image area to be corrected,</li><li>the step of replacing the representation of the image zone to be corrected by the corrected representation of the image zone to be corrected so as to obtain a transformed image zone.</li></ul>

Clip in case of blur

Preferably, according to the invention, the method further comprises the step of calculating an image having a controlled noise level, from the transformed image, by implementing a function whose purpose is to modify the luminance of the digital image and having at least as arguments:<ul id="ul0008" list-style="dash" compact="compact"><li>the luminance of a point of the transformed digital image,</li><li>the luminances of an area around the corresponding point of the digital image,</li><li>noise characteristic data.</li></ul>

It results from the combination of technical features that we obtain a blurred image and having a controlled noise level.

Variable characteristics affecting noise and / or blurring

The formatted information can depend on variable characteristic values ​​according to the digital image, in particular the size of the digital image. Preferably in this case, according to the invention, the method further comprises the step of determining the value (s) of the variable characteristics, for the digital image.

Thus, the implementation of the method for formatted information comprising characteristic data of the noise depending on variable characteristics according to the digital image is reduced to the implementation of the method for characteristic data of the noise not depending on any variable characteristic.

Reduction of the dynamics in the case of a restitution device

Preferably, according to the invention, the method is more particularly intended for calculating a transformed image from a digital image and formatted information relating to faults in a device chain comprising at least one device for restoring 'picture. The restitution device has a dynamic. The transformed image has a dynamic. The method further includes the step of adapting the dynamics of the transformed image to the dynamics of said rendering apparatus. It results from the combination of technical features that the reproduction, by the reproduction apparatus, of the transformed image exhibits reinforced high frequencies. It also results from the combination of technical features that the rendering device can reproduce character images with less blurring.

Noise correction and / or poly-chromatic blurring

The invention applies to the case of a digital image composed of color planes. The application consists in applying the method according to the invention to each color plane. A transformed image is thus obtained from the digital image. It results from the combination of technical features that the transformed image has the desired characteristics and a controlled noise level.

System

The invention relates to a system for calculating a transformed image from a digital image and formatted information relating to faults in an appliance chain. The device chain includes image capture devices and / or image rendering devices. The device chain includes at least one device. The system includes computer processing means for automatically determining characteristic data from the formatted information and / or the digital image. The characteristic data are hereinafter referred to as the noise characteristic data.

The transformed image does not present any visible or annoying defect, in particular of defects linked to noise, for its subsequent use.

Noise estimation according to the image

Preferably, according to the invention, the computer processing means for determining the characteristic data of the noise comprise<ul id="ul0009" list-style="dash" compact="compact"><li>selection means for selecting analysis zones on the digital image, in particular as a function of the devices in the device chain and / or of formatted information,</li><li>calculation means for calculating local variations in luminance over the analysis areas,</li><li>deduction means for deducing the characteristic noise data as a function of a statistical calculation of the occurrence of local variations over all of the analysis zones</li></ul>

Noise estimation from the image Luminance variation histogram

Preferably, according to the invention, the means of deduction further comprise:<ul id="ul0010" list-style="dash" compact="compact"><li>means for constructing a histogram of the occurrences of local variations in luminance,</li><li>selection means for selecting on the histogram at least a part of the part situated before the first local maximum, including the latter.</li></ul>

Noise estimation from image Noise as a function of luminance

Preferably, according to the invention, the System further comprises, for selecting on the digital image of the analysis zones, classification means for classifying the analysis zones according to their average luminance, so as to obtain classes. The system further comprises computer processing means for:<ul id="ul0011" list-style="dash" compact="compact"><li>deduce the noise characteristic data for the analysis zones belonging to the same class,</li><li>iterate the previous step for each of the classes.</li></ul>

Formatted information includes characteristic noise data

Preferably, according to the invention, the formatted information includes the characteristic data of the noise.

Clipping - Problem raised

Preferably, according to the invention, the system further comprises computer processing means implementing a transformation algorithm to produce an intermediate digital image. The algorithm has the advantage of making desired modifications to the digital image but has the disadvantage of increasing the noise of the intermediate digital image.

Clipping - Solution

Preferably, according to the invention for calculating a transformed image from the intermediate digital image obtained from the digital image, the system comprises calculation means implementing a function whose purpose is to modify the luminance of the digital image and having at least as arguments:<ul id="ul0012" list-style="dash" compact="compact"><li>the luminance of a point in the intermediate digital image,</li><li>the luminances of an area around the corresponding point of the digital image,</li><li>noise characteristic data.</li></ul>

Preferably, according to the invention the intermediate digital image is composed of the digital image.

Blur correction

Preferably, according to the invention, the system is more particularly intended for calculating a transformed image corrected for all or part of the blur. The system further includes:<ul id="ul0013" list-style="dash" compact="compact"><li>selection means for selecting in the digital image areas of images to be corrected,</li><li>calculation means for constructing, for each image area to be corrected thus selected, an enhancement profile from formatted information and noise characteristic data,</li></ul>

The system further comprises computer processing means for:<ul id="ul0014" list-style="dash" compact="compact"><li>correct each image area to be corrected thus selected as a function of the enhancement profile, so as to obtain a transformed image area, and for</li><li>combine the transformed image areas so as to obtain the transformed image of the digital image.</li></ul>

Calculation of the enhancement profile

Preferably, according to the invention, the formatted information making it possible to determine, for each image area to be corrected, an image representation and a reference representation in a base relating to the image area to be corrected. The system is such that the calculation means for constructing an enhancement profile from the formatted information and from the noise also comprise means for determining:<ul id="ul0015" list-style="dash" compact="compact"><li>a profile, if necessary taking noise into account, from the image representation and the reference representation</li><li>a configured operator allowing to pass from the image representation to the profile.</li></ul>

Correction of blurring from the enhancement profile

Preferably, according to the invention, the computer processing means for correcting each image area to be corrected as a function of the enhancement profile comprise calculation means for:<ul id="ul0016" list-style="dash" compact="compact"><li>represent at least in part the image area to be corrected in the base,</li><li>apply the configured operator to the representation of the image area to be corrected, so as to obtain a corrected representation of the image area to be corrected,</li><li>substitute the representation of the image area to be corrected by the corrected representation of the image area to be corrected so as to obtain a transformed image area.</li></ul>

Clip in case of blur

Preferably, according to the invention, the system further comprises calculation means for calculating an image having a controlled noise level, from the transformed image, by implementing a function having the object of modifying the luminance of the digital image and having at least as arguments:<ul id="ul0017" list-style="dash" compact="compact"><li>the luminance of a point of the transformed digital image,</li><li>the luminances of an area around the corresponding point of the digital image,</li><li>noise characteristic data.</li></ul>

Variable characteristics affecting noise and / or blurring

Preferably, according to the invention, the formatted information depends on variable characteristic values ​​according to the digital image, in particular the size of the digital image. The system further comprises calculation means for determining the value (s) of the variable characteristics, for the digital image.

Reduction of the dynamics in the case of a restitution device

Preferably, according to the invention, the system is more particularly intended for calculating a transformed image from a digital image and formatted information relating to faults in a device chain comprising at least one device for restoring 'picture. The restitution device has a dynamic. The transformed image has a dynamic. The system further comprises computer processing means for adapting the dynamics of the transformed image to the dynamics of the rendering apparatus.

detailed description

Other characteristics and advantages of the invention will appear on reading the description of the alternative embodiments of the invention given by way of indicative and non-limiting example, and from:<ul id="ul0018" list-style="none" compact="compact"><li>the <figref idref="f0001">figure 1</figref> which represents a transformed image calculated from a digital image and an intermediate image,</li><li>the <figref idref="f0001">figure 2</figref> which represents defects in the digital image,</li><li>the <figref idref="f0002">figure 3</figref> which represents a selection on the digital image of analysis zones,</li><li>the <figref idref="f0003">figure 4a</figref> which represents a local variation in luminance over an analysis area,</li><li>the <figref idref="f0003">figure 4b</figref> which represents a histogram of the occurrences of local variations in luminance,</li><li>the <figref idref="f0003">figure 4c</figref> which represents a part of the histogram located before the first local maximum of the histogram</li><li>the <figref idref="f0003">figure 5</figref> which represents classes of analysis zones according to their average luminance,</li><li>the <figref idref="f0004">figure 6</figref> which represents the modification of the luminance of the digital image,</li><li>the <figref idref="f0005">figure 7a</figref> which represents the correction of an image area transformed as a function of an enhancement profile,</li><li>the <figref idref="f0006">figure 7b</figref> which represents an example of creating a blurred image at a controlled noise level.</li><li>the <figref idref="f0007">Figures 8a and 8b</figref> which represent the construction of an enhancement profile from noise,</li><li>the <figref idref="f0008">Figures 9a, 9b, 9c and 9d</figref> which present the adaptation of the dynamics of the transformed image to the dynamics of a restitution device</li><li><figref idref="f0009">figure 10</figref> : IF formatted information related to P5 faults of a P25 device of a P3 device chain.</li></ul>

Apparatus

With particular reference to the <figref idref="f0009">figure 10</figref>, we will describe the notion of P25 device. Within the meaning of the invention, a P25 device can in particular be:<ul id="ul0019" list-style="dash" compact="compact"><li>an image capture device, such as a disposable camera, a digital camera, an SLR camera, a scanner, a fax machine, an endoscope, a camcorder, a surveillance camera, a webcam, an integrated or linked camera to a phone, personal assistant or computer, thermal imaging camera, ultrasound machine,</li><li>an image reproduction device such as a screen, a projector, a television set, virtual reality glasses or a printer,</li><li>a human being with vision defects, for example astigmatism,</li><li>a device that we want to look like, to produce images having for example an appearance similar to those produced by a Leica brand device,</li><li>an image processing device, for example a zoom software which has the side effect of adding blur,</li><li>a virtual device equivalent to several P25 devices,</li></ul>

A more complex P25 device such as a scanner / fax / printer, a Minilab for photo printing, a video conference device can be considered as a P25 device or several P25 devices.

Device chain

With particular reference to the <figref idref="f0009">figure 10</figref>, we will now describe the concept of P3 device chain. A set of P25 devices is called the device chain P3. The notion of chain of apparatuses P3 can also include a notion of order.

The following examples constitute P3 device chains:<ul id="ul0020" list-style="dash" compact="compact"><li>a single P25 device,</li><li>an image capture device and an image rendering device,</li><li>a camera, a scanner, a printer for example in a Minilab of photo printing,</li><li>a digital camera, a printer for example in a Minilab for photo printing,</li><li>a scanner, a screen or a printer, for example in a computer,</li><li>a screen or projector and the eye of a human being,</li><li>a device and another device that we want to look like,</li><li>a camera and a scanner,</li><li>an image capture device, image processing software,</li><li>image processing software, image rendering apparatus,</li><li>a combination of the previous examples,</li><li>another set of P25 devices.</li></ul>

Default

With particular reference to the <figref idref="f0009">figure 10</figref>, we will now describe the notion of default P5. A fault P5 of the device P25 is called a fault linked to the characteristics of the optics and / or the sensor and / or the electronics and / or the software integrated in a device P25; examples of P5 defects are, for example, distortion, blurring, vignetting, chromatic aberrations, color rendering, flash uniformity, sensor noise, graininess, astigmatism, spherical aberration.

Digital image

With particular reference to the <figref idref="f0009">figure 10</figref>, we will now describe the notion of digital image INUM. An image captured or modified or reproduced by a P25 device is called a digital image INUM. The INUM digital image can come from a P25 device from the P3 device chain. The INUM digital image can be intended for a P25 device in the P3 device chain. More generally, the digital image INUM can come from and / or be intended for the chain of apparatuses P3. In the case of animated images, for example video, consisting of a sequence in time of still images, the digital image INUM is called: a still image of the sequence of images.

Formatted information

With particular reference to the <figref idref="f0009">figure 10</figref>, we will now describe the notion of IF formatted information. Information formatted IF is called data related to the faults P5 of one or more apparatuses P25 of the chain of apparatuses P3 and making it possible to calculate a transformed image I-Transf taking account of the faults P5 of the apparatus P25. To produce the IF formatted information, various methods can be used based on measurements, and / or captures or restitution of references, and / or simulations.

To produce the IF formatted information, one can for example use the method described in the international patent application filed the same day as the present application on behalf of the company Vision IQ and under the title: "Method and system for producing formatted information linked to faults in at least one device in a chain, in particular blurring. " In this application, a method is described for producing formatted information linked to the devices of a chain of devices. The device chain notably comprises at least one image capture device and / or at least one image rendering device. The method includes the step of producing formatted information related to faults of at least one device in the chain. Preferably, the device making it possible to capture or restore an image (I). The device comprises at least one fixed characteristic and / or one variable characteristic depending on the image (I). The fixed and / or variable characteristics are likely to be associated with one or more characteristic values, in particular the focal length and / or the focus and their associated characteristic values. The method includes the step of producing formatted measured information related to device faults from a measured field D (H). The formatted information may include the measured formatted information.

To produce the IF formatted information, one can for example use the process described in the international patent application filed the same day as the present application in the name of the company Vision IQ and under the title: "Method and system for providing, in a standard format, formatted information to image processing means." In this application, a method is described for supplying, in a standard format, information formatted IF to image processing means, in particular software and / or components. IF formatted information is related to faults in a P3 device chain. The chain of apparatuses P3 notably comprises at least one image capturing apparatus and / or an image restitution apparatus. The image processing means use the formatted information IF to modify the quality of at least one image originating from or intended for the chain of apparatuses P3. The formatted information IF includes data characterizing defects P5 of the image capturing apparatus, in particular the distortion characteristics, and / or data characterizing faults of the image reproduction apparatus, in particular the distortion characteristics.

The method includes the step of filling in at least one field of the standard format with the formatted information IF. The field is designated by a field name. The field contains at least one field value.

To search for IF formatted information, one can for example use the method described in the international patent application filed on the same day as this application on behalf of the company Vision IQ and under the title: "Method and system for modifying the quality of 'at least one image from or to a chain of devices. " In this application, a method is described for modifying the quality of at least one image originating from or intended for a given chain of devices. The determined device chain comprises at least one image capturing device and / or at least one image rendering device. Image capture devices and / or image rendering devices, progressively put on the market by distinct economic players, belong to an indeterminate set of devices. The devices in the device set have faults which can be characterized by formatted information. The method comprises, for the image concerned, the following steps:<ul id="ul0021" list-style="dash" compact="compact"><li>the step of listing formatted sources of information relating to the devices of the set of devices,</li><li>the step of automatically searching, among the formatted information thus listed, specific formatted information relating to the determined device chain,</li><li>the step of automatically modifying the image by means of image processing software and / or image processing components taking into account the specific formatted information thus obtained.</li></ul>

To produce the IF formatted information, it is possible, for example, to use the method described in the international patent application filed on the same day as the present application on behalf of the company Vision IQ and under the title: "Method and system for reducing the frequency of updates to image processing facilities. " In this application, a method is described for reducing the frequency of updates to image processing means, in particular software and / or a component. The image processing means making it possible to modify the quality of the digital images originating from or intended for a chain of devices. The device chain includes at least one image capturing device and / or at least one image rendering device. The image processing means implement formatted information linked to the faults of at least one device in the device chain. The formatted information depends on at least one variable. Formatted information allowing to establish a correspondence between a part of the variables and identifiers. Identifiers make it possible to determine the value of the variable corresponding to the identifier, taking account of the identifier and the image. It results from the combination of technical features that it is possible to determine the value of a variable, in particular in the case where the physical meaning and / or the content of the variable are known only after the dissemination of the processing means. image. It also results from the combination of technical features that the time between two updates of the correction software can be spaced. It also results from the combination of technical features that the various economic actors who produce image processing apparatus and / or means can update their products independently of the other economic actors, even if the latter radically change the characteristics of their product or cannot force their client to update their product. It also results from the combination of technical features that a new functionality can be gradually deployed starting with a limited number of economic players and pioneer users.

Variable characteristic

We will now describe the notion of variable characteristic CC. According to the invention, variable characteristic CC is called a measurable and variable factor from one digital image INUM to another captured, modified or restored by the same device P25, and having an influence on the defect P5 of the captured image, modified or restored by the P25 device, in particular:<ul id="ul0022" list-style="dash" compact="compact"><li>a global variable, fixed for a given digital INUM image, for example a characteristic of the device P25 at the time of the capture or restitution of the image linked to a user setting or linked to an automatic device P25,</li><li>a local variable, variable in a given digital image INUM, for example coordinates x, y or ro, theta in the image, making it possible to apply, if necessary, a different local treatment according to the area of ​​the digital image RHUM.</li></ul>

Is not generally considered as a variable characteristic CC: a measurable factor and variable from one P25 device to another but fixed from one INUM digital image to another captured, modified or restored by the same P25 device, for example the focal length for a P25 camera with fixed focal length.

The IF formatted information can depend on at least one variable characteristic CC.

By variable characteristic CC, one can hear in particular:<ul id="ul0023" list-style="dash" compact="compact"><li>the focal length of the optics,</li><li>the resizing applied to the image (digital zoom factor: enlargement of a part of the image; and / or the sub-sampling: reduction in the number of pixels in the image),</li><li>non-linear luminance correction, for example gamma correction,</li><li>the contour enhancement, for example the leveling level applied by the device P25,</li><li>noise from the sensor and electronics,</li><li>the opening of the optics,</li><li>focusing distance,</li><li>the number of the view on a film,</li><li>over or under exposure,</li><li>film or sensor sensitivity,</li><li>the type of paper used in a printer,</li><li>the position of the center of the sensor in the image,</li><li>the rotation of the image relative to the sensor,</li><li>the position of a projector relative to the screen,</li><li>the white balance used,</li><li>the activation of the flash and / or its power,</li><li>the exposure time,</li><li>the gain of the sensor,</li><li>the compression,</li><li>the contrast,</li><li>another setting applied by the user of the device P25, for example an operating mode,</li><li>another automatic setting of the P25 device,</li><li>another measurement made by the P25 device.</li></ul>

Variable characteristic value

We will now describe the concept of variable characteristic value VCC. The value of the variable characteristic VCC is called the value of the variable characteristic CC at the time of capturing, modifying or restoring a determined image.

Calculation of the transformed image

An INUM digital image comprises a set of image elements called pixels Px-num.1 to Px-num.n regularly distributed over the surface of the RHUM image. On the<figref idref="f0001">figure 1</figref>, these pixels have the shape of squares but they could have any other shape, circular or hexagonal for example; it depends on the design of the surfaces intended to carry the image in the image capture and restitution devices. In addition, on the<figref idref="f0001">figure 1</figref>, the pixels have been shown contiguously but in reality, generally there is spacing between the pixels. The luminance associated at any point Px-num is vx-num.

The intermediate image I-Int comprises a set of pixels, similar to that of the image INUM but not necessarily, called intermediate pixels Px-int.1 to Px-int. n each intermediate pixel is characterized by an intermediate position Px-int and an intermediate value vx-int.

The transformed image I-Trarisf also includes a set of pixels called transformed pixels Px-tr.1 to Px-tr.n each transformed pixel is characterized by a transformed position Px-tr and a transformed value vx-tr.

A transformed image is a corrected or modified image which is obtained by applying a transformation to a RHUM image. This transformation, which can be a photometric transformation, is carried out by integrating into the calculation<ul id="ul0024" list-style="dash" compact="compact"><li>the INUM image,</li><li>DcB noise data characteristic of INUM,</li><li>the I-Int image,</li><li>formatted information which takes into account, for example, faults in the devices used and / or characteristics which one wishes to introduce into the image.</li></ul>

It will be noted that the formatted information can relate to a limited number of transformed pixels and / or integrate values ​​of variable characteristics depending on the image (for example the focal length, the focusing, the aperture, etc.), in this case there may be an additional step carried out for example by interpolation so as to be reduced to simple formatted information such as that of a device having no variable characteristics, so that the case of apparatuses in particular with variable focal length is reduced to the case of a apparatus with fixed focal length.

It will be noted that the formatted information can relate to a limited number of transformed pixels and / or values ​​of characteristics variable according to the image, in this case there can be an additional step carried out for example by interpolation. In the example of a function x ', y' = f (x, y, t) where t is a variable characteristic (focal length for example), the formatted information can consist of a limited number of values ​​(xi, yi, ti, f (xi, yi, ti)). It is then necessary to calculate an approximation for the other values ​​of x, y, t other than the measurement points. This approximation can be done using simple interpolation techniques or through configurable models (polynomials, splines, Bezier functions) of higher or lower order depending on the desired final precision. With an analogous formalism t could be a vector and simultaneously include several variable characteristics (focal, focus, zoom, etc.).

In the case of noise and / or blurring, the formatted information could possibly consist of vectors making it possible to characterize the noise and / or blurring relating to a device and / or a chain of devices, and this for all of the combinations of variable device parameters, in particular by using characteristic profiles of the defect in particular representation bases, in particular the frequency representations such as for example the Fourier transforms, the wavelet transforms .... Indeed, those skilled in the art know that the frequency representations are compact and suitable domains for representing physical phenomena related to noise and / or blurring.

It is also possible to combine the formatted information IF relating to several apparatuses P25 of a chain of apparatuses P3, to obtain formatted information relating to a virtual apparatus exhibiting the faults of said several apparatuses P25; so that it is possible to calculate in one step the transformed image I-Transf from the image INUM for all of said several devices P25; so that said calculation is faster than if the method according to the invention is applied successively to each apparatus P25; said combination, in the example of a frequency representation such as for example the Fourier transform, can be achieved by cumulating the characteristic profiles of the defect of each device, for example by multiplication.

The formatted information may include data studied in a preliminary phase and relating to the cameras used, but also any information in the Exif or other format style which would provide information on the camera's settings at the time of shooting (focal length, focus , aperture, speed, flash ..).

Let us assume that the digital image INUM represents for example the capture of the monochromatic image of a white square on a black background. On the<figref idref="f0001">figure 2</figref>, we have represented the luminances of a line of the INUM image. Due to the noise and / or blurring generated by the capture and / or restitution chain, the ideal profile (a stair step) is distorted. The method of the invention makes it possible, using CAPP calculation means incorporating approximations according to, among other things, a desired final precision, to obtain on the transformed image I-Transf, a square whose luminance value vx-tr at each of the points px-tr is corrected to the nearest approximations.

Note that the application of the CAPP algorithm can in the case of noise and / or blurring bring the original INUM image back to a perfect or almost perfect image. The same algorithm can also bring the INUM image to another possibly distorted image, but differently, so as to produce an image with close resemblance to a known type of noise and / or blurring of image (retro noise effect. .). The same process also makes it possible to reduce the INUM image to an imperfect image (in the sense of a white square on a black background as on the <figref idref="f0001">figure 2</figref>) but optimal in the eyes of the observer so that it is possible to possibly compensate for defects in perception of the human eye.

Noise estimation

For certain types of APP devices, in particular image capture, it is possible to deduce data characteristic of DcB noise from formatted information. For example, this is particularly the case for devices allowing to enter variable characteristics influencing noise such as gain, ISO, etc. The dependence between noise and these characteristics will be indicated in the information formatted in particular by means of functions. polynomials.

Insofar as formatted information does not allow direct or indirect deduction of noise characteristic data, it will be necessary to deduce this characteristic data. We will therefore describe, within the meaning of the present invention, an exemplary embodiment making it possible to produce data characteristic of DcB noise and relating to an INUM image.

The INUM image is subdivided into a series of analysis zones (ZAN) which are not necessarily contiguous and which may, if necessary, overlap. The<figref idref="f0002">figure 3</figref> represents an example of cutting. A ZAN analysis zone can be of any shape and it is not necessarily necessary to analyze all the points inscribed in said ZAN analysis zone. For each ZAN analysis zone, for example a square window of size (3X3 pixels, 5X5 pixels), the method performs a measurement of local luminance variation (VLL). The set of measurements of local variations in luminances for all the ZAN analysis zones is then analyzed statistically to produce one or more data characteristics of the noise DcB and relating to the RHUM image.

An example of measurement of local variation of luminance VLL can be carried out by calculating on a zone of analysis ZAN, the maximum difference in luminance between all the points. On the<figref idref="f0003">figure 4a</figref>, VLL is 29, which represents the maximum difference between two pixels in the area. Another way could be to calculate the standard deviation of the distribution relative to the variation in luminance.

The set of measurements of local variation in luminance VLL can be analyzed statistically by creating a histogram of the frequencies of appearance of the variations. This histogram, an example of which is shown in<figref idref="f0003">figure 4b</figref> on the abscissa is a quantification of the differences in luminance VLL according to the measurement accuracy on noise. On the ordinate, the number of appearance of an analysis area ZAN giving the value VLL is added up. In the example, there were 22 ZAN analysis zones for which the measurement of local variation in luminance gave the value 50.

The profile of this histogram for a natural image, for example a landscape image comprising a random distribution of patterns of different luminance, but of homogeneous luminance over small analysis zones, comprises a characteristic zone situated before the first local maximum (<figref idref="f0003">figure 4b, 4c</figref>). If we admit that a natural image has a large number of reduced size zones (size of a ZAN analysis zone) for which the lighting is almost uniform, then the first local maximum of the histogram (d 'abscissa xm and ordinate fm) characterizes the average noise of the INUM image. For the image with very little noise we will have many VLL measurements with small luminance differences and the abscissa of the first mode will approach the origin; on the other hand, if the image incorporates a lot of noise from the various devices in the chain, each VLL measurement carried out on theoretically homogeneous areas will generate high values ​​and move the abscissa of the first mode of the histogram from the origin.

The characteristic data of the noise of the INUM image can consist of all the values ​​of the histogram up to the first mode. Another way to extract more synthetic information from the noise characteristic is, as shown in the<figref idref="f0003">figure 4c</figref>, to assign an average noise value BM as being the abscissa xb, between the origin and the first mode of the histogram (xm), for which the ordinate is a fraction of fm (typically its half).

The <figref idref="f0003">figure 5</figref> represents a variant of calculation of the characteristic data of the noise DcB. According to a similar procedure for analyzing the ZAN analysis zones, the invention provides for simultaneously estimating information relating to the average luminance in said ZAN analysis zone (for example the algebraic average of the luminances on the area). The method also provides, depending on the quantification of the luminance images, to create classes which subdivide the luminance scale in a linear fashion or not. For 8-bit quantization, the maximum class is 255; typically we will use between 5 and 10 classes (C1 .. Cn) of luminance cutting. In an exemplary embodiment of the method, the choice of the division may be a function of the histogram of the luminances of the INUM image. Each class will correspond to a histogram of cumulative frequency of appearance of a VLL, in such a way that the noise contained in the INUM image is analyzed by luminance slice.

On the <figref idref="f0003">figure 5</figref> we have described three examples of ZAN analysis zones, for an analysis of noise characteristics in three classes. For the zone ZAN-i the mean luminance is 5.8, this zone therefore belongs to class C1 and the measurement of VLL (which is equal to 11) will therefore be accumulated in the histogram HC1 relative to C1. A similar approach is made for the ZAN-j and ZAN-p analysis zones which, given their average luminance measurement, belong to classes C2 and C3 respectively. When the set of ZAN analysis zones constituting the INUM image is analyzed, we obtain as many histograms as classes. In a similar way to the previous description it is possible to output a characteristic noise value by histogram and therefore by class and thus to constitute a data set DcB = [(C1, BM1), (C2, BM2), ... ( Cn, BMn)] characteristics of the noise of INUM.

Clipping

Either an INUM digital image within the meaning of the present invention, or also a transformation applicable to INUM so as to produce an intermediate image which, according to certain aspects, has the advantage of making the desired modifications but, on the other hand, has the disadvantage, in certain areas, to increase the noise of the image. This transformation, as we will see later, could be for example a transformation reducing the blurring, a transformation increasing the contrast, a transformation allowing the creation of mosaics of images, or any other transformation likely to modify the noise characteristics between the image INUM and I-Int. The process shown in<figref idref="f0004">figure 6</figref>, is called clipping in the sense that, in the context of the invention, it consists in taking portions of images. The calculation of the luminance vx-tr of a transformed pixel Px-tr-j requires relative information:<ul id="ul0025" list-style="dash" compact="compact"><li>at pixel Px-num-j and an analysis zone ZAN-j around the point</li><li>at pixel Px-int-j</li><li>DcB noise characteristic data</li></ul>

Analysis of the mean luminance and of the local variation of luminance VLL in the zone ZAN-j makes it possible to determine the class Cj of noise belonging, and to extract from the data DcB the noise BM-j. In one way, we can calculate a normalized ratio Rj between BM-j and VLL. As shown in the<figref idref="f0004">figure 6</figref> if Rj tends to 1 (case where the local variation in luminance VLL is substantially of the same order as BM-j, that is to say that noise is measured, then the luminance vx-tr of the transformed pixel Px-tr-j is mainly taken from INUM. The luminance value of a transformed pixel can then be expressed as a function of the luminance of the pixel vx-num, the luminance of the pixel vx-int and the characteristic data of the noise. may be the following rule: <maths id="math0001"><math display="block"><mrow><mi>vx</mi><mo>-</mo><mi>tr</mi><mo>=</mo><mfenced><mi>Rj</mi></mfenced><mspace width="1em" /><mi>vx</mi><mo>-</mo><mi>num</mi><mo>+</mo><mfenced><mn>1</mn><mo>-</mo><mi>Rj</mi></mfenced><mspace width="1em" /><mi>vx</mi><mo>-</mo><mi>int</mi></mrow></math><img file="EP1410331B1_D0001.tif" /></maths>where vx-num and vx-int represent the respective luminances of Px-num-j and Px-int-j. In the contrary case, (the local variation of luminance VLL is strong in front of BM-j i.e. we are in signal) the ratio Rj tends towards 0 and the luminance vx-tr of the transformed pixel Px-tr-j is taken mainly in the intermediate image I-Int.

More generally, the luminance value of a transformed pixel can be expressed as a function of the luminances of the pixel vx-num and of its neighbors, of the luminances of the pixel vx-int and of its neighbors and finally of the characteristic data of the noise. .

It will thus be possible for example to deduce the transformed image from the intermediate image by carrying out a more or less strong filtering operation in the latter from the noise measured in INUM.

This method has the advantage of only taking relevant information from the intermediate image, excluding points for which the noise analyzed in the original INUM image is too large in the sense of a comprehensive statistical noise study. by DcB data.

It is understood that it is possible during the clipping operation to apply between the INUM and I-Int images any passing relationship, in particular linear or non-linear transformations.

The system according to the invention comprises in <figref idref="f0002">figure 3</figref>, a device for selecting SZ analysis zones. In<figref idref="f0004">figure 6</figref> it includes a calculation device MC1 for calculating an intermediate pixel from a pixel Pi of the image INUM. Furthermore, a dcb calculation device makes it possible to calculate the characteristic data of the noise DcB and to provide a coefficient Rj. The calculation device MC2 makes it possible to calculate the value of a transformed pixel, that is to say its luminance, from the values ​​of the corresponding digital and intermediate pixels and the coefficient Rj.

Blur correction

We will now describe an exemplary embodiment of a method more particularly intended for calculating a corrected transformed image of all or part of the blur. The description of this process is based on the exemplary embodiment of the system of the<figref idref="f0005">figure 7a</figref>. The digital image INUM is subdivided into image zone to be corrected ZIC. All of these areas cover the entire INUM image and, where appropriate, these areas may possibly overlap to reduce certain disturbing effects more known to those skilled in the art under the term of edge effect. The creation of a ZIC * transformed image zone and corrected for blurring defect implements a process which requires, as an argument, without limitation, the following parameters:<ul id="ul0026" list-style="dash" compact="compact"><li>knowledge of the values ​​of the variable parameters of the device or of the chain of devices for capturing and / or restoring images at the time of shooting.</li><li>the luminance at each point Px-num belonging to the ZIC zone,</li><li>INUM DcB noise characteristic data</li><li>the formatted information relating to a modeling of the blurring of the device and / or of the device chain, and possibly previously modeled by means of a configurable model.</li></ul>

For a configuration of given arguments (focal length, focus, zoom, aperture, ..., DcB, ZIC area), the configurable model of the formatted information allows access to profiles characteristic of the blur relating to an RI image representation and a RR reference representation. These profiles are expressed in a particular base notably a frequency base B using for example a Fourier transform, a wavelet transform ...

The base B will be implicit or else filled in the formatted information. Within the meaning of the present invention, a person skilled in the art understands that it is possible to represent a digital image (for example INUM) in a vector space of dimension equal to the number of pixels. Base B is understood, and this in a non-exclusive way, a base in the mathematical sense of the term of this vector space and / or a vector subspace of this.

Thereafter, frequency is called an identifier relating to each element of the base. Those skilled in the art understand Fourier transformations and / or wavelet transforms as basic changes in image space. In the case of an APP device for which the blurring defects significantly affect only one subspace of the vector space of the images, it will be necessary to correct only the components of the INUM image belonging to this subset. space. Thus the base B will preferably be chosen as a base for representing this subspace.

Another way of implementing the method within the meaning of the invention is to choose a base for representing the optimal image in the sense for example of that of the calculation time. This base could be chosen of small dimension, each element of the base having a support of a few pixels spatially located in the INUM image (for example the Splines or the set of operators of local variations Laplacien, Laplacien de Laplacien or derived from higher order ...)

The measurement of the local variation of luminance VLL on the zone ZIC makes it possible thanks to the characteristic data of the noise DcB of INUM to calculate a coefficient Rj (device dcb2). This coefficient will be coupled to the RI and RR representations (pr device) to generate a PR enhancement frequency profile relating to the ZIC area. This profile indicates the gain to be made at each frequency relating to the luminance information contained in the zone to be corrected ZIC, to remove all or part of the blur.

The <figref idref="f0005">figure 7a</figref> shows that it then suffices to express the zone ZIC in a base B, in particular an adequate frequency base B (ZIC), to apply the enhancement function for all or part of the frequencies B (ZIC *) = B (ZIC) * PR, then by an inverse transform to find the transformed image area. All of the transformed image areas are then combined so as to obtain the deflected transformed image (I-Transf ID). This combination makes it possible, for example, to provide solutions in the event of overlapping of ZICs in particular to limit side effects;

The creation of the image (I-Transf ID) as previously described, has the advantage of making the necessary modifications to the INUM image with regard to blurring, but has the disadvantage of increasing the noise in certain zones (especially relatively uniform zones).

A second implementation of a method of the present invention is based on the exemplary embodiment of the system of the <figref idref="f0006">figure 7b</figref>. It makes it possible to produce a blurred image (I-Transf IDBC) having a controlled noise level. The creation of the transformed image (I-Transf IDBC) implements a clipping procedure similar to that described previously in<figref idref="f0004">figure 6</figref>, using the dcb1 device and the clipping device. In this case the intermediate image, as defined in the<figref idref="f0004">figure 6</figref>, is none other than the deflated image (I-Transf ID).

The <figref idref="f0007">figure 8</figref> describes more precisely the obtaining of the PR enhancement profile for a determined ZIC zone. The image representations RI and reference RR extracted from the information formatted and relating to an image area to be corrected ZIC are characteristic of the blur introduced by the acquisition and / or restitution system for a configuration of variable parameters given at the time of taking view (10mm focal length, infinite focus, aperture f / 2 ...). These RR and RI representations express the following notions<ul id="ul0027" list-style="dash" compact="compact"><li>RI is the frequency profile of a ZIC zone of a reference scene as it was generated by the device and containing blur,</li><li>RR is the optimal frequency profile of the same ZIC zone as it should have been generated if the device had not generated blurring.</li></ul>

We see that the relationship between these two profiles can indicate the gain for each frequency to bring to RI to find RR. On the other hand, it turns out that the direct application of the gain to be calculated between RI and RR can generate undesirable behaviors, in particular at high frequencies when the zone to be corrected ZIC has a high level of noise. These phenomena are known to those skilled in the art by the effect of luminance oscillations called “ringing”. According to the invention, the method will estimate an RH profile comprised between RR and RI and the position of which is parameterized as a function of the noise in the zone analyzed ZIC.

The <figref idref="f0007">Figures 8a and 8b</figref> show two examples of PR profiles which it is possible to generate according to the invention. The difference between the RI and RR profiles shows the frequency loss introduced by the blurring inherent in the device.

The <figref idref="f0007">figure 8a</figref> deals with the case of a significant noise level in the ZIC zone; it will be advantageous to choose an RH profile between RI and RR and such that its effect is less towards the high frequencies (the end of RH will be confused with RI) which in this case carry the information related to noise in the picture.

The <figref idref="f0007">figure 8b</figref> on the contrary, deals with the case of a very low noise level in the ZIC zone; the high frequencies of profile RI therefore represent signal and no longer noise. We will then have an interest in choosing an RH profile between RI and RR such that the gain between RH and RI remains significant even at high frequencies in order to reinforce the perception of details in the ZIC area.

In no case can RH exceed RR which is the ideal profile of the device but does not correspond to an image achievable by a real device. In view of the above description, it is possible to choose multiple functions which configure a curve of the RH profile between RI and RR. On the<figref idref="f0007">Figures 8a, 8b</figref> the basis of representation chosen for the RR and RI representations is that of Fourier. The abscissa axis carries the frequencies of the signal, that of the ordinate carries the logarithm of the module of the Fourier transform. One way in particular of proceeding to calculate a representation of profile of RH is to remain tangent in low frequency to the profile RR then (<figref idref="f0007">Figures 8a, 8b</figref>) use a straight line to the extreme point characterizing the high frequencies.

The construction of the frequency enhancement PR profile is immediately carried out by calculating the RH / RI ratio for all the frequencies.

Polychromatic noise and / or blur correction

The method of the invention is applicable to the processing of color images. A color image is considered from the point of view of software image processing as comprising as many images (or color planes) as there are basic colors in the image. This is how an IMrvb image is considered to include the three color planes Im-red, Im-green, Im-blue. Similarly, an IMcmjn image can be considered to have 4 color planes Im-cyan, Im-magenta, Im-yellow, Imnoir. In the method described above, each color plane will be treated independently so as to obtain n transformed images which will recompose the different color planes of the final transformed image.

Reduction of dynamics before a restitution device

The method of the invention is applicable to the calculation of a digital transformed image I-Transf, intended to be viewed via a known dynamic restitution means (<figref idref="f0008">figure 9a</figref>) to create an I-REST image. This restitution means, for example a projector, intrinsically introduces blur at the time of restitution, which results in the<figref idref="f0008">figure 9b</figref> for example, by reducing the profile of a staircase transition. In order to obtain a more favorable refund we have interest (<figref idref="f0008">figure 9c</figref>) to modify the dynamics of the transformed image upstream so that the projected image has a profile closer to the ideal profile. This dynamic modification is not always possible due to the quantization of the transformed image (generally 8 bits). To overcome this difficulty, the process can reduce the overall dynamic of the transformed image (the image becomes less contrasted therefore less energetic). We can apply to it the transformations necessary to take into account the blurring of the restitution device while remaining within the admissible dynamic range of the image (<figref idref="f0008">figure 9c</figref>) and compensate for the drop in energy at the level of the restitution device itself which no longer has a quantification problem, by increasing for example the energy at the level of the lamps for a lamp projection device (<figref idref="f0008">figure 9d</figref>). It results from this combination of technical means that the rendering apparatus can reproduce images of less blurred details, in particular characters.

Application of the invention to cost reduction

Cost reduction is a method and system for reducing the cost of a P25 device or a chain of P3 devices, in particular the cost of the optics of a device or a chain of devices; the process consisting in:<ul id="ul0028" list-style="dash" compact="compact"><li>decrease the number of lenses, and / or</li><li>simplify the shape of the lenses, and / or</li><li>design or choose from a catalog an optic having P5 defects greater than those desired for the device or the device chain, and / or</li><li>use materials, components, treatments or manufacturing processes that are less costly for the device or the device chain, adding defects P5.</li></ul>

The method and system according to the invention can be used to reduce the cost of an appliance or a chain of appliances: one can conceive a digital optics, produce formatted information IF relating to the defects P5 of the device or the chain of devices, use this formatted information to allow image processing means, integrated or not, to modify the quality of the images coming from or intended for the apparatus or the chain of apparatuses, so that the combination of the apparatus or the chain of apparatuses and image processing means makes it possible to capture, modify or restore images of the desired quality at a reduced cost.