Method of controlling an action, such as a sharpness modification, using a colour digital image

CA 02834883 2013-11-26 1 PROCESS FOR COMMANDING AN ACTION, IN PARTICULAR A SHARP MODIFICATION, FROM A DIGITAL COLOR IMAGE.

Field of the invention:

The invention relates to a method for controlling an action, in particular a modification of sharpness, from a digital color image.

It relates more particularly, but not exclusively, to an improvement in the sharpness of at least one color of a digital image.

The invention also relates to a system implementing such a method as well as an image generated by such a method.

The invention also relates to a method for producing an image capture and / or reproduction device which comprises an optical system for capturing and / or image reproduction, an image sensor and / or generator, and / or a servo system, the image being processed, with a view to improving it, by digital image processing means.

The invention also relates to an apparatus obtained by such a production method.

Problem addressed The satisfactory visualization of an image requires a sharpness which is all the more important as this image presents details of reduced dimensions.

CA 02834883 2013-11-26 2 As a result, it is known to seek to improve the sharpness of at least one color in a digital image according to methods such as those described below:

i) In the specific case of cameras:

- We can use an optical focusing device (focus) which moves optical elements pelmettant to vary the range of distances for which the image is sharp.

Such a device, manual or motorized, often comprises a servo system allowing the movement to be chosen as a function of the distances of the objects in the scene.

The applications of such a process are cameras, cameras.

It has the drawback of having a limited depth of field, especially with a large aperture, and a cost and a size that is difficult to adapt to devices with small dimensions, such as telephones.

- A solution of the wavefront coding type can be used, which adds a specific optical element in the optical system to allow reconstruction by calculation of the sharpness with a large depth of field.

The applications of such a method are limited (microscopy) and have the drawbacks of requiring a specific optical element as well as a modification of the equipment, including the optical system.

- We can implement a solution which adds a specific optical element consisting of a fixed defolmable liquid lens relative to the optics.

Such a method presents a servo-control system making it possible to choose the shape of said lens as a function of the distances from the objects in the scene.

The applications of this solution (for cameraphones or cameras) have the drawback of being a specific industrial process, of having a cost and bulk for the optical element, of requiring a hardware modification.

CA 02834883 2013-11-26 3 ii) In a more general framework, the solutions are:

- the algorithms of deflouement on the luminance or a color, by increasing the sharpness by sharpen or by another method of calculation.

The applications of such a method (all cameras) have the drawbacks of a limited increase in sharpness and therefore of a very small increase in depth of field.

Furthermore, the known techniques for designing or producing such devices for capturing and / or restoring images, such as digital or film cameras, consist in first selecting the properties of the material elements of the device. , in particular the optical system, the sensor and the servo system.

Then, where appropriate, digital image processing means are provided to correct the defects of at least one of the hardware elements of the device.

In particular, to design an optical device system, we first establish a specification, that is to say we specify the size, the focal length ranges, the aperture ranges, the field covered, the performances expressed, either in size of the image spot, or in value of MTF (modulation transfer function), and the cost.

From these specifications, a type of optical system is selected and, using an optical calculation software tool, such as the Zemax tool, the parameters of this system are selected to best meet the requirements. specifications of the specifications.

This development of the optical system is carried out interactively.

Generally, an optical system is designed to have the best quality at the center of the image and usually the quality at the edges of the image is lower.

In addition, the usual techniques are such that the optical system is designed so as to obtain a determined level of distortion, vignetting, blurring and depth of field CA 02834883 2013-11-26 4, so that the optical system can be compared to other optical systems.

In addition, for digital cameras, the characteristics of the sensor are also specified, namely: the quality of the pixels, the surface area of ​​the pixels, the number of pixels, the array of microlenses, the anti-alias filters, the geometry of the pixels. , and the arrangement of pixels.

The usual technique consists in selecting the sensor of an image capture device independently of the other elements of the device and, in particular, of the image processing system.

Capture devices and / or image generators also usually include one or more servo systems such as an exposure system and / or a focusing system (automatic focus or autofocus) and / or a focusing system. flash control.

Thus, to specify an exposure system which controls the aperture and the exposure time, possibly the gain of the sensor, the measurement modes are determined, in particular the areas of the image on which the exposure will be measured is determined. as well as the weight assigned to each zone.

For a focusing system, one determines the number and the position of the zones of the image which will be used to carry out the focusing.

A motor displacement setpoint is also specified, for example.

In all cases, these specifications are carried out regardless of the presence or absence of digital image processing means.

The invention:

Findings specific to the invention:

The invention results from the combination of the following observations, which are specific to it:

CA 02834883 2013-11-26 i) The devices for capturing and / or processing images generate on these images a variable sharpness which depends on the color considered as described below with the aid of figures la and lb.

5 In FIG. 1a is shown the converging lens 1 of an optical device (not shown) provided with a sensor 2 located on a focusing point 3.2 associated with a wavelength A2.

Thus, the color defined by this length A2 is clear on an image formed by this lens when the image represents an object at a very great distance.

However, such a setting has three problems:

- First, the focal point 3.2 of the lens is specific to the color defined by this wavelength A2, so that a focusing point 3.1 specific to another color defined by a wavelength Al is located upstream of the sensor.

Consequently, the image formed by this second color (A1) at the level of the sensor is less sharp than the image formed by the first color (A2), which reduces the sharpness of the overall image formed by the sensor.

- Second, the focal point of the lens for a wavelength is variable depending on the distance at which the object 4 shown in the image is located.

Thus, figure lb shows the new locations 4.1 and 4.2 of the focal points associated, respectively, with the wavelengths A1 and A2 when the object represented has passed from a very large distance (figure la) to a closer distance (figure lb).

In the latter case, it appears that the sensor is located on the focal point of the color (A1) which, previously, did not form a clear image.

- Third, the focal point of the lens for a wavelength and an object distance is variable depending on the position in the image of the object represented.

CA 02834883 2013-11-26 6 ii) As shown in figure 2 which is an example of the spectral distribution of an image along the 6.1 axis, the images are generally composed of several colors whose intensities (y-axis 6.2 ) can be close.

In this example are shown components 5.1 blue (wavelength around 450 nm), 5.2 green (wavelength around 550 nm) and red (wavelength around 600 nm) close, but it is clear that the invention applies to an image independently of its distribution of colors and of the wavelengths considered (for example infrared or ultraviolet).

iii) Conventional sharpening techniques do not take advantage of the fact that one of the colors may be sharper than the others depending on the distance from the object shown in the image.

iv) Furthermore, the invention starts from the observation that the conventional techniques for designing or producing apparatuses do not make it possible to take full advantage of the possibilities offered by digital image processing means.

The invention:

This is why the invention relates, in general, to a method for improving the sharpness of at least one color of a digital image comprising the steps of - choosing from among the colors of the image at least one color. referred to as the sharp color, - to reflect the sharpness of the sharp color on at least one other improved color, so that the improved color has increased sharpness.

Thanks to the invention, it is thus possible:

- increase the perceived sharpness of the image, - increase the depth of field of a capture device, - create a macro device, CA 02834883 2013-11-26, 7 - control the depth of field independently of the exposure, - measure the distance of objects in a scene imaged from an image, - improve the devices for controlling exposure and / or focusing and / or flash, - reducing the costs of the device capture - reduce, at equal performance, the size of the capture device, - read bar codes and / or business cards and / or text and / or take portraits and / or make landscapes using the same lens, having a focus fixed, such as for example that of a camera, - design and / or choose an optic giving the device increased characteristics in terms of aperture and depth of field, - make effects on the image according to the relative sharpness between at least two colors and / or the distance of the objects of the imaged scene, - allow the user to digitally change the focus of the image of the relative sharpness between at least two colors and / or of the distance from the objects of the imaged scene, - reducing the time between the capture request and the actual capture of the image, by eliminating or simplifying the optical focusing system.

The invention also relates to a method for producing a capture apparatus which comprises an optical capture system, and a sensor, and / or a servo system, the image being processed, with a view to improving it, by means of digital image processing means;

method in which the parameters of the optical system and / or of the sensor and / or of the servo system are determined or selected, from the capacities of the digital means CA 02834883 2013-11-26 8 for processing images, so as to minimize production costs and / or optimize the performance of the capture device.

In one embodiment, the method further comprises the step of decomposing the digital image into regions; said choice of the sharp color being made for each region.

In one embodiment, said choice of the sharp color consists of choosing the sharpest color according to a predetermined rule.

In one embodiment, said choice of sharp color is predetermined.

In one embodiment, said digital image comes from a capture device and said choice of the sharp color is a function of the distance between the capture device and at least one object of the scene captured to obtain said digital image.

In one embodiment, said image capture device comprising a macro mode, said choice of the sharp color is a function of the activation of the macro mode.

In one embodiment, said digital image coming from a capture device, said method further comprises the step of determining the distance between the capture device and at least one object of the captured scene from the sharpness of the image. at least two colors in an image region of said object.

In one embodiment, the method further comprises the step of reducing the sharpness of at least one color in at least one image region.

In one embodiment, the method further comprises the step of determining a slaving instruction of said capture device from the sharpness of at least two colors; so that focusing occurs in fewer steps and is accelerated.

In one embodiment, said digital image coming from a capture device comprising an optic, said method further comprises the step of choosing an optic from among CA 02834883 2013-11-26 9 a set of predetermined optics; said optics exhibiting characteristics such that images of an object at at least two predetermined distances exhibit distinct sharp colors; so that the depth of field is improved and / or the cost of the optics is decreased.

In one embodiment, said digital image coming from a capture device comprising an optic, said method further comprises the step of designing an optic taking into account the method according to the invention; said optics exhibiting characteristics such that the images of an object at at least two predetermined distances exhibit distinct distinct colors, - so that the depth of field and / or the aperture and / or any other optical characteristic is improved and / or the cost of the optics is reduced, - so that the mechanical focusing can be done with fewer positions.

In one embodiment, the repercussion of the sharpness of the sharp color on at least one other improved color is carried out using a calculation of the type CA = ON + F (CO - ON) where CA is representative of the improved color. , CO is representative of the improved color before treatment, ON is representative of the sharp color and F is a filter, in particular a low pass filter.

The invention also relates to a method for producing an apparatus (20) for capturing and / or restoring images which comprises an optical system (22, 22 ') for capturing and / or restoring images, a sensor (24) and / or generator (24 ') of images, and / or a servo system (26), the image being processed, with a view to improving it, by digital means (28, 28') of image processing, the method being such that the parameters of the optical system and / or of the sensor and / or of the image generator and / or of the servo system are determined or selected, from the capacities of the digital image processing means , and CA 02834883 2013-11-26 in particular the improvement of the sharpness of a color as a function of the sharpness of another color according to a process according to one of the preceding claims, - so as to minimize the production costs and / or to optimize the performance of the device for capturing and / or restoring images.

The invention also relates to an apparatus for capturing and / or restoring images using a color improvement method according to one of the preceding embodiments 10 and / or obtained by a production method according to the preceding embodiment.

The invention also relates to a digital image obtained according to a method conforming to one of the preceding embodiments or from an apparatus conforming to the preceding embodiment.

Finally, the invention also relates to a digital image processing device implementing a method according to one of the preceding embodiments.

Definitions:

The meaning of the various teLmes used is specified here:

- By digital image is meant an image in digital form. The image may be from an image capture device.

The digital image can be represented by a set of digital values, hereinafter called gray level, each digital value being associated with a color sensitivity and a relative geometric position on a surface or a volume.

For the purposes of the invention, the term “color” is used to refer to all the digital values ​​associated with the same color sensitivity.

The digital image is preferably the raw image of the sensor (raw format in English) before demosaicing operation. The digital image can also CA 02834883 2013-11-26 11 have undergone a treatment, for example demosaicing, white balance.

Preferably according to the invention, the digital image has not undergone any downsampling.

- When the digital image comes from an image capture device, the image capture device includes a sensor equipped with sensitive elements.

By sensitive element is meant an element of the sensor capable of converting a flow of energy into an electrical signal.

The energy flow can take the fauna, in particular a light flow, X-rays, a magnetic field, an electromagnetic field or sound waves.

The sensitive elements can be, depending on the case, juxtaposed on a surface and / or superimposed in a volume.

Sensitive elements can be arranged in a rectangular matrix, a hexagonal matrix or other geometry.

The invention applies to sensors comprising sensitive elements of at least two different types, each type having a color sensitivity, each color sensitivity corresponding to the part of the energy flow converted into an electrical signal by the sensitive element of the sensor.

In the case of a visible image sensor, the sensors generally have a sensitivity in 3 colors and the digital image has 3 colors:

red 5.1, green 5.2 and blue 5.3 shown in figure 2 which shows on the vertical axis 6.2 the amount of energy converted and on the horizontal axis 6.1 the wavelength.

Some sensors have a sensitivity in 4 colors red, green, emerald, blue.

The term color also means a combination, in particular linear, of the signals delivered by the sensor.

- The invention applies with the various known definitions of known sharpness.

For example, the sharpness of a color may correspond to the measurement of a value called BXU which is a measure of the blur spot area, as described in the article published in the Proceedings of IEEE, International Conference of Image Processing, Singapore 2004, and entitled Uniqueness of Blur Measure by Jérôme BUZZI and Frédéric GUIMARD.

CA 02834883 2013-11-26 12 In a simplified way, the blur of an optical system is measured from the image, called the impulse response, of an infinitely small point located in the sharpness plane.

The BXU parameter is the variance of the impulse response (i.e. its mean area).

Processing capabilities can be limited to a maximum value of BXU.

Various methods of such sharp measurements are described in manuals and publications such as, for example, the Handbook of Image & Video processing edited by Al Bovik and published by Academic press, pages 415 to 430.

A parameter relating to the quality of an image as generally accepted is meant.

In one embodiment, the sharpness of a color is obtained by calculating a gradient.

For example, the sharpness of a color can be obtained by a gradient calculation of 9 gray levels taken in neighboring geometric positions in the color considered.

The invention mentions the sharpness of at least two colors.

According to one embodiment, the sharpness of at least two colors is only considered relatively relative to one another.

For this realization, a gradient makes it possible to simply calculate a relative sharpness between two colors independently of the content of the image.

The invention mentions choosing from among the colors at least one color referred to as a clear color.

According to one embodiment, this choice can be made by determining which of at least two colors is the clearest.

For this realization, a gradient makes it possible to simply determine the clearest color among at least two colors.

In one implementation, - An image capture device is, for example, a disposable camera, a digital camera, an SLR camera (digital or not), a scanner, a fax machine, an endoscope, a camera, a camcorder , a surveillance camera, a toy, a camera or an integrated camera or connected to a telephone, to CA 02834883 2013-11-26 13 a personal assistant or to a computer, a thermal camera, an ultrasound machine, an MRI (magnetic resonance) imaging machine, an x-ray radiography machine.

It should be noted that the invention relates to such devices when they process images comprising at least two colors.

The expression optical system for capturing images is understood to mean the optical means allowing the reproduction of images on a sensor.

By image sensor is meant mechanical, chemical or electronic means allowing the capture and / or recording of an image.

The term “servo-control system” is understood to mean means of the mechanical, chemical, electronic or computer type allowing elements or parameters of the device to comply with a set point.

These include the automatic focusing system (autofocus), automatic white balance control, automatic exposure control, optical element control, in order, for example, to maintain single-lens quality. images, an image stabilization system, an optical and / or digital zoom factor control system, or a saturation control system, or a contrast control system.

- Digital image processing means can take various forms depending on the application.

The digital image processing means can be integrated, in whole or in part, into the device, as in the following examples:

- An image capture device which produces modified images, for example a digital camera which incorporates image processing means.

- An image restitution device which displays or prints modified images, for example a video projector or a printer including image processing means.

CA 02834883 2013-11-26 14 - A mixed device which corrects the defects of its elements, for example a scanner / printer / fax machine including image processing means.

- A professional image capture device which produces modified images, for example an endoscope including image processing means.

According to one realization:

the digital image processing means comprise a means for improving the image quality by acting on at least one of the parameters of the group comprising:

geometric distortions of the optical system, chromatic aberrations of the optical system, parallax compensation, depth of field, vignetting of the optical system and / or sensor and / or image generator, lack of sharpness of the optical system and / or of the sensor and / or of the image generator, the noise, the moiré phenomena, and / or the contrast, - and / or the determined or selected parameters of the optical system are chosen from the group comprising: the number of optical elements of the system, the nature of the materials composing the optical elements of the optical system, the cost of the materials of the optical system, the treatment of the optical surfaces, the assembly tolerances, the value of the parallax as a function of the focal length, aperture characteristics, aperture mechanisms, possible focal length range, focusing characteristics, focusing mechanisms, anti-alias filters, bulk, the depth of field, the characteristics relating the focal length and the focus, the geometric distortions, the chromatic aberrations, the decentering, the vignetting, the characteristics of sharpness, - and / or the determined or selected parameters of the sensor and / or generator of images are chosen from the group comprising: the quality of the pixels, the surface area of ​​the pixels, the number of pixels, the array of microlenses, the anti-alias filters, the geometry of the pixels, the arrangement of the pixels, CA 02834883 2013-11-26 - and / or the determined parameters or selected from the servo system are chosen from the group comprising: the focusing measurement, the exposure measurement, the white balance measurement, the focusing setpoint, 5 the opening setpoint, the exposure time setpoint, the sensor gain setpoint, the setpoint flash.

For the servo system allowing automatic focusing, it should be recalled that the focusing can be carried out in various ways, in particular by controlling the position of movable elements of the optical system or by controlling the geometry of the optical system. defolmable optical elements.

- The performances of a capture device are in particular, its cost, its size, the minimum quantity of light that it can receive or emit, the quality of the image, in particular its sharpness, the technical characteristics of the optics , sensor and servo control as well as its depth of field.

For this purpose, it should be noted that the depth of field can be defined as the range of distances in which the object generates a sharp image, that is to say whose sharpness is greater than a given threshold for a color. , generally green, or alternatively, or COhne the distance between the closest object plane and the most distant object plane for which the blur spot does not exceed predetermined dimensions.

Since the color green predominates in defining the sharpness of an image, as explained later, it is common to use the color green to define the depth of field.

The invention also relates to an apparatus obtained by the production method as defined above.

According to other characteristics of the invention which can be used independently of, or in combination with, those described above:

CA 02834883 2013-11-26 16 The invention relates to a method for controlling an action from a measurement carried out on at least one digital image, having at least two colors, coming from an image capture device, in which :

- the relative sharpness between at least two colors is measured on at least one region R of the image, and - at least one action is ordered as a function of the measured relative sharpness.

By region is meant part or all of the image.

A region comprises one or more pixels, contiguous or not.

Thus the action is in particular adapted to the distance between the imaged object and the capture device or is adapted to the relative depth between two imaged objects.

We can measure relative sharpness in different ways, for example (without the list being exhaustive):

- one can determine the clearest color, and / or - one can choose among the colors at least one color called "clear color", and / or - one can compare the sharpness between the colors, and / or - one can calculate a difference in sharpness, and / or - the relative sharpness can be calculated directly.

Various examples of relative sharpness measurements will be presented below, illustrated in particular by FIGS. 3a, 3b, 4, 5, 6, 7, 8, 9 and 10.

The relative sharpness and / or the measure of relative sharpness in a region can be expressed by a single numerical value, for example reflecting the average relative sharpness in the region, or by several numerical values ​​reflecting the relative sharpness in different parts of the region.

According to the invention, at least one action is controlled as a function of the measured relative sharpness.

This action is in particular (without the list being exhaustive):

- direct or indirect processing (notably via the supply of processing parameters or information CA 02834883 2013-11-26 17 of distance and / or position and / or direction) of the digital image and / or of another digital image, and / or - a measurement of distance and / or direction and / or position and / or size and / or orientation and / or geometric shape of at least part of at least one object or subject of the scene, and / or - information linked directly or indirectly to the geometry of the scene imaged in three dimensions, and / or - detection of an object including a face and / or the main subject (s) and / or - recognition and / or authentication of an object, for example a face, and / or - a measurement of the position and / or movement of the device, and / or - a servo control of the device or '' another device such as a robot, and / or - automatic framing of the main subject, and / or - a modification of the setting of the device, and / or - the production or triggering of a signal, and / or - an addition, deletion or modification of an object in the digital image or another image digital, and / or - any other action using directly or indirectly the relative sharpness measurement.

According to one realization, the action implements:

- the digital image, and / or - another digital image, and / or - a choice of the user of the device, and / or - at least one characteristic of the capture device during the shooting and / or - another data.

In the event that the action relates to direct or indirect processing, the processing may consist (without the list being exhaustive) of one of the following actions:

- digitally change the focus, and / or CA 02834883 2013-11-26 18 - make effects on the image depending on the relative sharpness between at least two colors and / or the distance from objects in the scene imaged, and / or - reduce the sharpness of at least one color in at least one region of an image, and / or - increase the sharpness of at least one color in at least one region of an image, and / or - compress, and / or - carry out any other treatment described in the present description.

The use of the measured relative sharpness to control the action thus makes it possible, in particular, to adapt the action to the distance between at least part of an imaged object and the measuring device, and / or to the geometry at least part of an object and / or the position and / or size of at least part of the object, and / or the direction of at least part of the object.

The known methods do not make it possible to control this type of action from a relative sharpness measurement of at least one region of the image, but require the use of a particular device in addition to the image sensor. to estimate a distance.

In addition, the known methods only allow distance measurement at one point or a limited number of points, while the invention makes it possible to measure the distance at a large number of points simultaneously.

According to one embodiment, the commanded action is included in the group comprising:

- a determination of the distance between the capture device and at least one object imaged by the digital image, and / or the determination of the relative distance between two imaged objects, CA 02834883 2013-11-26 19 - an action depending on said distance and / or said relative distance, - processing on at least one zone Z 'of the digital image and / or another digital image, - servo-control of the capture device and / or servo-control of 'another device, - the supply of an indication and / or alarm and / or alert signal to a user, - the detection of a part of the image, - a modification of the sharpness of a color, - a determination of the position and / or movement of the capture device, - the determination of the position in the image of a subject, - a modification of at least one characteristic of the image, - a modification of all or part of the image, - the determination of an area of ​​interest in the image, in particular in order to provide a servo control signal, - the modification of the resolution of all or part of the image, - the supply of information relating to the image, - the supply of information to a capture device of sound (s), - the setting of a compression, - a modification of all or part of the image, - at least one setting of the capture device.

CA 02834883 2013-11-26 According to one embodiment, the commanded action comprises processing on at least one zone Z 'of the digital image and / or of another digital image.

Zone Z 'may or may not be part of the digital image on which the relative sharpness measurement was taken.

As an example of processing carried out on a digital image distinct from that on which the relative sharpness between at least two colors is measured, the first point is the taking of a video sequence for which the following image can be processed. , or another image, this processing consisting in increasing (also by way of example) the sharpness.

Indeed, it is possible to increase the sharpness of a following image, it being understood that it is based on the measurement of a previous image which differs little from this following image.

15 Thus, it is not necessary to keep the current digital image in memory.

In another example: the sharpness measurement is, in a digital camera, carried out on the image displayed before shooting and the image taken subsequently taken at full resolution is processed (while the measurement carried out on the image displayed before shooting, is usually at lower resolution) from the last measurement or a combination of the last measurements.

In one embodiment, the zone Z 'constitutes all or part of the region (on which the relative sharpness measurement was carried out) of the digital image, and / or the entire digital image, and / or a distinct zone of the region of the digital image, and / or an area of ​​another digital image, and / or an entire digital image.

When the zone Z 'constitutes all or part of the region of the digital image, for example when one wishes to increase the depth of field, the zone Z' is a pixel and one defines a region of N pixels on which one measures relative sharpness and, as a function of this relative sharpness, a filter is applied which conveys the sharpness of the sharpest color to the other color so that the sharpness of the pixel is increased.

Thus by repeating this operation for each pixel, the depth of field is increased.

The zone Z ′ on which the processing is carried out can constitute an entire digital image, in particular when the sharpness is increased over the entire image.

By way of example of processing on a zone distinct from the region of the digital image, mention will be made of the case where the relative sharpness measurement is carried out on a region and the processing is applied to a centered image portion corresponding to digital zoom.

By way of example of processing applied to an area of ​​another digital image and / or to another entire digital image, the above example of a video sequence is recalled, the other digital image being, for example , an image following a video image; the other image is also, for example, the digital image taken at full resolution for a camera, while the image on which the measurement is made is at low resolution.

In one embodiment, the zone Z ′ for which processing is controlled comprises at least one pixel of an image and the region comprises a predetermined neighborhood of the corresponding pixel in the digital image. The processed image can be the digital image. The processed image can also be another image, for example an image coming from the same sensor and captured after the digital image.

In this case, the correspondence between the pixels of the two images can be done by associating the pixels of the two images located at the same place.

This case has the advantage of avoiding the storage of the digital image between the measurement and the processing without disturbing artifacts if the images are captured with a short time interval, for example 1 / 15s.

In one embodiment, this processing is applied to all the pixels of an image. The processed image can be the digital image. The processed image can also be another CA 02834883 2013-11-26 22 image, for example an image coming from the same sensor and captured after the digital image.

In one embodiment, the processing on at least the zone Z 'comprises the modification of at least one characteristic of the image belonging to the group comprising: the sharpness, the contrast, the luminosity, the details, the color, the type of compression , compression rate, image content, resolution.

Example of contrast modification:

The contrast of nearby objects is increased and the contrast of background objects is reduced, for example in the case of a video conference.

Conversely, you can reduce the contrast of nearby objects and increase the contrast of background objects to reduce the effect of fog.

Example of brightness modification:

The processing may consist of illuminating nearby objects and darkening the background, for example for a video conference.

Conversely for an image taken with a flash, the processing of the brightness will consist in lighting up the background and darkening the closest objects to compensate for the effect of the flash.

Example of modification of details:

For a video conference, the details of the background objects can be reduced, in order to allow a higher compression for these background objects, while keeping a maximum quality for the main subject.

Example of color modification:

The color saturation of the regions where the relative sharpness is greater than a threshold is reduced to eliminate excessive longitudinal chromatic aherrations, sometimes called "pu / pie fringing".

Example of modification of the compression type:

For example, for a video conference, an MPEG4 codec is provided with a near object / distant object segmentation, in order to allow CA 02834883 2013-11-26 23 to strongly compress the distant object to keep maximum quality of the main subject which is close.

Example of modification of the compression ratio:

As above in the case of a video conference, the compression rate may be higher for the background than for the main subject.

Example of content modification:

The treatment consists of replacing a background with a landscape or a setting.

In one embodiment, the processing comprises a modification of sharpness for each pixel of zone Z 'by means of a filter mixing the values ​​attached to the pixel over a predetermined neighborhood to each pixel, the parameters of the filter being a function of the relative sharpness measured.

In one embodiment, the zone Z ′ is determined from the measured relative sharpness.

For example, the zone Z 'corresponds to parts of images where the relative sharpness is included in a given range corresponding to parts of the image containing objects lying in a given range of distances, which allows, for example , to treat a foreground and background differently.

Within the meaning of the invention, background and background correspond to the same concept.

In one embodiment, the zone Z 'constitutes a background of an image, in particular intended to be transmitted remotely, in particular by a video or video conference system. The processed image can be the digital image. The processed image can also be another image, for example an image coming from the same sensor and captured after the digital image.

According to one embodiment, the processing comprises the supply of information depending on the distance between the imaged object and the capture device for all or part of the pixels of the zone Z ′ and a storage and / or a transmission and / or a transmission is ordered. or a use of this information function CA 02834883 2013-11-26 24 of the distance, the storage being carried out in particular in a computer file, in particular in an image file.

It will be recalled that the zone Z ′ can constitute a point and / or a region and / or several regions and / or a complete image and / or a main subject and / or a background.

The information depending on the distance can be a distance, for example with an indication of precision, or a range of distance values ​​such as, for example, a distance less than a centimeter, a distance between 1 and 10 centimeters then between 10 centimeters and 1 meter, and beyond 1 meter. The information as a function of distance can also be represented by a criterion of the type "too close", "near", "close", "far", or "macro". The information as a function of distance can also be translated into information on the nature of objects or subjects such as "portrait" or "landscape".

It is thus possible to provide a map of the distances of the various parts of the image.

It is also possible to provide the position of the zone with respect to the capture device.

The distance-dependent information may also include the values ​​of the distances of the various elements of the image such as the minimum distance, the maximum distance, the mean and the standard deviation.

It is important to note that the invention makes it possible to measure several distances in a scene from a single image while the prior art requires complex means such as the use of several cameras arranged in several places to perform the image. stereoscopy, or a moving camera, or a laser rangefinder or even an ultrasound sonar that does not provide a visible image.

In one embodiment, the commanded action comprises a slaving command of the capture device included in the group consisting of: a focus servo, an exposure servo, a flash servo, an image framing servo, a white balance servo, an image stabilization servo, CA 02834883 2013-11-26 a servo-control of another device or device linked to the capture device, such as the guidance of a robot.

Example of a focus servo control:

The main subject or areas of interest can be detected by distance measurements, from sharpness, with the main subject or area of ​​interest then being the closest area.

A focusing servo-control made from measurements taken directly on a single digital image is particularly advantageous over known focusing servo-controls, or "autofocus", for which it is necessary to perform measurements on images. successive.

In addition, a known servo-control of focusing consists in pressing on a trigger member until mid-stroke then in moving the framing before pressing fully, whereas with the invention the focusing can be carried out. fully automatically; the invention therefore saves time and improves the image.

Example of exposure servo control:

20 As with the focus servo, the exposure adjustment is performed on the main subject which is detected automatically; thus the exposure can be correct regardless of the position of the main subject in the frame of the image.

In other words, as with focusing, the user does not need to aim at the subject then press halfway and then move the framing.

Example of slaving of a flash:

Given that the invention makes it possible to determine the main subject, the lighting control can be carried out as a function of this main subject while with the prior art the power of the flash is adjusted as a function of the focus. without determining the main subject, that is to say, in particular, the closest subject.

As noted above, less illuminated subjects can be processed digitally by brightening.

CA 02834883 2013-11-26 26 Example of ordering another device:

When a mobile robot has to move, the regions closest to the mobile robot are determined and from the objects closest to the mobile robot a path free of any obstacle is determined.

In one embodiment, the commanded action comprises providing a signal such as a signal indicating the main object of interest of the digital image, and / or a focus area, and / or an alarm signal indicating a modification of the monitored and digitally imaged scene, and / or of distance of at least part of the imaged scene from the capture device.

For example, in a digital camera, it is possible to have a frame, in particular of a predeteiminated shape, around the main subject in order to indicate to the photographer which is the main subject detected by the device during the shooting.

This main subject indication signal can be used in particular before the actual shooting to indicate to the photographer which subject or object will be the clearest.

This signal can also be an indication that the nearest object or subject is too close to the camera to be in focus.

In this case, the signal is formed, for example, by the clear message "Foreground too close", or by an exaggeration of the blur of the foreground, or even by a visible modification of the color of the foreground.

The signal indicating that the scene or the foreground object is too short a distance can take into account the final use of the image which will be taken, in particular the resolution chosen for this use.

For example, a subject that would be out of focus on a television receiver or computer screen may be in focus on a small screen such as that of a camera.

Likewise, a fuzzy subject for a print on paper measuring 24cm x 30cm is not necessarily so for a print of 10cm x 15cm.

CA 02834883 2013-11-26 27 The blur indication signal can also take the subject into account.

For example, the detection of a bar code is more tolerant of blur than a natural image.

Example of an alarm signal provided by a camera:

In an object video surveillance system, the camera is set to monitor two regions.

The first of these regions is where the object is located and the second region is the entire field of view of the camera.

If an object in the field of view approaches the object to be monitored, then an alarm is triggered.

In one embodiment, the commanded action is made to depend on at least one characteristic of the capture device during the shooting, in particular the focal length, the aperture, the focusing distance, the exposure parameters, white balance settings, resolution, compression, or a user-made setting.

Indeed, the commanded action is a function of the measured relative sharpness and this relative sharpness between at least two colors depends on the setting of the camera, in particular the focal length, the aperture and the focusing distance. point.

In one embodiment, the digital image constitutes a raw image coming from the sensor of the capture device.

This arrangement facilitates the measurement of relative sharpness because if a raw or "raw" image is used, the measurement is not affected by processing such as demosaicing, sharpness improvement filtering, color space change or the tone curve.

The raw image coming from the sensor may however have undergone processing such as denoising, digital gain, black level compensation.

CA 02834883 2013-11-26 28 The relative sharpness measurement and / or the commanded action can (can) be performed in the capture device.

The relative sharpness measurement can be performed outside the capture device, for example on a computer after transfer of the digital image, and / or an action is controlled which is performed outside the capture device.

It is in fact possible to carry out a relative sharpness measurement outside the capture device; likewise the action can be carried out outside the capture apparatus, as already mentioned.

For example, a processing program carried out on a computer determines, from the sharpness measurements, the focusing distance and / or the depth of field in order to carry out treatments depending on this distance and / or the depth of field .

In one embodiment, the command comprises a command for the detection and / or recognition of a part of the image, such as a detection and / or recognition of a face.

For example, we know that a face has a determined size.

The method according to the invention makes it possible to determine the distance between objects or subjects and the capture device;

moreover, from this information of distance, focal length and size of the object in the image, it is possible to deduce the presence of the face (which has a size within a determined range).

The object size criterion can be supplemented by other criteria such as, for example, the colors.

Object detection, such as face detection, can be used in particular for, during a teleconference, automatically performing strong compression of the background.

This method can also be used for the detection of the defect, in order to correct it, of red eyes, or for face recognition (biometric applications).

In one embodiment, the commanded action comprises a measurement of the position and / or movement of the capture device.

CA 02834883 2013-11-26 29 In one embodiment, one or more objects intended to remain fixed in a scene of a captured image are kept in memory and the motion or position detection is carried out by determining the variation of the relative sharpness over time.

This arrangement can, for example, be used to produce a computer interface of the three-dimensional visual "mouse" type.

In one embodiment, the commanded action comprises determining the position in the image of the main subject (s).

The criterion for determining the main subject in a digital image will be the shortest distance from the capture device.

However, this criterion can be combined with other factors.

For example, it is possible to eliminate, by automatic processing, objects at the edge of the image which would be close to the capture device.

As previously described, it is also possible to take account of a criterion of the size of the object, this size being a function of the focal length and of the distance between the capture device and the object.

In one embodiment, the commanded action further comprises the automatic framing, in particular the centering, or the cropping of the digital image and / or of another image on the main subject of the digital image. The cropped image may be the digital image. The cropped image can also be another image, for example an image coming from the same sensor and captured after the digital image.

For example, it is possible to provide a "close-up" mode which automatically ensures a framing on a foreground object.

It is also possible to provide a "bust" mode which automatically ensures the framing of a face according to the so-called three-thirds rule, for example positioned at one-third of the height and the width of the image.

In one embodiment, the commanded action comprises the application of a treatment which is a function, on the one hand, of CA 02834883 2013-11-26 the relative sharpness and, on the other hand, of a criterion selected by the 'user.

For example, the selected criterion is the following:

favor the parts of the image which are closest to the capture device.

Thus the command can consist in increasing the sharpness of these parts of the image and reducing the sharpness of the rest of the image in order to create a depth of field less than that obtained in reality.

Under these conditions it is possible to simulate the behavior of a lens with variable focus and 10 aperture in an image obtained with an objective without control neither of focusing nor of aperture, as in a "cameraphone.

In one embodiment, the controlled action comprises changing the contrast and / or the brightness and / or the color and / or the sharpness of an image depending on the relative sharpness variation in the image.

Thus we can simulate localized lighting such as that of a flash; it is also possible to reduce the effect of a flash, for example to reduce the effects of backlighting or flat spots.

20 A scene is lit by one or more natural or artificial sources as well as possibly by one (or more) flash (s) controlled by the device.

It is known that an image capture apparatus performs exposure control (exposure time, sensor gain and, where appropriate, aperture), white balance control (gain of each color in the image). 'entire image) and possibly the flash (duration and power of the flash) as a function of measurements in a digital image of the scene (for example analysis of saturated areas, histogram analysis, 30 average color analysis) and / or measurements made with a complementary device: infrared range finder, flash pre-flash focus servo allowing to find the focus producing the sharpest image by comparing the sharpness of several images taken with different focus.

These controls modify the contrast and / or the brightness and / or the color of the image CA 02834883 2013-11-26 31 but do not use a measure of the relative sharpness between at least two colors on at least one region R of the image. the image.

On the other hand, known treatments such as the tone curve and color rendering modify the contrast and / or the luminosity and / or the color of the image but do not use a measure of the relative sharpness between at least two colors. on at least one region R of the image.

These known methods are limited by the absence of information on the geometry of the scene.

For example, it is difficult to distinguish a naturally dark object from a poorly lit object.

For example again, a flash cannot properly illuminate several subjects, if they are at different distances.

In one embodiment, the commanded action comprises providing, to an exposure and / or white balance and / or focusing servo system, the position of at least one area of ​​interest to be taken. into account, this area of ​​interest being determined by comparing at least two relative sharpness measurements.

For example, the exposure control can be carried out on the part closest to the capture device, possibly by combining with another criterion such as the elimination of nearby object (s), at the limit of 'image (edge ​​of field).

The slaving of the white balance could be carried out for example on a subject of significant size in the center of the image, possibly to the detriment of a background lit differently.

Alternatively, the method consists of determining a near part in the image and a far part and the white balance control takes separate measurements on these two regions in order to determine the presence or not of several lights, and to perform separate compensations. for each of these regions.

CA 02834883 2013-11-26 32 If the position of the area of ​​interest is supplied to the focusing control, the focusing action will be faster and the main subject (area of ​​interest) can be tracking, even if it is in motion.

In one embodiment, the commanded action comprises providing a signal, intended for the user, indicating that the image is too close to be sharp.

In one embodiment, the commanded action comprises modifying the resolution of an image as a function of the measured relative sharpness. The image can be the digital image. The image can also be another image, for example an image coming from the same sensor and captured after the digital image.

For example, the resolution is reduced when the image is taken at a distance from the capture device which is too small to obtain a sharp image at full resolution, the final resolution being chosen to obtain a sharp image.

In one embodiment, the commanded action includes providing an infoLmation, or signal, used for automatic indexing of the digital image.

For example, if the image includes subjects or objects at a distance less than a limit and of a size greater than a threshold, then the indexing could consist of the supply of a signal indicating that it is about 'a portrait or a group of people.

The distinction between these two situations is made according to whether the imaged scene comprises one or more objects or close subjects.

If the distance of the objects or subjects is greater than a predetermined limit, then the image can be considered to represent a landscape.

According to one embodiment, the commanded action comprises the supply to a sound capture device (s) of distance and / or direction information with respect to the capture device, of a subject or object in the field. digital image.

CA 02834883 2013-11-26 33 Thus, in a camcorder or a video camera, we can determine the main subject (s), determine the distances and / or directions of these main subjects and focus the capture sound on the main subject or subjects and thus eliminate background noise.

The directivity control of sound capture can be performed using two microphones and a phase shift between the signals from these microphones.

A particular application of the latter arrangement is, in a video conference, the use of a wide-angle image capture device and automatic tracking of the subject who is speaking orally.

In one embodiment, the commanded action includes setting high compression for the background and compression for the main subject (s), which main subject (s). ) being determined as constituting an area of ​​the image satisfying criteria based on the measured relative sharpness.

Thus, for example during a videoconference, the throughput can be minimized while maintaining satisfactory visibility of the main subject.

The latter is determined as constituting the part of the image closest to the camera and determined differently as described in the present application.

In one embodiment, the capture apparatus comprises a sensor having pixels provided with color filters of at least two kinds, these filters being chosen so that their spectral responses exhibit little overlap.

Under these conditions, it is possible to maximize the difference in sharpness between two colors and therefore optimize the precision of the relative sharpness measurement.

In one embodiment, the capture apparatus includes a sensor having pixels primarily used for producing the image and other pixels primarily for measuring relative sharpness.

CA 02834883 2013-11-26 34 In one embodiment, the pixels serving mainly to measure the relative sharpness have a spectral response in a spectral band which has little overlap with the spectral band, pixels serving mainly to produce the image.

In one embodiment, the pixels primarily used for producing the image have a spectral response primarily in the range visible to the human eye and the other pixels have a spectral response primarily outside the range visible to the human eye.

The invention also relates to a sensor thus defined, independently of a capture device and of the method, according to the invention, defined above.

The invention also relates to a capture device comprising such a sensor, this capture device also being able to be used independently of the method defined above.

The invention also relates, according to an arrangement which can be used in combination with (or independently of) the arrangements defined above, a digital image capture apparatus which comprises a sensor having, on the one hand, pixels. whose spectral response is mainly in the range visible to the human eye and, on the other hand, additional pixels having a spectral response, mainly outside the spectrum visible to the human eye, this sensor was such that the image part resulting from the additional pixels exhibits a sharpness, in at least a range of distances between the capture device and the imaged scene, greater than the sharpness of the part of the image resulting from the pixels spectral response mainly in the visible range.

The additional pixels can be sensitive to infrared and / or ultraviolet radiation.

Pixels sensitive to ultraviolet radiation can be used to improve sharpness at short distances, while pixels sensitive to infrared radiation can be used to improve sharpness at long distances.

By infrared and / or ultraviolet, CA 02834883 2013-11-26 can be understood to mean any part of the spectrum beyond or below the visible spectrum, in particular the near infrared such as 700 to 800 or 700 to 900nm, or the near ultra violet close to 400nm.

In one embodiment, the capture device is provided with a fixed optic, that is to say without mechanical elements for focusing.

Under these conditions, it is possible to focus by digital processing.

In one embodiment, the capture device is provided with a variable focal length optic without a moving or deformable focusing element, the relative sharpness between at least two colors on at least one region R of the image being variable according to the focal length and / or the position of the imaged object with respect to the device.

15 There is thus obtained a device provided with a simpler zoom, which can reduce the size and the cost, and increase the reliability.

The variable focal length optic comprises for example a single movable or defolmable optical group.

20 It is known that a zoom is achieved with at least two moving groups, for example one or two for the focal length and the other for the focusing.

In general, the focus and focal length are independent, that is, when the focal length varies, it is not necessary to change the focus.

25 This eliminates the time required for focusing.

There are also optics with variable focal length, known as varifocal, less expensive, in which the focus must be modified when the focal length varies.

Finally, there are afocal zooms in which two intricately linked movable optical groups are used to vary the focal length, focusing being achieved by a third group.

In one embodiment, the digital image comes from at least two sensors.

CA 02834883 2013-11-26 36 For example, each sensor is dedicated to a determined color.

It is for example possible to use a sensor of the tri-CCD type with common imaging optics on these sensors.

In one embodiment, the commanded action includes adding an object to an image and / or replacing part of an image based on the relative sharpness measured on the digital image.

For example, the process makes it possible to add a character next to the main subject.

We can also, by way of example, add an object in a given position in the image; the object will have the correct size in the image if we take into account the distance from the scene imaged at this position.

You can also modify the background or possibly hide it.

It is also possible to extract part of the image, such as the main subject, and insert it into another image, natural or synthetic, for example as part of a game.

It is also possible to add advertising information at a determined location and a fixed distance from the stage, for example behind the main subject.

In one embodiment, the method comprises capturing a sequence of images, the digital image forming part of the sequence and the commanded action being performed on at least one other image of the sequence.

Thus, as already described, the estimation of the relative sharpness can be carried out on preview images before shooting, at lower resolution, while the correction can be carried out on a definitively stored image, for example by means of a choice of filters resulting from a measurement made on the preview images.

In one embodiment, the commanded action comprises the modification of an adjustment of the capture device, in particular the focal length, the aperture, the focusing distance.

Thus, the camera may include an automatic adjustment program such that the aperture is CA 02834883 2013-11-26 37 increased if the main subject is in front of a background, the background may then be blurred. .

The adjustment program can also automatically adjust the aperture to the distance of subjects in a group so that there is sufficient depth of field for all subjects in the group to be in focus.

It will be noted that, in the latter case, a function performed automatically is obtained whereas it is performed manually in the state of the art.

In one embodiment the commanded action comprises producing a modified raw image.

The digital image is preferably the raw image of the sensor (“raw” format in English) before the demosaicing operation. The digital image may also have undergone processing, for example, white balance.

Preferably, the digital image has not undergone any downsampling.

An assembly with optical system, sensor and image processing means is thus obtained which produces a raw image exhibiting better quality or particular characteristics, for example an extension of the depth of field, while retaining characteristics similar to a raw image directly coming from the sensor and in particular compatibility with known functional blocks or components performing the function of converting raw image to visible image (“image pipe” or “image signal processor” in English) .

As a variant, the raw image has undergone demosaicing.

In one embodiment, the optics of the capture device exhibits strong longitudinal chromatic aberrations, for example such that, for a given focus, aperture and focal length, there is at least one color for which the object distance of best sharpness is less than f2 k being a coefficient less than 0.7, preferably less than 0.5, f being the focal length, 0 the aperture and P the most CA 02834883 2013-11-26 38 small (parsnip all the colors of the image) of the diameters of the blur spot of an object point located at infinity.

In one embodiment, the relative sharpness measurement between two colors is obtained by comparing the results of a first measurement M applied to the first color and the result of the second measurement applied to the second color, each measurement M providing a function value , on the one hand, the sharpness of the color and, on the other hand, the content of the digital image, so that the comparison can be freed from the content of the digital image.

Example of definition and realization of relative sharpness measurement:

The comparison of sharpness is performed using an M measurement on pixels of the digital image.

The measurement M in a given pixel P, for a given color channel C corresponds to the gradient of the variation of C in a neighborhood P.

It is obtained by the following calculation:

For a given color C, we consider V (P) a neighborhood of pixel P.

We denote by CM the average of the amplitude of the gradients on the neighborhood V (P), and SM the average of the amplitude of the differences between GM and the gradients on the neighborhood V (P).

A gradient is calculated by the magnitude of the difference in values ​​of two pixels of the same color.

The gradients in the neighborhood V (P) correspond to the gradients involving a predetermined number of pairs of pixels in the neighborhood V (P).

The measurement M at the pixel P having a color C can be defined by the ratio between SM and CM.

We thus obtain a value M (P, C).

This measurement does not, in itself, precisely and completely characterize the sharpness of the color C.

Indeed, it depends on the content of the image (type of scene imaged: textures, gradients, etc. ") in the neighborhood V (P) CA 02834883 2013-11-26 39 of pixel P.

A sharp transition in the imaged scene for the same color sharpness will generate a higher M measure than a smooth transition in the imaged scene.

On natural images, a transition will be present in the same way in each color, thus affecting the M measurement in the same way between colors.

In other words when a clear transition appears on a color C, the same type of transition appears on the other colors.

Thus, the comparison of the measurements M enables the relative sharpness to be established between a color C1 and a color C2.

The relative sharpness, between two colors C1 and 02, measured at a pixel P can be defined for example as a comparison between the two measurements M (P, C1) and M (P, 02).

Thus M (P, 01)> M (P, C2) implies that Cl is sharper than C2.

For example also, one could use one of the following fo / mules:

M (P, C1) -M (P, C2), M (P, C1) / M (P, C2), Or any other function F (M (P, C1), M (P, C2) adapted to the comparison between the two measurements.

The relative sharpness in an R region of the image can be set using the M measure on all P pixels in the R region.

The relative sharpness in an R region of the image can be all or a subset of the relative sharpnesses measured for the P pixels of the R region.

It can also be defined as a single value such as the sum S of the measurements on all the pixels P of the region R for each of the colors.

Thus, for two colors C1 and C2, we can for example consider that S (C1)> S (C2) implies that Cl is sharper than 02 on average over the region R.

One can also use any other function G (S (C1), S (02)) pe / putting the comparison of these two measurements.

CA 02834883 2013-11-26 In one embodiment, when the remote action 5 consists in determining the position of the main subject in the image, the remote action further comprises the automatic framing, in particular the centering of the image on the main subject.

The method can be implemented in an image capture or image processing apparatus or device.

These devices or devices form part of the group comprising: an electronic component, whether or not integrating a sensor, an electronic sub-assembly incorporating an optic, a sensor and possibly an image processing module ("camera module") 15 or any other form such as defined above.

Other characteristics and advantages of the invention will become apparent with the description of some of its embodiments, the latter being made with reference to the accompanying drawings in which:

20 - Figures la and lb, already described, are representative diagrams of the longitudinal chromatic aberration of a converging lens, - Figure 2, already described, is the color spectral diagram of an image, - Figures 3a and 3b are diagrams showing the improvement of the sharpness of a color by means of the same sharp color according to the invention, - figure 4 is a diagram showing the improvement of the sharpness of a color by means of different sharp colors associated with distinct regions of an image according to the invention, - figures 5, 6 and 7 are diagrams showing the improvement in the sharpness of a color by means of different sharp colors associated with the whole of an image according to the invention, CA 02834883 2013-11-26 41 - figure 8 is a diagram representing the control of an apparatus as a function of a difference in sharpness between the clear color and the color to be improved according to the invention, - figure 9 is a diagram representing the choice of a clear color from a distance measured between an object and a device capturing the image of that object, - figure 10 is a diagram showing the reduction in the sharpness of at least one color in at least one region of the image, - figure 11 is a diagram of an apparatus obtained by the method according to the invention, - FIG. 12 is a diagram showing steps of the method according to the invention, - FIG. 13 shows an adjustment mode according to the invention, - FIG. 14a and 14b form a set of diagrams showing adjustments used in the context of the invention, - Figures 15, 15a, and 15b illustrate a property of an image capture apparatus according to the invention and of a conventional apparatus, - Figures 16a to 16d are diagrams showing the properties of an optical system of an apparatus according to the invention and of a conventional apparatus, - Figures 17a and 17b are diagrams showing an example of selection of optical system for an apparatus according to the invention, - Figure 18 is a diagram illustrating the characteristics of a camera according to the invention, - Figures 18.1 and 18.2 represent means for implementing the method according to the invention, CA 02834883 2013-11-26 42 - Figures 19.1, 19.2 and 19.3 show steps of the method according to the invention according to several variant embodiments, and - Figures 20.1 and 20.2 show other embodiments of the invention.

In accordance with the invention, the method described below improves the sharpness of at least one color of a digital image by choosing from among the colors of the image at least one color called sharp color and by reflecting the sharpness of the sharp color on the image. at least one other improved color, as shown below using Figures 3a and 3b.

More precisely, in FIG. 3a is represented the sharpness (axis 7.2 of the ordinates) of two colors 13.1 and 13.2 as a function of the distance of the objects which they represent on the image considered vis-à-vis the device having captured the image (axis 7.1 of the abscissa).

As explained previously, the sharpness of these two colors varies differently as a function of this distance but overall, in this example, the first color 13.2 has better sharpness than that of a second color 13.1 of this same image.

This is why, according to the method in accordance with the invention, the sharpness of the first color 13.2 is reflected in order to effect the improvement 14 in the sharpness of the second color 13.1 which, after this improvement, has an increased sharpness 13.3.

In this example, CA, CO and CN are respectively representative values ​​of the enhanced color, the original (or to be enhanced) color and the sharp color.

In this example, the sharp color is the first color.

The original color and the improved color correspond to the second color before and after processing.

CA 02834883 2013-11-26 43 The repercussion of the sharpness on the second color is effected by using an F filter, according to a formula of the type:

CA = ON + F (CO - ON) Typically, the F filter will have the particularity of removing details from the image to which it is applied.

For this, we can use a linear low-pass filter (or averaging).

One can also use one of the many known nonlinear filters having the particularity of removing details such as for example a median filter.

At this point, it should be remembered that the human retina has a particularly high sensitivity, vis-à-vis the details of an image, for the color green so that the adjustment of optical systems is generally aimed at obtaining high sharpness for this. color for a certain focus range (see, for example, pages 30 to 33 of the book Color appearance models, by Mark D.

Fairchild edited by Addison Wesley).

Thus, according to a finding specific to the invention, an optical device delivering images whose sharpness is not satisfactory for the human eye can have a sharpness which is satisfactory for one of its colors, such as blue or red, for which the eye exhibits less sensitivity when considering details.

Typically, for a focusing optics at great distances (hyperfocal), considering an image presenting a near object and a distant object, it appears that the sharpness of the distant object is generally favored with a green color while the sharpness of the near object is improved by considering the blue color.

It then appears important to be able to improve regions of an image according to different sharp colors as a function of the relative sharpness between two colors.

In fact, in one embodiment of the invention, the sharp color used to improve the sharpness of a color is chosen as a function of regions of the image, such a method being described below. with the aid of FIG. 4 which shows an image comprising two regions 11.1 and 11.2.

In these two regions are present two colors 8.2 5 and 8.3.

However, the sharpness (y-axis 7.2) of these colors is such that in region 11.1 the color 8.2 is the sharpest while in region 11.2 the color 8.3 is the sharpest.

Therefore, the improvement of a color in the region 11.2 is done by considering the color 8.3 as a clear color while the enhancement of a color in the region 11.1 is done by considering the color 8.2 as a clear color .

At this point, it should be noted that the regions of an image may or may not be predetermined.

For example, in the case of a digital image made up of pixels, a region may be a spatial area delimited by one or more pixels.

In addition, it is possible to choose a sharp color to enhance another color by performing a simple comparison of the sharpness of one color against the others, at least one other, regardless of any notion of distance such as that represented by l 'axis 7.1.

In this case, such an analysis is presented under the fauna, for example, of a table:

Zone 11.1 11.2 Sharpness 8.2> 8.3 8.3> 8.2 In this case, the color 8.2 is selected as the sharp color in the region 11.1 while the color 8.3 is the sharp color in the zone 11.2.

Regardless of the use of regions in an image, it may be advantageous to consider different sharp colors CA 02834883 2013-11-26 to enhance a color in an image, as described below using Figures 5, 6 and 7.

More precisely, the diagram of FIG. 5 represents the sharpness (ordinate axis 7.2) of two colors 5 8.2 and 8.3 as a function of the distance (7.1) between at least one object of the scene captured to obtain said image and the device capture.

It appears that in the range 9.1, the color 8.3 exhibits a sharpness greater than the sharpness of the color 8.2 10 while the situation is the reverse for greater distances (range 9.2).

In this case, a method according to the invention may consider the color 8.3 as the sharp color, serving to correct the sharpness of a color, over the distance range 9.1, while the color 8.2 is considered to be the sharp color for. improve a color from an object in the captured scene to obtain the image located at a distance from the capture device within the range 9.2.

As a result of such corrections, the sharpness of the colors in the image can be improved towards a profile as shown in Diagram 6, namely the juxtaposition of the sharpest colors in the image.

It is clear that, analogously to the description of Fig. 4, it is possible to choose a sharp color to enhance another color by making a simple comparison of the sharpness of one color against the others, at least one color. other, independently of any notion of distance such as that represented by axis 7.1.

The sharpness curves represented in FIGS. 30 already described 3a, 3b, 4, 5 and 6 and described later 7 to 10, can vary according to the geometric position of the region considered of the image and / or other capture parameters. image such as focal length, aperture, focus, etc_ CA 02834883 2013-11-26 46 To determine the clearest color within the meaning of the invention, it is not necessary to know the parameters indicated above.

In other cases, and in particular to determine the distance according to the invention and / or to control the depth of field, it is necessary to know some of these parameters as well as the sharpness curves, at least partially. (s) or approximate for certain values ​​of these parameters.

Furthermore, the choice of the sharp color can also be determined by the software activation of at least one image capture mode such as a macro mode as described later.

In such a framework, we can consider the image as a single region.

It should be noted that in these Figures 5 and 6 is shown a threshold 8.1 which indicates the required level of sharpness, beyond which the image is considered blurred.

In a conventional processing, shown in FIG. 7, such a threshold 8.1 defines the depth of field, that is to say the range 9.2 of distances between at least one object of the scene captured to obtain said image, and the device capture, such that the image of the object is sharp.

One consequence of the invention is therefore to allow an extension of the depth of field of an optical system as detailed below with the aid of FIG. 9.

In this figure, the depth of field of a capture device, initially limited by the sharpness of the color 8.2 and the sharpness threshold 8.1, is increased by using a second color 8.3 having a satisfactory sharpness (below the threshold 8.1) over a new range of distances between at least one object of the captured scene to obtain said image and the capture device.

Concretely, such an application is implemented in fixed focus cameras, such as CA 02834883 2013-11-26 47 camera cameras.

Indeed, the optical design of these devices provides a range of sharpness for large distances up to a few tens of centimeters at best on the basis of a green color, similar to the color 8.2 in Figure 5.

Furthermore, since the color blue does not focus in the same way, it can have a sharpness at shorter distances than the color green, in a manner analogous to the color 8.3.

Consequently, the invention makes it possible to increase the sharpness of an image at a short distance from a cameraphone by attributing to the color green, and to the other colors, the sharpness of the color blue, correspondingly increasing the depth of field. of the device.

In one embodiment of the invention, shown with the aid of FIG. 8, more particularly adapted to a capture device provided with an autofocus, the method determines a control setpoint of the capture device considered at starting from the sharpness of at least two colors of the captured image so that the focusing is done in fewer steps and therefore more quickly.

For example, a distance 17.1 between at least one object of the imaged scene and the optical system 1 capturing the image can be determined using the different levels of sharpness (axis 7.2 of the ordinates) of the colors 8.2 and 8.3 used in the image. region 11.3 relating to the image of the object.

Knowing such a distance between the object 4 and the system 1, 11 is then possible to determine a setpoint 5 for the servo-control of the capture device 6.

This FIG. 8 will be described in more detail below.

According to another embodiment of the invention, shown with the aid of FIG. 10, the sharpness of at least one color is reduced in at least one region of the image.

Macro application CA 02834883 2013-11-26 48 We will now describe, based on FIGS. 5, 6 and 7, an embodiment of the method and system according to the invention, more particularly suited to the implementation of a Macro function. without the need for a particular mechanical device for a known image capture device.

A macro function is intended to make it possible to produce an image of objects close to the capture device in a predetermined range of distances, called the macro distance range 9.1, to the device.

Usually, a capture device can move all or part of the optics to achieve the macro function.

The method or system which is the subject of the invention can dispense with such a displacement.

According to the invention, the clearest color is predetermined / mine for the macro distance range 9.1, for example by measuring the sharpness 8.2 and 8.3 of the colors of the digital images obtained by the capture device for each color by making digital images from objects located at different distances from the capture device.

The clearest color (figure 5) is that corresponding to measure 8.3.

This predetelmination can be carried out definitively, for example at the time of design of the device (or of the series of devices).

Subsequently, when using the device, when the Macro function is activated, the sharpness of the net color thus predetermined / mined is passed on to the other colors, as described above.

When the Macro function is deactivated, the sharpness of the digital image can be calculated by a conventional method or by using the method according to the invention applied to the distance range 9.2.

A macro function is thus obtained which is compatible with a fixed focus optic without any moving mechanism, therefore without changing the size of the image capture device or adding any additional material cost.

The macro mode can thus be activated by software at the level of the device or any other device processing the image.

This software activation can thus be done in a conventional manner before capturing the image but also after this capture and on a local or remote device from the capturing device.

According to a variant, the activation of the macro mode can be done automatically, for example by determining the clearest image between the image generated in normal mode and the image generated in macro mode.

The macro function carried out according to the invention also benefits a device comprising variable parameters at the time of capture of the digital image and having an influence on the sharpness of the colors, in particular a capture device with zoom, and / or a optics with variable focus and / or variable aperture.

One then uses the curves of sharpness 8.2 and 8.3 corresponding to the value of the variable parameters according to the digital image.

The addition of a macro function allows the shooting of bar codes, business cards, or manuscripts containing text and / or diagram by an image capture device, in particular a telephone or a camera.

Application of depth of field extension We will now describe, based on FIGS. 4, 5, 6 and 7, an embodiment of the method and system according to the invention, more particularly adapted to the extension of the depth of field without necessity. of a particular mechanical device for a known image capture device.

Depth of field is the range of distances between objects in the scene and the image capture device, allowing a sharp digital image to be obtained.

Usually, a capture device has a limited depth of field CA 02834883 2013-11-26 and the smaller the opening of the optics is large.

According to the invention and as shown in FIG. 4, the digital image is broken down into regions 11.1 and 11.2, for example into square regions corresponding to 9 sensitive elements neighboring the sensor, or, more generally, into regions corresponding to X by Y sensitive elements or in regions of predetermined shape or calculated according to the digital image. The sharpest color is then chosen for each region, for example as the color corresponding to the lowest value among the values ​​obtained by calculating a gradient for each color from the gray levels corresponding to the color and the color. region considered.

In FIG. 4, the color 15 corresponding to curve 8.3 is sharper for region 11.2 while the color corresponding to curve 8.2 is sharper for region 11.1.

The sharpness of the sharp color thus chosen is then reflected for each region on the other colors.

By relying on Figure 5, it can be seen that the digital image of near objects -distance 5 to the capture device included in the distance range 9.1- is sharp for the color corresponding to the curve 8.3. (eg blue) while it is less so for the color corresponding to curve 8.2 (eg green).

It can also be seen that the digital image of distant objects - of distances to the capture device included in the distance range 9.2 - is clear for the color corresponding to the curve 8.2 whereas it is less so for the color corresponding to the curve 8.2. the color corresponding to the curve 8.3. As the eye is much more sensitive to sharpness in green than blue, it will perceive a sharpness corresponding to curve 8.5 30 in figure 7.

If 8.1 matches the sharpness threshold for the eye, the image will be sharp only for objects located at a distance from the capture device in the range 9.2.

Figure 6 CA 02834883 2013-11-26 51 represents, by curve 8.4, the sharpness obtained in each color after using the method according to the invention: the blue made it possible to obtain a sharpness better than the threshold 8.1 for the objects objects, located in the distance range 9.1, while green provided a better sharpness than the threshold 8.1 for distant objects, located in the distance range 9.2.

This results in a sharp digital image for all colors over a wide range of depth of field.

This is an example where the choice of the sharp color is made according to a predetermined rule namely the choice of the sharpest color in each region.

Thus the depth of field is increased without increasing the cost, the complexity or the bulk of the optics and / or without having to change the exposure, therefore without reducing the aperture, nor increasing the noise level or increasing the motion blur.

The increase in depth of field produced according to the invention benefits in particular fixed optics, in particular telephones. The increase in depth of field allows both barcode, business card, or manuscript containing text and / or diagram to be taken as well as portraits or landscapes by an image capture device, in particular phone or camera.

This is possible without using an expensive autofocus or macro device.

In addition, this device, compared to a manual macro-mechanical device, is produced fully automatically without any intervention by the user.

The increase in depth of field produced according to the invention also benefits an apparatus comprising variable parameters at the time of the digital image capture and having an influence on the sharpness of the colors, CA 02834883 2013-11-26 52 in particular a capture device with zoom, and / or an optic with variable focus and / or a variable aperture.

One then uses the curves of sharpness 8.2 and 8.3 corresponding to the value of the variable parameters according to the digital image.

The method and device according to the invention then makes it possible to choose or design, as described later with the aid of FIGS. 11 to 17b, during the design of the capture device, an optic with a more limited number of positioning positions. developed, which has the advantage of reducing optical design constraints and therefore reducing costs.

This also has the advantage of eg bringing faster and less expensive focusing by reducing the necessary precision of the servo mechanism.

For example, to obtain an optic with a great depth of field, it is possible to choose or design an optic having the characteristic of having the union of the ranges of clear distances for each of the colors as large as possible.

For example, to obtain an optic with a large aperture, it is possible to choose or design an optic having the characteristic of having a single clear color in each of the ranges of distances and such that the union of the ranges of clear distances for each of the colors corresponds to desired depth of field.

For example again, it is also possible to optimize both the aperture of the apparatus and the depth of field of the image.

A method and a device are also obtained which make it possible to reduce the longitudinal chromatic aberrations of a digital image.

There is also obtained a method and device which can increase the sharpness of an image without knowing the capture apparatus used to produce it.

CA 02834883 2013-11-26 53 Application to the measurement of the distance of objects in a scene from a single image We will now describe, based on FIG. 8, an embodiment of the method and system according to the invention , more particularly suitable for measuring the distance of objects in a scene from a single image without the need for a hardware telemetry measuring device.

The method then makes it possible to obtain an estimate of the distance of the objects present in each region of the digital image.

Usually, a capture device uses a hardware device to measure the distance of objects in a scene based on a laser, infrared device, pre-flash of flash ...

According to the invention, and as shown in FIG. 8, the digital image is broken down into regions 11.3, for example into square regions corresponding to 9 sensitive elements neighboring the sensor, or, more generally, into regions corresponding to X by Y sensitive elements or in regions of predetermined or calculated shape according to the digital image.

The sharpness of at least two colors is then measured for each region 11.3; the measured values, or the measured relative values ​​16.1 and 16.2, are then transferred to the corresponding sharpness curves 8.2 and 8.3 of the capture device.

A distance 17.2 is then obtained corresponding to an estimate of the distance 17.1 between the part of the object 4 shown in region 11.3 and the capture device.

The distance measurement carried out according to the invention benefits in particular fixed optics, in particular telephones.

The distance measurement carried out according to the invention also benefits an apparatus comprising variable parameters at CA 02834883 2013-11-26 54 when the digital image is captured and having an influence on the sharpness of the colors, in particular a camera. capture with zoom, and / or optics with variable focus and / or variable aperture.

One then uses the curves of sharpness 8.2 and 8.3 corresponding to the value of the variable parameters according to the digital image.

The method then makes it possible to obtain an estimate of the distance of the objects present in each region of the digital image.

This allows :

- To build a real-time and low-cost range finder device using a sensor and standard optics which delivers an image and distance information correlated to the image;

usually, several shots are necessary or a specific hardware device is necessary and the association of image / distance information is complex;

- for example the distance is displayed in real time on the image, - for example the distance information makes it possible to guide a robot, - to accelerate the development of capture devices with variable focus or focal length: it is in fact possible to determine, from a single image, the servo setpoint to be applied to obtain the desired focusing, for example on the central subject or in a focusing zone selected by the user, - to take into account the distance of the various objects in a scene to adjust the power of a flash and in particular the main subject or the subject in the focusing zone, CA 02834883 2013-11-26 - to take into account the distance of the various objects in a scene for the self-exposure device of the capture device, in order for example to favor the main subject or the subject in the area focus selected by the user for a portrait, - to automatically define the main subject without needing to ask the user to define it.

10 Application to the control of depth of field, independently of the exposure With reference to FIGS. 4, 5, 6 and 7, an embodiment of the method and system according to the invention will now be described, more particularly suited to the control of the field. depth of field, without the need for a particular mechanical device for a known image capture device.

The method then makes it possible to obtain a sharp image for objects located at a distance from the capture apparatus corresponding to a range of sharpness and a blurred image for the other objects.

Usually, a capturing device has a limited depth of field and the smaller the aperture of the optics is, and therefore the depth of field and exposure are related so that a choice must be made in low light between 25 depth of field, noise and camera shake. Depending on the embodiment, it is possible to control exposure and depth of field separately.

According to the invention and as represented in FIG. 4, the digital image is broken down into regions 11.1 and 11.2, for example into square regions corresponding to 9 sensitive elements neighboring the sensor, or, more generally into regions corresponding to X by Y sensitive elements or in regions of predetermined or calculated shape according to the digital image.

The clearest color is then chosen for each region, by CA 02834883 2013-11-26 56 example as the color corresponding to the lowest value among the values ​​obtained by calculating a gradient for each color from the levels of gray corresponding to the color and the region considered.

In FIG. 4, the color corresponding to curve 8.2 is sharper for region 11.2 while the color corresponding to curve 8.3 is sharper for region 11.1.

The sharpness of the sharp color thus chosen is then reflected for each region on the other colors as described previously.

As we have seen previously, a sharp digital image is thus obtained for all the colors in a large range of depth of field.

To determine the distance between the capture device and the objects of the scene captured in the region of the digital image, we use:

- either, as described above, the sharpness of at least two colors for each region, - or another more precise distance measuring method or device.

The sharpness can then be reduced, for example by a Gaussian filter, or by a filter simulating a bokeh, in the regions and / or in parts of the field containing objects located at distances outside the desired sharpness range.

For example, for a portrait, we can obtain a blurred background which highlights the face without the need for a wide aperture optic.

For example for a landscape, one can obtain a great depth of field except possibly for isolated objects in the corners which can harm the readability of the image.

For example, for a scene with close objects in the corner due to poor framing, these nearby objects can be blurred.

For example, the depth of field can be left to the user, either CA 02834883 2013-11-26 57 in the device, or during post-processing on a computer.

Thus the depth of field is controlled without having to change the exposure, therefore without modifying the aperture, nor increasing the noise level or increasing camera shake.

The depth of field control carried out according to the invention benefits in particular fixed optics, in particular telephones.

The depth-of-field control allows both barcode, business card, or manuscript containing text and / or diagram to be taken as well as portraits or landscapes by an image capture device, in particular a phone or camera.

This is possible without using an expensive optical device with a large aperture.

In addition, this device can be produced fully automatically without any intervention by the user.

The depth-of-field control carried out according to the invention also benefits an apparatus comprising mobile optics, in particular a zoom.

An experienced hobbyist can thus directly or indirectly independently control depth of field and exposure.

FIG. 11 is a diagram illustrating the architecture of an apparatus for capturing or restoring images.

Such an apparatus, for example for capturing Images, comprises, on the one hand, an optical system 122, in particular with one or more optical elements such as lenses, intended to form an image on a sensor 124.

Although the examples mainly relate to a sensor 124 of the electronic type, this sensor can be of another type, for example a photographic film in the case of a so-called film camera.

CA 02834883 2013-11-26 58 Such a device also comprises a servo system 126 acting on the optical system 122 and / or on the sensor 124 to carry out a focusing so that the image plane is on the sensor 124, and / or so that the quantity of light received on the sensor is optimal by adjusting the exposure time and / or aperture, and / or so that the colors obtained are correct, by slaving the white balance.

Finally, the apparatus comprises digital image processing means 128.

As a variant, these digital image processing means are separate from the apparatus 120.

It is also possible to provide a part of the image processing means in the apparatus 120 and a part outside the apparatus 120.

The digital processing of the image is carried out after the recording of images by the sensor 124.

An image reproduction apparatus has a structure similar to an image capture apparatus. Instead of a sensor 124, there is provided an image generator 124 'receiving images from digital image processing means 128' and supplying the images to an optical system 122 ', such as a projection optical system. .

In what follows, for clarity of the description, reference will only be made to image capture devices.

The invention consists, according to one of its aspects, which can be used independently of the aspects previously described, from the capacities of the digital image processing means 128, 128 'to determine or select the parameters of the optical system 122, 122' , and / or the image sensor or generator 124, 124 'and / or the servo system 126.

The level of performance that can be achieved with each of the components of the apparatus when they are associated with digital image processing means has been shown in the diagram of FIG. 12.

These levels are represented by CA 02834883 2013-11-26 59 by the broken line 130 for the optical system, the broken line 132 for the sensor, the broken line 134 for the control, and the broken line 136 for the device.

Starting from these performance levels that can be obtained with digital image processing means, it is possible to choose performance levels for each of the components of the device which are, before processing, significantly lower than the performance levels obtained. after application of the treatment means.

It can thus be seen that the level of the performance of the optical system can be set at level 130 ', the levels of the performance of the sensor and of the servo system can be set at the levels, respectively 132' and 134 '.

Under these conditions, in the absence of digital processing, the performance level of the apparatus would be at the lowest level, for example level 136 'corresponding to the lowest level 130' for the optical system.

The digital image processing means are preferably those described in the following documents:

- Patent application EP 02751241.7 entitled:

Method and system for producing formatted information related to device faults in a chain of devices and formatted information intended for image processing means.

- Patent application EP 02743349.9 for: Method and system for modifying the qualities of at least one image coming from or intended for a chain of devices.

- Patent application EP 02747504.5 for: Method and system for reducing the frequency of updates of image processing means.

- Patent application EP 02748934.3 for: Method and system for correcting the chromatic aberrations of a color image produced by means of an optical system.

CA 02834883 2013-11-26 - Patent application EP 02743348.1 for: "Method and system for producing formatted information related to geometric distortions.

- Patent application EP 02748933.5 for: "Method and system for providing, in a standard format, formatted information to image processing means".

- Patent application EP 02747503.7 for: Method and system for calculating a transformed image from a digital image and formulated information relating to a geometric transformation.

- Patent application EP 02747506.0 for: Method and system for producing formatted information related to faults of at least one device in a chain, in particular blurring.

15 - Patent application EP 02745485.9 for: Method and system for modifying a digital image by taking its noise into account.

- Patent application PCT / FR 2004/050455 for:

Method and system for modifying a digital image in a differentiated and almost regular manner per pixel.

These digital image processing means make it possible to improve the quality of the images by acting on at least one of the following parameters:

- Geometric distortions of the optical system.

It will be recalled that an optical system can distort the images in such a way that a rectangle can be deflated into a cushion, with a convex edge on each of the sides or into a barrel with a concave shape on each of the sides.

- Chromatic aberrations of the optical system: if an object point is represented by three colored spots having precise positions relative to each other, chromatic aberration results in a variation in the position of these spots relative to each other , the aberrations being, in general, all the more important as one moves away from the center of the image.

CA 02834883 2013-11-26 61 - Parallax: when an adjustment is carried out by deformation or displacement of an optical element of the optical system, the image obtained on the image plane may shift.

The adjustment is, for example, a focal length adjustment, or a focus adjustment.

This defect is illustrated by FIG. 13 in which there is shown an optical system 140 with three lenses in which the center of the image has the position 142 when the lens 144 has the position shown in solid lines.

As lens 144 moves to assume position 144 ', shown in broken lines, the center of the image assumes position 142'.

- Depth of field: when the optical system is focused on a defined object plane, the images of this plane remain sharp as well as the images of objects close to this plane.

We call depth of field the distance between the closest object plane and the most distant object plane for which the images remain sharp.

- Vignetting: in general, the brightness of the image is maximum in the center and decreases as one moves away from the center.

Vignetting is measured by the difference, in percent, between the brightness at a point and the maximum brightness.

- The lack of sharpness of the optical system and / or of the sensor and / or of the image generator is measured for example by the parameter BXU as defined above.

- The noise of the image is generally defined by its standard deviation, its shape, and the size of the noise spot and its coloring.

- The moiré phenomenon is a distortion of the image which occurs when there are high spatial frequencies.

Moiré is corrected by configuring the anti-alias filters.

- Contrast is the ratio between the highest and lowest image brightness values ​​for which image details are still visible.

CA 02834883 2013-11-26 62 As shown in Figures 14a and 14b, it is possible to improve the contrast (figure 14a) of an image, that is to say to extend (figure 14b) the range of luminosities over which we can distinguish details.

This extension is carried out using in particular a contrast and noise correction algorithm.

A description will now be given, in relation to FIG. 15, of an embodiment of unifolmiser sharpness in the field of the image.

First of all, it is recalled that the image surface of an object plane does not constitute a perfect plane but has a curvature, called field curvature.

This curvature varies depending on various parameters including focal length and focus.

Thus, the position of the image plane 150 depends on the zone on which the focusing is carried out.

In the example shown in FIG. 15, the plane 150 corresponds to a focus at the center 152 of the image.

For focusing on an area 154 close to the edge of the image, the image plane 156 is closer to the optical system 122 than the image plane 150.

To simplify the servo system for focusing, the image plane is placed in a position 158, intermediate between positions 154 (corresponding to focusing on a zone close to the edge of the image), and 150 ( corresponding to a focus on an area in the center of the image).

The combination of the digital image processing means 128 with the servo control 126 for focusing makes it possible to limit the displacement of the plane 158 for focusing, which reduces the energy consumption of the servo system and makes it possible to reduce the volume of its components.

The blur properties have been shown in the diagram of FIG. 15a with a conventional focusing servo system in which the maximum sharpness is obtained at the center of the image.

Thus, in this diagram of FIG. 15a, the field of the image is plotted on the abscissa and the CA 02834883 2013-11-26 63 CA 02834883 2013-11-26 63 value expressed in BXU have been plotted.

With this classic servo system, the blur is, at the center, 1.3 and, at the edge of the image, 6.6.

FIG. 15b is a diagram similar to that of FIG. 15a showing the properties of a control of a device produced according to the invention, starting from the assumption that the digital image processing means make it possible to correct the blur up to 'to a value of BXU equal to 14.

The curve represented on this diagram of figure 15b thus presents, in the center of the image, a value BXU = 2.6 and the value of BXU decreases when one moves away from the center to then go up again until a value of 4 towards the edge of the image.

It will be recalled that this value is the limit so that the blur can be corrected by the digital processing means.

Thus, it is possible to obtain a clear image over the entire field of the image, whereas this is not the case with a device equipped with a conventional system.

In one embodiment, the digital image processing means comprise means for improving the sharpness such that they make it possible to dispense with a focus servo.

By way of comparative example, the diagrams of FIGS. 16a, 16b, 16c, and 16d show the characteristics of an apparatus obtained according to the conventional technique and those of an apparatus obtained with the method according to the invention.

The conventional device is a digital camera integrated in a mobile phone having a VGA sensor, that is to say a resolution 640 x 480 without focusing system.

The conventional apparatus has an opening of 2.8 while the apparatus obtained with the method according to the invention has an opening of 1.4.

FIG. 16a, which corresponds to the conventional apparatus, is a diagram on which the percentage of field of the image is represented on the abscissa, the origin corresponding to the center of the image. The ordinate represents V vignetting.

CA 02834883 2013-11-26 64 Figure 16b is a similar diagram for an apparatus obtained according to the invention.

In the diagram of figure 16a (conventional apparatus) the vignetting reaches the value 0.7 at the edge of the image while in the diagram of figure 16b it can be seen that the optical system of the apparatus according to the invention has a markedly greater vignetting, of the order of 0.3.

The correction limit of the algorithm used is 0.25.

In other words, thanks to the correction algorithm, it is possible to use a substantially larger vignetting optic.

FIG. 16c is a diagram representing on the ordinate the blurring, expressed in BXU, as a function of the field of the image (on the abscissa) for a conventional apparatus.

In this classic camera, the blur characteristic is 1.5 at the center and 4 at the edge of the image.

The diagram of FIG. 16d also represents the blurring for the optics of the apparatus obtained with the method according to the invention.

On the abscissa of this diagram of FIG. 16d, the field of the image is also represented and on the ordinate the blurring expressed in BXU.

It can be seen that in this diagram of FIG. 16d, the blur in the center of the image is of the order of 2.2.

It is therefore greater than the blur of the diagram in FIG. 16c.

On the other hand, on the edges, a blurring of the order of 3 was chosen, taking into account the limit of the correction algorithm.

In other words, surprisingly, we chose a degraded optics as regards the sharpness in the center, whereas we obtain the same results as with the conventional apparatus, with, in addition, a higher aperture.

It should also be noted that on the edges, the optics of the apparatus according to the invention represent a quality similar to that of conventional optics, this result being obtainable due to the degradation of the vignetting with respect to the classic optics.

The diagrams of FIGS. 17a and 17b show the characteristics of different optical systems CA 02834883 2013-11-26 between which the choice must be made in order to 'produce a capture device using the method according to the invention.

In the example shown in FIG. 17a, the optical system provides an image spot 1100 of small dimensions.

This system has a modulation transfer function (MTF) represented by a diagram where the spatial frequencies are on the abscissa.

The cutoff frequency value is fc.

The FTM function comprises a plateau 1110 in the vicinity of zero frequencies and a portion 10 rapidly decreasing towards the value fc.

The optic represented by the diagram of FIG. 17b, presents an image spot 1114 of dimensions substantially greater than the image spot 1100 and its MTF has the same cut-off frequency fc as in the case of FIG. 17a.

On the other hand, the variation of this MTF as a function of the spatial frequency is different: this frequency decreases relatively regularly from the origin towards the cutoff frequency.

For the choice of the optical system, it is based on the fact that the correction algorithm of the modulation transfer function is effective from a value of 0.3.

Under these conditions, it can be seen that with the optics of FIG. 17b, a correction is obtained which allows the MTF to be increased up to a value f2, for example, of the order of 0.8 fc whereas with 25 the optics of FIG. 17a, the correction is only possible up to a frequency fl of the order of 0.5 fc.

In other words, with a correction algorithm, the optic shown in figure 17b will provide more detail than the optic shown in figure 17a, and this despite the fact that the image spot is of larger dimensions than in the case of of Figure 17a.

The optics corresponding to FIG. 17b will therefore be chosen.

Application to the increase in depth of field CA 02834883 2013-11-26 66 We will now describe an alternative embodiment of the method for which the sensor and / or the optical system are more particularly suited to the increase in the depth of field .

Conventional CMOS or CCD sensors are often sensors formed from a so-called Bayer mosaic of pixels.

The Bayer mosaic consists of a succession of 2x2 pixels, formed of 2 green pixels (i.e. a photosite sensitive to light in a spectral range around 550nm), of a red pixel (spectral range around 600nm) and a blue pixel (spectral range around 450nm).

The spectral ranges are shown in Figure 2.

Depending on the sensors, the spectral bands of green, red and blue differ and have a greater or lesser overlap.

A strong overlap between these three bands has the consequence of reducing the sensitivity of the sensor to colors (it becomes "color blind"), but increases its general sensitivity to light and vice versa.

A strong overlap between the three bands also reduces the differences in sharpness between the colors, thus reducing in particular the range of distances at which at least one of the three colors is sharp.

Also advantageously, according to the invention, the spectral bands can be adapted, for example by reducing their overlap, so as to increase the range of distances for which at least one of the three colors is clear.

This adaptation could be carried out jointly with the design of the optics and, depending on the constraints relating to the digital processing of the image.

Description of a sensor optimizing the method according to the invention In an alternative embodiment of the method, the sensor and / or the optical system are more particularly suited to applications making it possible to provide precise indications of the distances of the imaged objects.

CA 02834883 2013-11-26 67 In this variant embodiment, a Bayer pixel mosaic is used.

It is common for sensors to have a large number of pixels to provide outlier digital values.

These pixels are commonly called "dead pixels" (or "burned pixels" in English).

Also, the digital image generation processing operations contain a step of filtering these outliers so that, on the generated image, the outliers of these pixels are erased and therefore not visible.

The accuracy of the distance measurements according to the method depends in particular on the variation in the relative sharpness as a function of the distance.

This variation depends on the quantity of chromatic aberration that it is possible to obtain with the capture system (sensor and optics).

However, the range of spectral frequencies of visible light, and therefore the light useful for a photograph, is relatively restricted: of the order of 400nm to 700nm.

Also, the variation in relative sharpness as a function of the distance is then limited with a conventional Bayer sensor.

Several ways of modifying a sensor to go beyond this limitation are possible.

A simple way is to use in addition to the three classic colors:

red, green and blue, a different spectral band, for example 800nm-900nm or any other band beyond and / or below the visible spectrum.

The pixels sensitive to this fourth spectral band will not necessarily be useful for the reconstruction of the visible image but will mainly be used to estimate the distance of the objects by comparison of the relative sharpness on this fourth spectral band with one, or several, of the three classic colors.

We can then advantageously arrange the pixels in the following way: starting from an arrangement of a classic red, green, blue Bayer, we substitute, all the NxM pixels, a few pixels by sensitive pixels in this fourth CA 02834883 2013- 11-26 68 spectral band.

By choosing N and M large enough (for example 64 each), and by substituting for example 9 pixels one thus ensures that only approximately 1 pixel per 1000 of the classic Bayer is affected.

Thus, during the construction of the image, these pixels can be considered as “dead pixels” and their values ​​filtered.

A photographic apparatus is thus obtained which makes it possible to provide more precise indications of the distance of the objects imaged every N × M pixels of the image.

Description of a second sensor optimizing the method according to the invention In another embodiment, shown in FIG. 20.2, one starts from a conventional Bayer in which three pixels R, G, B and one pixel U corresponding to one are provided. part of UV or infrared spectral band.

By infrared and / or ultraviolet, one can understand any part of the spectrum beyond or below the visible spectrum, in particular the near infrared such as 700 to 800 or 700 to 900nm, or the near ultra violet close to 400nm.

This U pixel is used to enhance the sharpness of visible colors as shown with the diagram in Figure 20.1.

On this diagram, we plotted: on the abscissa the distances "d" of the objects imaged to the capture device, and on the ordinate the diameter "D" of the blur spot.

The curves, 20.3, 20.4, 20.5, and 20.6 represent the variation of the diameter "D" as a function of the distance "d" for, respectively, the red "R", the green "G", the blue "B" and the ultraviolet "U".

Line 20.7 represents the sharpness threshold defining the depth of field.

Thus the distance “d1” represents the depth of field limit for a capture device comprising “RGB” pixels and not the U pixels while using the sharpness improvement method according to the invention.

The distance “d2” represents the depth of field limit obtained with a capture device comprising the sensor shown in FIG. 20.2 and making use of the sharpness improvement method according to CA 02834883 2013-11-26 69 of the invention.

The "U" pixels are only used to reflect the sharpness of the "U" color to the "RGB" colors for objects located between the distances "d1" and "d2".

Thus the final image will only include the three "RGB" colors (or any other known visible color distribution).

Alternatively, near infrared sensitive pixels are added to improve sharpness at greater distances.

Increase in longitudinal chromatic aberrations According to the invention, advantage is taken of the existence of variations in relative sharpness between two colors as a function of the distance from the objects.

Also, it is possible to design optics exhibiting relative sharpness between the three color planes which are very different depending on the distance.

Such optics are said to exhibit strong longitudinal chromatic aberrations.

In a practical way, we can for example design the optics so that over a large range of distances:

the smallest of the spot diagram diameters (diameter of the blur spot) between the three colors is below a first predetermined threshold and the largest of the spot diagram diameters between the three colors is below a second predetermined threshold.

Alternatively, we can use the value of the BxU instead of the diameter of the spot diagram.

The two thresholds are determined as a function, for example, of the capacities and constraints of the digital processing operations for generating the image, on the one hand (such as for example the size of the filter "F" described below), and of the characteristics of the sensor, on the other hand.

Figure 18 shows an example of the BxU (ordinate axis) measurements for the three RGB color planes as a function of distance (abscissa axis) for an optic designed for this purpose.

The values ​​shown are those in the center of the image field.

At each point of the image field, different but similar curves can be measured. S1 and S2 denote the two thresholds CA 02834883 2013-11-26 described above.

The range of distances satisfying the two criteria described above is then, for this optic, approximately 12cm-infinity (d1-> infinity in figure 18) which means that we will be able to reconstruct a clear image for 5 imaged scenes. within this range of distances.

With conventional optics we would have ended up with three curves close to the curve of the red color R in figure 18 and therefore to an optical system allowing sharp image reconstructions only for objects at greater distances 10 25cm - infinity (d2 H infinite in figure 18).

It is also possible to use an optic having longitudinal chromatic aberrations such that, for a given focus, aperture and focal length, there is at least one color for which the object distance of better sharpness is less than f2 than k --- , k being a coefficient less than 0.7, preferably 0.P 20 less than 0.5, f being the focal length, 0 the aperture and P the smallest (among all the colors of the image) of the diameters of the blur spot of an object point located at infinity.

Application to Automatic Resolution Adaptation We will now describe an alternative embodiment of the invention making it possible to automatically adapt the resolution of the image to any blurring associated with taking a picture outside the depth of field of the camera. capture.

When the imaged scene is too close (below the depth of field), the image is blurred, i.e. the spot diagram occupies a spot larger than X pixels in diameter , X being a predetermined parameter defining the depth of field limit.

A under CA 02834883 2013-11-26 71 digital image sampling (zoom out), will reduce the size of the blur spot by a factor depending on the type of sub-sampling used, but, typically, of the order magnitude of the sub-sampling factor considered.

We can then generate from the digital image a clear image, but of lower resolution by choosing the downsampling factor so that the blur spot is less, once the image is downsampled, than one. given threshold.

As a variant, in order to minimize the calculations, the first step is to carry out the sub-sampling described above before carrying out the increase in sharpness according to the invention.

Application to the modification of sharpness - Filtering As for the distance measurement (figure 8), the expected sharpness of each color of the digital image can be extracted from the relative sharpness measurement between two color planes.

According to an alternative embodiment of the invention, the processing comprises a modification of sharpness for each pixel of the zone Z 'by means of a filter mixing the values ​​of pixels over a predetermined neighborhood of each pixel, the parameters of the filter being a function of of the measured relative sharpness.

Indeed, an image capture device provided with an optic may present different sharpnesses according to the color planes and according to the distance of the imaged objects.

The dependence of the sharpness (or blur) on the distance of the imaged objects makes it impossible to increase the sharpness by means of a predetermined processing such as a predetermined sharpness filtering.

An alternative embodiment of the invention consists in choosing or adapting the sharpness filters to the relative sharpnesses measured.

A particular case, already described, of the adaptation of the filtering or modification of sharpness consists of the repercussion CA 02834883 2013-11-26 72 of the sharpness of the sharp color on at least one other improved color, this repercussion being carried out at the same time. 'using a calculation of the type CA = ON + F (CO - ON), where CA is representative of this improved color, CO is representative of the improved color before treatment, CN is representative of the clear color and F a filter, including a low pass filter.

But, more generally, it is possible to use a sharpness filter involving all (or a subset of all) the colors.

Thus, in the case of an RGB (or RGB) digital image to process the value of a pixel P, the filter M will be able to modify the value of the pixel P as a function of the values ​​of the pixels on a neighborhood of the pixel P on the set of three colors.

For example, by noting RN, GN, BN the digital data relating to the red, green and blue colors of the digital image, and RA, GA, BA the digital data relating to the colors of the enhanced image, we can choose the filter M conne an operator performing the following operations:

GA = GN + c GG * M GG (GN) + c GR * M GR (RN) + c GB * M GB (BN) RA = RN + c RG * MRG (GN) + c RR * MRR (RN) + c RB * MRB (BN) BA = EN + c BG * MEG (GN) + c BR * MER (RN) + c BB * M BB (BN), Where:

The M_ {R, G, B} {R, G, B} represent filters, which can be chosen as zero-sum linear filters, such as for example high-pass frequency filters.

The c_IR, G, BI {R, G, B} represent coefficients weighting the impact of each filter M_ {R, G, B) - {R, G, B}.

This filtering example can also reflect the sharpness of the sharpest color on others.

For example, assuming that only the color blue is sharp, the high pass filters M_IR, G, BI {R, G, B} will give values ​​close to 0, when applied to the colors green and red which are CA 02834883 2013-11-26 73 blurry in the example.

In this particular case, GA will therefore be equal to GN plus c GB * M GB (BN), that is to say GN plus the high frequencies of blue.

The green color thus inherits the sharpness of the net color (blue).

It is the same for the color red.

In practice the sharpness of the colors is not a binary factor, so the filters M IR, G, B1 {R, G, B} and the coefficients c_ {R, G, B} {R, G, B} can be adapted to the different possible values ​​of color sharpness.

An exemplary embodiment of such an adaptation, in the context of RGB images from a given capture device, is as follows:

We consider the relative sharpness values ​​of red with respect to green, and of blue with respect to green: V_GR, V_GB.

These values ​​are quantified in such a way that the values ​​thus quantified constitute an entry in a 2D array of reasonable size. With each input (couple of quantified V_BR, V BG values), we associate a set of filters M {R, G, BI {R, G, B} and a set of coefficients c (R, G, B} {R, G, B} adapted.

In the particular case, where one seeks to improve the sharpness of the digital image, one will be able, for each input, to predetelminate the filters M {R, G, B) {R, G, B} and a set of coefficients c_ {R, G, B} {R, G, B} such that a digital image, taken by the capture device and having the relative sharpness corresponding to the input is perfectly corrected for sharpness by the application of the M.

We can also refine the filter M so as to take into account the fact that the three colors do not vary on a large scale in the same way, in particular in colored areas (as opposed to black, gray or white areas) of the image. digital.

For this, it is possible for example to weight, in each pixel P of the zone Z ', the actions of the filters M_ {R, G, B} {R, G, B} by the amplitude of the relative variations of the colors on a neighborhood of pixel P.

CA 02834883 2013-11-26 74 In certain cases, the association table between relative sharpness considered and set of filters may include other entries such as, for example, the position of zone Z 'in the image field or shooting parameters such as the value of the focal length, aperture, focusing distance, etc., of the optical system at the time of shooting.

Indeed, it is usual that the sharpness characteristics of a digital image also depend on these factors.

Also, for the correction of sharpness of a digital image, the image field will first be cut into several zones Z ′ and the method will be applied to each of the zones.

The cutting will preferably be carried out as a function of the sharpness characteristics of the colors so that in each zone the sharpness of the colors is relatively homogeneous.

With this embodiment, an automatic adaptation of the sharpness filtering applied to the digital image is thus obtained, to the distance between the scene imaged and the capture device.

It should be noted that, thanks to the use of relative sharpness, this automatic adaptation to the distance can be done without the explicit knowledge of this distance.

Beyond the modification of the sharpness of the digital image, this realization of the method also allows the automatic adaptation of treatments aiming for example at the correction of optical defects and / or of the sensors whose effects on the image depend on the distance between imaged scene and capture device.

Blurring (or loss of sharpness) is one example, but other optical and / or sensor (s) defects, such as geometric distortions or vignetting, are other examples.

Principle of the invention Description of Figures 19.1, 19.2, 19.3 Figures 19.1, 19.2 and 19.3 represent steps of the method according to the invention according to several embodiments.

In FIG. 19.1, there is shown an image 10 comprising a region R and having two colors 195 and 196, a CA 02834883 2013-11-26 relative sharpness measurement 190 between the two colors 195 and 196 on the region R of the image 10, an action 191 controlled as a function of the measured relative sharpness.

Optionally the controlled action also depends on a mode 193 corresponding for example to a choice of the user of the apparatus, and / or a characteristic of the capture apparatus during the shooting.

In FIG. 19.2 there is shown an image 10 comprising a region R and having two colors 195 and 196, a relative sharpness measurement 190 between the two colors 195 and 196 on the region R of the image 10, an action 191 controlled as a function of the measured relative sharpness including processing the image 10 and producing a processed image 192.

As an option, the controlled action also depends on a mode 193 corresponding for example to a choice of the user of the device, and / or a characteristic of the capture device during the shooting.

In FIG. 19.3 there is shown an image 10 comprising a region R and having two colors 195 and 196, a relative sharpness measurement 190 between the two colors 195 and 196 on the region R of the image 10, an action 191 controlled according to the measured relative sharpness including processing another image 194 and producing a processed image 198.

As an option, the commanded action also depends on a mode 193 corresponding for example to a choice of the user of the apparatus, and / or a characteristic of the capture apparatus during the shooting.

Application to the modification of the contrast and / or of the luminosity and / or of the color and / or of the sharpness An embodiment of the invention will now be described in which the action control consists in modifying the contrast and / or the brightness and / or the color of the image as a function of the relative sharpness between at least two colors out of at least one region R of the image.

CA 02834883 2013-11-26 76 The use of the relative sharpness between at least two colors on at least one region R of the image, directly or indirectly (for example with a step of estimating the geometry, in 3 dimensions , of the scene), allows for example to simulate the addition of localized lighting, for example a flash positioned anywhere in the scene, and / or, conversely, to reduce the effect of a flash or lighting of various colors in the scene.

Thus we can reduce the effects of backlighting, and flattening of light associated with the flash.

In one embodiment, the digital image is divided into regions according to the relative sharpness between at least two colors, so that each region images a part of the scene lying within a given distance range and which is oriented in a given direction.

An indication of the direction can be obtained from the local variation of the relative sharpness in the image.

An indication of the distance can be obtained from the relative sharpness as previously described.

Relative sharpness and its variation can also be used directly without going through a distance and an orientation.

To add or modify lighting, we can then determine, for each region, the quantity and color of light to add or subtract at each point since we know the distance to the simulated source from the imaged point and the orientation. of the imaged object relative to the source.

In one embodiment, the three-dimensional geometry of the scene is reconstructed by measuring the distance at a large number of points in the image.

A technique known in image synthesis is then used to add lighting to the scene (ray tracing or other).

In one embodiment, lighting is added on the main subject (s) adapted to each subject to cause a "fill-in" effect simulating one or more flash (s) positioned in front. or on the side of each subject.

This operation can be carried out automatically and independently for each subject.

With the known technique, adding lighting for each subject is only possible with studio lighting.

Similarly, the flash output can be determined based on the closest subject to illuminate it properly, and supplement the lighting of other subjects by adding simulated lighting.

It is also possible to determine for each region the color of the lighting by a known method of estimating white balance and then make the color of the lighting of the scene uniform.

In the state of the art, the white balance is estimated globally for lack of information on the 3-dimensional geometry of the scene.

Description of Figures 18.1 and 18.2 Figure 18.1 shows a sensor 2, producing a raw image 180 subject to a pre-processing 181, for example a white balance, and / or a black level compensation, and / or noise reduction to produce a preprocessed image 182.

There has also been shown a relative sharpness measurement 190 knowing an action 191 corresponding to a processing implementing the preprocessed image 182 and the relative sharpness measurement 190, to produce a processed image 192.

Finally, a downstream processing of the processed image 192 has been shown, corresponding for example to demosaicing or to other processing necessary to convert a raw image into a visible image.

Figure 18.2 shows a sensor 2, producing a raw image 180.

There is also shown a relative sharpness measurement 190 controlling an action 191 corresponding to a processing implementing the raw image 180 and the relative sharpness measurement 190, to produce a processed image 192.

Finally, a downstream processing of the processed image 192 has been shown, corresponding, for example, to demosaicing or other processing necessary to convert a raw image into a visible image.

CA 02834883 2013-11-26 78 In one variant, the action implements processing on a visible image.

Simplification of optics The invention applies to an apparatus comprising variable parameters when the digital image is captured and having an influence on the sharpness of the colors, in particular a capture apparatus with zoom, and / or an optic with variable point and / or a variable opening.

One then uses the curves of sharpness 8.2 and 8.3 corresponding to the value of the variable parameters according to the digital image.

As described, the invention makes it possible to restore the focus digitally without a moving group and instantaneously, which therefore makes it possible to reduce the complexity of a zoom by eliminating at least one moving part.

For example, depending on the distance from the subject and the focal length, the relative sharpness between two colors can be variable, whereas this is not acceptable in known optics.