Device and method for superimposing patterns on images in real-time, particularly for guiding by localisation

The invention relates to the field of image data processing, specifically the registration in real time image data representative of known observation on image patterns.

In a number of areas, it is important to know precisely, at every moment, how one is positioned relative to a place or object, or more generally a region.

This is for example the case in the field of surgery, particularly when it implements a technique known as "minimally invasive." In this case, the operating field is observed by an endoscopic camera inserted into the patient's body and delivering images on a or more monitors (or observation glasses) against which (which) is installed a surgeon. In the case of a guided procedure by a robot, the remote pilot surgeon manipulative robot arm the ends of which are also introduced into the patient's body. This is particularly the case of the Da Vinci ® installation Intuitive Surgical Inc., which has, firstly, a stereoscopic display, allowing the surgeon to visualize three-dimensional (3D) the intervention area, and secondly, steering levers and foot pedals allowing the surgeon to direct control instructions and control the robot.

This surgical technique is particularly useful for the patient insofar where it is minimally invasive. But it is particularly difficult to implement because it offers the surgeon, first, a partial vision and deformed part of the area in which he has to intervene because of the use an endoscopic camera, and secondly, a small intervening space and hindered by the manipulator arms of the robot and the endoscopic camera. Moreover, some regions, such as the heart, being animated, it increases the difficulty of the operation.

To improve the situation, it was proposed to implement a preoperative modeling phase. In such a pre-operative phase, firstly performs a three-dimensional modeling, and optionally time, of the object area of ​​intervention from images obtained by medical imaging. In the case of a heart, it also determines the coronary network using angiographic sequences and then performs a resynchronization coronary this network on the surface of the core obtained by MRI. then determining an anatomical model of the part of the patient body containing the object region of the intervention, always from images obtained by medical imaging.

Then, it is determined, firstly, the optimal cut point given the anatomical model and parameters such as dexterity and accessibility of the target area, and secondly, the optimal configuration of the arm robot, so as to avoid collisions and to obtain maximum separation, in particular.

Once the pre-operative phase is complete, the surgeon can perform the procedure. The patient is then installed on the operating table, and then the endoscope is calibrated using a grid placed on the operating table and observed from different points of view. The patient's body is then incised at points of optimal incisions previously determined. Then, the robot arm are placed in the previously determined optimal configuration, and ends, like that of the endoscopic camera, are introduced into the patient's body through the incisions. The procedure can begin.

Despite the pre-operative phase, the surgeon may still have difficulty pinpointing the area of ​​intervention. This can especially occur when working on a body such as the heart. It may indeed be difficult to locate the coronary artery due to an excessive amount of fat on the surface of the epicardium. It is also possible to confuse the marginal branch of the circumflex artery or the diagonal branch (highly developed) with coronary artery due to the magnification of the endoscopic camera and / or the smaller field of view and / or low recoil possible and / or poor positioning of the opening made in the pericardium.

In addition to these difficulties in identifying operating targets, the surgeon encounter difficulty locating the endoscopic camera and therefore to estimate the angle at which he observes the intervention region. Finally, lacking tactile feedback, it is not possible to determine the area to be operated by exerting on it the pressure with the ends of the robot arms.

In summary, the surgeon is experiencing real difficulties in determining the precise position of the zone (or portion) of the observed region in which it is to intervene in relation to the known positions of the ends of the arms of the robot intervention.

To further improve the situation, it has been proposed to assist the surgeon by superimposing on the endoscopic images of the region observed a pattern representative of a feature of the area portion which has to perform the procedure, or an adjacent portion. These patterns are previously extracted from the numerical models, made from images obtained by medical imaging. But such assistance is not enough, especially when the intervention area has several characteristics substantially identical elements (either naturally or because of the viewing angle), as in the case of the coronary network.

Similar difficulties of accurately determining the positions of places or objects, via real images portions, also occur in other technical fields, especially in the field of urban guide.

The invention therefore aims to remedy all or part of the aforementioned drawbacks.

It proposes for this purpose a dedicated device registration in real time patterns (for example three-dimensional (3D)) known characteristics of a region, on images (real) in this region.

This device is characterized in that it comprises, on one hand, a memory in which are stored patterns representative of portions of a selected region and position and orientation known in relation to a common reference, and secondly, processing means, on receiving the designation of at least one observation image portion of the selected region, taken on a selected angle, and at least one representative attribute of this portion to determine in the memory a pattern representative of the designated part, given the designated attribute and then readjust the determined pattern on the designated image portion considering the selected angle.

The device of the invention can comprise other characteristics that can be taken separately or in combination, including:<ul><li>* Processing means capable of performing the resetting by successive designations of observation of images of portions of the selected region and / or attributes representative of these portions,</li><li>* A memory capable of storing a representative three-dimensional model of the area within the selected repository. In this case, the processing means are preferably responsible for carrying out their (s) registration (s) from, in particular, the stored three-dimensional model,</li><li>* Processing means capable of determining at least one measurement equation from the designation of an image portion, of an attribute of at least one hypothesis, and proceed to the determination of the pattern and its resetting according to the determined measurement equation. Several equations of measurement can be determined from the designation of several portions of images, attributes and assumptions. In this case, the processing means carry out the determination of the pattern and its registration as a function of a combination of measurement equations determined. Resetting is for example of rigid type by minimizing a criterion selected taking into account the measurement equations derived from the hypotheses. Furthermore, the hypotheses can be transmitted to the processing means by the user performing the designations, or be directly determined by the processing means,</li><li>* Processing means capable of delivering image data representative of a motifrecalé so that we can observe the same time as the observation image, and superposed on the corresponding designated image portion, once the registration done,</li><li>* A memory capable of storing a table of correspondence between patterns and information data representing them. In this case, the processing means are advantageously responsible for issuing, whenever they are ordered, the information data representing a pattern flunked,</li><li>* Processing means able to deliver substantially the same time, whenever they are ordered, representative image data of a pattern flunked and information data representative of that reason.</li></ul>

The invention further relates to a location by guide means comprising, firstly, means of observation can deliver images of a selected region under a selected angle, a second hand, a display (e.g. one or more computer monitors or observation glasses) to display these images, thirdly, a man / machine interface enabling a user to designate at least a portion of the region represented by the images displayed and at least one attribute representative of that portion, and fourthly, a registration device of the type described above, fed portion (s) and attribute (s) designated by the man / machine interface and outputting representative image data of a so that the latter motifrecalé is superimposed on the display on the displayed designated portion, once the registration made.

The observation means may include means for acquiring, eg endoscopic type, whose position is known at every moment against a calibration reference, from which is defined the position of the observed region, and able to delivering observation images on the display.

When the installation is only used for guidance, for example urban, it preferably comprises control means, on receiving a request designating a pattern of the observed region to order the registration device to determine data position representative of the position of the pattern relative to the calibration reference, given the resetting and then determine control instructions for guiding the user to the portion which corresponds to this motif.

When the installation is used for surgical procedures, it may include an intervention robot having arms whose respective positions relative to the calibration reference are known at every moment and that can be remotely controlled by instructions provided by a user via man / machine interface. But it also includes control means coupled to the registration device and the human / machine interface and loaded, when they receive a request designating a pattern of the observed region, first, to order the device resetting to determine position data indicative of the position of the pattern relative to the calibration reference given the registration, and secondly, to determine control instructions for moving the robot arm in the vicinity of the portion region corresponding to the designated pattern.

The invention also relates to a process dedicated to registration in real time patterns (eg 3D) known characteristics of a region, on images of this region, and of:<ul><li>* Store in a memory representative patterns of portions of a selected region and position and orientation known in relation to a common reference,</li><li>* Observe an angle selected the chosen region and deliver real-time observation of this region images,</li><li>* Designate at least one portion of image observation of the selected region and at least one representative attribute of this portion,</li><li>* Determine the memory a pattern representative of the designated part, given the designated attribute, and</li><li>* Readjust the pattern on the designated image portion considering the chosen angle.</li></ul>

The method according to the invention can comprise other characteristics that can be taken separately or in combination, including:<ul><li>* Can perform the resetting by successive designations of observation of images of portions of the selected region and / or attributes representative of these portions,</li><li>* Can be stored in memory a 3D model representative of the region in a reference frame, and perform each registration at least from the 3D model,</li><li>* We can make a rigid type registration by minimizing a selected criterion. In this case, preferably determining at least one measurement equation from the designation of an image portion, of an attribute of at least one hypothesis, and one proceeds to the determination pattern and rigid registration depending on the determined measurement equation. We can determine several measurement equations from the designation of several portions of images, attributes and assumptions constituent constraints, and make the determination of the cause and its rigid registration based on a combination of measurement equations determined. At least some of the assumptions may be transmitted by the user performing the designations, such as attributes,</li><li>* Can be delivered representative image data of a pattern readjusted so that it is observed together with the observation image, and superposed on the corresponding designated image portion,</li><li>* Can be stored in the memory a table of correspondence between patterns and information data representing these reasons, and deliver on request of information data representative of a pattern,</li><li>* Representative can view the image data of a pattern flunked,</li><li>* We can make the appointments via a man / machine interface,</li><li>* When receiving a request from a user designating a pattern within the observed region, one can determine position data representative of the position of the pattern relative to the calibration reference (from which is defined the position of the observed region), given the registration and then determine control instructions to guide the user to the portion corresponding to said designated pattern,</li><li>* It can provide an intervention robot having arms whose respective positions relative to the chosen reference are known at every moment and can be remotely controlled by instructions provided by a user via the man / machine interface, and when one receives a user request designating a pattern of the selected region, one can determine position data indicative of the position of the pattern relative to the calibration reference, given resetting, then determine control instructions for moving the robot arm in the vicinity of the area portion corresponding to the designated pattern.</li></ul>

Other features and advantages of the invention will appear on examining the detailed description below and the appended drawings, in which: <ul><li>the <figref idrefs="f0001">figure 1</figref> schematically illustrates an embodiment of an installation according to the invention, suitable for application in the field of minimally invasive telesurgery,</li><li>the <figref idrefs="f0002">2</figref> schematically illustrates a 3D pattern representative of a coronary artery,</li><li>the <figref idrefs="f0002">3</figref> schematically illustrates a 3D pattern representative of three adjacent coronary arteries,</li><li>the <figref idrefs="f0002">4</figref> schematically illustrates a 3D pattern representative of a coronary artery and three bifurcations,</li><li>the <figref idrefs="f0002">5</figref> schematically illustrates a 3D pattern representative of an angular sector at a bifurcation between two coronary arteries, and</li><li>the <figref idrefs="f0002">6</figref> is a picture of a heart on which was superimposed a 3D model, after resetting.</li></ul>

The attached drawings may not only serve to complete the invention, but also contribute to its definition, if appropriate.

The invention relates generally retiming real-time image data representative of known units, for example three-dimensional (3D), characteristic of a region, on the observation of this region images. But it also concerns localization guide installations using such a registration, such as guiding urban facilities and telesurgery, especially type "minimally invasive."

firstly refers to the <figref idrefs="f0001">figure 1</figref> to describe an exemplary embodiment, without limitation, a plant of the invention suitable for the minimally invasive telesurgery.

The illustrated telesurgery system consists for example from the facility Da Vinci ® company Intuitive Surgical Inc. She schematically includes a PC control station with a chair 1 C allowing a surgeon to settle at a console equipped with a keypad (not shown), a first manual control 2 for the left hand, a second manual control 3 to the right hand, a display device 4, here stereoscopic type and a set of five control pedals.

Each manual control 2, 3 comprises for example a control lever 6, 7 (type "joystick") for driving one of the manipulator arms 8, 9 of a robot 10, which will be discussed further, and the manipulator arm 11 of a stereo camera 12, on which also return later, and one or more control buttons 13,14 (for sensitive types, or push button, or "mouse").

The entire pedal 5 has such a pedal for combining manual control 2, 3 steering the intervention robot 10, a pedal for combining manual control 2, 3 to the camera control 12 and a pedal for combining manual control 2, 3 of the steering control module 15 of the installation, which will be discussed further.

The keypad, the manual controls 2 and 3, and all 5 control pedals are the man / machine interface.

The display device 4 comprises, here, a first screen 16 for display of two-dimensional (2D) images actual delivered by a first path of the camera 12 and for the left eye surgeon C, and a second screen 17 for display of two-dimensional (2D) images actual delivered by a second route of the camera 12 and intended for the right eye surgeon C.

The intervention robot 10 is designed to be placed in proximity of the operating table 18, which is installed on the patient P to be operated minimally invasive manner. It typically includes two control arms 8 and 9 provided with ends adapted to the operation and intended to be introduced into the patient's body P via the incisions.

The stereo camera 12 comprises a manipulator arm 11 whose end supports two optical fibers 19 defining two channels endoscopic image acquisition.

The intervention robot 10 and the endoscopic camera 12 can be combined to form a "master robot".

The installation further comprises a control assembly 20, for example arranged in the form of a workstation, comprising the control module 15 and a registration device D of the invention, which will be discussed further. The control module 15 is coupled to the console of the PC control station, the intervention robot 10, the stereo camera 12 and registration device D.

The registration device D, according to the invention is intended, in general, to fail in real time of known patterns that are characteristic of a region (here in which must perform an operation), on pictures (actual) in this region. In what follows, it is assumed that the patterns are three-dimensional (3D), but they could be two-dimensional (2D), at least for some of them.

This device D comprises first memory 21 stores three-dimensional patterns (3D) representative characteristics portions of the region in which the intervention is to take place, and known position and orientation relative to a common reference ( or preoperative repository). The device D also comprises a processing unit 22 coupled to memory 21 and loaded, when it receives the designations of the one part, at least part of an image observation of the intervention area, taken under an angle chosen by the endoscopic camera 12 and delivered by the control module 15, and secondly, at least one attribute representative of the designated part, to determine in the memory 21 a 3D pattern which is representative of this portion designated, given the designated attribute and the selected shooting angle, and to reset the 3D pattern determined on the designated image portion.

It is important to note that a set of 3D patterns can be a 3D model. Therefore, the registration may include not only a 3D pattern, but also on a 3D model.

For example in the case of the heart, a 3D model can be representative of the coronary tree, this 3D model then being constituted of a plurality of 3D patterns representative of characteristic structures of the coronary tree, such as arteries, junctions and bifurcations. In fact, in the case of a coronary tree, we define two types of structures. One type includes graphs showing the arteries, while a second type consists of features, such as junctions or bifurcations. These different types are illustrated in<figref idrefs="f0002">Figures 2 to 5</figref>.

More specifically, the <figref idrefs="f0002">2</figref> illustrates a representative pattern of a coronary artery, <figref idrefs="f0002">3</figref> illustrates a representative pattern of a configuration of three coronary arteries, the <figref idrefs="f0002">4</figref> illustrates a representative pattern of a configuration of three bifurcations on a coronary artery, and <figref idrefs="f0002">5</figref> illustrates a pattern representative of an angle α characteristic of a bifurcation between two coronary arteries.

For example, a bifurcation is defined by a first hand by an index (integer identifying the bifurcation), second hand by two index (Art1 and Art2 that identify both affected arteries), and a third hand, by a point (coordinate triplet of the bifurcation in the preoperative repository). Similarly, an artery is defined, firstly, by an index (integer identifying the artery), and secondly, by a set of parameters of a B-spline component centerline of the artery ( quadruplet (xi, yi, zi, ui) wherein (xi, yi, zi) is a triplet defining each artery checkpoint in the preoperative repository, and ui denotes a node of the B-spline whose value is between 0 and 1).

Obtaining 3D patterns not covered by the invention, it will not be described here in detail. A precise description of a method of production is for example given in the article<nplcit id="ncit0001" npl-type="b"><text>Eve Coste-Way et al "Optimal Planning of robotically Assisted Heart Surgery: Transfer Precision in the Operating Room" B. Siciliano and Dario P. (Eds.): Experimental Robotics VIII, STAR 5, pp. 424-434, 2003</text></nplcit>, Or in the document <nplcit id="ncit0002" npl-type="b"><text>Fabien Mourgues et al "3D + t Modeling of coronary artery tree from standard non-simultaneous angiograms" Proc. of MICCAI, Volume 2208 of LNCS, Springer (2001), 1320-1322</text></nplcit>.

Suffice to say that getting 3D patterns of an area of ​​intervention, such as an organ such as the heart, first of all requires a three-dimensional modeling, and possibly temporal (3D + t), the area of ​​intervention from images obtained by imaging (MRI, scanner, etc.). For example, in the case of a core, obtainable by MRI its volume (3D) model at a given instant of the cardiac cycle. For a complete 3D model + t, it animates the volume model (3D) from a 3D model + t coronary heart modeled network, obtained from angiographic image sequences (X-rays), also called coronary angiography , taken from different angles and for several cycles, and synchronized with an electrocardiogram (ECG).

In the above example of a 3D model of coronary tree, 3D motifs are fractions (or portions) of the tree characteristics (or network) 3D coronary whose positions are known relative to the volume model (3D) the patient's heart P.

Preoperative reference against which are defined the positions of 3D patterns and the heart is usually the one of the patient's outer shell P. This is in effect with respect to this outer envelope that can be calibrated the intervention robot 10 and the endoscopic camera 12. further, the reporting of the position of the heart, and therefore its coronary network, with respect to the outer casing enables to readjust the patient relative to the robot in the operating room.

It also determines a 3D anatomical model of the patient's body, at least in part a vast region containing the object of intervention, always from images obtained by medical imaging.

The positions, orientations and configurations reasons (and models) 3D are stored in the memory 21 with respect to a common reference frame defined from markings on the patient's outer shell P.

Preferably, it also stores in memory 21 the 3D model of the patient's heart P and its 3D anatomical model.

The invention not having to subject the pre-operative planning stage of the surgical operation, the latter will not be described here. Therefore, described in the following implementation of the device D according to the invention, in an installation according to the invention, in the operating phase.

Suffice to say that the preoperative planning stage, which is optional, is inter alia to determine the optimal incision three points, which will allow to bring the ends of the manipulator arms 8, 9 and 11 of 10 intervention robot and the endoscopic camera 12, given the 3D anatomical model of the patient's body and P parameters, such as dexterity and accessibility of the target area. It also involves determining the optimal configuration of the manipulator arms 8 and 9 of the intervention robot 10 to avoid collisions and to obtain maximum separation.

The device D is involved in the installation once the arm 11 and the endoscopic camera 12 (possibly endoscopy) were calibrated. It is, here, to precisely determine the optical parameters of the camera and its position in the reference arm 11. An example of endoscopic calibration method is described in the document<nplcit id="ncit0003" npl-type="b"><text>F. Mourgues et al "Flexible calibration of actuated stereoscopic endoscope for overlay in robot assisted surgery", Proc. of MICCAI, Volume 2488 of LNCS, Springer (2002), 25-34</text></nplcit>. The calibration data are preferably stored in a memory of the control module 15.

Is then carried out incisions in the body of the patient P, at the incisions optimal points determined during the preoperative planning phase. The control arms 8 and 9 of intervention robot 10 and that 11 of the endoscopic camera are then placed in the optimal configuration, also determined during the preoperative planning phase, and their respective ends are introduced into the patient's body P by the incisions. It is recalled that the determination of the optimal incision points is a preferred option, but not mandatory, the positioning of the robot arm and the endoscope can be empirically determined by the surgeon.

Both lanes of the endoscopic camera 12 deliver their respective 2D image sequences to the control module 15, which transmits them to the display device 4 so that they are displayed on the screens 16 and 17. The surgeon can C then observe the region in which is located the heart H purpose of the intervention, in terms of observation of the endoscopic camera 12. the <figref idrefs="f0002">6</figref> is an image of the heart H of the kind that are observed by the surgeon C on screens 16 and 17.

The intervention region being known, one can from the beginning of the intervention propose a first overlay of modèle3D (here of the coronary tree) on the observation images displayed. This can be done manually or by an external patient registration, installed on the operating table, with its pre-operative model. The external resetting consists firstly to point with the end of the robot more radiopaque markers previously pasted on the patient's chest, and previously segmented in scanners images. Then, one calculates the rigid transformation between the pointed markers and segmented markers. This initial rigid transformation toggles between preoperative repository (repository where are represented the grounds of the 3D model to fail) to the repository of the robot base, and the repository of the endoscopic camera (then using the calibration of the endoscope). This external recalibration technique is described in the document<nplcit id="ncit0004" npl-type="b"><text>E. Coste-Way et al "Optimal Planning ofrobotically assisted heart surgery: Transfer accuracy in the operating room, B. Siciliano and Dario P., eds, Springer Tracts In Advanced Robotics, Experimental Robotics VIII, Volume 5, Springer (2002) 424-434</text></nplcit>.

Once the external recalibration eventually made it pinpoints the area where the intervention is to take place. To do this, the surgeon C denotes at least a portion of the images displayed on the screens 16 and 17 and at least one representative attribute of each image portion using the man / machine interface. The designation of a portion is performed for example by selecting the portion of the image with a mouse. The appointment of an attribute is carried out either by voice command or by selecting from a list displayed on the screens 16 and 17 or on an auxiliary display. In the example illustrated in<figref idrefs="f0002">4</figref>, Squares A1 to A3 materialize the places in which the surgeon C "click" with the mouse to designate three portions of the region that seem significant to him. Similarly, in the example illustrated in<figref idrefs="f0002">5</figref>The square B materializes the place where the surgeon C "click" with the mouse to designate the portion of the area which seems significant to him.

The attributes are information types (or classes) that describe each known local characteristic or a known local configuration, or general information leading to the processing module 22 of the device D to determine the memory 21 3D pattern matching .

Once in possession of the designations provided by the surgeon C, the device D transmits them to its processing module 22, so that it determines in the memory 21 a 3D pattern which seems to him the representative or designated portions, given (or) the attribute (s) designated (s).

This determination is performed by a 23 extraction module coupled to the memory 21. Then, the processing module 22 must perform the registration of this pattern (and possibly of the entire 3D model to which it belongs). This is to determine how to orient and position the pattern so that it can be superposed on the portion designated by the surgeon, taking into account the angle at which the intervention area is observed by the endoscopic camera.

The registration is preferably performed by a registration module 24 of the processing module 22. Furthermore, this recalibration is preferably rigid, for minimizing a criterion constructed from measurement equations. The measurement equations are deduced here on assumptions provided by the surgeon, when he thinks recognize a portion, such as an artery or arteries bifurcation between known or directly determined by the registration module 24. Of course, another type of registration could be considered, including a registration refined.

For example, the rigid registration is to determine at least one measurement equation from the designation of an image portion, an attribute and at least a hypothesis about the identity of the pattern determined by the modulus extraction 23. This resetting can also take into account the 3D model of the heart when it is stored in the memory 21.

Several equations of measurement can be determined from the designation of several portions of images, and several attributes of one or more assumptions that act as constraints. In this case, the registration module 24 generates several sets of corresponding equations to the various possible hypotheses and then optimizes the registration parameters by minimizing a criterion. It then classifies readjustments obtained for relevance and selects the best registration. The method of estimation parameters is particularly described in the document<nplcit id="ncit0005" npl-type="s"><text>Vieville T. et al "Implementing a multi-model estimation method", The International Journal of Computer Vision, 44 (2001) 41-64</text></nplcit>.

But any other known parameter estimation technique can be used.

Once the rigid registration of 3D pattern performed, it is possible to reset the entire 3D model which includes said pattern. Thus, the entire 3D model, seen from the angle of observation of the endoscopic camera, not just the 3D pattern determined and readjusted, which can be superimposed on the image displayed or observed. Of course, some portions of the 3D model may not be visible due to the angle of observation.

The processing module 22 can then transmit to the control module 15 the position data of the 3D flunked pattern (or the entire 3D model to which it belongs) in the standard (calibration) of the displayed image (or observed when the surgeon is equipped with observation spectacles), and the image data that define this pattern (or model) 3D, so he orders the display device 4 a superposed display of the images observation issued by the endoscopic camera 12 (or observed in glasses).

A superposition of a large 3D pattern on a core image is illustrated in <figref idrefs="f0002">6</figref>. The square D1 and D2 will materialize the places in which the surgeon C. clicked his mouse to designate two servings in the intervention area which it considered significant, and the fraction of the coronary network superimposed on the heart of the image H materializes there 3D pattern flunked by the device D.

With this superposition, the surgeon knows immediately how it is positioned relative to the area that it must operate.

In some situations, the designations made by the C surgeon may not allow the extraction module 23 to determine the 3D pattern that corresponds to the selected portion or the registration module 24 to perform proper registration while causing poor superposition of the pattern (or model) 3D on the displayed image. Therefore, the processing module 22 may be arranged to precisely determine the position of the observed region in the repository by successive recalibrations each resting on the extraction of a new 3D pattern consecutively to the designation of at least one another portion of observation images and at least one attribute representing this other portion.

In this situation, two cases can be considered. In one case, the 3D pattern determined by the extraction module 23 is not superimposed on the image of the observed region to the selected portion of the C surgeon or the 3D pattern is superimposed flunked, but it is not to the structure observed by the surgeon on the selected portion. The surgeon must then complete a new designation for converging the registration operation. In a second case, the control module 15 that automatically discovers the error and sends a message to the surgeon, requiring its share of new designations.

Upon receipt of these new designations, the resetting device D reiterates the treatments discussed above, taking into account the new and old designations. It may in particular when it is arranged for this purpose, calculate new hypotheses and measurement equations.

One can also consider storing in the memory 21 a table of correspondence between the 3D patterns and information data representing them. This may be for example the name of the pattern, such as the name of an artery and / or its possible ramifications, or the coordinates of a target point, or recorded operational data during the phase planning, such as the identification of a stenosis or calcification area.

In this case, the processing module 22 may be configured to output the information data associated with a 3D pattern (readjusted or not) when it receives the order of the control module 15 (e.g. in case of asks the surgeon C). But it is also possible an automatic operation in which the processing module 22, each time he flunked a 3D pattern and he is about to deliver the position data and image control module 15 , extracted from the memory 21 the information data associated so as to communicate substantially simultaneously.

The control module 15 may also be designed, each time it receives a request designating a 3D pattern of the observed region, so as to, on the one hand, direct the registration device D to determine position data indicative of the position of the 3D pattern with respect to the calibration reference, considering the registration, and secondly, determine control instructions for moving the robot arms 8 and 9 of intervention robot 10 in the vicinity of the portion of region corresponding to the designated 3D pattern.

The system presented above can be used for other types of operation, such as liver or breast open surgery. Generally, the installation of the invention can guide a user in accomplishing its task in an environment where it has only a partial and / or distorted vision, and / or in an environment difficult.

Furthermore, as indicated previously, the registration device D according to the invention can be used in other applications than those presented above. It can especially be used in dedicated facilities at single guide by location (without intervention from a ROV), particularly in urban areas.

It is possible to envisage an embedded installation in a motor vehicle, and comprising one or more cameras, providing a real image control module of the environment so that they are displayed on screens in the passenger compartment, and a registration device D coupled to said control module. Installation can be also coupled to an onboard guide device by satellites, the GPS type.

In this application, the registration D device stores in its memory representative environmental reasons (or region) in which can move the vehicle and determined from digital recordings previously realized. These patterns whose positions are defined in a chosen benchmark, for example building facades, or notable buildings or sites, or statues or artwork. The memory may also include a solid model of the environment, and a correlation table between patterns and information data on these grounds.

Once the cameras calibrated against a calibration reference, the installation can be used.

The control module is in charge here, when it receives a passenger vehicle a request designating a pattern of the observed region (it has selected in the list of patterns stored using a mouse or exercising pressure on a touch screen display), to order the registration device D to determine position data representative of the position of the pattern relative to the calibration reference, given induced by resetting the offsets between the reference selected and the calibration mark, and then determine control instructions to guide the vehicle driver to the portion of the region corresponding to that ground. As indicated above, the control module may optionally be based on the vehicle position data delivered by the GPS device, to determine the control instructions. When the vehicle arrived on site, the pattern can be optionally superimposed on the real image of the area portion, and information associated with said data pattern can be issued to the passenger.

But it is also possible an automatic operation in which the registration D device receives a passenger vehicle designations of at least a portion of observed and displayed region (he selected with a mouse or by pressing on the touch screen display) and at least one representative attribute of this portion, such as the nature of a building (home, school, town hall, museum, church) or place (garden, park, square) or object (statue, sculpture, artwork), and determines in its memory the pattern that best matches the designated part, given the attribute nominated and selected shooting angle. Then he realigned the determined pattern on the designated image portion. He then deduces the position data flunked reason, the repository of the displayed image (calibration reference), and transmits them, along with the image data that define the pattern and any associated information data, the control module ordering them on-screen display, superimposed manner on the observation images delivered by the cameras. This operation is similar to that described previously in the surgical application.

One can also consider an installation variant adapted to guide users by locating pedestrians in an environment, for example urban. In this case, the device according to the invention is advantageously implemented in a communication device, such as a mobile phone, equipped with a function of location, for example by triangulation or GPS, and a camera, as well as 'possibly an inertial unit.

The processing module 22 of the resetting device D and the control module 15 of the facility may be implemented in the form of electronic circuits, software modules (or computer), or a combination of software modules and electronic circuits.

The invention also relates to a method dedicated to real-time registration of known patterns, region characteristic features, on pictures in this region.

It can be implemented using the registration device and facilities discussed above. The functions and sub-functions main and optional of the steps of the method being substantially identical to those provided by the various means constituting the device and facilities, only be summarized below the steps implementing the main functions of the method the invention.

This method is characterized in that it comprises:<ul><li>* Stored in a memory 21 representative patterns of portions of a selected region and position and orientation known in relation to a common reference,</li><li>* Observe an angle selected the chosen region and deliver real-time observation of this region images,</li><li>* Designate at least one portion of image observation of the selected region and at least one representative attribute of this portion,</li><li>* Determine the memory a pattern representative of the designated part, given the designated attribute, and</li><li>* Readjust the pattern on the designated image portion considering the chosen angle.</li></ul>

The invention is not limited to the device embodiments, systems and process described above, only as examples, but encompasses all the variants that may be envisaged by the art in the context of the claims below.