Computerized specimen encoder device

110257/2

NeoPath, Inc. C.94205

Computerized specimen encoder device

COMPUTERIZED MICROSCOPE SPECIMEN ENCODER

Field of the Invention 5 This invention relates to computerized location marking of information on microscope specimens such as on microscope slides, with particular utility in pathological examination and re-examination of the slides. 10

BACKGROUND OF THE INVENTION

Specimen slides such as Pap smears are examined by microscope users for specific types of events such as 15 cancer cells. These events are, however, relatively rare and once found are difficult to re-locate for confirmation by others. This is a severe drawback, since an important requirement for cancer screening laboratories is certification by a pathologist, with the 20 need for independent relocation.

The prevailing method in aiding relocation is the placement of an ink dot on the slide near the location of the event. This method has proven to be awkward, time 25 consuming and inaccurate. In addition, with this method, it is not possible to ascertain if the entire specimen area of the slide has been uniformly examined or if the areas of the specimen have or have not been scanned. It is accordingly often the case, that if the user is 30 interrupted, it is necessary to restart slide examination. With microscope examination of items, such as diamonds or other types of jewels, for identifying characteristics, the use of ink dots can actually detrimentally mar the appearance of the item. 35

Ink dot marking and other similar location methods also do not provide any information at all regarding the rate (an important quality control factor) at which each area 2 has been examined. The dots merely function as markers and cannot provide a description of the event or any scan history record. As a result of the realization that cancer screening is inaccurate, because areas are missed, 5 there is a major effort in place to improve the quality of screening laboratories by requiring re-screening of randomly selected slides.

In accordance with current ink dot marking practice, 10 slides are marked with ink dots, at particular events of interest, with electrically controlled pens or marking, with the push of a button. These marking devices are exemplified by those disclosed in US Patent Nos. 3,600,057; 4,262,426; 4,690,521; and 5,076,679. 15

More sophisticated location methods, with event location coordinates, include those, such as disclosed in US Patent No. 4,190,314, wherein microscope slides or cover slips are marked with a grid of lines which is visible 20 through the microscope lens. However, it is necessary to continually interrupt scanning to write down coordinates. Furthermore, no scan history is possible and special slides are required for each specimen. 25 Similarly, US Patent Nos. 1,996,141 and 3,514,180 provide viewable stage coordinates and modern microscopes are available with scales to determine stage position However, there is no scan history and the user must stop scanning to interpret the scale and write down the 30 coordinates of events of interest.

Motor controlled stage or slide holder devices such as embodied in US Patent Nos. 3,999,047; 4,012,112; 4,833,382 and 5,072,382, generally provide a visible 35 indicia on a slide for observation by the user with coordinates set into the device. The slide is moved in a meander pattern until an event of interest is discovered, 3 at which point the slide is stopped and the position is recorded. While this provides both recording of position and a recording of scan history, it is limited by the need for the user to wait until the event comes into the 5 field of view to stop the meander, rather than by controlling movement by hand. Such devices, besides being very expensive, also interfere with normal use of the microscope and the motion control devices thereof detract from the feel of a normal microscope stage io positioning control.

Very recently, a new device has been developed which utilizes personal computers to record slide data. This device, with video input to specially modified 15 microscopes, superimposes the display of a computer output on the microscope field for viewing by the user. A mouse is used to mark off events of interest. The device is described in Cytometry. vol. 13, pages 109-166 (1992) and Analytical & Quantitative Cytology & 20 Histology, vol. 14, August 1992. While useful, such devices are inherently expensive and complex in requiring extensive modification of microscopes, thereby limiting their utility with respect to different field-available microscopes. As a result, widespread reviewing 25 capability is severely restricted.

SUMMARY OF THE INVENTION

Generally, the present invention comprises an economical 30 encoding method for microscope examination of specimens, particularly of specimen slides, and a computerized encoder device for use in such method. The method and device are suitable for encoding any microscope examined specimen which presents one or more generally planar 35 surfaces for viewing by the microscope lens. The method and device have particular utility in the encoding of pathological slide specimens and the following discussion 4 is with reference to such utility.

The method and device are operable with universally available personal computers or with an inexpensive 5 modified computer device to be described hereinafter.

Both method and device utilize the standard x-y plane motion of microscope slide stages and specimens such as specimen slides thereon, to effect encoding. The encoding is effected by correlating motion of the slide 10 stage and of the slide to pixel locations on a computer generated image of the slide, for direct translation into a corresponding computer readable and storage retrievable replication. Such replication includes the motion of the slide during examination, as well as position location, 15 on the slide, of events, and their relative degree of interest. The correlation replication and encoding is made initially with reference to preset location areas on slides to be examined. Accordingly, on review of the slide, the image and replication is retrieved by the re- 20 initiation at such preset location area.

The encoder device of the present invention comprises, in conjunction with computer means or with integrated computer means, correlation means for translating 25 movement of the microscope stage, with mounted slide, in the x-y plane, to retrievable corresponding computer pixel locations and cursor movement, which are stored on computer storage media linked to the computer means. The encoder device further comprises means for marking 30 selected microscope viewing areas, as correlated to the computer pixel locations, with indicia, at pre-selected time intervals. Means are further included for marking the indicia with ratings of areas of Interest and/or digitized vocalized or audio accounts. The encoder 35 device further comprises reviewing means whereby a stored image of the indicia is reviewable, upon demand, and wherein movement of the microscope stage causes 5 corresponding movement, of a computer cursor on the image and the selected marked indicia.

With cursor location on a specific area of interest, and 5 positioning of the specimen slide on the microscope stage, the area is directly viewable under the microscope and a voice record associated with the event can be made or retrieved. 10 In accordance with the present invention, the slide encoder is attached to the movable stage of essentially any microscope with a moving stage, whereby x-y plane movement and location, is correlated to examination of an identified slide, with information marking and location 15 being directly correspondingly written on computer storage media, during or directly after the examination. The x-y plane described herein corresponds to the two dimensional viewed area of a specimen slide being examined. 20

Subsequent use of the computer-stored information, coupled with the slide encoder, in a slide re-examination, permits independent retrieval of such information and location on the same slide previously 25 examined. The computer storage media may also include digitized vocal transcription of history of the slide and events, made during the original examination and/or subsequent re-examinations. 30 In a preferred economical embodiment of the present invention, the slide encoder includes a standard computer mouse pen affixed to the movable stage of the microscope. A stationary surface (i.e. mouse pad), for the mouse pen to roll on, may be the supporting surface for the 35 microscope, a surface element separately affixed to the non-moving frame of the microscope stage, or any other stationary element positioned in contact with the rolling 6 ball of the mouse. Examination movement of the slide causes a corresponding movement of the roller ball of the computer mouse pen on the rolling surface. Such movement is readily translated and stored in the computer storage 5 media and is displayed in the form of a representative slide image on a direct viewing screen. Alternatively, the actual control movement mechanism of the slide holder is directly linked to a computer bus for a direct, software controlled, recordation of slide examination and 10 information location markings.

In accordance with the slide encoding method of the present invention, the computer is instructed to and generates time controlled markings such as dots, which 15 correlate to slide viewing areas, on the representative slide image. For example, at time intervals of one second, as regulated by the internal clock of the computer, a dot is generated as being representative of the full area being viewed at that time. As a result, 20 with a subsequent review of the slide, there is a specific correlation of the computer stored dot notation on the representative slide image, directly with an actual full viewing area of the slide. It is also readily ascertainable which areas have and have not been 25 viewed. In addition, the density of dots is a direct indication of the rate at which the slide had been examined. Thus, closely spaced dots indicate a slow scan and widely dispersed dots indicate a more rapid scan.

Such information is critical for quality control 30 evaluations.

With the marking generation or subsequent review, each marked area is readily labelled, such as with a numerical grading, to inform a reviewer of specific events and 35 their relative importance. If desired, a digitized voice record describing the particular events being labelled may also be generated and stored for correlative 7 retrieval upon review of the slide.

It is an object of the present invention to provide a slide encoding device for a microscope, wherein movement 5 of the microscope stage, and slide, provides a corresponding record on computer storage media of slide examination, wherein, in effect, the microscope stage functions as a computer mouse device. 10 It is a further object of the present invention to provide computer generated markings, on a simulated corresponding slide image, stored on the computer storage medium, which can be retrieved and reviewed with direct relation to a review of the specific actual slide 15 viewing, with the markings being further selectively marked for events of interest.

It is a still further object of the present invention to provide a retrievable record of events of interest, 20 original scanning rate, specific areas scanned and a record of events of interest.

These and other objects features and advantages of the present invention will become more evident from the 25 following discussion and drawings in which:

SHORT DESCRIPTION OF THE DRAWINGS

Figure 1 is an isometric view of a microscope with an 30 embodiment of the present invention in the form of a standard mouse pen affixed to the movable microscope stage;

Figure 2A-C are bottom views of the movable microscope 35 stage of the microscope of Figure l, showing direct computer connection to the movement mechanism of the microscope stage in alternative embodiments; 8

Figure 3 is a representative view of a computer device, with viewing screen and storage capability, having marking controls, voice input and retrieval review, which is used to monitor slides with computer correlative 5 information and to provide markings and slide scanning history;

Figure 4 is a flow chart illustrating the operative method of the present invention; 10

Figure 5a is a representation of the computer view screen of Figure 3, wherein specimen area dimensions are selected to begin the operative method as shown in Figure 4; 15

Figure 5b is a further representation of the view screen of Figure 3, with selection of a particular slide by name and number identification; 20 Figure 5c is illustrative of the view screen awaiting placement of the selected slide to be examined, on the slide stage, and positioning the slide stage to the predetermined start point; 25 Figure 5d is a computer generated non-magnified representation of the pathological slide selected in Figure 5b, showing the predetermined starting point, corresponding to the actual magnified microscope viewing area, as the upper left hand corner; 30

Figure 5e shows the slide representation of Figure 5d after slide screening, wherein dots have been generated to correspond to microscope viewing areas, with several areas having been marked as being of interest;

Figure 6a represents the screen of Figure 3 during slide information review and retrieval for slide selection; 35 9

Figures 6b and 6c represent the screen immediately thereafter with the slide having been placed on the microscope stage and the stage being positioned at the starting point; and 5'

Figure 6d represents additional areas of interest having been entered on review.

DETAILED DESCRIPTION OF THE INVENTION 10

In accordance with the method of the present invention, scanning examination of a microscope slide is effected in the following manner: 15 1) operatively linking a computer, with a movable slide stage of a microscope, to correlatively record movement of the slide stage and a specimen such as a specimen carried on a slide, placed on the slide stage; in the x-y plane in which the slide stage and 20 slide are movable; 2) fixedly placing an identified specimen slide on the movable stage; 25 3) moving the stage, with concomitant slide movement, until the microscope viewing area is fixed on a predetermined starting location point of the slide's cover slip, e.g. the upper left hand corner of the slide's cover slip; 30 4) initiating correlative start location by the computer, operatively linked to said movable stage, to record subsequent movement in the plane in which the stage and slide are movable; 35 5) moving the stage, with concomitant slide movement, away from the starting location point, 10 with microscope examination of the slide at various locations on the slide, with the linked computer recording such motion and various locations on computer storage medium; 5 6) causing the computer to automatically record, on the computer storage medium, indicia (such as black dots), at pre-determined time intervals, with said indicia correlating to the microscope viewing area 10 location on the slide, at the pre-determined time intervals; and 7) marking indicia of interest with a distinguishing marking for subsequent retrieval and recognition. 15

The marking of the indicia of interest is either in the form of a recognizable numerical grade indicating its relative degree of importance or the indicia is marked with a digitized voice or audio record, describing the 20 characteristics or importance of the area being so marked. The particular indicia can be so marked at any time during slide examination or subsequent review, when the particular indica is in the microscope viewing area. 25 With re-examination of the slide, the slide is placed on any similarly equipped microscope slide stage (review is however independent of the type of microscope being used). At the appropriate slide viewing area start point, the recording of the particular identified slide 30 is recovered from the computer storage medium to a viewing screen. Beginning from the reference start point correlation, a cursor on the viewing screen provides a continual correlation of position thereon to the actual microscope viewing area. Movement of the microscope 35 stage causes concomitant movement of the cursor. As a result, the cursor can be made to fall on a selected marked indicia which correlates to the area of interest 11 on the slide which was so marked. Such area can then be directly viewed or any audio record can be played.

The density of marked indicia on the viewing screen 5 provides an indication of the rate at which the slide was originally examined. The denser the indicia, the slower and presumably the more careful the original examination. Absence of indicia in a particular area of the slide is indicative of such area not having been originally 10 scanned and examined.

It is understood that the present method and device are applicable to microscope slide examination in fields other than pathological determinations, such as metal 15 stress analysis, fingerprint analysis, etc. in which areas or events of interest are marked for subsequent review. The present method and device are also applicable to microscope examination of specimens which do not require a slide carrier, but which are fixed into 20 position, such as with a clamp, in replicable positions. For example, identification examination and reexamination of diamonds or other gems is possible with separate x-y plane examination of the various facets for specifically located identification markings. Similarly 25 any three dimensional object, e.g. semiconductor chips, of appropriate dimensions, can be examined and reexamined for ” events of interest" in each/or any number of its surface planes. 30

DETAILED DESCRIPTION OF THE DRAWINGS

AND THE PREFERRED EMBODIMENT

With specific reference to the drawings, in Figure 1,. a 35 typical microscope 1 (Olympus BH-2), is shown with a slide stage 2, movable in the x-y plane shown, mounted on relatively fixed frame 3. Slide 5b, with pathological specimen to be examined, is located and fixed into 12 position on slide stage 2, with slide holder 5a. In operation, focus control knob' 9 causes frame member 3 to move in the z-direction (up and down) for focussing relative to selected lens 25. At the point of proper 5 focus, frame 3 is fixed in position and slide stage 2 moved in the x-y direction (the two dimensional viewing area of slide 5b) for full scanning of the pathological sample on slide 5b. Concentric knob control 6 is geared, as is more clearly shown in Figure 2, to effect such x-y 10 direction movement.

In accordance with the embodiment of the present invention shown in Figure 1, a first "mouse pad" element or plate bracket 4 is affixed, in a cantilevered manner 15 to "fixed" frame 3. A standard mouse pen 7 is affixed via bracket 5 to movable slide stage 2, whereby the ball 8 of mouse pen 7 is in rolling contact with plate ι bracket 4. The rolling ball 8, in accordance with standard computer mouse technology is in contact, within 20 the pen, with the surfaces of each of two cylinders with perpendicular axes. The motion of ball 8 in the x-axis direction causes one cylinder to rotate and its motion in the y-axis direction causes the other cylinder to rotate. Each cylinder is coupled with a multi-slotted disk which 25 interrupts light between two light sources and two photo detectors and as the disks rotate, the photo detector signals are transformed by internal circuitry to the amount and directions of each of the x and y direction motions of the ball. An integrated circuit chip (e.g., 30 Motorola PIC 1.6C54-XT/P) in thg.mpuse pen 7 provides a sequencSTof Joytes encoding this motion through connecting wire 10 to the serial bus connection on a personal computer. Standard "mouse driver" software in the computer's operating system converts these signals into 35 the computer display screen's cursor position such as shown in Figure 5d. 13

Figures 2A and 2B depict the geared rack movement mechanism of slide stage 2 for translation into mouse pen movement in the embodiment shown in Figure l. Each of the inner and outer knobs of control 6 (shown in the 5 right hand control position) is comprised of a rotary gear (not shown), with the outer knob gear engaging geared rack 52 to cause platform 53 (and slide stage 2 thereon) to move in the y-direction relative to stage frame 3. The inner gear of control 6 (not shown) engages 10 geared rack 54 which in turn causes bar 55, to move in the x-direction relative to platform 53, and causing attached block 56 to move in the x-direction. Slide holder 5a, attached, in turn to block 56, is moved, ^thereby moving slide 5b in the x-direction. 15

An alternative movement and monitoring geared rack mechanism for direct, rather than the intermediate mouse monitoring movement shown in Figure l, is also depicted in Figure 2A, wherein geared rack 57 is mounted on stage 20 frame 3, whereby it causes a gear in assembly 59, mounted on platform 53 (in the left hand mount position of control 6), to rotate as the slide 5b is moved in the y-direction. A second geared rack 58 is attached to bar 55 to move the slide in the x-direction. Two concentric 25 gears in assembly 59 cause its inner and outer shafts to rotate with the slide's x and y motion to, in turn, rotate the inner and outer shafts of a concentric shaft encoder 59a in assembly 59, such as is available from Clarostat Sensors and Controls in Plano, Texas. Each of 30 the x and y output signals from encoder 59a is decoded into a direction signal level and incremental counts proportional to distance using a "mouse" decoder chip or a personal computer board such as is available from Technology 80, Inc. in Minneapolis, Minnesota. An 35 integrated circuit decoder chip provides a sequence of bytes encoding the motion through wire 10 to a serial bus connection on a personal computer. As with the other 14 embodiment, "mouse driver" software in the computer's operating system converts these signals into the computer display screen's cursor position. If the computer board is used, the bytes are made available directly in the 5 computer's memory.

In a further embodiment of the present invention, shown in Figure 2B, encoder 59a is a single shaft encoder such as is available from Dynamic Research Corporation, in 10 Wilmington, MA, with a gear engaging geared rack 57. In such embodiment, in place of geared rack 58, a rack 61 is mounted to block 56 so as to move with the slide's x motion. A second encoder 60, mounted to platform 53, has a geared shaft engaging geared rack 61. Encoder 60 15 thus encodes the x motion of the slide. The two independent encoder signals are treated as described above.

In a still further embodiment of the present invention, 20 shown in Figure 2C, linear encoders 70 and 71 are mounted between the frame 3 and the platform 53 and between the platform 53 and the bar 55 whereby differential motion between the slide and frame is monitored. Linear encoders of this type are available from Dynamic Research 25 Corporation and computer boards such as from Technology 80, Inc. are used to interface them with a personal computer.

The device of the present invention can be embodied as a 30 stage position encoding module which is plugged into one of the serial or parallel ports of a personal computer. Alternatively, in place of a standard computer with a monitor screen, the monitoring of the encoder output, via wires 10, can be effected by use of the integrated device 35 11 shown in Figure 3. The device ll integrates a personal computer board, as described above, with a small display panel 12, a special keyboard 13, an integral 15 fixed storage device 14 such as a floppy disk or a flash card reader, and a vocal input 21. If desired, the keyboard may be a separated but connected element. 5 With reference to the flow chart of operation shown in Figure 4, a disk with slide information (such as patient identification) is placed into the device of Figure 3, and the screen, as shown in Figure 5a, queries regarding the slide area to be scanned. The various menu screens io are accessed by means of key 22 and appropriate selection movement is effected by use of keys 16 and 17. With selection of the desired slide area from menu list 109 (by means of entry key 18), the slide specimen list 110 (pre-entered with patient name or other identification) 15 appears for appropriate selection as shown in Figure 5b. With selection of the appropriate slide, the actual corresponding slide is placed on the slide stage 2, shown in Figure 1., with the device waiting (Figure 5c) until such placement and positioning is effected to the 20 preselected start point on the slide cover slip by / viewing through the microscope lens. As shown in Figure' 5d, a slide image 5b' appears on the screen with the U crosshair cursor 23, appearing on the slide image on I screen 12, at the predetermined recordation starting \ 25 position (shown as the upper left hand corner of image 5b', in Figure 5d) and scanning is initiated. As the slide stage is moved, the crosshair cursor moves correspondingly, and at pre-set time intervals, the computer enters a black dot 90 on the slide image which 30 corresponds, in time to the microscope viewing area being scanned at the time and in area to such microscope viewing area. At any of the black dot areas 90, a numerical interest rating can be entered via the keyboard number keys 19. Alternatively, or in addition, a voice 35 record may be made by pressing key 20 to activate microphone 21 and speaking into microphone 21. The numerical rating or voice record as well as the entered 16 dots are recorded as a history of the slide for subsequent recall when the slide is re-examined. Thus, as shown in Figure 5e, at the end of a scanning session, two microscope viewing areas 103 and 105 have been marked 5 with interest levels of 3 and 5 and a vocal history was recorded and marked with a "V" at a third point 104.

Because of the density of dots, it is evident that the upper portion of the slide has been examined more 10 intensively than the lower portion. with re-examination of an already scanned slide, such as by a certifying pathologist, a slide which is identified, from menu list 110, as in Figure 6a is recalled from 15 computer storage. The actual slide is placed on a microscope slide stage of a similarly equipped microscope. When the stage is positioned at the predetermined starting point, review is initiated, as shown in Figures 6b and 6c. To view the marked points of 20 interest (or for that matter any area), the microscope stage is moved while cursor position is monitored on the viewing screen. When the position of the cursor corresponds to that of the particular marked area of interest, it is in the viewing area of the microscope for 25 the direct examination thereof. If a vocal history has been made, it can be replayed at such position. In addition, the slide may be re-examined during review, with additional areas being scanned or, as shown in Figure 6d, additional areas of interest e.g. at;106 and 30 107 can be appropriately marked or voice records made.

It is understood that the above description, drawings and examples are merely illustrative of the present invention and that changes in procedure and components may be made 35 without departing from the scope of the present invention as defined in the following claims.