Video recording and image capture device
13 claims: 1 independent, 12 dependent
- 1Device for medical video recording comprising:an endoscope (15);a digital media (120, 125);an imager (100) in communication with said endoscope (15), said imager (100) converting energy received from said endoscope to signals that are contemporaneously stored onto said digital media (125);and a housing (105), said housing (105) enclosing said imager (100), characterized by a touch screen (145) for entering control commands for said imager (100) and said endoscope (15) and said touchscreen (145) is responsive to a touch by a user (10), said touch screen (145) is at least partially retractable within a footprint of said housing (105) and the touch screen (145) is slidable at least partially out of the footprint of said housing (105).
97 paragraphs, as filed
Field Of The Invention
0001The invention relates generally to device for video recording and image capture, and more particularly to such a device for image based documentation of surgical procedures. The invention also relates to means to process signals into an MPEG layer stream.
Background Of The Invention
0002The maxim that "a picture is worth a thousand words" is more true than ever in image based documentation of surgical procedures. There patients' lives and physicians' reputations are at stake. The early history of image based documentation is littered with crude attempts at utilizing photographic film to capture anatomical curiosities and groundbreaking surgical advances for teaching purposes. However, little attempt was made to provide documentation of routine cases. Average patients simply did not merit this special consideration.
0003With the advent of analog cameras coupled to fiber optics, physicians began displaying some surgical patient images on video monitors in the operating room. This was especially true in endoscopic surgeries where physicians peer inside patients' body cavities normally only viewable with open surgery. Attempts made to systematically record all patient procedures on videotape required large and cumbersome archives of videotapes.
0004Although analog cameras are still used, solid state image cameras coupled to endoscopes and other medical instruments are now standard. These cameras produce images in digital format. Such format allows convenient and efficient image transfer, review, and archival.
0005One method for archiving digital images is suggested in <patcit id="pcit0001" dnum="US6067075A"><text>U.S. Patent 6,067,075 to Pelanek</text></patcit>. Therein, a workstation is disclosed for transferring and archiving patient images previously stored in the memory of diagnostic equipment. However, this method does not disclose contemporaneously saving images in a permanent location, meaning that the equipment is subject to "downtime" as data is extracted from its memory.
0006Once image data has been extracted and archived it must be made available to those in a position to best make use of it, the treating physician. Ironically in a circular manner, the person to whom eventual delivery of images is made is the same person who used the diagnostic equipment to record the images in the first place who must now await it being made available to him. Thus, it would be advantageous to the physician to have the data available at the conclusion of the surgical procedure or series of surgeries. Consequently, equipment downtime for data extraction also is eliminated increasing efficiency of equipment and facility utilization.
0007Image data availability, however, is only of value when the data can be read and images displayed readily. Thus, exotic and unusual methods of storing and reading image data chain physicians. Most convenient are displays in NTSC and PAL standards and devices that can play image data on these displays. NTSC is the common television standard in the U.S., while PAL standards predominate in Europe and in parts of Asia. Additionally, most convenient for medical personal, is for images and video to be viewable utilizing common off the-shelf optical media players (i.e. set-top DVD players) and/or standard personal computers.
0008<patcit id="pcit0002" dnum="US5045955A"><text>U.S. Patent 5,045,955 to Ikeda</text></patcit> discloses an apparatus coupled to a digital tape drive for recording and reproducing high-definition medical images having specific analog matrix exceeding that of NTSC or PAL standards. Typically, these images can only be produced by specialized imaging equipment and can only be viewed on highly specialized monitors. They are incompatible with NTSC or PAL standards and thus are inapplicable to the use of documenting procedures where analog or digital cameras provide a video signal to be displayed.
0009Other suggestions have been made to integrate images into a medical setting. Therein, imaging is primarily related to diagnosis rather than documentation; resulting in files solely consisting of visually relevant components and omitting narration by the treating physician.
0010Suggestions to incorporate equipment in surgical suites fail to consider issues relevant to logistics of placing equipment. Surgical suites include a plurality of equipment. Vertically stacking equipment in shelves, i.e. racking, advantageously conserves space. However, this limits the readily available user-operator access to front face plates. Equipment that incorporates space saving advantages while providing maximum user-operator access, therefore, is highly desired.
0011<patcit id="pcit0003" dnum="US2002101507A1"><text>US 2002/101507 A1</text></patcit> is directed at an endoscopic imaging system, where a signal representing an object image produced by a scope and projected by a camera head is processed by a CCU and displayed as an endoscopic image on a TV monitor. The object image is stored as digital image data on a memory in the CCU, read as image data of a still image, and recorded on a PC card mounted in a PC card slot. The PC card slot is formed in the front panel or the like of the CCU. A lid member or the like functioning as an anti-liquid invasion member and shield can be located at an opening of the slot.
0012Therefore, what is desired is a device that produces and saves a surgical documentary recording file that incorporates both video and audio to a transportable and common media contemporaneously during the surgery.
0013What is also desired is a device that formats a video signal into a plurality of video formats for display, displays an image stream, and allows a user to capture still images and save same as a still graphic file to the same media as the documentary recording file.
0014It is further desired that the device and the parts that the user interacts with are convenient and accessible.
0015Also, it is desired that the device is capable of "feed through" of real-time video signals to a plurality of display devices, in a plurality of video formats, when the device is in a standby mode, when the device is off, or in the event when the device malfunctions.
0016Further, what is desired is an interactive tutorial which allows a user to manipulate the files saved.
Summery Of The Invention
0017A solution to these objects is provided, according to the present invention, by a device for medical recording according to independent claim 1.
0018In accordance with one embodiment, a device for medical video recording includes an endoscope, and an imager for converting energy such as electromagnetic energy, direct current energy and the like, which is received from said endoscope to signals that are contemporaneously stored onto said media.
0019In accordance with one embodiment of the invention, a device is provided that encodes a video signal into a documentary recording file formatted in an MPEG-2 layer.
0020In accordance with one embodiment of the invention, a multimedia interface to process a video signal and audio signal into a first and second digital data stream having common characteristics is provided.
0021In accordance with one embodiment of the invention, a recording mode for recording a video stream during a surgical procedure is provide for surgical documentation.
0022In accordance with one embodiment of the invention, a means for a video signal to be displayed by a display unit during recording mode, or when the device is in a standby mode, or when the device is off, or when the device has malfunction, is provided.
0023In accordance with one embodiment of the invention, an archival mode is provided for review, edit and playback of a documentary recording file and graphic file.
0024In accordance with one embodiment of the invention, a medical instrument having touch screen control includes a touch screen for entering control commands, a processor for receiving the control commands and for generating control signals to operate the medical instrument, a housing for enclosing the processor. The touch screen is movable between a first position protected by the housing and second position extended from the housing for entry of the control commands.
0025In accordance with one embodiment of the invention, a multimedia interface for processing a video signal for recording video into a multiple frame layer includes a controller for an inter-ic bus for providing a multiple master digital connection, an analog to digital converter for converting a video signal to a first digital stream, the converter operably connected to the inter-ic bus, a video compression and decompression integrated circuit for encoding the first digital stream into a second digital stream having frames and decoding a second digital stream, the video compression and decompression integrated circuit operably connected to the inter-ic bus a programmable buffer for selectively saving frames handled by the controller, the buffer operably connected to the controller and the video compression and decompression integrated circuit and the buffer inserting the frame into the second digital stream for decoding.
0026In accordance with one embodiment of the invention, a method for recording an MPEG layer file for documenting surgical procedures while displaying an MPEG layer stream and n number selecting still image files corresponding to the MPEG layer stream, includes the steps of providing a first digital data stream comprising a video signal, providing a second digital data stream comprising an audio signal, multiplexing an MPEG layer data stream from the first and second digital data stream, streaming the MPEG layer data stream to a hard drive and an optical media drive operably - connected on a vertically stacked bus, writing the MPEG layer stream to the hard drive and the optical media drive, displaying the MPEG layer stream on a display unit, selecting n number of frames from the MPEG layer stream,converting n frames to n still image files, and multiplexing the output signal to the display unit by adding n still image files.
0027The terms "communicate", "communicating" and "communication" as used herein include both conveying data from a source to a destination, as well as delivering data to a communications medium, system or link to be conveyed to a destination. The term "communication" as used herein means the act of communicating or the data communicated, as appropriate.
0028The invention and its particular features and advantages will become more apparent from the following detailed description considered with reference to the accompanying drawings.
Brief Description Of The Drawings
0029<figref idref="f0001">Fig. <b>1</b></figref> is a schematic diagram of a Video Recording and Image Capture Device used in a surgical patient setting in accordance with the present invention;
0030<figref idref="f0002">Fig. <b>2</b></figref> depicts a front panel elevation view of a Video Recording and Image Capture Device of <figref idref="f0001">Fig. 1</figref>;
0031<figref idref="f0003">Figs. <b>3</b></figref> depicts a rear panel elevation view of a Video Recording and image Capture Device of <figref idref="f0001">Fig. 1</figref>;
0032<figref idref="f0004">Fig. <b>4</b></figref> is a schematic diagram of a Video Recording and Image Capture Device of <figref idref="f0001">Fig. 1</figref>;
0033<figref idref="f0005">Figs. <b>5a</b></figref>, <figref idref="f0006"><b>5b</b></figref>, and <figref idref="f0007"><b>5c</b></figref> depict a Video Recording and image Capture Device of <figref idref="f0001">Fig. 1</figref>;
0034<figref idref="f0008">Fig. <b>6</b></figref> is a schematic diagram of a multimedia interface of <figref idref="f0001">Fig.1</figref>;
0035<figref idref="f0009">Fig. <b>7</b></figref> depicts a schematic of interconnecting Video Recording and Image Capture Device and a display unit of <figref idref="f0001">Fig. 1</figref>;
0036<figref idref="f0010 f0011 f0012 f0013 f0014 f0015">Figs. <b>8</b> through <b>13</b></figref> depict use of a Video Recording and Image Capture Device through the interactive display of the touchscreen of <figref idref="f0001">Fig. 1</figref>;
0037<figref idref="f0016">Fig. <b>14a</b></figref> and <figref idref="f0017"><b>14b</b></figref> depict a plan view a track advance system for a touchscreen of a Video Recording and Image Capture Device of <figref idref="f0001">Fig. 1</figref>; and
0038<figref idref="f0018">Fig. <b>15</b></figref> depict a Video Recording and Image Capture Device of <figref idref="f0001">Fig. 1</figref> with touch screen fully retracted, touch screen extended and touch screen extended and deflected.
Detailed Description Of The Drawings
0039The present invention will be described as used in connection with one or more endoscopes. It will be understood that other applications may be used equally effectively. Various frame displays and icons having particular graphical representations are described in connection with the functions of the present invention. It will be understood that other graphical representations may be used equally effectively in communicating to users the function performed therein.
0040Herein, image and image stream refer to the perception of the user utilizing their audiovisual capabilities; an image stream may also include an auditory component. Video signal, digital signal, analog signal, data stream, or the like refer to electrical signals in the present invention and electrical equipment operably connected and/or associated with the present invention.
0041<figref idref="f0001">Fig. 1</figref> is a schematic diagram of a Video Recording and Image Capture Device used in a surgical patient setting in accordance with one embodiment of the present invention. Patent <b>5</b> is being examined by user <b>10</b> with an endoscope <b>15</b>. User <b>10</b> may be a physician, surgeon, nurse, or other qualified professional or para-professional. Video Recording and Image Capture Device (imager) <b>100</b> has been placed into a video recording mode. A camera head <b>20</b>, such as a charge coupled device (CCD), CMOS, or CCl chip, is coupled to endoscope <b>15</b> to receive and produce a video signal <b>300</b> (see <figref idref="f0004">Fig. 4</figref>) of a body cavity of patient 5. Video signal <b>300</b> results in an image stream <b>25</b> when displayed on a display unit.
0042As user <b>10</b> manipulates endoscope <b>15</b>, user <b>10</b> controls camera head <b>20</b> with camera control unit (CCU) <b>30</b> and observes image stream <b>25</b> on a display unit, for example external video screen <b>35</b>. Imager <b>100</b> is operably connected to CCU <b>30</b> to receive video signal <b>300</b>. Device is also operably connected to external video screen <b>35</b>, microphone <b>40</b>, printer <b>45</b>, and network <b>50</b> through standard ports. In one embodiment, imager <b>100</b> is operably connected to camera head <b>20</b> to receive video signal <b>300</b> and control camera head <b>20</b>, obviating the need for CCU <b>30</b>.
0043As user <b>10</b> examines and treats patient <b>5</b>, imager <b>100</b> is recording a digital data stream representing image stream <b>25</b> on optical media drive (OMD) <b>125</b> (see <figref idref="f0002">Fig. 2</figref>) and saving a backup copy to hard drive (HDD) <b>120</b> (see <figref idref="f0004">Fig. 4</figref>)for a plurality of purposes. During this period user <b>10</b> may provide narration to the examination. Microphone <b>40</b> or an independent audio recording system captures the narration and converts it into audio signal <b>314i</b> (see <figref idref="f0008">Fig. 6</figref>) or <b>316i</b> (see <figref idref="f0008">Fig. 6</figref>), respectively, collectively audio signal <b>315</b> (see <figref idref="f0008">Fig. 6</figref>).
0044Upon encountering a situation requiring further investigation, user <b>10</b> utilizes an image capture mode to save and freeze a first image from image stream <b>25</b> for further investigation. Using imager <b>100</b>, a second image substantially like the first image can be superimposed on image stream <b>25</b>. Upon encountering further situations that attracts the attention of user <b>10</b>, a plurality of second images substantially corresponding to respective first images found of interest can be saved and concurrently superimposed on image stream <b>25</b>. A file corresponding to second image can be printed with printer <b>45</b> and/or forwarded over network <b>50</b> to other persons for consultation.
0045<figref idref="f0002">Figs. <b>2</b></figref> and <figref idref="f0003"><b>3</b></figref> depict a front panel and rear panel elevation view, respectively, of a Video Recording and Image Capture Device in accordance with one embodiment of the present invention. <figref idref="f0004">Fig. <b>4</b></figref> is a schematic diagram of a Video Recording and Image Capture Device in accordance with one embodiment of the present invention. Imager <b>100</b> is substantially housed in chassis <b>105</b> and includes a plurality of components: power supply <b>110</b>, microprocessor main board (MMB) <b>115</b>, hard drive (HDD) <b>120</b>, optical media drive (OMD) <b>125</b>, communication interface <b>130</b>, multimedia interface <b>135</b>, speaker <b>140</b>, digital video interface <b>150</b>. Chassis <b>105</b> includes an add-on tray <b>105a</b> that advantageously houses touchscreen <b>145</b>.
0046Power supply <b>110</b> is connected to an AC power supply with an electric cord at power port <b>110a</b> located on chassis back panel <b>105b</b>. When power switch <b>110b</b> of chassis front panel <b>105c</b> is engaged, power supply <b>110</b> provides suitable power to components of imager <b>100</b>. Power supply <b>110</b> conforms to IEC 601-1 isolation requirements, provides power meeting medical grade standards of <b>145</b> Watts, includes a power correction factor, and is capable of automatically handling automatically an AC voltage range of 100 to 250 VAC, 50-60 Hz.
0047Microprocessor main board (MMB) <b>115</b> executes software and includes a central processor unit ("processor"), random access memory, a plurality of expansion slots, a BIOS chip, and on-board sound and graphics capabilities on a footprint convenient for chassis <b>105</b>. Therein, it may be that an Intel Pentium III with a processing speed of at least 800 MHz; an Intel 815 chip set; at least three (3) PCI and one (1) AGP slot; SDRAM of at least 128 MB; on-board AGP graphics; and a flash upgradeable BIOS chip wherein a customized logo may be loaded, is provided on an ATX motherboard. Further, MMB <b>115</b> preferably includes port panel <b>115a</b> to connect to other devices and or networks via serial, parallel, USB and ethernet, and to connect to mouse, keyboard, and video monitor, a Intel 40-pin DVO motherboard connector, or similar, and support for the connector.
0048Hard drive (HDD) <b>120</b> and optical media drive (OMD) <b>125</b> maybe IDE or Enhanced IDE (alternatively IDE) compatible data storage media drives operably connected MMB <b>115</b> by a vertically stacked IDE bus <b>116</b>. HDD <b>120</b> maybe a magnetic hard drive having at least 40 Gigabytes of storage and minimal access time, as such a drive from Maxtor having EIDE access time of 8.0 ms, 7200 rpm disk speed is preferred. OMD <b>125</b> is preferably a DVD+RW drive capable of reading and writing data to an optical media disk, such as a compact disk (CD) and/or a digital versatile disk (DVD).
0049Communication interface <b>130</b> is any kind of network or proprietary communications interface. Therein, for example, a network card that is capable of interfacing with the Karl Storz®, Inc. Storz Communication Bus (SCB) is preferred.
0050Multimedia interface <b>135</b> connects to MMB <b>115</b> via a Peripheral Component Interconnect (PCI) local bus to provide record and write image stream <b>25</b> originating from camera head <b>20</b>. Speaker <b>140</b> is housed in chassis 105 and provides auditory capabilities for imager <b>100</b>.
0051Digital video interface <b>150</b> is operably connected to MMB <b>115</b> to create a video signal conforming to the digital visual interface (DVI) standard. Therein, a digital video interface <b>150</b> is preferably physically implemented as a printed circuit board having an Intel 40-pin DVO connector, or similar, to MMB <b>115</b> and a DVI-D output connector in expansion slot <b>105d</b>.
0052A touchscreen <b>145</b> is provided as illustrated on <figref idref="f0005">Figs. <b>5a</b></figref>, <figref idref="f0006"><b>5b</b></figref>, and <figref idref="f0007"><b>5c</b></figref> in accordance with one embodiment of the present invention. A surface <b>145a</b> responsive to touch by an operator, such as user <b>10</b>, and a display <b>400</b> capable of showing the functions of imager <b>100</b> are included in touchscreen <b>145</b>. Preferred is a touchscreen manufactured by Optrex Inc. of Plymouth, MI. A video connection to MMB <b>115</b> is made to the corresponding DVI-D output and a data and power connection is made between standard RS-232 serial ports.
0053Touchscreen <b>145</b> is advantageously housed in tray <b>105a</b> (see <figref idref="f0016">Fig. 14a</figref>) of imager <b>100</b> within chassis <b>105</b>. Therein, it is mounted within a track advance system (TAS) <b>146</b> (see <figref idref="f0016">Fig. 14 a)</figref> so that touchscreen <b>145</b> is secure and is relatively flush with a front face of chassis <b>105</b> as depicted in accordance with one embodiment of the present invention in Fig <b>18</b>.
0054TAS <b>146</b> allows user <b>10</b> to advantageously use the space available when equipment is racked. User <b>10</b> presses a button or other releasing mechanism to release touchscreen <b>145</b> and slides touchscreen <b>145</b> away from chassis <b>105</b> and out of tray <b>105a</b> as indicated by the arrows as depicted in accordance with one embodiment of the present invention in <figref idref="f0006">Fig. <b>5b</b></figref>. To make touchscreen <b>145</b> more user friendly and visible, user <b>10</b> is able to tilt touchscreen <b>145</b> at an angle as indicated by the arcuate arrows in accordance with one embodiment of the present invention in <figref idref="f0007">Fig. 5c</figref>.
0055<figref idref="f0018">Fig. 15</figref> discloses the touch screen in various states of deployment. For example, <b>101</b> shows one embodiment in which the touch screen is fully retracted, however, other embodiments will not have the touch screen fully retract. <b>102</b> shows the touch screen extended out and <b>103</b> shows the touch screen extended out and deflected from the plane of the housing.
0056<figref idref="f0016">Figs. <b>14a</b></figref> and <figref idref="f0017"><b>14b</b></figref> depict a plan view of a track advance system for a touchscreen of a Video Recording and Image Capture Device in accordance with one embodiment of the present invention. A platform <b>144</b> (shown advanced out of tray <b>105a</b>) disposed to tray <b>105a</b> is operable with rails <b>143a</b> and <b>143b</b> and is guided by tracks <b>147a</b> and <b>147b</b> by bearings or other friction reducing mechanisms as depicted in accordance with one embodiment of the present invention in <figref idref="f0016">Fig.<b>14a</b></figref>. Releasing mechanism <b>143d</b> includes one or more components that selective to user desire, release touchscreen <b>145</b> in order to advance it. Releasing mechanism <b>143d</b> further may include one or more springs <b>143e</b>.
0057Platform <b>144</b> may be of any suitable design to hold touchscreen <b>145</b> even when placed at an angle from horizontal. Touchscreen <b>145</b> is operative with imager <b>100</b> through cables and connectors.
0058Rails <b>143a</b> and <b>143b</b> have stops <b>143c</b> (only partially shown for clarity) operable with tracks <b>147a</b> and <b>147b</b> to prevent advancing touchscreen 145 beyond a predetermined point.
0059TAS <b>146</b> includes on or more hinge <b>142</b>, such as edged hinge <b>142a</b> and <b>142b</b>, or center hinge <b>143c</b> a center hinge as depicted in accordance with one embodiment of the present invention in <figref idref="f0017">Fig.<b>14b</b></figref><b>.</b> Hinge <b>142a</b> and <b>142b</b> cooperate to angle platform <b>144</b> so that user <b>10</b> can during the surgery interactively access touchscreen <b>145</b>. To prevent changing angle of the platform <b>144</b> or accidental closure, one or more tapping mechanisms are disposed with hinge <b>142</b>, that make it resistant to unintentional movement.
0060<figref idref="f0008">Fig. <b>6</b></figref> is a schematic diagram of multimedia interface <b>135</b> in accordance with one embodiment of the present invention. Multimedia interface <b>135</b> is physically implemented on one or more printed circuit boards, also commonly referred to as expansion cards, having a plurality of integrated circuits and a PCI bus connection <b>136</b> to MBB <b>115</b> for data transfer. Through bus <b>136</b> multimedia interface <b>135</b> also receives electric power. Audio and video connection ports, i.e. input and output ports, are physically implemented in expansion slots <b>105</b> and integrally connected to multimedia interface <b>135.</b> In one embodiment, multimedia interface <b>135</b> is implemented as a PCI expansion card with bus connection <b>136</b> to MBB <b>115</b> and a further printed circuit board connecting directly to the PCI expansion card.
0061Resident on multimedia interface <b>135</b> is an Inter-IC bus operatively controlled by controller <b>200</b>. Controller <b>200</b> implements the Inter-IC bus as a multi-master bus to connect circuits to PCI interface <b>210</b>. Router <b>220</b> is provided as a field programmable gate array and has programmable read only memory that is accessible via a port, such as a JTAG port.
0062In recording mode, imager <b>100</b> receives the input of video signal 300 and audio signal <b>315</b> and contemporaneously encodes both signals while writing the data stream to HDD <b>120</b> (see <figref idref="f0004">Fig. 4</figref>) and/or media in OMD <b>125</b> (see <figref idref="f0004">Fig. 4</figref>).
0063Video signal <b>300</b> is received by multimedia interface <b>135</b> as analog video signal <b>304i</b> or digital video signal <b>306i</b> in input ports located in expansion slot <b>105d</b> (see <figref idref="f0003">Fig. 3</figref>). The active input port is automatically detected by multimedia interface <b>135</b>, as are active output ports.
0064A relay <b>205</b> is operative to bypass video signal <b>300</b> to an exit port corresponding to the input port for display by an external monitor when imager <b>100</b> is not powered or imager <b>100</b> is in standby mode. Therein, relay <b>205</b> includes a receiver for receiving a sense signal from software to indicate that the recording mode is engaged. If the sense signal is not received with a predetermined time, video signal <b>300</b> is bypassed. If imager <b>100</b> is not powered, a default mode of relay <b>205</b> bypasses the signal to the corresponding port, i.e. video signal <b>300</b> input on the S-video input is bypassed to an S-video output port.
0065Analog video signal <b>304i</b> is received using composite video input or S-Video input. If analog video signal <b>304i</b> is received when imager <b>100</b> is not electrically powered or recording mode is not engaged, video signal <b>300</b> is bypassed using bypass relay <b>205</b>. Received analog video signal 304i is digitized using analog to digital converter <b>240</b>. Therein, the NTSC or PAL standard analog video signal <b>304i</b> is digitized to first internal data stream, digital data stream (DDS) <b>310</b>. A digitized data stream having 4:2:2 luminance to chrominance (YUV) pixels is preferred. Other data stream configurations may also be used. In particular, a data stream having 8:2:2 YUV may be advantageously used in connection with CCU 30 (see <figref idref="f0001">Fig. 1</figref>) which is capable of producing higher analog video frame quality. DDS <b>310</b> is then streamed via router <b>220</b> to video media compression and decompression integrated circuit (V-codec) <b>230</b> where it is encoded.
0066Digital video signal <b>306i</b> is received using the SDI digital video signal input. If digital video signal <b>306i</b> is received when imager <b>100</b> is not electrically powered or recording mode is not engaged, video signal <b>300</b> is bypassed using bypass relay <b>205</b>. Received signal <b>306i</b> is processed by SDI receiver <b>250</b> and converted to a first internal data stream, DDS <b>310</b>. A digitized data stream having a 4:2:2 luminance to chrominance (YUV) pixels is preferred; although other data stream configurations may also be used. DDS <b>310</b> is streamed via router <b>220</b> to V-codec <b>230</b> where it is encoded.
0067Inputs for audio signal <b>314i</b> originating from a stereo line and audio signal <b>316i</b> originating from microphone <b>40</b> are provided for recording a desired audio signal such as dictation. Both inputs may be combined to create a single audio signal <b>315</b> or used individually. Audio signal <b>315</b> is received by audio compression and decompression integrated circuit (A-codec) <b>260</b> and converted to a second internal data stream, digital data stream for audio (DDSA) <b>312</b>, and passed to automatic gain control <b>270</b> disposed on the Inter-IC bus. DDSA <b>312</b> is then streamed to V-codec <b>230</b>.
0068V-codec <b>230</b> is provided as embedded software, preferably operating on a LINUX or UNIX embedded platform, on one or more integrated circuits on the Inter-IC bus and operatively connected to controller <b>200</b>. Also operatively connected are one or more buffer memories, such as SDRAM integrated circuits. Configuration settings of V-codec <b>230</b> are stored on an associated erasable electronic programmable read only memory (EEPROM or EPROM) circuit, or similar. The embedded software of V-codec <b>230</b> executes an algorithm to decode and/or encode a Motion Picture Experts Group (MPEG) layer, such as MPEG, MPEG-2 and/or MPEG-4, data stream (MDS) <b>320</b>. MDS <b>320</b> advantageously utilizes high compression rates achieved by using an index frame and noting a sequence of changes to that index frame in subsequent images. Therein, MDS <b>320</b> is comprised of a stream of frames <b>322</b>.
0069Algorithms of this type are known to execute file formatting appropriate for CD-R and a plurality of DVD media storage and play. Typically, MDS <b>320</b> is preferred to encode in a file format designated as "VOB" applicable to DVD+RW media storage and play. It should be appreciated that the present invention may be adapted readily to utilize other compression and decompression algorithms.
0070V-codec <b>230</b> multiplexes DDS <b>310</b> and DDSA <b>312</b> into MDS <b>320</b> which is then streamed to MMB <b>115</b>. From there, MDS <b>320</b> is streamed to HDD <b>120</b> and to OMD <b>125</b> via vertically stacked IDE bus <b>116</b>. Therein, HDD <b>120</b> writes MDS <b>320</b> to a magnetic disk and streams a copy to OMD <b>125</b> for writing to an optical disk such as a CD-R or DVD media to create contemporaneous documentary recording file while the surgery is being conducted by user 10. If the write process on OMD <b>125</b> is interrupted or fails, a backup copy resides on HDD <b>120</b> for a subsequent write attempt after the conclusion of the surgery.
0071A means for a video signal to be displayed by a display unit during recording mode is provided. The display unit may be external video screen <b>35</b> or touchscreen <b>145</b>. <figref idref="f0009">Fig. <b>7</b></figref> depicts a schematic of interconnecting imager <b>100</b> and a display unit in accordance with one or more embodiments of the present invention. An embodiment of video screen <b>35</b>, external video screen <b>35a</b>, is operable to receive an analog and/or digital signal using connectors for S-video, composite and/or SDI format. Therein, multimedia interface <b>135</b> senses the active output port and router <b>220</b> streams DDS <b>310</b> accordingly to either SDI transmitter <b>280</b> to produce digital video signal 306x or to digital to analog converter <b>290</b> to produce analog video signal 304x, respectively. In converter <b>290</b>, common data stream DDS <b>310</b> is automatically converted to the appropriate NTSC or PAL standard analog signal in use by video screen <b>35a</b>.
0072Touchscreen <b>145</b> is operable to receive a video signal in digital visual interface (DVI) format, and an embodiment of video screen <b>35</b>, external video screen <b>35b</b> having touchscreen capabilities, is operable to receive a video signal in digital visual interface (DVI) format or analog monitor format. Therein, MMB <b>115</b> streams a copy of MDS <b>320</b> to resident AGP graphics support and/or DVI graphics support and digital video interface <b>150</b> for output to touchscreen <b>145</b> and video screen <b>35b</b> as required by the active output.
0073Image capture mode is provided for saving a frame <b>322</b> (see <figref idref="f0008">Fig. 6</figref>) from MDS <b>320</b> substantially like image of image stream <b>25</b>. User <b>10</b> views an image of interest in image stream <b>25</b> and initiates instruction to "freeze" the image of interest. Since both reaction time on behalf user <b>10</b> and processing time on behalf of imager <b>100</b> has passed when such an instruction has been received, a frame <b>322</b> substantially like image <b>26</b> is saved, wherein the substantiality is temporal. The freeze instruction is forwarded to multimedia interface <b>135</b>. Therein, controller <b>200</b> instructs PCI interface <b>210</b> to forward the current frame <b>322</b> being handled to frame buffer <b>295</b>, also known as frame grabber, which stores frame <b>322</b>.
0074Frame buffer <b>295</b> can be implemented as an integrated circuit, capable of storing and identifying a plurality n of frames <b>322</b> corresponding to maximum number n frames capable of being stored according to the buffer size (hereinafter frame <b>322<sub>1</sub></b> designating the first frame <b>322</b> stored, frame <b>322<sub>2</sub></b> the second, ...., and frame <b>322<sub>n</sub></b> the last frame stored). As shown, identification may be implemented by designating a first-in-first-out order, or similar order. Upon receipt of frame <b>322<sub>1</sub></b>, buffer <b>295</b> forwards a copy to router <b>220</b>. Router <b>220</b> multiplexes frame <b>322<sub>1</sub></b> with DDS <b>310</b> and streams the combined signal to the active output port for display as combined image. As described further herein in connection with image capture mode, user <b>10</b> is able to manipulate the position and size of the combined image within image stream <b>25</b> and on a display unit.
0075A copy of frame <b>322<sub>1</sub></b> is saved to HDD <b>120</b>, preferably in file format Joint Picture Experts Group JPEG layer corresponding to the MPEG layer such as MPEG, MPEG-2 and/or MPEG-4 layer, as a graphic file. A software application for conversion to other formats such as "TIFF," "BMP," and/or "FPX", or similar, is provided.
0076Upon additional instruction by user <b>10</b> to capture further images, further corresponding frames <b>322</b> are saved in the manner described. Therein, frame <b>322<sub>2</sub></b> is multiplexed with frame <b>322<sub>1</sub></b> and DDS <b>310</b> and the combined signal streamed to the active output for display on a display unit.
0077An archival mode is provided for review, edit and playback of documentary recording file and a graphic file. Therein, the recording file is accessed and a digital data stream corresponding to MDS <b>320</b> is read from media of HDD <b>120</b> and/or OMD <b>125</b> and is streamed to V-codec <b>230</b> and is decompressed into a digital data stream corresponding to DDS <b>310</b>. The decompressed digital data stream is then routed to the active port for display or transferred to MMB <b>115</b> for display on a display unit.
0078<figref idref="f0010 f0011 f0012 f0013 f0014 f0015">Figs. <b>8</b> through 13</figref> depict use of imager <b>100</b> through interactive display <b>400</b> of touchscreen <b>145</b> in accordance with one embodiment of the present invention. It should be appreciated that similar usage is achieved through use of other input devices, as for example mouse and keyboard, coupled to a display unit.
0079Upon powering of imager <b>100,</b> it is in standby mode and page <b>410</b> is shown as depicted in <figref idref="f0010">Fig. <b>8</b></figref> in accordance with one embodiment of the present invention. Display <b>400</b> is shown as page <b>410</b> providing a plurality of fields. The fields depicted include fields grouped to display surgical information <b>600</b>, multimedia status <b>610</b>, image panel <b>620</b>, and control panel <b>630</b>. In the embodiment illustrated, multimedia status <b>610</b> indicates that the optical media includes data saved from a previous surgical session. To use imager <b>100</b> for the current session and input patient data, user <b>10</b> touches the desired field in surgical information block <b>600</b> to include the requisite data.
0080Surgical information block <b>600</b> includes fields for biographical patient information, identity of treating physician, and additional information as may be convenient and necessary. If the field requires an alpha-numeric entry, alpha-numeric keyboard page <b>420</b> is displayed on top of page <b>410</b>, as depicted in <figref idref="f0011">Fig. <b>9</b></figref>. User <b>10</b> enters the data by touch. Field <b>422</b> contemporaneously displays the entry made. The user confirms the entry and is taken back to page <b>410</b> to make the next selection. If the desired field of frame <b>410</b> requires a numeric entry, numeric keyboard page <b>430</b> is displayed on top of page <b>410</b>, as depicted in <figref idref="f0012">Fig. <b>10</b></figref> according to one embodiment of the present invention. User <b>10</b> enters the data by touch. Field <b>422</b> contemporaneously displays the entry made. The user confirms the entry and is taken back to page <b>410</b>, now modified with additional date as depicted in <figref idref="f0013">Fig. <b>11</b></figref> according to one embodiment of the present invention, to make the next selection.
0081Returning to <figref idref="f0010">Fig. <b>8</b></figref>, control panel <b>630</b> provides functions to control imager <b>100</b> including recording video, capturing images, muting audio input, and conforming choices. Other functions as may be convenient and/or necessary may also be included.
0082To place device in recording mode to record a video file to a disk properly loaded in OMD <b>125</b>, user <b>10</b> touches the designated icon, here illustrated as icon <b>632</b>, in control panel icon <b>630</b>. In response, icon <b>632</b> may change design, such as becoming highlighted. Concurrently, in image panel <b>620</b>, image <b>25</b> is displayed in image window <b>624</b> and image status bar <b>622</b> displays confirmation of recording and elapsed recording as depicted in <figref idref="f0014">Fig. <b>12</b></figref>. Similarly if recording is initiated via CCU <b>30</b>, icon <b>632</b> becomes highlighted, image <b>25</b> is displayed in image window <b>624</b> and image status bar <b>622</b> displays confirmation of recording and elapsed recording, also as depicted in <figref idref="f0014">Fig. <b>12</b></figref> according to one embodiment of the present invention. To cease recording, user <b>10</b> touches highlighted icon <b>632</b> or engages the appropriate control on CCU <b>30</b>.
0083To mute audio recording or conversely engage audio recording, user <b>10</b> touches the designated icon, here illustrated as icon <b>636</b>, in control panel <b>630</b>. In response, icon <b>636</b> may change design, such as becoming highlighted. To reverse the action, user <b>10</b> touches icon <b>636</b> which in response may change design, preferably returning to its previous design.
0084To capture an image that is being displayed in image window <b>624</b>, user <b>10</b> touches the designated icon, here illustrated as icon <b>634</b>, in control panel <b>630</b> and imager <b>100</b> is placed in image capture mode and the instruction is processed as described above. In response, icon <b>634</b> may change design, such as becoming highlighted, and a confirmation message that the desired action is being performed will be displayed. Similarly, image capture mode can be accessed if user <b>10</b> engages the appropriate control on CCU <b>30</b> to capture image <b>26</b> that is being displayed in image window <b>624</b>. Instruction is processed as described above and icon - <b>634</b> may change design, such as becoming highlighted, and a confirmation message that the desired action is being performed will be displayed.
0085To allow user <b>10</b> to monitor available resources on a multimedia disc, multimedia status <b>610</b> includes fields identifying the remaining capacity of disc and the number of images captured and the length of time of image stream <b>25</b> recorded.
0086The archival mode is accessible from control panel <b>630</b> to review and/or manipulate saved documentary recording files and graphics file. User <b>10</b> touches the designated icon, here illustrated as icon <b>638</b>, and review page <b>440</b> is displayed as depicted in <figref idref="f0015">Fig. <b>13</b></figref> according to one embodiment of the present invention. Page <b>440</b> preferably includes a control panel <b>630a</b>, note area <b>670</b>, file directory <b>660</b>, image panel <b>620</b> including image window <b>624</b>, image status bar <b>622</b>, and controls <b>626</b>. Controls <b>626</b> preferably include pause, playback, print and confirmation means as well as elapsed time and other controls as deemed useful and/or necessary.
0087Therein, a graphic file may be printed to printer <b>45</b> located in the surgical suite so that it is immediately available for review by user <b>10</b>. Software is provided to enlarge and view and manipulate the graphic file such as providing visual contrast, inserting a scaled file object corresponding to a physical prosthesis and thus determining whether such prosthesis fits patient <b>5</b>.
0088In accordance with one embodiment of the present invention, sensor <b>318</b> (see <figref idref="f0004">FIG. 4</figref>) is provided for detecting an infrared remote control signal. Sensor <b>318</b> enables user <b>10</b> to remotely operate imager <b>100</b>. Sensor <b>318</b> is in communication with the processor or Microprocessor main board (MMB) <b>115</b>, the sensor receiving control signals to operate the medical instrument.
0089In accordance with one embodiment of the present invention, a system for speech recognition is provided for user <b>10</b> interaction with imager <b>100</b>. Microphone <b>40</b> receives user <b>10</b> voice commands, which are routed to speech recognition module <b>319</b>. Speech recognition module <b>319</b> is software that executes on Microprocessor main board (MMB) <b>115</b> to receive voice signals that control the medical instrument.
0090In accordance with one embodiment of the present invention, database module <b>321</b> provides a database server and/or database management application that is operatively provided to enable user <b>10</b> to access, manage and/or updates data in imager <b>100</b>. Database module 321 is software that executes on Microprocessor main board (MMB) <b>115</b> to establish control signals to operate the hard drive and the optical media drive.
0091In accordance with one embodiment of the present invention, expert system <b>323</b> is provided wherein characteristics of video signal <b>300</b> meeting predetermined characteristics are automatically brought to the attention of user <b>10</b> and/or saved as graphic files <b>810</b>. Expert system <b>323</b> is software that executes on Microprocessor main board (MMB) <b>115</b> to generate control signals to operate the medical instrument.
0092In accordance with one embodiment of the invention, a stereoscopic module <b>324</b> is provided for associating a plurality of video signals <b>300</b> with each other, as such providing stereoscopic images on a display unit. Stereoscopic module <b>324</b> is software that executes on Microprocessor main board (MMB) <b>115</b> to associate a plurality of files to provide stereoscopic images on a multimedia interface.
18 sheets
Sheet 1 Sheet 2 Sheet 3 Sheet 4 Sheet 5 Sheet 6 Sheet 7 Sheet 8 Sheet 9 Sheet 10 Sheet 11 Sheet 12 Sheet 13 Sheet 14 Sheet 15 Sheet 16 Sheet 17 Sheet 18
Every citation, both ways
| Document | Relation | Office | Cited during |
|---|---|---|---|
| US12167829B2 | Cited by | United States of America | Applicant |
| WO0008585A | Cites | World Intellectual Property Organization (WIPO) | – |
| US5029016A | Cites | United States of America | – |
| US5592216A | Cites | United States of America | – |
| US5697885A | Cites | United States of America | – |
| US5740801A | Cites | United States of America | – |
| US2002101507A1 | Cites | United States of America | – |
| US6184922B1 | Cites | United States of America | – |
| US6471649B1 | Cites | United States of America | – |
21 members in 6 offices
Priority claims3
| Document | Office | Kind | Date |
|---|---|---|---|
| 410456P | United States of America | – | |
| 41045602 | United States of America | P | |
| 0329012 | United States of America | W |
Members21
| Document | Office | Kind | |
|---|---|---|---|
| CA2498578A1 | Canada | A1 | |
| CA2752536A1 | Canada | A1 | |
| CA2752652A1 | Canada | A1 | |
| WO2004025963A1 | World Intellectual Property Organization (WIPO) | A1 | |
| AU2003274984A1 | Australia | A1 | |
| WO2004025963A9 | World Intellectual Property Organization (WIPO) | A9 | |
| US2004133072A1 | United States of America | A1 | |
| EP1550311A1 | European Patent Office (EPO) | A1 | |
| JP2005538779A | Japan | A | |
| US2006015008A1 | United States of America | A1 | |
| US2006050144A1 | United States of America | A1 | |
| JP2009081857A | Japan | A | |
| EP1550311A4 | European Patent Office (EPO) | A4 | |
| EP1550311B1This record | European Patent Office (EPO) | B1 | |
| CA2498578C | Canada | C | |
| JP2012142949A | Japan | A | |
| JP5613693B2 | Japan | B2 | |
| CA2752536C | Canada | C | |
| CA2752652C | Canada | C | |
| US8992418B2 | United States of America | B2 | |
| US9936862B2 | United States of America | B2 |
34 legal events, as 4 offices reported them to INPADOC
Over the term
Point at a mark for the eventEvents
| Event | Code | Office | |
|---|---|---|---|
| Lapsed in a contracting state [announced via postgrant information from national office to epo]LapsedPG25 | PG25 | EP | |
| Lapsed in a contracting state [announced via postgrant information from national office to epo]LapsedPG25 | PG25 | EP | |
| Patent expired after termination of 20 yearsExpiredPE20 | PE20 | GB | |
| Expiry of rightR071 | R071 | DE | |
| Opt-out of the competence of the unified patent court (upc) registeredP01 | P01 | EP | |
| Annual fee paid to national office [announced via postgrant information from national office to epo]GrantedPGFP | PGFP | EP | |
| Annual fee paid to national office [announced via postgrant information from national office to epo]GrantedPGFP | PGFP | EP | |
| Annual fee paid to national office [announced via postgrant information from national office to epo]GrantedPGFP | PGFP | EP | |
| Annual fee paid to national office [announced via postgrant information from national office to epo]GrantedPGFP | PGFP | EP | |
| Fee paymentPLFP | PLFP | FR | |
| Fee paymentPLFP | PLFP | FR | |
| Fee paymentPLFP | PLFP | FR | |
| Fee paymentPLFP | PLFP | FR | |
| No opposition filed against granted patent, or epo opposition proceedings concluded without decisionGrantedR097 | R097 | DE | |
| No opposition filedOpposition26N | 26N | EP | |
| No opposition filed within time limitOppositionORIGINAL CODE: 0009261PLBE | PLBE | EP | |
| Information on the status of an ep patent application or granted ep patentGrantedSTATUS: NO OPPOSITION FILED WITHIN TIME LIMITSTAA | STAA | EP | |
| Dpma publication of mentioned ep patent grantGrantedR096 | R096 | DE | |
| Designated contracting statesAK | AK | EP | |
| European patent grantedGrantedFG4D | FG4D | GB | |
| (expected) grantORIGINAL CODE: 0009210GRAA | GRAA | EP | |
| Grant fee paidORIGINAL CODE: EPIDOSNIGR3GRAS | GRAS | EP | |
| Despatch of communication of intention to grant a patentORIGINAL CODE: EPIDOSNIGR1GRAP | GRAP | EP | |
| First examination report despatched17Q | 17Q | EP | |
| Supplementary search report drawn up and despatchedA4 | A4 | EP | |
| Information provided on ipc code assigned before grantRIC1 | RIC1 | EP | |
| Information provided on ipc code assigned before grantRIC1 | RIC1 | EP | |
| Information provided on ipc code assigned before grantRIC1 | RIC1 | EP | |
| Request for extension of the european patent (deleted)DAX | DAX | EP | |
| Designated contracting states (corrected)RBV | RBV | EP | |
| Request for examination filed17P | 17P | EP | |
| Designated contracting statesAK | AK | EP | |
| Request for extension of the european patentAX | AX | EP | |
| Public reference made under article 153(3) epc to a published international application that has entered the european phaseORIGINAL CODE: 0009012PUAI | PUAI | EP |
Numbers
- Publication
- 1550311
- Application
- 37592532
Titles3
- German
- VIDEOAUFZEICHNUNGS- UND BILDERFASSUNGSEINRICHTUNG
- English
- VIDEO RECORDING AND IMAGE CAPTURE DEVICE
- French
- DISPOSITIF D'ENREGISTREMENT VIDEO ET DE SAISIE D'IMAGES
Classification
- CPC, 13
- A61B1/042
- A61B1/00022
- A61B1/00045
- A61B1/0005
- H04N5/765
- H04N5/77
- H04N5/775
- H04N5/781
- H04N5/85
- H04N7/183
- H04N9/8042
- A61B1/0004
- A61B1/00042
- IPC, 10
- H04N7 18
- H04N9 47
- A61B1 04
- H04N5 76
- H04N5 765
- H04N5 77
- H04N5 775
- H04N5 781
- H04N5 85
- H04N9 804
Designated states4
- Contracting states, 4
- Germany
- France
- United Kingdom
- Italy
