Universal input device

The present invention relates to a computer input device for producing smooth electronic ink. More specifically, the present invention relates to input devices that can be used on a variety of platforms while providing a general user interface.

Computing systems are dramatically changing the way people live. The first wave of computers was exorbitantly expensive and cost-effective only in the business environment. As computers became more affordable, the use of personal computers at work and at home became so widespread that computers became as common as desks in the office and kitchen tables at home. Microprocessors are incorporated into every aspect of people's daily lives, from their use in television and other entertainment systems to devices for coordinating the behavior of automobiles.

The evolution of computing devices, from data crunching devices that occupy the entire floor of a large office facility to laptop computers or other portable computing devices, is documented and informative. Has a dramatic effect on how is remembered. With such mobile computing, individuals can use their computing devices to type letters, draft memorandums, take notes, create images, and do a lot of work outside the office. It became possible to do. Professionals and non-specialists alike are empowered to work on the move with devices that meet their computing demands everywhere.

Computer systems that use regular computer systems, especially graphical user interface (GUI) systems such as Microsoft Windows®, use a keyboard (text) with one or more buttons for user selection. Optimized to accept user input from one or more individual input devices, such as (for input) and pointing devices (such as a mouse).

One of the first goals of the computing world was to have computers on every desk. For the most part, this goal has been achieved by the ubiquity of personal computers in the office workplace. With the advent of notebook computers and advanced personal data assistants (PDAs), office workplaces have been expanded to include a variety of unconventional sites where work can be reached. Increasingly, computer users must become masters of the diverse user interfaces for each of their computing devices. From the mouse and keyboard interface to a standard personal computer, to the simplified resilient stylus of the personal data assistant, to the minimal keys of a cellular phone, users have the basic skills. You are faced with various user interfaces that need to be mastered before they can be used.

Despite technological advances, most users tend to use paper-printed documents as their primary editing tool. Some of the advantages of printed paper include its readability and portability. Other benefits include the ability to share annotated paper documents and the ease with which printed paper can be archived. One user interface that bridges the gap between high-performance computing systems and the functionality of printed paper is the stylus-based user interface. One approach to stylus-based user interfaces is to use resistance techniques (common in today's PDAs). Another approach is to use an active sensor (active sensor) in the notebook computer. One of the next goals of the computing world is to give users a user interface for operating computers.

A drawback associated with the use of styluses is that such devices are tied to computing devices, including sensor boards. In other words, the stylus can only be used to generate an input when used in conjunction with a required sensor board. In addition, stylus detection is affected by the degree of proximity of the stylus to the sensing board (sensor board).

In the art, there is a need for portable computing devices that can function as input devices for any one of the various computing devices and can operate in a variety of situations.

<p> The present invention addresses one or more of the issues identified above, thereby providing users with a common user interface across a variety of computing platforms.</p>

<p> Aspects of the invention provide an input device for generating electronic ink and / or other inputs, independent of the device to which the data is directed. The input device can be formed in the shape of a pen and may or may not include an ink cartridge that facilitates the movement of the input device in a familiar manner.</p><p> The above overview of aspects of the invention, as well as the following detailed description of the various embodiments, will be better understood by reading in conjunction with the accompanying drawings. The accompanying drawings are included for illustration purposes, not as a limitation with respect to the claimed invention.</p>

Aspects of the invention are electronic inks that can control a desktop or notebook computer, write on a whiteboard, control a PDA or cellular phone, or be portable to any of the above platforms. With respect to input devices that can be used in a variety of different computing platforms. The following explanation is divided into the following several items. That is, terms, general-purpose operating environment, general-purpose pens and general-purpose cameras, active coding, passive coding, inertial sensors, additional components, and embodiments as samples.

Term Pen: Any writing tool that may or may not include the ability to store ink. In some examples, a stylus (needle brush) that has no ink capacity can be used as a pen according to an embodiment of the present invention.

Camera: Image capture system Active coding: Within an object (object) on which the input device is located to determine the position and / or movement of the input device using appropriate processing algorithms. Capture the surface code.

Passive coding: An image obtained from an object or surface on which the input device is moved using appropriate processing algorithms, other than the code captured to detect the movement / position of the input device. Use the data to detect the movement / position of the input device.

Input device: A device for inputting information that can be configured to generate and process information Active input device (active input device): Indicators using sensors built into the input device ( An input device that actively measures (signal) and produces data that indicates the position and / or movement of the input device.

Passive input device (passive input device): An input device whose motion is detected using sensors built outside the input device. Computing device: Any device configured to process information, including desktop computers, laptop computers, Tablet PCs , personal data assistants, phones, or input devices.

General-purpose operating environment FIG. 1 is a functional block diagram showing an example of a general-purpose digital computing environment that can be used to implement various aspects of the present invention. In FIG. 1, the computer 100 includes a processing unit 110, a system memory 120, and a system bus 130 connecting various system components including the system memory to the processing unit 110. The system bus 130 can be any of several types of bus structures, including memory buses or memory controllers, peripheral buses, and local buses that use any of the various bus architectures. System memory 120 includes read-only memory (ROM) 140 and random access memory (RAM) 150.

A basic input / output system 160 (BIOS) containing basic routines that help transfer information between elements inside computer 100, such as during boot, is stored in ROM 140. The computer 100 also includes a hard disk drive 170 for reading and writing to a hard disk (not shown), a magnetic disk drive 180 for reading and writing to a removable magnetic disk 190, and a CD. It also includes an optical disk drive 191 for reading and writing to a removable optical disk 192 such as ROM or other optical medium. The hard disk drive 170, the magnetic disk drive 180, and the optical disk drive 191 are connected to the system bus 130 by the hard disk drive interface 192, the magnetic disk drive interface 193, and the optical disk drive interface 194, respectively. The above drives and associated computer readable media provide the personal computer 100 with non-volatile storage of computer readable instructions, data structures, program modules, and other data. Other types of computer-readable media that can store computer-accessible data, such as magnetic cassettes, flash memory cards, digital video disks, Bernoulli cartridges, random access memory (RAM), and read-only memory (ROM), are also available. It will be recognized by engineers in the field that it can be used in an exemplary operating environment.

Several program modules are stored on hard disk drive 170, magnetic disk 190, optical disk 192, ROM 140, or RAM 150, including operating system 195, one or more application programs 196, other program modules 197, and program data 198. It is possible that it has been done. The user can enter commands and information into the computer 100 via an input device such as the keyboard 101 or the pointing device 102. Other input devices (not shown) can include microphones, joysticks, gamepads, satellite dishes (satellite dishes), scanners and the like. These input devices, and other input devices, are often connected to the processing unit 110 via a serial port interface 106 connected to the system bus, but are a parallel port, game port, or universal serial. You may connect with other interfaces such as Bus) (USB). Further, the above devices may be directly connected to the system bus 130 via an appropriate interface (not shown). The monitor 107, or other type of display device, is also connected to the system bus 130 via an interface such as the video adapter 108. In addition to monitors, personal computers typically also include other peripheral output devices (not shown) such as speakers and printers. In a preferred embodiment, a pen digitizer 165 and an accompanying pen or stylus 166 are provided to digitally capture handwritten input. Although showing a direct connection between the pen digitizer 165 and the serial port, in practice the pen digitizer 165 processes through a parallel port or other interface, and system bus 130, as is well known in the art. It is also possible to connect directly to the unit 110. Further, although the digitizer 165 is shown separately from the monitor 107, it is preferable that the usable input area of the digitizer 165 has the same spread as the display area of the monitor 107. In addition, the digitizer 165 can exist as a separate device that can be integrated into the monitor 107 or attached to or otherwise attached to the monitor.

Computer 100 can operate in a networked environment that uses logical connections to one or more remote computers, such as remote computer 109. The remote computer 109 can be a server, router, network PC, peer device, or other common network node, and is typically many of the aforementioned elements associated with computer 100, or Although all are included, only the memory storage device 111 is shown in FIG. The logical connection depicted in Figure 1 includes a local area network (LAN) 112 and a wide area network (WAN) 113. Such networking environments are common in offices, enterprise-wide computer networks, intranets (corporate networks), and the Internet.

When used in a LAN networking environment, the computer 100 is connected to the local network 112 via a network interface or network adapter 114. When used in a WAN networking environment, the personal computer 100 typically includes a modem 115, or other means, for establishing communication over a wide area network 113, such as the Internet. The modem 115, which can be internal or external, is connected to system bus 130 via the serial port interface 106. In a networked environment, the program module drawn in connection with the personal computer 100, or a portion of the program module, can be stored in a remote memory storage device.

It will be appreciated that the network connections shown are exemplary and other techniques for establishing communication links between computers can also be used. The existence of any of a variety of well-known protocols such as TCP / IP, Ethernet, FTP, HTTP, etc. is envisioned, and the system allows users to retrieve web pages from web-based servers. It is possible to operate in a client-server configuration so that it becomes. Any of a variety of traditional web browsers can be used to view and manipulate data on web pages.

General Purpose Pens and Cameras Figure 2 presents exemplary embodiments of input devices for use in accordance with various aspects of the invention. Some different elements and / or sensors are described below. Aspects of the invention can be practiced using a combination of various sensors. In addition, additional sensors can be included, including magnetic sensors, accelerometers, gyroscopes, microphones, or any sensor capable of detecting the position of the input device with respect to a surface or object (object). In FIG. 2, the pen 201 is an ink cartridge 202, a pressure sensor 203, a camera 204, an inductive element 205, a processor 206, a memory 207, a transceiver 208, a power supply 209, a docking station 210, and a cap 211. And the display 212. Various components can be electrically connected as needed, for example using a bus (not shown). Pen 201 is a desktop computer, laptop computer, Tablet It can act as an input device for a set of devices, including a PC , a personal data assistant, a telephone, or any device capable of processing and / or displaying information.

The input device 201 can include a standard pen and an ink cartridge 202 for writing or drawing on paper. In addition, the user can use the device to generate electronic ink while manipulating the input device in a pen-specific manner. Thus, the ink cartridge 202 provides a comfortable and familiar medium for generating handwritten strokes on paper, while pen movements are recorded and used to generate electronic ink. The ink cartridge 202 can be moved from the retracted position to the writing position using any of several well-known techniques. Alternatively, the ink cartridge 202 does not contain ink, such as a plastic cartridge with a rounded tip, but is replaced with a cartridge that allows the user to move the pen to the surface without damaging the pen or surface. Can be done. Further, for example, an inductive element or a group of inductive elements may be included to help detect the relative movement of the input device by providing a signal indicating the input device in a manner similar to the signal generated by the stylus. It is possible. It also includes a pressure sensor 203 to indicate the input that would be shown when the pen is pressed while located on the object, as it can be reached, for example, by selecting a mouse button input. It is possible to facilitate the selection or instruction of objects. Alternatively, the pressure sensor 203 can use a pen to detect the depressing force (pressing pressure) that forms the stroke for use in varying the width of the electronic ink produced. .. Further, the sensor 203 can activate the operation of the camera. In an alternative mode, the camera 204 may operate independently of the settings of the pressure sensor 203.

Further, in addition to a pressure sensor that can act as a switch, it is possible to include additional switches that make various settings to control the operation of the input device. For example, one or more switches are provided on the outside of the input device to power on the input device, activate the camera or light source, control the sensitivity of the sensor, or the brightness of the light source, to the text. Set the input device to sketch mode (drawing mode) where conversion is not performed, set the device to store the input data internally, process and store the input data, the other party with whom the input device can communicate It can be used to transmit data to processing units such as computing devices, or to control any configuration possible.

A camera 204 can be included to capture an image of the surface on which the pen is moved at the top. The inductive element 205 can also be included to improve the performance of the pen when used as a stylus in an inductive system. Processor 206 can consist of any well-known processor for performing functions related to various aspects of the invention, as described in more detail below. Similarly, memory 207 can include any memory device for controlling RAM, ROM, and / or devices for storing data, or for storing software for processing data. .. The input device can further include transceiver 208. Transceivers allow the exchange of information with other devices. For example, Bluetooth technology, or other wireless technology, can be used to facilitate communication. Other devices can include computing devices that can further include input devices.

A power source 209 can be included to power the device on which data is processed, stored, and / or displayed when the pen 201 is used independently of or remotely from the host device. Can be supplied. The power supply 209 can be integrated into the input device 201 at any position and is arranged for immediate replacement if the power supply is replaceable, or facilitates charging if the power supply is rechargeable. Can be arranged as such. As an alternative, the pen can be used as an alternative power source, such as an adapter for electrically connecting the pen 201 to a car battery or a charger connected to a wall outlet, a computer power source, or any other power source. You can connect.

Docking station link 212 can be used to transfer information between an input device and a second device, such as an external host computer. The docking station link can also include a structure for recharging the power supply 209 when connected to a docking station (not shown) or when connected to a power source. A USB or other connection can be made to detach the input device to the host computer via the docking station link or through an alternative port. Alternatively, the pen can be connected to the device using a hard wire connection to transfer or receive data. In wired configurations, the docking station link is removed when you choose to connect the input device directly to the host by wire. Docking station links can be excluded or replaced using another system (eg, Bluetooth 802.116) to communicate with another device.

The input device 201 is removable and may include a metal tip to facilitate resistance-based sensing so that the input device 201 can be used, for example, with a device that includes a sensing board. It is possible to further include cap 211. The body of the input device 201 can be composed of plastic, metal, resin, or a combination thereof, or any material that can provide protection to the components of the input device or to the overall structure. is there. The chassis can include metal dividers to electrically shield some or all of the sensitive electronic components of the device. The input device can have an elongated shape that can accommodate the shape of the pen. However, the device can be formed in a variety of shapes suitable for use as an input device and / or an ink generating device.

3A-3C depict three exemplary embodiments of a camera for use in accordance with aspects of the invention. As mentioned above, the input device 201 can be used, for example, to generate electronic ink by detecting the movement of the pen using a camera. The camera 304 can capture an image of the surface on which the pen moves at the top and include it to detect the amount of movement of the pen on the surface being scanned through image analysis. The movement is correlated with the document, and it is possible to electronically transfer, add, or associate electronic ink to the document (eg, remember the annotations entered separately from the original document). ..

As shown in FIG. 3A, in one embodiment, the camera 304 includes, for example, an image sensor 320 composed of an image sensing element array (imaging element array). For example, a camera can consist of a CMOS image sensor capable of scanning a square area (local, area) 1.79 mm long and 1.79 mm wide with a resolution of 32 pixels long and 32 pixels wide. The minimum exposure frame rate (screen rewrite speed) for one such image sensor is approximately 330 Hz, while an exemplary image sensor can operate at a processing speed of 110 Hz. The image sensor selected may include a color image sensor, a grayscale image sensor, or behave to detect brightness above a single threshold. However, the choice of camera, or camera component, is necessary to take into account performance, cost, etc., or to accurately calculate the location of the input device, based on the desired operating parameters associated with the camera. It will vary based on other considerations that may be dictated by factors such as the alleged resolution.

Light source 321 can illuminate the surface on which the input device moves. The light source can consist of, for example, a single light emitting diode (LED), an LED array, or other light emitting device. The light source can produce monochromatic light, including white, or can produce multiple colors. A half mirror 322 can be included inside the camera to direct the light as desired. The camera 304 may further include one or more optical devices 323 to focus the light from the light source 321 to the scanned surface 324 and / or to focus the light reflected from that surface to the image sensor 320. It is possible.

As shown in FIG. 3A, the light emitted from the light source 321 is reflected by the half mirror 322, which is a mirror that reflects or transmits the light depending on the direction of the incident light. The reflected light is then directed to the lens system 323 and sent to the reflective surface below it. The light is then reflected on its surface, passes through the lens system 323, hits the half mirror 322 at a transmission angle through the mirror, and enters the sensing array 320. Of course, image data can be captured using a camera that includes a wide variety of components, including a camera that incorporates fewer or more components. There can be many variations in the configuration of the components. To give just one example, in a simplified configuration, the light source and the sensing array can be placed together so that both the light source and the sensing array are located on the surface where the image should be captured. In that case, no reflection is needed inside the camera, so the half mirror can be removed from the system. As shown in FIG. 3B, in a simplified configuration, the light source 321 is located at some distance from the lens 323 and the sensor 320. In a more simplified configuration, as shown in Figure 3C, the light source can be removed and the peripheral light reflected by the surface of the object is focused on the sensor 320 by the lens 323.

Therefore, the components incorporated in the camera or the modification of the arrangement of the components can be used in a manner suitable for the aspects of the present invention. For example, the placement and / or orientation of the camera and / or cartridge can differ from that shown in FIG. 2 to allow for a wide variety of camera and / or ink configurations and orientations. For example, the camera 304, or any component of the camera 304 component, can be placed in an opening adjacent to the opening provided for the ink cartridge, rather than in the same opening as shown. .. As a further example, the camera 304 can be placed in the center of the input device and the ink cartridges can be placed beside the camera. Similarly, the light source 321 can be incorporated into a structure that houses the remaining components of the camera, or one or more components can be placed separately from the other components. In addition, light projecting using light sources and / or optics, with additional components and / or software as needed, or with modifications. feature) can also be enabled.

Actively Coding To aid in the detection and / or positioning of an input device, the surface of the object on which the input device is placed can contain image data representing the relative position of the area on that surface. In one exemplary embodiment, the scanned surface can include a display of a host computer or other external computing device, which display is a desktop computer, laptop computer, Tablet PC . It can accommodate monitors for personal data assistants, phones, digital cameras, or any device that can display information. Therefore, Tablet The writing paper or other image generated on the screen of a PC corresponds to a sign representing the relative position of that part of the document within the entire document, or contains data about any other part of the image. It is possible. The information can consist of an image that can contain alphanumeric characters, coding patterns, or any discriminable pattern of image data that can be used to indicate relative position. The image or group of images selected for use in specifying the location of an area within the surface of an object is the pixel resolution of the sensor and / or the pixel resolution of the image data contained within the surface being scanned. It is possible to depend on the sensitivity of the scanning device built into the camera. The location information extracted from the object can then be used to track the movement of the input device on the object. That information can be used to accurately generate electronic ink or other information that corresponds to the movement of the input device. The position information can be used to detect the position in the image where the input should be made and to provide an indication of the movement of the input device on the surface of the object. For example, the information provided can be used interactively with word processing software to generate changes in a document.

In an alternative embodiment, the object used in combination with the input device can consist of, for example, paper with location information in the background. Positional information is sensed by sensors associated with the input device and can be incorporated in any form of code, optical representation, or other form that can be used to represent the relative position of a particular part of the paper. Is.

FIG. 4 shows an exemplary technique for coding the position of a document. In this example, the background of the image can include thin lines that form a maze-like pattern when viewed in large chunks. Each collection of lines in a maze design consisting of several thin lines with unique orientations and relative positions can indicate, for example, the position of that part of the maze pattern with respect to other parts of the document. Decoding of the maze pattern seen in the captured image can be performed according to a number of decoding schemes. In one embodiment, a particular configuration and collection of lines can be decoded to generate location information. In another embodiment, a lookup table containing data that identifies the location of the captured data, extracts a code corresponding to the sampled pattern from the image, and uses that code to locate the area. It can be derived by referring to the address of.

Passive coding Even in the absence of a position code, the image captured by the image sensor can be analyzed to determine the position of the input device at the time of image capture. Successive images can be used to calculate the relative position of the input device at different points in time. Correlation of this information can provide an accurate trajectory of the input device on the substrate. This locus information can be used, for example, to generate electronic ink that accurately represents a handwritten stroke.

FIG. 5 illustrates a locus pattern that can generate electronic ink based on this. In this example, the first captured image is the first time t₁First position of the input device in p₁It is possible to include a part of the maze pattern indicating. The next captured image is at the second time t₂Second position in p₂In this example, it is possible to include a part of encoded image data which is a different part of the maze pattern, which provides the position information of. The third captured image contains the third part of the maze pattern, thereby the third time t.₃Third position in p₃It is possible to indicate the location of the input device at. Using this data, the above three points are at time t₁From t₃It is possible to show the trajectory of the input device up to. An algorithm for estimating the inking pattern followed by an input device can be applied to generate electronic ink. The complexity of the applied algorithm can specify the accuracy of the ink produced. For example, a basic inking algorithm can simply connect dots with straight lines of the same thickness. Processes an algorithm that takes into account the previous sampling point, the time between samplings, or other data indicating the speed or acceleration at which the input was moved, data indicating the pressing force used, or any other reasonable data. It is possible to provide electronic ink that more accurately represents the actual movement of the input device (eg, from another sensor).

The optical scan performed by the camera 304 can generate the data needed to locate and identify the input device at various points in time, and that information can be used to generate electronic ink. In one exemplary embodiment, time t₁The image captured in the time t₂By comparing with the image captured in₁From t₂It is possible to provide data showing the distance the pen has moved from one point to another during the period up to. The above two data points, and / or the relative distances moved, can then be used to generate a locus of movement of the input device to generate an electronic ink that represents a handwritten stroke. Comparing parts of two or more images, or captured images, to calculate relative movement can also be performed by different analyzes. In that case, features appearing in multiple images can be compared, and the relative movement of features or groups of features from one location to another in those images gives an accurate indication of pen movement. It is possible to be provided. If irregular sampling cycles are used, modify the processing algorithm to compensate for fluctuations in the sampling cycle to more accurately correlate the movement of the input device with the actual time required for each movement. be able to. Information indicating the speed of movement may help to produce an electronic ink of the appropriate thickness.

According to such an embodiment, an object or image data indicating the movement of the input device on the display, mouse pad, desktop, or the surface of the computing device is extracted from the surface on which the input device moves. It is possible to include any non-uniform reflective surface that is possible. The tracking algorithm that can process the captured image data can be fixed or can vary depending on the characteristics of the captured image. Using a simple tracking algorithm, the processor can detect, for example, the grain of wood on a desktop, relative to a particular pattern of grain in a continuous image based on a comparison of image sequences captured by the camera. Positions can be used to detect the position of the input at various points in time and / or the relative movement of the input device on its surface. If the features in the image are not so easily discerned and the image is uniform, a more complex tracking algorithm may be needed.

Hardware Architecture FIG. 6 is a diagram showing a hardware architecture of a system according to an embodiment of the present invention. Many of the same or related components as shown in the above embodiments are represented using similar reference numerals. Processor 606 can consist of any well-known processor for performing functions related to various aspects of the invention. For example, the processor is FPSLIC It can include the AT94S40 and can consist of an FPGA with an AVR core. This particular device contains a 20MHz clock and is capable of operating at speeds of 20MIPS. Of course, the choice of processor for use inside the input device 601 may be dictated by system cost and / or processing speed requirements. Processor 606 can perform such analysis when image analysis is performed inside the input device. Alternatively, the processing can be done by a second processor, such as a digital signal processor (DSP) built into device 601. Processor 606 should take essential steps to reduce power consumption to preserve the power stored in power supply 609, such as shutting down the power of various components when the input device is inactive. It is possible to operate further and the inactivity of the input device can be based on data indicating the movement and / or position of the device. Processor 606 calibrates and adjusts the performance of various components, including, for example, adjusting to the brightness of the light source or to the sensitivity of the camera's sensing array 604. It is possible to work further. Also, the processor, or coupled digital signal processor, can choose from a plurality of stored image processing algorithms and, for example, move according to the surface-related properties in which the device is moved at the top. It can be controlled to select the most suitable image analysis algorithm to detect. Therefore, the algorithm can be automatically selected based on the behavioral considerations programmed into the input device. Alternatively, the input device may be controlled and the settings established based on the user's choice, for example, entered via the activation of a force sensor or based on the handwritten stroke corresponding to the command. Possible

In one embodiment, memory 607 stores one or more RAMs, ROMs, flash memories, or software for storing data, software for controlling devices, or software for processing data. It is possible to include any memory device or group of memory devices to do so. As described above, the data representing the location information can be processed inside the input device 601 and stored in the memory 607 for transfer to the host computer 620. Alternatively, the captured image data can be buffered in memory 607 inside the input device 601 for processing or for transfer to the host device 620 for another purpose.

The transceiver, that is, the communication unit 608, can include a transmitting unit and a receiving unit. As mentioned above, the information representing the movement of the input device processed in a form suitable for generating and / or displaying electronic ink, or processed in another form, is the above-mentioned desktop computer, laptop computer. , Tablet PC , personal data assistants, phones, or other such devices where user input and electronic ink may be useful can be transmitted to the host computer 620. Transceiver 608 is an external device for short-range wireless communication, infrared communication, using any wireless communication technology, including Bluetooth technology, or using cellular technology or even other long-range wireless technology. Can communicate with. Alternatively, the transceiver 608 can be used via a direct link to the host computer, such as via a USB connection, or docking. It is possible to indirectly control the transmission of data via cradle) 630. The input device can also be wired to a specific host computer using a dedicated connection. The transceiver 608 can also be used to receive information and / or software, which in one embodiment can be used to improve the performance of the input device. For example, program information for updating the control function of the processor can be uploaded via any of the techniques described above. In addition, software can be transmitted to the input device and can be downloaded from an external device, including software for analyzing image data and / or calibrating the input device.

Processor 606 can operate according to the interaction model. The interaction model can be implemented in the form of software to ensure that the unit maintains a consistent experience of producing electronic ink regardless of the external device acting as the input device. Interaction models are desktop computers, laptop computers, tablets On any host device, including PC , personal data assistants, phones, whiteboards, or on any device capable of storing, viewing, or recording data entered through an input device. The captured data can be processed to convert it into a form that is generally suitable for use. The processor 606 can recognize the device to which it is connected, or the device that is intended for data representing handwritten input, and based on such recognition, the appropriate form for the recognized specific device. You can select the process to convert the input data to. In that case, the conversion to a form useful to the computing device that may be the recipient of each is contained within the input device and provided as needed. Recognition of the receiving device of interest can be achieved as a result of communication between the devices if they are connected wirelessly or directly. Alternatively, the user can enter the identification of the device or group of devices to which the data is directed directly into the input device. Of course, if the input device includes a display, the data can be processed using default processing algorithms suitable for use with that display and / or many other devices.

Inertia sensor As shown in FIG. 7, the input device 701 may include one or more inertial sensors 715 for sensing the movement, position, or orientation of the pen, in addition to the aforementioned components represented by similar references. It is possible. For example, the input device 701 can include a gyroscope to provide data representing the angular velocity of the pen in multiple directions. The input device 701 can include one or more accelerometers, or a set of accelerometers, that measure the acceleration or gravity applied to the pen. Data representing the movement of the pen can also be obtained using a magnetic sensor that measures the movement of the pen by detecting fluctuations in the measured values of the earth's magnetic field. Since this sensor detects the movement of the input device based on data other than image data, it is described herein as an inertial sensor. Either with an input device that can include a gyroscope, accelerometer, magnetic sensor, inductive element, or any device or group of devices for measuring the movement of the input device, or an inertial sensor built into the input device. , Or data from any one can be used in combination with data from the camera to obtain data representing the movement or position of the input device, thereby generating data for generating electronic ink. Can be done.

Embodiments as Samples I. Input devices can operate using active coding A. Active coding provides position information about an entry of input information on a display or on another writing surface. As mentioned above, the surface of an object on which the input device is placed and / or moved can contain encoded image data indicating the location or relative position of each area within that surface. .. The object can include the display of a computing device such as a laptop computer. In one embodiment, the document can be recalled from memory and displayed on the screen. In the document, it is possible that coded information indicating the position of each area of the document is encoded in the background or the like. For example, the background of the document is a maze pattern (maze) It is possible that a sufficiently large portion of the pattern, including pattern), uniquely identifies each area in the entire document. Using the input device in combination with this encoded location information, even if the laptop display does not include a sensor to detect the movement of the input device on the screen, you can annotate or annotate the document at the specified location. It is possible to make edits. Thus, an input device can act as an "active input device" so that the sensors associated with the input device generate data indicating the location or location of the device.

In one example, an image sensor embedded in the input device captures image data representing the surface of the display screen on which the input device is placed and / or moved. The sensor captures an image containing a position code indicating the relative position of the input device. When the user moves over the displayed image, annotates and / or edits the displayed electronic document, the input device picks up the input and the location in the document where the input should be incorporated. Generates a signal that represents the data to be represented. Control of the laptop can also be done using an input device instead of a mouse, or can perform other standard input functions, including cursor movement and selection activation.

The input device can be used in combination with word processing software to edit the document, for example by deleting the text and inserting new text. To edit a document displayed on the screen of a computing device, the user places the input device on the screen in the desired position. To delete the text, the user can move the device by locating the input device near the screen and canceling the image of the displayed text. By sensing the position code, the image can be processed to determine that the pen was moved in a undo motion, and the text corresponding to where the user moved the input device can be identified. .. Therefore, the input can be used to erase that data.

The user may then want to insert new text. In a familiar way, the user can draw a "carrot", a symbol for inserting text, such as an upside-down "V", where the new text should be inserted. The processing software for converting the input stored in the input device or host computer into image data and / or commands recognizes the symbol as a control signal for inserting text. Then, with the help of the input device, the user can manually write the text to be inserted.

In an alternative embodiment, the user can add a note with a highlight pointing to the source text to which the annotation is relevant. For example, the user can select the text (sentence) to be highlighted using the pull-down menu displayed on the screen or the highlight button. The input device is then dragged onto the text to be selected for highlighting. The comment that should be associated with the highlighted / selected text can then be written on the screen of the display at a position adjacent to the highlighted text. When the operation is complete, the user can select the prompt required to complete the annotation entry. All of the above changes to the document can be made using the input device, whether or not the display includes a sensor to detect the movement of the input device.

Changes to the document can be displayed and / or incorporated within the document in the form of image data, electronic ink, or data that is converted to text. Converting the input to text is done in a way that is invisible to the user, so that the text can appear in the display of the document on the screen as it is typed. Alternatively, the user's handwriting can appear within the body of the document. To achieve immediate display of edits, information representing pen movements and the location of such edits can be continuously transmitted to the laptop device.

As mentioned above, the identity of the individual making the input can also be recorded. For example, an input device can generate information that identifies a user and / or a particular input device. The identification information can be added to the generated input data. Alternatively, such identification information can be provided as a separate signal transmitted to the host device.

The exemplary embodiment described above identifies the surface on which the input device is moved as the display of the laptop device, but the input device is embedded within the surface of any object in which the device can be moved on top. It is also possible to function to detect the position using the code. Therefore, the input device can be used in combination with a desktop computer, Tablet PC , personal data assistant, telephone, or monitor of any device capable of displaying information to generate an image incorporating a position code. And / or can be edited. The encoded information can also be incorporated within a transparent sheet placed on the display screen of such a device, or within a surface that can be used in combination with a display, including a protective film. Is.

In addition, the encoded information can be incorporated on a writing surface such as paper or on a writing material so as to uniquely identify a position on the surface thereof. For example, it is possible to incorporate location information into the background of paper. As mentioned above, the position information can include any form of indication or code that represents the relative position of a particular part on the paper. Therefore, the input device can be used in connection with the encoded paper to record the information corresponding to the user's handwriting in the appropriate position. For example, you can use the input device to draft a letter to a customer while you are in a taxi, equipped with only an input device and a writing surface that incorporates coded location information. When writing on paper using an input device, gestures corresponding to text or other input information are recognized by detecting changes in the position of the input device at a given point in time. The input can then be converted to electronic ink or other electronic data for use in generating the information corresponding to that gesture. Input conversion can occur as the input is generated, either inside the input device or when received by a host computing device connected to the input device. Alternatively, such a conversion can be done at a later point in time. For example, information generated using an input device can be stored in memory for proper processing at a later point in time and transmitted to the recipient and / or host computer.

The data generated using the input device, whether it is a handwritten letter, symbol, word, or other written image, is the position identified by the position code. Can be incorporated into the document with. Therefore, even if the formatted template does not exist, the layout of a document such as a letter described above can be realized by using the position information that determines the position in the document in which the information should be input. For example, the address of the drafter, the address of the recipient, the body of the letter, and the remaining components can be entered on the form in the appropriate positions. The position information captured and encoded by the scanner (image reader) is used to incorporate words or other images that form the content corresponding to the electronic document in the appropriate position.

The detected location information can also be used by the input device to interact with the host computing device to enter commands, make selections, and so on. If the computing device is a mobile camera or phone with web browsing characteristics, the input device can be used like a stylus or mouse to make selections from the displayed buttons or menus. Therefore, you can use the input device to launch a browser on the host computer and select the option to retrieve files such as the documents mentioned above, even files that are stored remotely. The input device allows the user to choose to download a file that contains the information they need. The user can then enter annotations on the file or set of files downloaded via the input device. If the input device is prepared to communicate with the remote computing device, those edits can be transmitted to the remote location from which the file was downloaded. Alternatively, assuming the input device is communicating with the host computing device, you can use that edit to edit the files stored inside the input device and / or the host computing device.

In another embodiment, the file displayed on the monitor of the host computing device can be a spreadsheet generated using spreadsheet software such as EXCEL . Position codes can be used to associate a position with a given cell in a spreadsheet (spreadsheet). The user can enter a number entry in the cell displayed on the screen. At that point, the input device captures the image associated with the location of the input device and transmits that information to the host computing device. For example, processing software that resides in a host computing device and is running in combination with spreadsheet software identifies the cell selected for entry based on the detected position code and spreads accordingly. Change the sheet document.

Input devices can also be used to recall images, or other pre-stored information, associated with a particular gesture, or a particular combination of gestures. For example, an input device can be used to draw a symbol that is programmed to be recognized by a processing algorithm device. A maze pattern can be used to accurately detect the movement of an input device on that pattern and to detect specific symbols associated with that movement. For example, a user can control an input device to draw on paper a symbol that he or she has previously identified to be associated with a company logo. The maze pattern allows the combination of movements corresponding to the letter "S" immediately following the letter "M" to be identified as an instruction to specify the input of the Microsoft Corporation logo. As a result, such pre-stored information can be entered into the document by inputting a sequence of previous inputs.

B. Active coding can provide location information if the host computing device includes a sensor that senses the movement of the input device. The input device can also be used as a passive input device. .. In this mode, the input device can be used, for example, in combination with a computing device that senses the movement of the input device using resistance-based sensing. Tablet When used in combination with a device that includes a sensor board to detect movement of the input device, such as a PC or personal data assistant, the input device can act like a stylus. Input devices allow the input device to be positioned very close to the screen to generate electronic ink or other image data. The control function can also be input in the same manner. In addition, the image displayed on the monitor of a particular computing device can also contain data corresponding to a code representing the relative position of that part of the document. The position information extracted from the object using the camera can then be used to track the movement of the input device as an alternative to or in combination with the movement detected using the sensors of the computing device. it can.

For example, the user is Tablet You may wish to generate or modify an image on a portable computing device that already includes the ability to detect the location of an input device such as a PC or Personal Data Assistant. The input device can only function as a passive input device so that information representing the movement of the input device is generated by the computing device. However, the sensor of the computing device may not have the ability to detect the range of pen movement required by the user in a given situation. For example, if the user is moving in an unstable vehicle, accurate detection of user input may be hindered. When a user edits a file by moving the input device on the screen of the computing device's display, the input device can be shaken and displaced considerably from the sensor board. Even if the signal generated by the sensor of the computing device is not very accurate, the image data captured by the input device is used to detect the movement of the input device in a plane horizontal to the surface of the computing device. be able to. Even if the sensor of the computer device can no longer detect the movement of the input device, the image sensor of the input device maintains an accurate representation of the movement of the input device to reflect the input intended by the user. It can generate enough information. Thus, an input device can function as a passive input device, or an active input device, even when used in combination with a computing device that includes the ability to sense the movement of the input device.

II. Input devices can operate using passive coding techniques A. Passive coding provides location information for inputting input information on a display or other writing surface. It is also possible to use the input device in connection with any paper, writing surface, or other carrier to record information corresponding to the user's handwriting. Again, equipped with only an input device and a writing surface, the input device can be used to draft a letter to the client. In this case, the motion of the input device is detected using a code other than the code embedded in the image of the surface of the carrier, and the gesture is detected by relying on passive coding. For example, the user can draft characters on plain paper. When the user writes using the input device, the image sensor captures the image on the paper. Objects in the image can be identified, and the movement of the objects in the captured image indicates the movement. The perceived objects can include watermarks on the paper, or other artifacts or other objects on the surface of the paper that can accommodate other imperfections. Alternatively, the paper can include borders that can also be used to calculate the movement of the pen on the surface. It is possible to identify the relative movement of the input device even in the absence of paper. It is possible to move the input device over the surface of the desk and the grain of wood to provide the objects needed to detect the relative movement of the input device. In a manner similar to that described above, the user can draft characters on paper or on any surface where motion can be optically detected on top. The movement of the input device can be stored in memory and / or converted into information representing its gesture.

In yet another embodiment, the mobile device can be used as an alternative to the mobile computing device. For example, while a technician is traveling by train to meet the rest of the design team, he has successfully devised a solution to a circuit failure associated with his company's pacemaker, but a laptop, or other compute. The ing device is not available and it is possible to use the own input device as a suitable alternative for recording one's thoughts. Making the most of the available time (and removing or capping the ink cartridges), the user sketches on the back of the chair in front of him, showing changes to the suspected electrical circuit. write. The user boots the input device, sets the input device to a good mode for generating sketches (which may include, for example, disabling the conversion), and provides a solution to the problem. Start sketching the simplified design to represent. The input device can then store a file representing the handwritten strokes. You can switch from sketch mode and write down annotations and references (references, references) next to the appropriate parts of the sketch, and the entries will be included in the image file. For example, the user can switch to an annotation mode in which the gesture corresponding to the character is recognized. This allows the user to incorporate a description of the solution he or she has proposed along with the sketch. Instead of waiting until they arrive at the medical research center, the operator draws a diagram for contemplation prior to the scheduled meeting.You can choose to transmit the face to the rest of the design team. Such transmission can be performed in any way, including uploading the modified document from the input device to a portable radio device such as a cellular phone. The information can then be used to generate image files such as VISIO documents.

Once transmitted to the rest of the team members, the aforementioned file corresponding to the sketch of the drawing can be displayed on the monitor of the team member's host computing device. For example, an image and accompanying text can be presented on the display of a desktop computer. By positioning the input device close to the image of the file displayed on the monitor, additional annotations can be added to the displayed annotations. In that case, the motion of the input device can be detected by measuring the relative motion of the object in the image captured by the optical sensor of the input device. The signal generated by the input device can be processed by software stored inside the input device or transmitted to the host computing device for processing. The processing of the detected motion can generate other data representing electronic ink, text, or annotations input through the input device.

B. Passive coding can provide location information if the host computing device includes a sensor to detect the movement of the input device. It can be used in conjunction with computing devices that have sensors for detecting motion. For example, the input device can be used as a source for generating handwritten annotations on personal data assistants or other computing devices designed for use with a stylus. Therefore, the user is in the process of completing an existing "to do" list. It is possible to be reminded of "list)" and want to add an item. The user retrieves a list stored in a host computing device such as a personal data assistant. Positioning the tip of the input device on the display of the Personal Data Assistant, the user can choose to traverse the menu and retrieve the desired list. When presented with the list, the user can enter a check in the empty box located next to the description of the already completed task on the screen of the host device. The input device captures an image of the screen containing the data corresponding to the box and transmits the data to the host computing device. Next, using a processing algorithm for analyzing the image data, the host computing device detects the shape of the box as an object that can be input. In order to successfully enter the checkmark, the image data is processed to detect the movement of the pen on and within the area of the box, which is a gesture that forms the recognized shape of the "check". The host device then modifies the files associated with the list to include a check indication in the box. Positioning the input device on the space following the last item in the list, the user enters text that describes the additional item. The sensor of the host device detects the movement of the input device and generates data representing the input. The input is converted to text and displayed to the user with an empty box.

Similarly, users of Microsoft Reader, such as students reading homework novels, may want to write down the annotations next to the relevant text. Annotations are written to the image displayed on the monitor of the mobile host device using the input device. For example, the user locates the input device on the monitor of a host computer, such as a Tablet PC , and enters a handwritten annotation next to the relevant text. This gesture is detected by the sensor of the host device, converted into image data, and stored as electronic data displayed on the screen. Annotations can remain in handwritten form or can be converted to alphanumeric characters. Annotations can be activated to view attached comments, or without activating additional features such as positioning the input device on top of highlighting the existence of the annotation or any other indication. Can not be seen. Therefore, the memorandum can be stored in a separate file or with a copy of the electronic version of the novel stored inside the host computer.

III. Additional sensors can generate additional information indicating the relative position of the input device. In yet another embodiment, information from additional sensors that form part of the input device is used to complement or completely replace other forms of motion detection. Is possible. Such additional sensors can detect linear acceleration, velocity, rotation, pressing force, tilt, electromagnetic field changes, or any perceived indication of the movement or position of the input device. Such information may be useful in efforts to generate more accurate motion detection. Alternatively, additional sensors can provide only the information available at a given time. For example, the input device can be used with an overall uniform surface, such as blank paper. In such cases, the image captured by the optical sensor may provide information that is insufficient to evenly and accurately detect the movement of the input device. When optical motion detection becomes more difficult, such as when it becomes more difficult to detect an object for tracking the motion of an input device, according to one embodiment for optically detecting motion. For example, additional information from additional sensors can be used to provide more accurate motion detection. Specifically, the algorithm or set of algorithms used to identify position and / or motion incorporates calculations that take into account additional information, which, in the case of optical motion detection, motion and / or It is possible to complement the position detection.

If optical detection cannot provide useful results, additional sensors can provide only the information used to detect motion. For example, if a user attempts to sketch a drawing on a uniform white tabletop of a laminated countertop, the optical sensing system may not be able to provide sufficient data to represent the motion. In that case, additional sensors can provide enough information to generate a tentatively satisfactory and accurate representation of the input information.

For example, if the input device is far enough away from the surface being scanned, the optical sensor unit may not capture an accurate representation of the provided image. In that case, additional information from additional sensors can be used to supplement the data acquired by the image of the object whose input device is moved at the top. Therefore, even if the input device is more than 1 inch (2.54 cm) away (z-axis) from the display on which the input device is moved, the sensors inside the input device are in the plane of the display, i.e. horizontally and vertically. It is possible to provide instructions for the movement of the pen in the direction.

For example, an input device used in conjunction with a laptop computer is placed on a folding table in front of the user. An image of the document with the maze pattern embedded in the background is displayed on the screen of the laptop. Annotations entered by the user can be shown in solid blue ink. The plane encounters eddy and the seatbelt sign is lit. As the user reaches for the keyboard of the laptop computer and adds another word to the annotation, the user's hand suddenly deviates from the surface of the screen. Image sensors may not be able to accurately detect the lines that form the displayed maze pattern, but x-axis and y-axis movements are measured by additional sensors built into the input device. ..

Effect of Input Device on Document Life Figure 8 shows the use of an input device according to an exemplary embodiment of the invention when a document is created, transmitted, and edited using the input device in a variety of environments. ing. The following description is merely an example of the use of an input device and does not limit the structure or function of the present invention.

Input devices can be used to extend the life of a document by allowing it to create and / or edit the document for use in connection with a large number of devices in a wide variety of environments. The input device 801 can be used to electronically create a document 802 on the screen of one computing device, such as the Tablet PC 803 illustrated. For example, an input device can be used to generate a handwritten draft of a document. Electronic ink corresponding to the information entered on the screen of the Tablet PC 803 is generated by the input device acting as a stylus for the Tablet PC 803. The electronic ink can be converted into text form and stored in the Tablet PC 803.

An electronic file representing a document can be transmitted to a second computing device such as a desktop PC 804. In this environment, the document can be edited on the screen of a desktop device using an input device that acts as an independent input unit. The input device senses the relative position of the input device in the displayed image of the document, so edits entered on the desktop device screen, even if the display does not contain elements to detect the position of the input device. Can be reflected in the electronic document. Edits generated using the input device can be transmitted to the desktop PC 804 as they are generated, or stored inside the input device for transmission to any PC at a later time. Edits can be entered into the version of the document stored in the desktop PC 804.

The created document can also be output as a hard-copy by a printer such as the printer 805 linked to the desktop PC 804. The hardcopy 806 version of the document can contain information or codes that specify the relative position of the input device at any position in the document, for example using a maze pattern. Hard copies can be modified by one or more users, each having an input device, and each user's edits are generated by a separate input device. Along with information representing the edit, information identifying the pen used to generate the edit can also be provided. For example, input can be reflected using underlined colored text, as seen in applications for tracking changes made to a document. Edit / input from desktop PC 804 to Tablet to incorporate into that document It can be transferred to PC803. Alternatively, the edits can be stored inside the input device and uploaded at a later time.

The document can also be printed on plain paper or on any carrier that does not include indications of the relative position of the input device. Again, the hard copy can be modified by one or more users with the input device, and each user's edits are generated by the input device. In this example, the position and movement of the pen can be identified using coding techniques for optically sensing the movement of the input device on the paper. As mentioned above, position / motion can be identified using a comparison algorithm that detects the relative position of the object within each frame of the image data and is used to determine the motion of the input device. The resulting edits can be transmitted, for example, to the computing device from which the document came from to update the original data file. Editing can be transmitted via wireless or wired communication, or through a computing device such as a pocket PC 807 for transmission to the destination device when the device containing the editing is docked to the computing device.

The electronic document can also be transmitted to a second computing device such as the illustrated Tablet PC. In that environment, the document can be edited on the screen of a tablet device using an input device such as a simple stylus. The input can be transferred from the Tablet PC to a computing device that remembers the original copy of the document, for example, as an annotation to the document or as an edit to incorporate into the document.

Additional components Although the above description and the accompanying figures depict embodiments utilizing specific components, the scope of the invention also includes the addition of components and / or the removal of any components depicted. Similarly, the rearrangement of various components within the input device structure can be performed without significantly affecting the accuracy with which the camera or inertial sensor detects the movement of the pen and produces electronic ink. For example, the image sensor can be replaced or complemented by a sensing device for detecting the properties of a surface or object on which the input device can be moved. Therefore, if a maze pattern is formed on the surface of an object so that the pattern can be detected based on the emission of energy outside the visible light spectrum, the reflectance of such energy transmitted to the object, or its reflectance. Such sensing techniques can be used to detect the perception of any property of the surface and to identify the position and / or movement of the input device on the surface of the object. As a further example, the microphone can use a microphone sensing system to detect acoustic reflections or emissions from an object on which the input device is placed.

The exemplary embodiments described and shown above have described input devices implemented in the form of pens. However, aspects of the invention are applicable to input devices of any shape and size.

The use of such input devices should enable personal computing everywhere. Thus, a user with the described input device can generate or edit a data file regardless of where they are. Documents and other information can be generated, edited, or recorded in an office environment, in the classroom, in a hotel, on the move, or even on the beach.

As mentioned above, the input device can include a suitable display. Alternatively, the display of the host computing device can be used to view the created documents and images. The user can select the format of the document before or after entering information such as text, or browse the document and make changes to the format of the document. In the context of the above example, by looking at the document created on such a display, the user can insert a header containing his address in the appropriate position.

The present invention is defined using the claims, but the present invention is in any combination or partial combination (sub). The scope of claims is exemplary, as it is possible to include the elements and steps described herein in combination). Accordingly, there are any alternative combinations to define the invention that incorporate one or more elements from the specification in various combinations, or partial combinations, including description, claims, drawings. To do. In view of the present specification, an alternative combination of aspects of the invention, independently or in combination with one or more elements or steps as defined herein, as a modified or alternative form of the invention, or the present invention. It will be clear to engineers in the art that it can be used as part of the invention. The described description of the invention contained herein can include all such modifications and alternatives. For example, in various embodiments, some order of data is shown. However, any sort of data is included by the present invention. Also, if characteristic units such as size (eg, bytes or bits) are used, all other assumptions are made.

<figref num="1">It is a block diagram which shows the general structure of the computer which can be used together with embodiment of this invention.</figref><figref num="2">It is a schematic diagram which shows the input device (including all the components) by an exemplary embodiment of this invention.</figref><figref num="3A">It is a schematic diagram which shows one of three exemplary embodiments of the camera system for use according to the aspect of this invention.</figref><figref num="3B">FIG. 6 is a schematic diagram showing another one of three exemplary embodiments of a camera system for use in accordance with aspects of the present invention.</figref><figref num="3C">FIG. 6 is a schematic diagram showing another one of three exemplary embodiments of a camera system for use in accordance with aspects of the present invention.</figref><figref num="4">It is a figure which shows the exemplary technique (maze pattern) for coding the position of a document.</figref><figref num="5">It is a figure which shows the locus pattern which can generate electronic ink based on this.</figref><figref num="6">It is a block diagram which shows the hardware architecture of the system by one Embodiment of this invention.</figref><figref num="7">FIG. 6 is a schematic diagram showing a further combination of components incorporated into an input device for producing electronic ink according to another exemplary embodiment.</figref><figref num="8">It is explanatory drawing which shows the use example of the input device by some exemplary embodiments of this invention.</figref>

Code description

101 Keyboard 102 Mouse 106 Serial Port Interface 107 Monitor 108 Video Adapter 109 Remote Computer 110 Processing Device 111 Memory 112 Local Area Network 113 Wide Area Network 114 Network Interface 115 Modem Mode 120 System Memory 130 System Bus 165 Digitizer 166 Stylus 192 Hard Disk Interface 193 Magnetic Disk Drive Interface 194 Optical Drive Interface 195 Operating System 196 Application Program 197 Other Program Modules 198 Program Data 201,701 Pen (Input Device) 202,702 Ink Cartridge 203,703 Pressure Sensor 204,704 Camera 205 Induction Element 206,706 Processor 207,707 Memory 208,708 Transceiver 209 Power 210 Docking Station 211 Cap 212 Docking Station Ring (Display) 304 Camera 320 Image Sensor (Sensing Array) 321 Light Source 322 Half Mirror 323 Lens System (Lens) 324 Surface 601 Input Device 604 Sensing Array (Image Sensor) 606 Processor 607 Memory 608 Transceiver (Communication Unit) 609 Power Supply 620 Host Device (Host Computer) 630 Docking Cradle 715 Inertivity Sensor 801 Input Device 802 Document 803 Tablet PC (First Computing Device) 804 Desktop PC (Second Computing Device) 805 Printer 806 Hard Copy 807 Pocket PC