Device and method for detecting coordinate

Description: TECHNICAL FIELD [Detailed description of the invention]

【0001】

[Industrial application field]

The present invention relates to a coordinate detection device and method, specifically, a coordinate detection device and method for detecting a coordinate position instructed by an operator.

【0002】

[Conventional technology]

Conventionally, this type of device has a built-in device (called a digitizer) that indicates the top of a dedicated tablet with a dedicated pen, and a rotary encoder that detects the rotating ball and the direction and amount of rotation of the ball. There are devices (eg mice and trackballs).

【0003】

All of these The person who sees the display screen = the person who operates the pointing device In many cases.

【0004】

By the way, recently, the display information has been digitized in advance, and various presentations have been made in front of a large number of people at the seat. In general, such a presentation is desired to be displayed on a relatively large screen, and is often performed by connecting a projector or a dedicated OHP device to an information processing device (for example, a personal computer).

【0005】

With such a system, it is attracting attention because it is possible to create presentation materials as if it were a word processor.

【0006】

[Problems to be Solved by the Invention]

However, for example, even if some menus are displayed in the displayed (or projected) image, it is the dedicated operator who selects the menus after all, and the viewer can only participate as a receiver. ..

【0007】

As a method for solving this, for example, there are Japanese Patent Application Laid-Open No. 4-18859 and Japanese Patent Application Laid-Open No. 4-18860. These proposals disclose a technique including means for irradiating a pointing light beam and means provided on a screen for detecting the irradiated position.

【0008】

Certainly, with such a configuration, it is possible to remotely specify the desired position on the screen, but the screen has a complicated configuration, which makes it difficult to manufacture and very expensive. become. Also, since the screen is essential, the place to give a presentation will be decided naturally.

【0009】

[Means for solving problems]

as well as [Action]

The present invention focuses on this point, and an object of the present invention is to provide a coordinate detection device and a method that make it possible to remotely specify a desired position while utilizing existing resources with a simple configuration. It is a thing.

【0010】

In order to solve this problem, for example, the coordinate detection device of the present invention has the following configuration. That is, it is a coordinate detection device that detects a designated coordinate position, and detects at least two markers that specify a coordinate input space in a pointing device provided with an imaging means and an image supplied from the pointing device. The means and a derivation means for deriving the position of the imaging means in the imaging direction with respect to the coordinate input space based on the detected position of the marker are provided.

【0011】

Further, according to a preferred embodiment of the present invention, it is desirable that the coordinate input space is a display screen. As a result, the coordinate input space can be constructed as long as the image can be displayed.

【0012】

Further, it is desirable that the marker in this case is an identification marker image displayed at a predetermined position on the display screen. As a result, the process of displaying the marker may be performed on the side that controls the display, and a general-purpose one can be used on the side that displays the image.

【0013】

Further, the marker may be a removable physical marker. In this case, the side that controls the display does not need to perform processing such as displaying a marker, and the coordinates can be detected as long as the marker is installed.

【0014】

Further, it is desirable that the pointing device and the detection means and the derivation means are at least separate from each other. As a result, the pointing device can be made very compact, allowing remote coordinate input at intervals such as holding a microphone.

【0015】

Further, it is desirable that the derivation means derives the coordinate position based on the two detected markers and the predetermined shape information that specifies the coordinate input space. As a result, the coordinates can be input regardless of the distance between the operator and the place where the coordinate input space is placed.

【0016】

Further, it is desirable that the shape information is an aspect ratio. As a result, the coordinate detection becomes extremely simple and the detection speed can be increased.

【0017】

Further, it is desirable to provide a general-purpose interface for sending out the coordinate data derived by the derivation means. As a result, coordinate input can be performed simply by connecting to a general-purpose information processing device.

【0018】

It is desirable that the pointing device includes at least one button switch and outputs the state information of the button switch together with the video signal. As a result, not only the input of coordinates but also various instructions can be given.

【0019】

Further, it is desirable to provide a means for adjusting the relationship between the position of the marker and the coordinate input space. As a result, normal coordinate input can be performed even in different coordinate input spaces.

【0020】

[Example]

Hereinafter, an embodiment according to the present invention will be described in detail with reference to the accompanying drawings.

【0021】

FIG. 1 shows the configuration of the system in the embodiment. In the figure, 1 is a CPU that controls the entire device, 2 is a ROM that stores a boot program, etc., and 3 is a RAM that is used as an OS, application, or work area processed by the CPU 1. 4 is an external storage device (for example, a hard disk device, a floppy disk device, a magneto-optical disk device, etc.) that stores an OS, an application program, and various information. 5 is a keyboard for inputting various commands and inputting characters. 6 is a VRAM that expands the image to be displayed on the display screen, and 7 is a display device that displays the image expanded on VRAM 6. When various image data are expanded on VRAM 6, the expanded image is displayed on the display device 7. Therefore, writing data to VRAM and displaying it are equivalent.

【0022】

Reference numeral 9 denotes a coordinate input unit operated by the operator, which is provided with an image sensor and buttons like a general mouse device and a coordinate input pen. The captured image data and the pressing information of this button are taken into the device via the illustrated interface 8.

【0023】

In the above configuration, when the power is turned on to this system, CPU1 reads the OS program from the external storage device 4 into RAM3 according to the boot program stored in ROM2 and runs the OS. Then, the operator starts various applications based on the instructions from the keyboard 5 and the coordinate input unit 9 of the embodiment, and executes the process. At this time, if the coordinate input unit 9 of the embodiment is used, the corresponding cursor is displayed on the display device 7, and the same work as that of a general mouse or digitizer can be performed. Since the cursor display control itself is known and is not directly related to the present invention, the description thereof will be omitted, but at least the OS receives coordinate data and button press information via the interface 8 and the cursor. Will be controlled.

【0024】

FIG. 2 shows an operation example of the system of the embodiment and the coordinate input unit 9.

【0025】

As shown in the figure, the coordinate input unit 9 in the embodiment is large enough to be held in a hand and enables the position of the cursor on the display screen to be controlled remotely. In the illustration, 9a is the coordinate input unit. A button provided on the 9 is shown, and 101 is a cursor linked to the operation of the coordinate input unit 9.

【0026】

Hereinafter, the principle for realizing the operation shown in FIG. 2 will be described in detail.

【0027】

As described above, the coordinate input unit 9 has a built-in solid-state image sensor (CCD). Then, based on the captured image, the display screen position on the extension pointed to by the coordinate input unit 9 is detected.

【0028】

In general, recognizing an captured image requires a very complicated process, and the inventor of the present application has realized such a process extremely easily.

【0029】

Specifically, a marker that can be distinguished from the display image is displayed at a predetermined position (at least two places) in the display area (hereinafter, screen area) of the display device 7. Reference numerals 102 and 103 in FIG. 2 are markers. Here, the marker may have a special color, or the blinking cycle may be set to a predetermined value. In any case, as long as the marker can be recognized, the function described below can be realized, so the method does not matter. Here, for the sake of simplicity, an example in which the marker 102 is red and the marker 103 is green will be described. However, in this case, the color used in the marker shall not be displayed in the screen area.

【0030】

FIG. 3 shows an example of imaging by the coordinate input unit 9. FIG. (A) shows an example in which the center position of imaging is outside the imaged screen area, and FIG. 3B shows an example in which the center position of imaging is inside the imaged screen area. Two markers (red marker and green marker) can be detected to determine whether or not the center position of the image is within the display screen, and the position of the imaged marker is uniquely determined in the captured image. The size and position of the screen area can be calculated, and the judgment can be made accordingly.

【0031】

For example, if the aspect ratio (generally called the aspect ratio) of the screen area of the display device 7 is determined, as long as two markers are detected, the marker position and the aspect ratio of the display screen are used to determine the center of imaging. The coordinates of the position with respect to the display screen can be obtained. The advantage of using the aspect ratio is that the distance of the operator of the coordinate input unit 9 from the display device 7 can be ignored (the distance between the operator and the display screen can be ignored).

【0032】

Therefore, in the state of the figure (A), the detected coordinate data may be processed as out of range, and in the state of the figure (B), the center position of the imaging with respect to the screen area determined by the marker. All you have to do is calculate the coordinates.

【0033】

The point to pay attention to here is how the coordinate input unit 9 is held in the operator's hand because it is only necessary to know the position of the directivity direction of the coordinate input unit 9 in the screen area. There is no need to consider it at all. That is, it is not obligatory to hold the button 9a so that it is always on top.

【0034】

A brief explanation of how to obtain the coordinate position will be as follows.

【0035】

FIG. 4 (A) shows the screen area captured by the coordinate input unit 9 (the area surrounded by points A, B, A', B').

【0036】

Here, it is assumed that the point A is a red marker and the point B is a green marker. Of course, the positions of points A'and B'are undecided at this point, but they can be easily obtained. That is, the angle θ1 from the horizontal axis to the point A can be easily detected (calculated) with the distance between the points A and B and the point B as a reference. Also, since ABA'(= θ2) is fixed, the angle θ3 from the horizon at point A'is θ3 = θ1-θ2 Will be. That is, in this case, the image displayed on the screen can be made upright by rotating the points A and P clockwise by θ3 with respect to the point B.

【0037】

As a result, points A'and B'can be simply obtained. That is, the points A'and B'are obtained by multiplying the length of the side AB by cos θ2 and sin θ2, respectively. More specifically, the x-coordinate of the point A'after the rotation process is the same as the x-coordinate of the point A after the rotation process, and the y-coordinate of the point B'after the rotation process is the point A after the rotation process. Same as the y coordinate. After that, it can be determined whether or not the point P is within the screen area only by comparing the magnitudes of the coordinates x and y of the point P and the points A, B, A', B'.

【0038】

That is, when the coordinates of each point after rotation are (XP, YP), the coordinates of point A are (XA, YA), and the coordinates of point B are (XB, YB), the following conditions are satisfied. Then, it can be determined that the point P is in the screen area.

【0039】

XB XP XA YB YP YA However, here, the lower left of the screen is the origin, the right direction is the positive direction of the X axis, and the upper direction is the positive direction of the Y axis. Needless to say, the positive direction of the axis is not limited to this.

【0040】

In addition, the maximum horizontal coordinate value XMAX and the maximum vertical coordinate YMAX (the number of dots or the virtual absolute coordinate value adopted by the information processing device) of the screen area of the coordinate space used inside the information processing device are fixed. Therefore, the coordinates X and Y used by the information processing at point P are X = XMAX (XP-XB) / (XA-XB) Y = YMAX (YP-YB) / (YA-YB) Can be obtained at.

【0041】

When the CPU 1 in the apparatus (system) of the embodiment receives an instruction to acquire the coordinate position from the OS or the application, the coordinates may be detected by the procedure shown in FIG. The processing procedure shown in the figure is initially stored in the external storage device 4, is read into the RAM 3 when the system is in operation, and is called at appropriate time intervals.

【0042】

First, in step S1, the captured image is input from the coordinate input unit 9. Then, in step S2, it is determined whether or not there are two markers in the two markers. If neither of them is detected, the coordinate input unit 9 is not heading toward the display screen, so the process proceeds to step S5 to perform error processing. The error processing includes, for example, notifying the caller that an error has occurred, or notifying the latest coordinate data determined to be in the screen area as a detection result. In the latter case, the cursor can be left visible in the screen area, providing a natural environment for the operator.

【0043】

Now, when two markers are detected, whether or not the center position of the entire captured image is included inside the captured screen area based on the position of each marker and the aspect ratio of the preset screen area. Is determined. If it is determined that the screen area is out of the screen area, the error processing in step S5 is performed.

【0044】

If it is determined that the image is within the screen area, the coordinates of the center position of the entire captured image with respect to the screen area are calculated, and the calculation result and the information indicating the pressed state of the button 9a are referred to this process. Return to the original.

【0045】

As a result of the above, according to the present embodiment, the desired position of the display image can be specified remotely and without using a special display device.

【0046】

In the above embodiment, the CPU1 that controls the entire information processing apparatus shows the coordinate position detection, but for example, the circuit for detecting the coordinates may be independent. Generally, an information processing device (for example, a personal computer) is provided with an interface called an expansion slot for expanding various functions, so that a card or board for performing the above coordinate detection may be mounted. In this way, the functions of the above-described embodiment can be realized with a general-purpose device, and the need for a special information processing device is eliminated.

【0047】

<Explanation of the Second Example> In the above embodiment, it has been described that the aspect ratio of the display screen of the display device 7 is constant, but it is not uncommon for the device to be able to fine-tune the ratio. Further, when the same presentation data is performed on different display devices of different devices, there is no guarantee that the aspect ratio is always constant between the display devices.

【0048】

Therefore, in the second embodiment, a process of determining the aspect ratio is added before actually operating the coordinate input unit 9.

【0049】

The device configuration is the same as in FIG. 1, and in the following, the process of starting when a start instruction of the fine adjustment process of coordinate input (for example, a start instruction of the corresponding program from the keyboard) is given will be described with reference to FIG. Before giving the processing start instruction, the operator shall fix the coordinate input unit 9 appropriately toward the display screen (do not touch it by hand).

【0050】

First, in step S11, a marker is displayed at the upper right corner position on the display device 7. In step S12, it is determined whether or not this marker is within the imaging range of the coordinate input unit 9. If not, adjust the direction of the coordinate input unit 9 until the judgment in step S12 is yes. If the coordinate input unit 9 is equipped with a zoom function, it is rare that the determination result in step S12 becomes NO as long as the magnification is set small.

【0051】

Now, when the marker in the upper right corner of the screen area is detected, the marker position in the entire captured image is stored and retained. Then, the process proceeds to step S14, in which a marker is displayed in the upper left corner of the screen area, and the same process is performed (steps S15 and 16).

【0052】

Then, a marker is also displayed in the lower right corner of the screen area to detect it (steps S17 and S18).

【0053】

In this way, when all the markers at the positions that specify the shape of the screen are detected, the aspect ratio of the screen is determined accordingly. After that, the coordinate detection may be performed according to the determined aspect ratio.

【0054】

Generally, it is desired that the aspect ratio of the display screen and the ratio of the number of dots in the vertical and horizontal directions of the display image match, but the aspect ratio of the display screen can be freely adjusted with the adjustment knob provided on the display device. It is more likely that they do not exactly match. That is, inside the information processing apparatus, for example, when processing is performed in the dot position or the coordinate space corresponding to the dot position, it is necessary to correct the center position of the detected captured image.

【0055】

To do this, for example, enter the distance between the upper right corner marker and the lower right corner marker in the coordinate system (absolute coordinate value used by the device) and the distance in the upper right corner and upper left corner coordinate system (similar) and correct it. It should be set as a parameter.

【0056】

As described above, according to the second embodiment, the fine adjustment process can be performed before the operation of the coordinate input unit 9, so that the coordinates can be input correctly even if the size and shape of the screen area are slightly different. Will be able to do.

【0057】

<Explanation of the Third Example> In the first and second embodiments described above, the information processing device itself has been described as also serving as the coordinate detection process, but the processing portion related to the coordinate detection is provided outside the information processing device. An example will be described as a third embodiment.

【0058】

FIG. 7 shows a schematic diagram of the usage pattern in the third embodiment. In the figure, 100 is the main body of the information processing device, and 200 is a projector that takes in the video signal from the information processing device and projects it on the screen 500. 300 is a coordinate detector and 400 is a coordinate input unit.

【0059】

In the third embodiment, the coordinate detection device 300 and the information processing device 100 are connected by a serial interface (for example, RS-232C). Since a personal computer has this kind of interface, the information processing device to be connected may be a general-purpose one with the above configuration.

【0060】

FIG. 8 shows a block configuration diagram of the coordinate detection device 300 of the third embodiment.

【0061】

In the figure, 301 is a CPU that controls the transmission of this device, 302 is a ROM that stores the operation processing procedure, and 303 is a RAM that is used as a work area. 304 is a serial interface for bidirectional communication with an information processing device, and 305 is an interface with a built-in converter that converts a video signal from the coordinate input unit 400 into a digital signal. A button press signal of the coordinate input unit 400 is also supplied to this interface. Reference numeral 306 is an operation switch for setting the environment of the present device, and is used for setting, for example, the baud rate and parity when communicating with the information processing device.

【0062】

In the above configuration, the coordinate detection processing content of the CPU 301 is the same as that of the first embodiment. However, the detected coordinate data will be output to the information processing apparatus side via the interface 304. Further, the adjustment process described in the second embodiment may be executed when an instruction command for adjustment is received from the information processing apparatus. Alternatively, send it when the interface is ready.

【0063】

As a result of the above, it becomes possible to utilize the existing device as it is and input coordinates remotely.

【0064】

<Explanation of the Fourth Example> In each of the first to third examples described above, a specific marker is displayed on the displayable area. That is, it was necessary for the information processing apparatus to display the marker at a predetermined position on the display image.

【0065】

However, this does not limit the invention of the present application. For example, as shown in FIG. 9, a physical marker is placed outside the projection plane (or display area) (it can be removed and can be used for existing screens and display devices by simply carrying the marker). This is because it can be realized by detecting.

【0066】

By doing so, a marker is installed, or a display device or screen with a marker is required, but special processing is not required on the information processing device side. However, when a marker is installed, fine adjustment processing is required first.

【0067】

<Explanation of Fifth Example> In the first embodiment, when the center position of the captured image is not within the screen area, error processing is performed even if the marker can be detected. In this case, the screen area can be specified, and if the coordinates are outside the screen area, the coordinates may be output to the information processing apparatus. On the information processing apparatus side, when the obtained coordinates exceed the display area, the direction in which the coordinate input unit 9 is oriented may be indicated as shown in FIG.

【0068】

<Explanation of the Sixth Example> In the first embodiment, the marker is displayed by the control of the information processing device side, but this part may be performed without the intervention of the CPU of the information processing device.

【0069】

FIG. 11 shows the main block configuration of the sixth embodiment. The composite circuit shown in the figure inputs the video signal to be displayed, synthesizes a marker image in the upper right corner and the lower left corner (not limited to this, but may be the upper right corner and the lower right corner) of the input video, and displays the display device ( It is output to a CRT display device, projector, or dedicated OHP device. Also. Further, the coordinate data from the coordinate input unit, the button pressing information, and the like are input and output to the information processing main body.

【0070】

With such a configuration, the CPU on the information processing device side does not have to perform special marker display control, and the load on the CPU can be reduced.

【0071】

<Explanation of the 7th Example> In the above-described embodiment, the display screen is assumed to be rectangular. However, if the operator is relatively close to the size of the display screen, and the operator is not in front of the display screen but in a biased location, the image captured by the coordinate input unit 9 is displayed. The screen area of is not always rectangular.

【0072】

Therefore, a separate fine adjustment start instruction button is provided in the coordinate input unit 9, and when the pressing of the button is detected, markers are sequentially displayed at the four corners of the screen to form a trapezoidal shape as a screen area in the captured image. May also be acceptable.

【0073】

Further, in the embodiment, it has been described as a pointing device for interlocking the cursor on the display screen. In other words, the target for coordinate input is the display screen or screen, but it is not limited to this. However, when giving a presentation or the like, it is desirable that the operation result be reflected on the display screen remotely.

【0074】

<Explanation of the eighth embodiment> In addition, the coordinate input unit 9 is equipped with an LED to determine whether or not two markers can be detected and whether or not the direction toward which the two markers are directed is within the screen area. The LED may be used to notify the operator and be user-friendly.

【0075】

Further, in this embodiment, as described above, it may be applied to one device or as separate devices. Of course, it may also be realized by supplying a program from the outside. Therefore, the present invention is limited to the above embodiment, and various improvements or applications are possible as long as it falls within the scope of the idea.

【0076】

[Effect of the invention]

As described above, according to the present invention, it is possible to remotely specify a desired position on the display screen while utilizing existing resources with a simple configuration.

【0077】

[Simple explanation of drawings]

[Figure 1]

It is a block block diagram of the apparatus in an Example.

[Figure 2]

It is a figure which shows the operation example of the coordinate input part of an Example.

[Fig. 3]

It is a figure for demonstrating the principle of coordinate detection in an Example.

[Fig. 4]

It is a figure for demonstrating the coordinate determination process and the coordinate detection process in an Example.

[Fig. 5]

It is a flowchart which shows the processing procedure of the coordinate detection in an Example.

[Fig. 6]

It is a flowchart which shows the fine adjustment processing procedure in 2nd Example.

[Fig. 7]

It is a figure which shows the operation example in 3rd Example.

[Fig. 8]

It is a figure which shows the block block diagram of the coordinate detection apparatus of 3rd Example.

[Fig. 9]

It is a figure which shows the screen in 4th Example.

[Fig. 10]

It is a figure which shows the display example of the screen in 5th Example.

[Fig. 11]

It is a figure which shows the structure of the 6th Example.

[Explanation of symbols]

1 CPU 2 ROM 3 RAM 4 External storage device 5 keyboard 6 VRAM 7 Display device 8 interface 9 Coordinate input section