Coordinate input apparatus, control method therefor and program

The present invention relates to a coordinate input device for optically detecting a coordinate position input to a coordinate input surface by an indicator such as a finger for inputting or selecting information, a control method thereof, and a program. In particular, the present invention relates to a detachable and portable coordinate input device, a control method thereof, and a program.

Conventionally, various types of coordinate input devices (touch panels and digitizers) have been proposed or commercialized as this type of coordinate input device. For example, a coordinate input device such as a touch panel that can easily operate an information processing device such as a PC (personal computer) by touching the screen with a finger without using a special instrument or the like is widely used. ing.

As the coordinate input method, there are various methods such as those using a resistance film and those using ultrasonic waves. As a coordinate input method using light, a method is known in which a retroreflective material is provided on the outside of the coordinate input surface, light from a light projecting unit is reflected by the retroreflective material, and the light amount distribution is detected by a light receiving unit. Then, in this method, the indicated position is calculated by instructing the inside of the coordinate input area with an indicator such as a finger to block the optical path and detecting the shaded direction (see, for example, Patent Documents 1 and 2). ).

The configuration of FIG. 10 is shown as an example of generalizing the configuration of Patent Document 1. FIG. 10 shows sensor units 1L and 1R arranged at both ends of the coordinate input surface, and a coordinate input effective area 3 which is a coordinate input surface used when inputting coordinates. Then, it has a retroreflective unit 4 that surrounds three sides of the coordinate input effective region 3 and recursively reflects the incoming light in the incoming direction.

The sensor units 1L and 1R have a light emitting unit and a light receiving unit (not shown). The light projecting unit irradiates light that spreads in a fan shape substantially parallel to the input surface of the coordinate input effective region 3, and the light receiving unit receives the returned light after the light is retroreflected by the retroreflective unit 4. The coordinate input device is input to the coordinate input effective region 3 based on the light shielding directions (shielding angles θL, θR) detected by the two sensor units 1L and 1R, respectively, and the distance between the sensor units 1L and 1R. The coordinate position can be calculated. In FIG. 10, 2 is a control circuit that controls the sensor units 1L and 1R, processes the acquired output signals of the sensor units 1L and 1R, or outputs the processing result to an external device, and 8 is retroreflection. It is a light-transmitting protective member for protecting the part 4.

Patent Document 2 shows an example of a specific configuration of a light emitting unit and a light receiving unit in a sensor unit in the optical shading type coordinate input device shown in Patent Document 1.

In Patent Document 3, instead of the retroreflective portion shown in Patent Documents 1 and 2, a light guide portion provided on three sides around the coordinate input effective region and emitting light on a side surface substantially perpendicular to the light guide direction is provided. It is shown.

Further, Patent Document 4 discloses a configuration for controlling the lighting of the light emitting unit in each sensor unit. Specifically, in order to prevent the light emitted from the light emitting unit of one sensor unit from being received as ambient light by the light receiving unit of the other sensor unit, the light emitted from each light emitting unit of the sensor unit is emitted. Is controlled so as to be performed alternately.

Further, in Patent Document 5, retroreflective portions are arranged on the upper side and the lower side of the coordinate input effective region, and the sensor unit is arranged with a gap between the retroreflective portion and the coordinate input effective region. It is shown.

<p><patcit num="1"><text>U.S. Pat. No. USP4507557</text></patcit><patcit num="2"><text>Patent Publication No. 2004-272353</text></patcit><patcit num="3"><text>Patent Publication No. 2003-280802</text></patcit><patcit num="4"><text>Patent Publication No. 2001-43021</text></patcit><patcit num="5"><text>Japanese Patent No. 4118664</text></patcit></p>

<p num="0011"> However, in the above-mentioned prior art, it is difficult to support a multi-display as shown in FIG. 11A, in other words, to support a large-sized coordinate input area and a horizontally long coordinate input area for the following reasons. Become. Note that FIG. 11A assumes that three front projectors are used to display one image on one large screen. First, when a plurality of coordinate input devices according to the prior art are arranged side by side to support multi-display (see FIG. 11B), the retroreflective unit 4 is indispensable at the joint portion of the plurality of coordinate input devices. Therefore, not only the displayed images are discontinuous, but also the coordinates cannot be continuously instructed / operated from the area a to the area b in FIG. 11B, and the operability is significantly reduced. That is, in the multi-display, the retroreflective portion 4 at the joint portion becomes an obstructive component.</p><p num="0012"> In order to solve the problem, as shown in FIG. 11A, the retroreflective portion 4 must be provided outside the display area of the multi-display. As is clear from FIG. 11 (A), the distance difference between the direction a in FIG. 11 (A) and the direction b in FIG. 11 (A) increases as the number of display surfaces increases. Therefore, in this type of coordinate input device in which the light projected by the light projecting unit reaches the retroreflective unit 4 and the retroreflected light is detected by the light receiving unit, the light is received by this distance difference (optical path difference). The difference in quantity becomes large.</p><p num="0013"> That is, even if the amount of light projected by the sensor unit 1L is constant regardless of the direction of light projection, a relatively large amount of light received in the direction a and the smallest amount received in the direction b can be obtained due to this distance difference. It disappears. It is generally difficult to keep this difference within the dynamic range of a light receiving element (photoelectric conversion element such as a CCD or CMOS). That is, if the light receiving amount in the direction a is set to be the maximum output value within the dynamic range range, the light in the direction b cannot be detected at all. Alternatively, if sufficient light is projected to detect the light in the direction b, the detection signal of the light from the direction a will be saturated, and it will be difficult to obtain correct data.</p><p num="0014"> In order to solve the problem, it is necessary to increase the amount of light projected in the direction b of FIG. 11A and change the amount of light projected according to the direction of light projected. By doing so, the amount of light received can be smoothed, but adverse effects such as cost increase and an increase in the size of the device cannot be avoided.</p><p num="0015"> As a configuration for reducing the optical path difference, as shown in FIG. 11C, a configuration in which more sensor units 1 and 1R are arranged to divide the region detected by each sensor unit can be considered. At this time, from the viewpoint of not deteriorating the operability described above, it is preferable that the display area does not have the retroreflective portion 4. For example, as shown in FIG. 11C, the upper side of the display area. , And the recurrence reflection unit 4 shall be provided only on the lower side. Although the number of sensor units increases, the optical path difference becomes smaller, and it becomes possible to detect a stable optical signal.</p><p num="0016"> The configuration of Patent Document 5 is a configuration in which retroreflective portions are arranged on two opposite sides of a display area, and a light emitting portion and a light receiving portion are provided outside the retroreflective portion. At this time, if the height of the light receiving and receiving unit and the retroreflective unit from the coordinate input surface (height of the coordinate input surface with respect to the normal direction) are the same, the retroreflective unit blocks the optical path. Therefore, the height of the light receiving / receiving unit and the retroreflective part from the coordinate input surface (height of the coordinate input surface with respect to the normal direction) are configured to be different, but the following new problems arise.</p><p num="0017"> As shown in FIG. 12A, in the configuration of Patent Document 5, a sensor unit 901 having a light emitting unit and a light receiving unit is arranged between the retroreflective unit 903 and the coordinate input surface 902. The light projected from the light projecting unit in the sensor unit 901 is retroreflected by the retroreflective unit 903 provided on the opposite side, and is received by the light receiving unit of the sensor unit 901. Therefore, the optical path becomes the hatched portion 908. If the indicator 907 is arranged as shown in the figure, the sensor unit 901 on the left side does not block the optical path of the hatched portion 908, so that the sensor unit 901 on the left side cannot detect the indicator 907. On the other hand, since the optical path of the hatched portion 908 of the sensor unit 901 on the right side is blocked by the indicator 907, the position information (direction) of the indicator 907 can be detected.</p><p num="0018"> That is, even if one sensor unit can detect the position information (direction), the other sensor unit cannot detect the position information (direction). Therefore, in the state of the indicator 907, the indicated position of the indicator 907 is calculated. I can't. The indicated position can be detected when the indicator 907 approaches the coordinate input surface 902 and sufficiently blocks the optical path of the sensor unit 901 on the left side, in other words, the position immediately before the indicator 907 touches the coordinate input surface 902. It can be said that it was when I came to. Therefore, in the configuration of Patent Document 5, if the indicator 907 is separated from the coordinate input surface 902, the position cannot be stably detected. Hereinafter, a function capable of detecting the position of the indicator 907 even at a position away from the coordinate input surface will be referred to as "proximity input".</p><p num="0019"> As a configuration corresponding to this problem (proximity input is not possible depending on the region), as shown in FIG. 12B, a configuration in which a retroreflective portion is provided between the light emitting portion and the light receiving portion of the sensor unit 901 can be considered. A retroreflective unit 903 is provided between the light projecting unit 910 and the light receiving unit 909 in the sensor unit 901 on the left side, and the optical path where the light projecting unit 910 reaches the facing retroreflective unit 903 and the light receiving unit 903 from the retroreflective unit 903 The optical path to reach is set as shown. When the indicator 907 is in the same position as in FIG. 12A, the light path from the retroreflective unit 903 to the light receiving unit 909 is not blocked, but the light projecting unit 910 reaches the opposing retroreflective unit 903. Since the optical path is blocked, the indicator 907 can be detected. That is, the left and right sensor units 901 can detect the sensor, and the proximity input is possible.</p><p num="0020"> FIG. 12C shows a cross-sectional state of the sensor unit 901, and in order to efficiently detect the retroreflected light with this configuration, it is preferable that the distance L between the light emitting unit 910 and the light receiving unit 909 is small. However, reducing the distance L is equivalent to reducing the width h of the retroreflective portion 903, and as a result of reducing the width h of the retroreflective portion, the retroreflected light is substantially proportional to the width h. It will be smaller by that amount.</p><p num="0021"> Therefore, in order to efficiently detect the retroreflected light, it is preferable to reduce the distance L, and in order to secure sufficient retroreflected light, the distance L must be increased. Must be resolved.</p><p num="0022"> The present invention has been made to solve the above problems, and an object of the present invention is to provide a coordinate input device capable of stable coordinate input regardless of an input position, a control method thereof, and a program. Another object of the present invention is to provide a coordinate input device capable of proximity input, a control method thereof, and a program in the entire coordinate input effective area including the vicinity of the sensor unit.</p>

<p num="0023"> The coordinate input device according to the present invention for achieving the above object has the following configuration. That is, A coordinate input device having a retroreflective portion provided on two opposite sides of a rectangular coordinate input effective region and a plurality of sensor means provided on each of the two sides. The sensor means A light receiving part that receives incoming light and Of the two sides, a light projecting unit that projects light to a retroreflective unit provided on the opposite side. It is equipped with a surface light emitting part that emits uniform diffused light from a band-shaped surface. In the sensor means, the light projected by the light receiving unit is retroreflected by the retroreflecting unit provided on the opposite side, and the plurality of sensors provided on the opposite side. The light emitted by the surface emitting unit of the sensor means of the above is simultaneously detected.</p>

<p num="0024"> As described above, according to the present invention, a coordinate input device and a control method thereof that enable stable coordinate input regardless of the input position and enable proximity input in the entire coordinate input effective area including the vicinity of the sensor unit. , Can provide the program.</p>

<figref num="1">It is explanatory drawing of the outline of the coordinate input device of Embodiment 1.</figref><figref num="2">It is explanatory drawing of the structure of the sensor unit 1 of Embodiment 1. FIG.</figref><figref num="3">It is explanatory drawing of the optical property of Embodiment 1. FIG.</figref><figref num="4">FIG. 3 is a block diagram and a timing chart diagram of the control / calculation unit of the first embodiment.</figref><figref num="5">It is explanatory drawing of the region which can detect the indicated position in the sensor unit of Embodiment 1. FIG.</figref><figref num="6">It is explanatory drawing of the position detection of the coordinate input effective area of Embodiment 1. FIG.</figref><figref num="7">It is a flowchart which shows the coordinate calculation process of Embodiment 1.</figref><figref num="8">It is explanatory drawing of the height relation of the sensor unit of Embodiment 1 and a retroreflective part.</figref><figref num="9">It is explanatory drawing of Embodiment 2. FIG.</figref><figref num="10">It is a schematic block diagram of an optical coordinate input device.</figref><figref num="11">It is explanatory drawing of the problem at the time of multi-display.</figref><figref num="12">It is explanatory drawing of the problem at the time of proximity input.</figref>

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings, based on its preferred embodiments. The configuration shown in the following embodiments is only an example, and the present invention is not limited to the illustrated configuration.

&lt;Embodiment 1&gt; FIG. 1 is an explanatory diagram for explaining an outline of the coordinate input device of the first embodiment.

In FIG. 1, sensor units 1a to 1d are provided in the vicinity of the four corners (near the corners) of the rectangular coordinate input effective region 3. Each of the sensor units 1a to 1d is provided with a light emitting unit, a light receiving unit, and a surface light emitting unit (details will be described later). Retroreflective parts 4a and 4b are provided on two opposing sides of the coordinate input effective region 3, and the light projected by the sensor units 1a and 1b or the light projecting units of the sensor units 1c and 1d provided on the opposite sides. Is retroreflected.

The retroreflected light is detected by the light receiving units of the sensor units 1a to 1d. The light receiving unit is configured to detect the retroreflected light and simultaneously detect the light of the surface emitting units of the sensor units 1a and 1b or the sensor units 1c and 1d provided on the opposite sides. That is, to explain the light receiving unit of the sensor unit 1a, the light retroreflected by the retroreflective unit 4b provided on the opposite side of the light of the light projecting unit of the sensor unit 1a and the surface light emitting unit of the sensor units 1c and 1d. Simultaneously detect the light emitted by. Further, when viewed from the sensor unit 1a, the regions 9a and 9b are joint portions for detecting the light of the surface light emitting portion and the retroreflected light. In order to reliably detect the light in the direction of the joint portion, the surface light emitting portion and the retroreflective portion 4 of the sensor unit at that portion overlap each other. Even when viewed from the sensor unit 1b, the joint portions of the regions 9a and 9b are set to overlap.

Further, 8a and 8b are coordinate input device housings that accommodate the sensor units 1a and 1b, the sensor units 1c and 1d, and the retroreflective units 4a and 4b, respectively, and are hereinafter referred to as sensor bars. The control / calculation unit 2 is further housed in the sensor bar 8a. Signals are transmitted and received between the sensor bars 8a and 8b by a communication unit such as wired or wireless, and the control / calculation unit 2 controls the light emitting unit, the light receiving unit, and the surface light emitting unit of the sensor units 1a to 1d. Further, the control / calculation unit 2 calculates the indicated position from the output information from each of the sensor units 1a to 1d, and outputs the result to an external device (for example, a PC).

In the following description, when the sensor units 1a to 1d are generically referred to, they are referred to as the sensor unit 1. Similarly, when the retroreflective parts 4a and 4b are generically referred to, they are referred to as the retroreflective parts 4.

&lt;Explanation of sensor unit 1&gt; FIG. 2 is an explanatory diagram illustrating the structure of the sensor unit 1 of the first embodiment.

FIG. 2A is a cross-sectional view of the sensor unit 1, and the optical system will be mainly described. Reference numeral 5 denotes a rectangular coordinate input surface 5 for inputting coordinates, and a light projecting unit 30, a light receiving unit 40, and a band-shaped surface emitting unit 50 are provided from the side closer to the coordinate input surface 5. The optical center line of the light projecting unit 30 is indicated by a line segment bb, the optical center line of the light receiving unit 40 is indicated by a line segment cc, and the optical center line of the surface light emitting unit 50 is indicated by a line segment dd. .. As shown in the figure, the line segment bb, which is the optical center line of the light projecting unit 30, is located at a distance L1 away from the line segment cc, which is the optical center line of the light receiving unit 40. It is provided on the line segment dd which is the optical center line of the surface light emitting unit 50.

2 (B) to 2 (D) are front views of the sensor unit 1 as viewed from the front (direction e of the arrow in FIG. 2 (A)), and FIGS. C) is a diagram for explaining the light receiving unit 40, and FIG. 2 (D) is a diagram for explaining the surface light emitting unit 50.

In FIG. 2B, reference numeral 31 denotes an infrared LED that emits infrared light, and the emitted light is projected into a range of approximately 90 ° by the light projecting lens 32. On the other hand, in the horizontal direction with respect to the coordinate input surface 5 (see the same 2 (A)), the light from the infrared LED 31 is projected as a luminous flux limited in the vertical direction, and is mainly directed to the retroreflective unit 4. It is configured so that light is projected. Reference numeral 33 denotes a light-transmitting adhesive, which fills the space between the light projecting lens 32 and the infrared LED 31 without a gap.

In FIG. 2C, 41 is a one-dimensional line CCD, 42 is a light receiving lens that acts as a condensing optical system, 43 is a diaphragm that limits the incident direction of incident light, and 44 is incident of extra light such as visible light. It is an infrared filter that prevents light. The light projected by the light projecting unit 30 is retroreflected by the retroreflective unit 4, passes through the infrared filter 44 and the diaphragm 43, and is collected on the detection surface of the line CCD 41 by the light receiving lens 42.

In the case of the first embodiment, the light projecting unit 30 and the light receiving unit 40 are arranged so as to overlap each other, and the distance L1 is from the light projecting unit 30 to the retroreflective unit 4 so that the observation angle with respect to the retroreflective unit 4 becomes small. It is set to a value sufficiently small compared to the distance. Therefore, even if the distance is L1, sufficient retroreflected light can be detected by the light receiving unit 40.

Further, when viewed from the front, the positions of the center of the diaphragm 43 and the center of light emission of the light projecting unit 30 are set to the same position. Therefore, the light of the light projecting unit 30 projected in the substantially 90 ° direction is retroreflected by the retroreflective unit 4, passes through the infrared filter 44 and the diaphragm 43, and is lined by the light receiving lens 42 according to the incident angle of the light. An image is formed on the pixels of the CCD 41. Therefore, since the output signal of the line CCD41 outputs the light amount distribution according to the incident angle of the reflected light, the pixel number of the line CCD41 indicates the angle information.

FIG. 2D is an explanatory view illustrating the outline of the surface light emitting unit 50, 51, 52 and 53 are infrared LEDs, 54, 55 and 56 are collimating lenses, and 57, 58 and 59 are light that adheres the two. It is a permeable adhesive. Reference numeral 60 denotes a diffuser plate, which diffuses the light emitted by the infrared LEDs 51, 52 and 53, and radiates the diffused light from the diffuser plate 60.

In the first embodiment, three infrared LEDs 51, 52 and 53 are used, but the present invention is not limited to this, and the light emission from the diffusion surface is made uniform according to the length of the diffusion plate 60. The number is set as appropriate.

Further, the present invention is not limited to this configuration as long as the diffused light is uniformly emitted from the light emitting surface, and for example, a method using a light guide plate may be used. I used the word uniform here, but I will add more explanation. The details will be described later, but as shown in FIG. 5A, the light receiving portion 40 of the sensor unit 1a has an angle θa as a visual field range. Therefore, in the range of the visual field range θa, the light from the retroreflective unit 4 and the surface light emitting unit 50 provided on the opposite sides is detected. Therefore, in FIG. 2D, the light must be detected by the light receiving unit 40 provided in the sensor unit 1 provided on the opposite sides evenly within the range of the light emitting surface length L3 of the surface light emitting unit 50. ..

Now, it is assumed that the surface light emitting unit 50 of FIG. 2D is provided in the sensor unit 1a of FIG. At point S, it is assumed that the directions of the light receiving portions 40 of the sensor units 1c and 1d provided on the opposite sides are M and N, respectively. Of the diffused light emitted at the point S, only the light in the direction M is detected by the sensor unit 1c, and only the light in the direction N is detected by the light receiving unit of the sensor unit 1d. Therefore, the diffused light emitted in the other direction is consumed as wasted energy in the coordinate input device. By using the surface light emitting unit 50 that emits diffused light in this way, although there is a loss of light energy, the required light can be inexpensively produced within the range of the light emitting surface length L3 without using a complicated optical system. Can be generated.

As described above, the surface light emitting unit 50 must continuously emit light to the light receiving units 40 of at least two sensor units 1 provided on the opposite sides within the range of the light emitting surface length L3. Therefore, the light intensity of each of the light receiving units 40 of the sensor units 1a to 1d is such that the light receiving units 40 can continuously detect light within the range of the light emitting surface length L3 of the surface light emitting unit 50. It is called "almost uniform diffused light" emitted by.

Further, in the present invention, for example, the optical system of the light projecting unit 30 is used as a component of the surface light emitting unit 50, and a cost reduction effect is obtained by standardizing the components.

As described above, the light receiving unit 40 has a structure sandwiched between the light emitting unit 30 and the surface light emitting unit 50. The distance L1 between the light projecting unit 30 and the light receiving unit 40 is set to a value sufficiently smaller than the distance from the light projecting unit 30 to the opposing retroreflective unit 4 so that the observation angle with respect to the opposing retroreflective unit 4 becomes small. Set. Similarly, the distance L2 between the surface light emitting unit 50 and the light receiving unit 40 is also set to a sufficiently small value. Therefore, by adopting a structure in which the light receiving unit 40 is sandwiched between the two, the light receiving unit 40 can efficiently detect both the retroreflected light by the light projecting unit 30 and the light of the surface light emitting unit 50.

Here, the signal detected by the light receiving unit 40 of the sensor unit 1 will be considered. Focusing on the sensor unit 1a of FIG. 1, the light projected by the light projecting unit 30 is retroreflected by the retroreflecting unit 4b, and the intensity of the retroreflected light detected by the light receiving unit 40 is mainly as follows. It depends on the factors.

Factor 1: Light projection distribution of the light projection unit 30 Factor 2: Incident angle characteristic of retroreflective efficiency incident on the retroreflective part 4b Factor 3: Distance between the light projecting unit 30 and the retroreflective unit 4b Factor 4: Light reception distribution of the light receiving unit 40 To explain in order, in FIG. 2B, if the main light ray direction of the light projecting unit 30 is f, the light radiation intensity in the direction f is generally strong and deviates in the direction g or the direction h. As a result, the light emission intensity decreases as shown in FIG. 3 (B).

Further, in FIG. 3A, if the direction of arrival of light with respect to the normal direction of the retroreflective unit 4b is defined as the incident angle α, the retroreflective efficiency of the retroreflective unit 4 is as shown in FIG. 3C. It reaches its maximum at an incident angle of 0 ° and decreases as the incident angle increases. Therefore, the retroreflective efficiency is good in the direction J of FIG. 3A, and the retroreflective efficiency decreases in the direction K. That is, the intensity of the retroreflected light is determined according to the incident angle of the light from the light projecting unit 30 that has reached the retroreflected portion 4b.

Further, in FIG. 3A, the distance between the light projecting unit 30 and the retroreflective unit 4b of the sensor unit 1a differs depending on the direction. That is, the distance is relatively short in the direction J, and the distance becomes longer as the direction K is reached. It is generally known that waves decay exponentially as the reach increases. Further, as shown in FIG. 2A, although the light projected by the light projecting unit 30 is collimated, it is difficult to make it completely parallel light (it is difficult to make it an ideal point light source). .. Therefore, when the reach to the retroreflective portion 4b increases, the light energy is lost by the amount of the spread of the light. Further, even in the light receiving optical system, the light from the center direction of the optical axis can be efficiently collected, but the light collection efficiency generally decreases as the distance from the center direction of the optical axis increases due to the characteristics of the lens optical system.

In FIG. 3A, the intensity of the retroreflected light detected between the direction J and the direction K is mainly determined by the multiplication of the above factors 1 to 4. Since the incident angle α is the largest in the direction K and the distance to the retroreflective unit 4 is the largest, the retroreflected light that can be detected by the light receiving unit 40 is the smallest due to factors 2 and 3. Therefore, by directing the main light direction of the light projecting unit 30 where the radiation intensity of the light projecting unit 30 is the largest in the direction K or the center direction of the optical axis of the light receiving optical system in the direction K, more retroreflected light can be produced. It becomes possible to detect. FIG. 3D shows the light receiving distribution output by the light receiving unit 40 at this time, and although the synchrotron radiation intensity of the light emitting unit 30 is strong in the direction K, the output is small due to factors 2 and 3. It becomes a value. Further, although the influences of factors 2 and 3 are small in the direction J, the output is still small because the radiation intensity of the light emitting unit 30 is weak or the light collecting efficiency of the light receiving unit 40 is lowered. Then, a point where the output is maximized is generated in the direction R between the directions J and the direction K.

Β shown in FIG. 3D indicates the dynamic range of the light receiving unit 40. In order to increase the output signals in the directions J and K, for example, if the current flowing through the infrared LED 31 of the light projecting unit 30 is increased, the output in the direction R also increases, and the output near the direction R has a dynamic range. The waveform is distorted beyond the range of. As a result, accurate position detection becomes impossible. Further, if sufficient output signals cannot be obtained in the directions J and K, stable position detection cannot be performed due to the influence of noise. On the outside of the direction J and the direction K (direction M and direction N in FIG. 3A), the output signal is further rapidly attenuated, so that the influence of noise becomes serious.

Although the details of the coordinate input device of the present invention will be described later, the sensor unit 1a has a direction M with the end of the coordinate input effective area 3 as a visual field range and an origin O of the coordinate input effective area 3. The direction N including the above must be the detection range. Therefore, it is necessary to obtain a stable signal over the entire detection range. Therefore, the sensor units 1a to 1d are provided with a light projecting unit 30 and a surface light emitting unit 50 that project light toward the retroreflective unit 4 provided on the opposite sides. Then, the control / calculation unit 2 causes the light emitting unit 30 of the sensor unit 1a to emit light, and at the same time, causes the surface light emitting units 50 of the opposing sensor units 1c and 1d to emit light, and emits both lights to the light receiving unit 40 of the sensor unit 1a. Detect with.

The light emitted by the surface light emitting unit 50 is diffused light as schematically shown in FIG. 2 (D). Therefore, only the light emitted in the direction of the facing sensor unit 1a is detected by the light receiving unit 40 of the sensor unit 1a, and the light emitted in the other direction is discarded as extra light. Therefore, the light emitted by the surface light emitting unit 50 detected by the light receiving unit 40 indicates the arrival direction of the light, that is, the angle information.

The detection signal waveform of the light receiving unit 40 of the sensor unit 1a obtained by this configuration is shown in FIG. 3 (E), and is a stable signal within the dynamic range range of the light receiving unit 40 in the range from the direction M to the direction N. Can be detected.

By using the surface light emitting unit 50, extra light is generated in addition to the required light (light toward the opposing sensor unit 1), but the following merits can be obtained. The retroreflected light by the light projecting unit 30 is a reciprocating route to the retroreflective unit 4 and the light receiving unit 40 facing from the light projecting unit 30, but the path of the surface light emitting unit 50 is one-way, and the light loss is small. .. Further, as shown in FIG. 3A, the surface emitting unit 50 of the sensor unit 1c is closer than the surface emitting unit 50 of the sensor unit 1d when viewed from the sensor unit 1a. Further, as shown in the drawing, the normal direction of the surface light emitting unit 50 of the sensor unit 1d faces the direction of the sensor unit 1a, and the surface light emitting unit 50 of the sensor unit 1c has a large angle difference from the normal direction. ing. Generally, the surface light emitting unit 50 shown in FIG. 2D emits diffused light from the light emitting surface, but the light intensity of the light emitting surface in the normal direction is strong, and gradually weakens when an angle difference occurs from the normal direction. Become.

Therefore, although the light energy from the surface emitting unit 50 of the sensor unit 1c toward the sensor unit 1a is not strong, the distance to the sensor unit 1a is short, so that the light energy can be detected by the light receiving unit 40 of the sensor unit 1a. On the other hand, although the distance from the sensor unit 1d to the sensor unit 1a is long, the energy of the light directed to the sensor unit 1a is strong, so that the light receiving unit 40 of the sensor unit 1a can detect the light.

From the viewpoint of the sensor unit 1b, this time, the normal direction of the light emitting surface of the sensor unit 1c is the direction of the sensor unit 1b, and the distance between the two becomes long. Therefore, as in the case of the sensor unit 1a, a stable signal is obtained. Is obtained.

That is, by making the normal direction of the light emitting surface of the sensor unit 1d (first sensor unit) substantially coincide with the direction of the sensor unit 1a (second sensor unit) provided on the opposite side, the sensor unit 1a And the sensor unit 1b can detect the light at the same time. Specifically, the main ray direction (optical axis direction) of the light projecting unit 30 of the sensor unit 1a is the direction K, which is the diagonal direction of the sensor unit 1a and is provided on the opposite side. It is substantially aligned with the normal direction of the light emitting surface of the surface emitting unit 50 of 1d.

With this configuration, when the sensor unit 1a detects and the sensor unit 1b detects, the drive control of the surface light emitting unit 50 of the sensor unit 1d (for example, the current value to the infrared LED and the light emitting time) Etc.) is not necessary. Therefore, since the sensor units 1a and 1b can perform simultaneous detection, the sampling rate for coordinate calculation can be improved (the sampling rate is doubled as compared with the case where the sensor units 1a, 1b, 1c and 1d each detect. Become).

FIG. 8A shows a cross section of the sensor unit 1 of FIG. 2A, and an explanation for explaining the height positional relationship between the sensor unit 1 and the retroreflective unit 4 (normal direction of the coordinate input surface 5). It is a figure.

The light receiving unit 40 (composed of a line CCD 41, a light receiving lens 42, an infrared filter 44, etc.) is sandwiched between a light emitting unit 30 and a surface light emitting unit 50 (composed of a collimating lens 54, a diffuser plate 60, etc.). It is composed. In the present invention, the upper end portion of the retroreflective portion 4 is located at substantially the same height as the upper end portion of the diffuser plate 60 of the surface light emitting portion 50, and the lower end of the retroreflective portion 4 is the lower end of the light projecting portion 30. It is at almost the same height position as. FIG. 8B shows the situation, and shows the height relationship of the retroreflective portion 4 provided on the side facing the sensor unit 1. The height positions of the retroreflective portions 4 provided on the two sides are the same for both, but the retroreflective portion 4 on the sensor unit 1 side is not specified in FIG. 8 (B).

FIG. 8B shows a light beam from the light projecting unit 30 to the retroreflective unit 4 provided on the opposite side, and a light ray reaching the light receiving unit 40 from the retroreflective unit 4. If any of these light rays is blocked, the sensor unit 1 can detect the direction thereof, so that even if the indicator is not in contact with the coordinate input surface 5, in other words, proximity input is possible.

Further, the sensor unit 1 in FIG. 8B receives the light of the surface light emitting unit 50'of the sensor unit 1 provided on the opposite side. Since the heights of the upper end of the diffuser plate 60 and the upper end of the retroreflective part 4 at the surface light emitting part 50'are the same, the height at which proximity input is possible is the connection between the retroreflective part 4 and the diffuser plate 60. It is kept constant. Therefore, it is possible to provide a coordinate input device that is easy for the operator to operate.

&lt;Explanation of control / calculation unit&gt; A CCD control signal for a CCD, a clock signal for a CCD, an output signal of a CCD, and an LED drive signal are transmitted and received between the control / calculation unit 2 of FIG. 1 and the sensor units 1a to 1d.

FIG. 4A is a block diagram of the control / calculation unit 2. The control signal for the CCD is output from the arithmetic control circuit (CPU) 71 composed of a one-chip microcomputer or the like, and controls the shutter timing of the CCD, data output, and the like. The clock signal for the CCD is transmitted from the clock generation circuit (CLK) 72 to the sensor units 1a to 1d, and is also input to the arithmetic control circuit 71 in order to perform various controls in synchronization with the line CCD 41. There is.

The LED drive signal is supplied from the arithmetic control circuit 71 to the infrared LED of the light projecting unit 30 or the surface light emitting unit 50 of the sensor units 1a to 1d via the LED drive circuits 74a to 74d. Further, the LED drive circuits 74a to 74d control the power supply to the respective light projecting units 30 or the surface light emitting unit 50 of the sensor units 1a to 1d at the timing described later.

The detection signals (analog signals) from the line CCD 41, which is the light receiving unit 40 of the sensor units 1a to 1d, are input to the AD converters 75a to 75d of the control / calculation unit 2, respectively, and are digital signals under the control of the calculation control circuit 71. Is converted to. The converted digital signal is stored in the memory 73 as needed, the angle is calculated and the coordinate value is calculated by the method described later, and the result is transmitted to a terminal such as an external PC as a communication interface (for example, USB interface) 76. Output via.

FIG. 4B is a timing chart of various signals.

81 and 82 are control signals for the CCD, and the shutter release time of the line CCD 41 is determined at intervals of the control signals 81. 82a, 82b, 82c and 82d are gate signals to the sensor units 1a, 1b, 1c and 1d, respectively, and are signals to transfer the electric charge of the photoelectric conversion unit inside the line CCD 41 to the reading unit. 83a, 83b, 83c and 83d are signals for driving the infrared LED 31 of the light projecting unit 30 of the respective sensor units 1a, 1b, 1c and 1d. Further, 84a, 84b, 84c and 84d are signals for driving the infrared LEDs 51 to 53 of the surface light emitting units 50 of the sensor units 1a, 1b, 1c and 1d, respectively.

First, in the first cycle of the control signal 81, the infrared LEDs 31 of the light projecting units 30 of the sensor units 1a and 1b provided on one side are turned on, and the surfaces of the sensor units 1c and 1d provided on the opposite sides are turned on. The infrared LEDs 51 to 53 of the light emitting unit 50 are turned on. Therefore, the light receiving unit 40 of the sensor unit 1a includes the light retroreflected by the retroreflective unit 4b provided on the opposite side of the light projected by the light projecting unit 30, and the sensor unit 1c and the sensor unit 1c provided on the opposite side. The light of the surface emitting unit 50 of 1d is detected. Similarly, the light receiving unit 40 of the sensor unit 1b detects the retroreflected light of the light projecting unit 30 of the sensor unit 1b and the light of the surface light emitting unit 50 of the sensor units 1c and 1d. Then, the detection result of the light receiving unit 40 of the sensor units 1a and 1b is output by the gate signals 82a and 82b.

The signal to be read is, for example, when there is no input by an indicator such as a finger, that is, when there is no light-shielding portion, the light amount distribution as shown in FIG. 3 (E) is output from the respective sensor units 1a to 1d. Is obtained. Of course, such a light amount distribution cannot always be obtained in any system, and it depends on the characteristics of the retroreflective unit 4, the characteristics of the light projecting unit 30 and the surface light emitting unit 50, and changes over time (dirt on the reflecting surface, etc.). This light intensity distribution changes.

Next, based on the next control signal 81, this time, the infrared rays of the light emitting units 30 (signals 83c and 83d) of the sensor units 1c and 1d and the surface light emitting units 50 (signals 84a and 84b) of the sensor units 1a and 1b. The LED is driven. Then, the detection results of the light receiving unit 40 of the sensor units 1c and 1d are output by the gate signals 82c and 82d.

The light emitting unit 30 of the sensor units 1a and 1b emits light at the same time, but the light of the light emitting unit 30 of the sensor unit 1a is not detected by the light receiving unit 40 of the sensor unit 1b. The configuration may be such that the projection range of the projection unit 30 is limited, or a light-shielding plate is provided between the sensor units 1a and 1b. Therefore, as shown in FIG. 4B, it is possible to detect the required signal in at least three cycles of the control signal 81. Further, if the sensor units 1a and 1b are illuminated during the reading period of the sensor units 1c and 1d, it is possible to acquire the control signal 81 in two cycles.

By controlling in this way, it is possible to obtain an excellent effect that enables a high-speed position detection sampling rate to be realized.

In FIG. 3 (E), assuming that the A level is the maximum detected light amount level and the B level is the lowest detected light amount level, the level obtained in the absence of the detected light is near the B level, and the detected light amount is reached. As the number increases, it approaches the A level. In this way, the analog signal output from the light receiving unit 40 is sequentially AD-converted and taken into the arithmetic control circuit 71 as a digital signal.

FIG. 3F is an example of an output when input is performed with an indicator such as a finger, that is, when the detection light is blocked. Since the reflected light is blocked by the indicator in the C level portion, the amount of light is reduced only in that portion.

The detection is performed by detecting this change in the amount of light distribution. Specifically, first, the initial state without input as shown in FIG. 3 (E) (hereinafter, the data obtained in the initial state is referred to as initial data (or reference data)) is stored in the memory 73 in advance. .. Then, by calculating the difference between the pixel data obtained in each sample period and the initial data stored in advance, it is determined whether or not there is a change as shown in FIG. 3 (F).

Specifically, the threshold value Vtha as shown in FIG. 3 (F) is compared with the output signal, and the pixel number equal to or lower than the threshold value Vtha is detected. Then, the pixel range in which the output level is equal to or less than the threshold value Vtha is calculated, and for example, the center thereof is defined as the direction in which the light-shielding object is located. The pixel number N of the line CCD41 indicates the direction of arrival of light (angle θ), as can be clearly seen in the optical system of FIG. 2C. Therefore, by calculating the relationship between the pixel number N and the angle θ in advance as a function at the time of assembly at a factory or the like, the output pixel number N can be converted into the angle θ.

&lt;Explanation of coordinate calculation method&gt; FIG. 5 is a diagram for explaining the visual field range of the sensor unit 1. FIG. 5A is a diagram for explaining the visual field range of the sensor unit 1a. The light of the light projecting unit 30 of the sensor unit 1a is retroreflected by the retroreflective unit 4b provided on the opposite side, and detected by the light receiving unit 40 of the sensor unit 1a. At the same time, the light of the surface emitting unit 50 of the sensor units 1b and 1c provided on the opposite sides is detected. Therefore, the range of the light arrival direction detected by the light receiving unit 40 of the sensor unit 1a is expressed by the angle θa. Therefore, when an input instruction is given by the indicator to the range of the hatched portion area-a in the coordinate input effective region 3, the light in that direction is blocked, and the sensor unit 1a changes its direction (angle) from the change in the light. Can be detected. However, even if the input instruction operation is performed outside the hatched portion area-a in the coordinate input effective region 3, the sensor unit 1a cannot detect it.

FIG. 5B is a diagram for explaining the visual field range of the sensor unit 1b. In this case, the field of view of the sensor unit 1b is represented by an angle θb. Therefore, when an input instruction is given by the indicator to the range of the hatched portion area-b in the coordinate input effective area 3, the light in that direction is blocked, and the sensor unit 1b can detect the direction (angle). It will be possible.

The hatched portion area-ab in FIG. 5C is an overlapping region of the visual field range of the light receiving portion 40 of the sensor units 1a and 1b (the overlapping region of the hatched portions of FIGS. 5A and 5B). Therefore, if the instruction operation by the indicator is performed in the area of the hatched portion area-ab, the direction can be detected by each of the sensor units 1a and 1b. Therefore, it is possible to geometrically calculate the indicated position using the distance between the sensor units 1a and 1b and the directions (angles) obtained by the respective sensor units 1a to 1d.

As shown in FIG. 1, the sensor units 1c and 1d are arranged at positions substantially symmetrical with the sensor units 1a and 1b, respectively, with the x-axis as the axis of symmetry. Similarly, the sensor units 1b and 1d are arranged at positions substantially symmetrical with the sensor units 1a and 1c with the Y axis as the axis of symmetry. Therefore, the position designated within the range of the hatched portion area-cd of the coordinate input effective region 3 of FIG. 6A can be calculated from the angle information obtained by the sensor units 1c and 1d. Similarly, in the hatched portion area-ac shown in FIG. 6B, the indicated position can be calculated using the angle information obtained by the sensor units 1a and 1c. Further, in the hatched portion area-bd shown in FIG. 6C, the indicated position can be calculated by using the angle information obtained by the sensor units 1b and 1d.

Further, in the area of the hatched portion area-accd shown in FIG. 6D, the indicated position can be calculated by using the detection results of the sensor units 1c and 1d or by using the detection results of the sensor units 1a and 1c. It is possible. Therefore, the position calculation accuracy can be improved by using the average value of the output results of both.

Furthermore, when the coordinates are continuously input from the hatched portion area-ac (FIG. 6 (B)) to the hatched portion area-cd (FIG. 6 (A)), a sensor for calculating the position in the middle of the coordinate input is performed. The combination of units 1 is changed. When the change occurs, for example, the detected coordinate values become discontinuous even though the continuous coordinate input operation is performed due to a factor such as a displacement of the sensor unit position or a measurement error. There is. In other words, the coordinate calculation resolution is lowered by changing the combination.

In order to prevent this, the output results of the sensor units 1a and 1c are adopted at the α point in FIG. 6D, and the output results of the sensor units 1c and 1d are added as the sensor units move to the β point. Then, only the output results of the sensor units 1c and 1d are adopted at the β point. In this way, by changing the weights of the output results of the sensor units 1a and 1c and the output results of the sensor units 1c and 1d according to the position (weighted average), an excellent effect of enabling faithful detection of the indicated position can be obtained. can get.

Further, the position of the hatched portion area-abcd shown in FIG. 6E can be calculated by any combination of the sensor units 1. Therefore, by instructing the origin O, it is possible to correct the coordinate system so that the output results output by each combination of the sensor units 1 are the same.

As described above, in the coordinate input device of the present invention, the visual field range of each sensor unit 1a to 1d is the entire area on the side opposite to the origin O and the coordinate input effective region 3. Then, in the region where the switching of the sensor units 1a to 1d occurs, the visual field range is set so that the position can be detected by the combination of the plurality of sensor units a to 1d.

&lt;Explanation of coordinate calculation process&gt; FIG. 7 is a flowchart showing the coordinate calculation process of the first embodiment.

First, when the power of the coordinate input device is turned on, in step S102, various initializations related to the coordinate input device such as port setting and timer setting of the control / calculation unit 2 are performed. In step S103, the pixel effective range of the line CCD 41 is set from, for example, a set value stored in advance in the memory 73. Further, the initial read count of the initial read operation of the line CCD 41 is set.

It should be noted that this initial reading operation is an operation for removing unnecessary charges of the line CCD 41, which is performed only when the coordinate input device is started up. In the line CCD41, unnecessary charges may be accumulated when the line CCD 41 is not operated, and if the coordinate input operation is executed while the charges are accumulated, it may become undetectable or cause erroneous detection. .. Therefore, in order to avoid this, in step S104, the reading operation is executed a predetermined number of times while the light emitting by the light emitting unit 30 and the surface light emitting unit 50 is stopped. As a result, unnecessary charges are removed.

In step S104, the read operation of the line CCD41 is executed. In step S105, it is determined whether or not the reading has been executed a predetermined number of times or more. If the reading has not been executed more than a predetermined number of times (NO in step S105), the process returns to step S104. On the other hand, when the reading is executed a predetermined number of times or more (YES in step S105), the process proceeds to step S106.

In step S106, the pixel data of the line CCD 41 in the non-illuminated state of the light projecting unit 30 and the surface light emitting unit 50 is taken in as the base data. In step S107, the base data is stored in the memory 73. Subsequently, in step S108, as reference data, pixel data corresponding to the initial light amount distribution of the line CCD 41 in the illuminated state of the light projecting unit 30 and the surface light emitting unit 50 is taken in. In step S109, the reference data is stored in the memory 73.

The reference data in the state with illumination is the illumination of the light projecting unit 30 of the sensor unit 1 itself and the illumination of the surface light emitting unit 50 of the two sensor units 1 provided on the opposite sides of the sensor unit 1. Is. Further, the set of the sensor units 1a and 1b on the upper side of the coordinate input effective region 3 and the set of the sensor units 1c and 1d on the lower side are illuminated at different timings to capture reference data. This is because the sensor units 1a and 1b on the upper side and the sensor units 1c and 1d on the lower side face each other, so that if they are illuminated at the same time, each other's illuminations will be detected by each other's light receiving units 40. This is to avoid it.

Then, in step S110, it is determined whether or not the acquisition of the reference data is completed in all the sensor units 1a to 1d. When the acquisition of the reference data is not completed in all the sensor units 1a to 1d (NO in step S110), steps S108 and S109 are repeated. On the other hand, when the acquisition of the reference data is completed in all the sensor units 1a to 1d (YES in step S110), the process proceeds to step S111.

The processing up to this point is the initial setting operation when the power is turned on. Needless to say, this initial setting operation may be configured to operate according to the intention of the operator by a reset switch or the like configured in the coordinate input device. After this initial setting operation, the state shifts to the normal coordinate input operation state (normal acquisition operation state) by the indicator.

In step S111, in the coordinate input sampling state, the normal acquisition operation of the line CCD41 is executed to acquire the pixel data (light amount distribution). In step S112, it is determined whether or not the acquisition of pixel data has been completed in all the sensor units 1a to 1d. When the acquisition of the pixel data of all the sensor units 1a to 1d is not completed (NO in step S112), step S111 is repeated. Then, when the acquisition of the pixel data of all the sensor units 1a to 1d is completed (YES in step S112), the reference data acquired at the time of initialization is compared with the pixel data in step 113, and the difference value is calculated. .. In step S114, it is determined whether or not there is a light-shielding portion (input). If it is determined in step S114 that there is no input (NO in step S114), the process returns to step S111, and the acquisition of pixel data is executed again. At this time, if this repetition period is set to about 10 [msec], sampling is performed 100 times / second. On the other hand, if it is determined that there is an input (YES in step S114), the process proceeds to step S115.

As described with reference to FIGS. 5 and 6, the region in the coordinate input effective region 3 is divided, and the combination of the sensor units 1a to 1d for detecting the light-shielded portion is determined for each region. Therefore, when the input operation is performed, the light-shielding portion should be detected by at least two sensor units, and in step S114, the sensor unit that has detected the light-shielding portion is specified. If no input is performed, there is no sensor unit that has detected the light-shielded portion, and the process returns to step S111 again to repeat the acquisition of pixel data.

In step S115, it is determined whether the sensor unit that has detected the light-shielding portion is larger than 2, that is, whether it indicates the overlapping area of FIG. 6 (D) or FIG. 6 (E) or the other area. To do. When the overlapping area is specified (YES in step S115), Flag = 1 is set. On the other hand, when a region other than the overlapping region is specified (NO in step S115), Flag = 0 is set.

In step S118, the combination of the two sensor units required for coordinate calculation is selected. In step S119, the indicated position (coordinate value) is calculated using the angle information output by the two selected sensor units and the distance information between the sensor units (first calculation). Next, in step S120, it is determined whether or not Flag = 0. When Flag = 0 (YES in step S120), the coordinate value calculated here is used as a definite coordinate value and output to an external device in step S123. Then, the process returns to step S111.

On the other hand, when Flag = 1 (NO in step S120), there are a plurality of combinations of the two sensor units whose indicated position can be calculated, and it is determined in step S121 whether or not the calculation is completed in all the combinations. If there is a combination of the remaining sensor units that can calculate the indicated position (NO in step S121), the process returns to step S118 and the indicated position calculation is executed again. As a result, since there are a plurality of indicated position calculation results in step S122, coordinate values such as the average value or the weighted average value are calculated (second calculation). Then, in step S123, the calculated coordinate value value is output to the external device as the definite coordinate.

As described above, according to the first embodiment, since a stable light output signal can be obtained at the light receiving portion of the sensor unit, the accuracy and resolution of the detected coordinate values are improved, and the sensor unit can be miniaturized. Therefore, an excellent effect that the device can be miniaturized can be obtained. Further, since the components (sensor unit, retroreflection unit) can be arranged only on the two opposite sides of the coordinate input effective region, it is possible to provide a large-sized and horizontally long coordinate input device capable of supporting multiple displays.

&lt;Embodiment 2&gt; According to the configuration of the present invention, the retroreflective portions 4a and 4b provided on the outer two sides of the coordinate input effective region 3 and the sensor units 1a to 1d provided near both ends of the retroreflective portions 4a and 4b are stable. Provided is a coordinate input device that detects light and has good operability.

With this configuration, the following new configurations are also possible.

FIG. 9A is an example in which the present invention is applied to a multi-display. Sensor units 1a, 1b, 1c and 1d, and retroreflective parts 4a and 4b are provided in order to set the range of the display area β as the coordinate input effective area. If this set of components is placed next to each other as shown in the figure (components are indicated by broken lines in the figure), a multi-display consisting of three display surfaces can be made interactive. Here, interactivity refers to a function that enables display control by directly touching an object displayed on the display surface and display of the trajectory by instructing the object. That is, the display surface can be enlarged as much as possible by connecting the configurations of the first embodiment of the present invention as a unit. Since there is no shield such as a retroreflective part at the joint of the display images, it is possible to continuously input coordinates across the display surface, and an excellent effect of providing an interactive display with good operability can be obtained. ..

Further, as shown in FIG. 1, the components of the present invention are housed in two sensor bars 8a and 8b. Therefore, as shown in FIG. 9B, by mounting two sensor bars 8a and 8b on an existing whiteboard, wall, or the like, the display device can be immediately made interactive. In this case, there are only two components, and the user can wear it "immediately", "easily", and "anywhere", and an excellent effect of greatly improving the convenience of the user can be obtained.

As described above, according to the second embodiment, in addition to the effect described in the first embodiment, the coordinate input device can be easily provided by making the component of the coordinate input device removable from the coordinate input surface. It becomes possible to carry it.

In other words, the coordinate input device integrated with the conventional screen increases in size according to the size of the display screen. Therefore, in the manufacturing stage, the distribution stage, and the sales stage, the management cost and the distribution cost are large, and the installation also requires a large cost. By adopting a product form in which the sensor part is attached later to a display that uses an existing whiteboard, wall surface, or show window as a screen, the input device can be miniaturized and the cost burden can be significantly reduced. Is possible. In addition, by making the configuration portable for the user, it is possible to obtain an excellent effect that the usage opportunity is increased and the convenience can be improved.

The present invention is also realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiment is supplied to the system or device via a network or various storage media, and the computer (or CPU, MPU, etc.) of the system or device reads the program. This is the process to be executed.