Action detecting interface

[0001] [Technical field to which the invention belongs] The present invention relates to a remote control device for electronic devices such as a television receiver, particularly an optical motion sensing remote control device for electronic devices. [0002] [Conventional technology] An infrared remote control device is used as a general device for remotely controlling a television receiver. However, conventional remote control devices have some drawbacks. One of them is that remote control devices tend to disappear. Also, users often mistake the remote control device for video cassette recorders for the remote control device for television receivers. In fact, many people own a large number of remote controls. The user also has to remember where and what button is located on the remote controller. Also, the remote control device must replace the battery on a regular basis. If the television receiver is equipped with a video camera and can read the user's gestures, the remote control device becomes unnecessary. However, in a television receiver, it is not easy to distinguish other movements from user gestures within the range captured by the video camera. For example, it would be inconvenient for a user to change channels each time they stood up to get a snack in the kitchen. [0003] [Problems to be Solved by the Invention] The present invention has been made in view of the above-mentioned circumstances, and the present invention comprises a display device that displays a plurality of icons indicating predictable processing in an electronic device, and a motion detection circuit that detects motion in a visual field. It is an object of the present invention to provide a remote control device. [0004] [Means for solving problems] The motion detection circuit tracks the movement of the user's hand after detecting a predetermined movement of the hand indicating that the user has started the remote control operation in the shape of the hand and the visual field. The object does not necessarily have to be a hand. The motion detection circuit outputs a cursor control signal indicating the movement of the hand. The display device and the motion detection circuit are connected to the control circuit. The control circuit receives a cursor control signal and displays a movable and movable visual indicating means, that is, a cursor on a display device. The cursor moves on the display device according to the movement of the user's hand. Further, when the control circuit controls the electronic device, the controlled electronic device performs an operation corresponding to the icon selected by the user by using the visual instruction means that the user can move. [0005] In some of the embodiments according to the present invention, when a particular icon overlaps with a movable visual indicator on the display device, the motion detection circuit detects a motion pattern of a predetermined hand, and the user's icon is displayed. Detect selection. For example, the motion detection circuit detects the pause of hand motion for a certain period of time after the particular icon and the movable visual indicator overlap on the display device as the user's selection of the icon. Alternatively, when the user performs an action such as pressing an icon, the motion detection circuit detects this action as an operation of pressing a button. [0006] In the first embodiment of the present invention, the motion detection circuit includes a video camera. The motion detection circuit also receives the video output signal output from the video camera and calculates the motion vector in each macroblock between two consecutive video frames. Each video frame consists of multiple blocks. Each block has a vector that represents the behavior of the detected object. The motion detection circuit groups adjacent vectors in substantially the same direction as one region. When deciding whether to track or not track an object for each frame, the motion detection circuit examines each region and whether or not this region satisfies the following two conditions (a) and (b). To judge. [0007] (a) This vector makes one rotation clockwise or counterclockwise. (b) Return to the position where this area was originally located. When this region satisfies the above conditions (a) and (b), the motion detection circuit locks this region. [0008] In a second embodiment of the invention, the motion detection circuit is at least one video camera, random access memory. It has an interface), a random access memory, and a CPU. The motion detection circuit detects motion by comparing the pixel values of each macroblock in two consecutive video frames output from the video camera. If the absolute value of the difference between two corresponding macroblocks in two consecutive video frames exceeds a certain value, the motion detection circuit determines that the motion was performed in this macroblock, and this region is the active region. Judge as. [0009] When deciding whether to track or not track the movement of the hand for each video frame, the motion detection circuit investigates the detected active region, and this active region has the following two conditions (a) and (b). Determine if it meets the requirements. [0010] (a) This active region moves in a straight line in the first direction. (b) Return to where this active region was originally located. When this active region satisfies the above conditions (a) and (b), the motion detection circuit locks this active region. In addition to the above-mentioned movements, all kinds of repetitive movements can be used as movements for instructing the locking of hand movements. [0011] Further, in the third embodiment of the present invention, the motion detection circuit compares the image portion selected by the user output from the video camera with the image stored in advance, and when the two images match, a video is obtained. It is determined that the image portion selected by the user output from the camera is the user's hand. In addition to the methods described above, various methods can be used by the user to select an image portion representing his or her hand. [0012] In all the embodiments of the present invention, the control circuit is used to keep the ratio between the moving distance of the hand and the moving distance of the movable visual indicating means always constant regardless of the length of the distance between the user and the motion detection circuit. Has an automatic cursor sensitivity adjustment function. The automatic cursor sensitivity adjustment adjustment mechanism calculates the ratio of the movement distance of the movable visual indicating means corresponding to the movement distance of the hand 20. Therefore, for example, when the user performs an operation of moving the hand in an oblique direction, the movable visual indicating means moves diagonally on the display screen regardless of the length of the distance between the user and the operation detection circuit. [0013] BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, the remote control device and the remote control method for electronic devices according to the present invention will be described in detail with reference to the drawings. [0014] In the first embodiment of the present invention, when the user performs an operation that is easy for the television receiver to detect, the television receiver locks such an image. Further, in the modification of the present invention, the remote control device visually recognizes and locks the user's hand. Although the term "hand" is used in this specification, other discriminable objects may be used instead of the user's hand. [0015] In all embodiments of the invention, when the hand image is locked, the television receiver tracks the movement of the hand and moves the cursor on the display device in the same direction as the movement of the hand. That is, the user can move the cursor with his / her hand, just as he / she moves the mouse of a personal computer. In addition, the user can move the cursor and select a desired icon from the icons on the display device. If you lose sight of this movement after locking the hand image, the television receiver will display a message to the user. When the message is displayed, the user either performs a particular hand action or reselects the image of the hand on the display. By the method described above, the television receiver relocks the image of the hand and tracks this movement. [0016] FIG. 1 is a block diagram showing a basic structure of a remote control device to which the present invention is applied. The parts shown in blocks 1 to 12 are the same as those of a conventional digital television receiver set. Antenna 1 receives the signal. The tuner 2 tunes the signal received by the antenna 1. The signal tuned in the tuner 2 is demodulated and error corrected in the demodulation and error correction block 3. The demultiplexer 4 separates the signal supplied from the block 3. Demultiplexed on-screen (hereinafter referred to as OSD) data is supplied to the OSD circuit 5, video data is supplied to the video decoder 6, and audio data is supplied to the audio decoder 10. The superimposer 7 mixes OSD data and demodulated video data and supplies them to the cathode ray tube (hereinafter referred to as CRT) block 8. The CRT block 8 displays the data supplied from the super imposer 7 on the CRT 9. The amplifier 11 amplifies the demodulated audio signal and outputs it to the loudspeaker 12. [0017] Blocks 13 to 16 are sites that perform a major function in all embodiments of the invention, and in each embodiment perform some different function. The video camera 13 is mounted on the CRT9, for example, when it captures an image of the user 18 in front of the television receiver set, it supplies the image to the motion detection circuit 15. The motion detection circuit 15 is connected to the RAM 14 and the CPU 16. [0018] In the first embodiment of the present invention, when it is desired to remotely control the television receiver, first, the user 18 uses the hand 20 to repeatedly perform a discriminable operation. Examples of this discriminable operation include the operation of drawing a circle as shown in FIG. 2 and the operation of moving the hand 20 from the position of 20 (a) to 20 (b) as shown in FIG. Is listed as. The television receiver distinguishes between the above-mentioned movements and other movements, and senses the intention of the user to perform remote control. At this time, the television receiver displays the menu button icon 22 on the screen of the CRT 9 as shown in FIGS. 3 and 21. When the image of the hand 20 is captured by the CPU 16, the CPU 16 locks this movement, and the cursor 24 displayed on the screen of the CRT 9 follows the movement of the locked hand 20. For example, when the user 18 moves the hand 20 to the right, the cursor 24 displayed on the screen of the CRT 9 also moves to the right at the same time. The hand 20 and the cursor 24 function similarly to the mouse and the cursor in a personal computer, respectively. The television receiver does not know the absolute position of the hand 20, but only senses the moving speed and the moving direction, and moves the cursor 24 accordingly. When the cursor 24 moves to the position of the menu button icon desired by the user 18, the user 18 keeps the hand 20 stopped at this position for about 2 or 3 seconds. The CPU 16 determines the above-mentioned operation as the operation of pressing the button, and executes the function represented by the menu button icon 22. If no action is detected within a certain period of time, the CPU 16 determines that the time has expired, deletes the menu button icon 22 from the screen, and starts detecting the predetermined action again. [0019] Next, the motion detection circuit 15 according to the first embodiment of the present invention will be described. First, the motion detection circuit 15 detects the motion vector of each macroblock between two consecutive frames. At this time, the motion detection circuit 15 deletes the motion vector that does not meet the specific threshold value. FIG. 4 is a diagram showing a specific example of the entire macroblock in one video frame. For the sake of brevity, the macroblocks in the figure are shown smaller and larger than they actually are. When the motion detection circuit 15 detects a motion vector, as shown in FIG. 4, the motion detection circuit 15 groups a plurality of vectors having substantially the same direction as one region. For example, FIG. 4 is a diagram showing that region 1 and region 2 are grouped. At this point, the motion detection circuit 15 does not determine to which region the image of the hand 20 belongs, and performs the above processing for all frames. [0020] When there is a region composed of vectors having substantially the same position as the region 1 and having substantially the same direction in the next frame, the motion detection circuit 15 groups this region as a region following the region 1. .. Other areas are grouped in the same way. The new area will be given a new name. If the region following the region in the previous frame is not detected, the motion detection circuit 15 deletes this region. 5 to 8 are diagrams showing snapshots of motion vectors, each having an interval of 0.5 seconds. It usually takes 3 seconds to draw a circle. As shown in FIG. 6, since the region 1 disappears within 3 seconds, the motion detection circuit 15 determines that the region 1 is not the motion of the hand 20. Region 2 remains as a candidate for the region to which the movement of hand 20 belongs. For all frames, the motion detection circuit 15 inspects whether each region satisfies the following conditions (1) and (2). [0021] [0021] (1) This vector makes one rotation clockwise or counterclockwise. (2) Return to the position where this area was originally located. [0022] When a certain region satisfies the above two conditions, the motion detection circuit 15 determines that this region is an image of the hand 20. [0023] When the motion detection circuit 15 determines that the region 2 is an image of the hand 20 as shown in FIG. 8, it then locks the region 2 and tracks this motion as shown in FIG. The motion detection circuit 15 notifies the CPU 16 that the movement of the hand 20 has been locked, and supplies information about the movement of the hand 20 to the CPU 16. The CPU 16 supplies the information received from the motion detection circuit 15 to the OSD circuit 5. When the OSD circuit 5 receives the information, it displays the cursor 24 on the CRT 9 and makes it follow the movement of the hand 20. [0024] When the motion detection circuit 15 loses track of the hand 20, the motion detection circuit 15 supplies the information to the CPU 16. When CPU16 receives the information, it displays the message "Please move your hand to the right" on CRT9. When the user 18 follows this message, the motion detection circuit 15 then controls the CPU 16 and displays the message "Please move your hand up" on the CRT9. When the user 18 follows this message again, the motion detection circuit 15 redetermines and locks the image of the hand 20 by detecting an image that first moves to the right and then moves up. [0025] The above-mentioned movement of the specific hand 20 is not limited to the movement of drawing a circle, and may be other identifiable movement. Also, when selecting the menu button icon 22, the user 18 can not only stop the hand 20 but also perform other specific actions. As a modification of the motion of drawing a circle, the user 18 may move the hand 20 several times in the diagonal direction, for example, from the lower left to the upper right, as shown in FIG. When the user 18 moves the hand 20 up, the motion vector points to the upper right, as shown in area 3 in FIG. 11, and when the user 18 moves the hand 20 down, it is shown in area 3 in FIG. As such, the motion vector points to the lower left. That is, each time the user 18 moves the hand 20, the motion vector points in the opposite direction. Therefore, the motion detection circuit 15 determines that the user has performed a specific motion by detecting a motion vector pointing from a predetermined direction to the opposite direction a predetermined number of times within a fixed time, and moves the hand. Lock 20 movements. [0026] The above-mentioned operation is an operation that is easier for the user 18 to perform than the operation for drawing a circle shown in FIG. 2, and is an operation that is easier for the operation detection circuit 15 to detect. However, this behavior is often unintentionally performed by the user 18, and is therefore often erroneously detected. For example, when the user 18 moves the leg up and down, the motion detection circuit 15 may erroneously detect the motion as the motion described above. This is a trade-off. [0027] As a variation of the cursor 24, the OSD circuit 5 may highlight the menu button icon 22, such as the digital satellite system graphical user interface. For example, when user 18 moves hand 20 up, OSD circuit 5 highlights the menu button icon 22 at the top. With the cursor on the highlighted menu button icon 22, user 18 stops hand 20 and selects this menu button icon 22. In the present specification and claims, the cursor 24 follows the movement of the hand 20 detected by the motion detection circuit 15, including highlighting the menu button icon 22 corresponding to the motion of the hand 20 as described above. It shall mean all changes on the display device. [0028] In the embodiment of the present invention, the method of image recognition is not limited to the detection of motion vectors. For example, the motion detection circuit 15 tracks the locus drawn by the color of the hand 20, and when this locus draws a circle, detects and locks the region of this color as an image of the hand 20. In the second embodiment of the present invention, the pattern matching method is used as a method for detecting the image of the hand 20. In the present invention, the most important point is that the user performs a predetermined identifiable operation such that the motion detection circuit 15 can easily discriminate the motion of the hand 20 and other motions. [0029] Further, the present invention can be applied not only to a digital television receiver but also to a general device including a video camera and a display device such as an analog television receiver or a personal computer television telephone. Further, as in the case of CRT9, for example, a liquid crystal display device or a projector can be used. [0030] MPEG or H261 video encoders are used for television conferences and television telephones. FIG. 13 is a diagram showing a general example of a video encoder. The video camera 100 supplies a signal to the discrete cosine transform circuit 102 via the subtraction node 101, and the discrete cosine transform circuit 102 compresses the received signal. In the case of a predicted frame, the subtraction node receives a signal from the video camera 100, subtracts this signal from the data in the restored frame, and then the discrete cosine transform circuit 102 processes the signal for discrete cosine transform. The quantization circuit 103 receives the signal processed by the discrete cosine transform, quantizes the signal, and outputs it as an encoded stream. The dequantization circuit 104 receives a signal from the quantum circuit, dequantizes the signal, and supplies the signal to the inverse discrete cosine transform circuit 105. The inverse discrete cosine transform circuit 105 compresses the received signal and supplies it to the frame 107 via the addition node 106. Frame 107 stores the received signal. In the case of a prediction frame, the addition node 106 receives a signal from the inverse discrete cosine transform circuit 105, adds the restored data in the frame to this signal, and then supplies the processed signal to the frame 107. [0031] The motion detection circuit 108 is connected to the frame memory 107, compares the current frame with the previous frame, and calculates the motion vector of each macroblock. As described above, the motion detection circuit 108 can detect the motion of drawing the circle of the hand 20 by calculating the motion vector and therefore calculating the motion vector in the given region. The motion detection circuit 108 supplies data indicating the motion of the hand 20 to the CPU 16. Blocks 1 to 12 are common to blocks 1 to 12 shown in FIG. Blocks 13 to 15 can replace this encoder variant. By integrating the motion detection circuit and the coding device, the size of the circuit can be reduced and the cost can be reduced. [0032] Further, when the camcorder 13 is motor driven, the CPU 16 automatically controls pan, tilt and zoom to position the image of the hand 20 in the optimal position, usually in the center, within the field of view of the camcorder 13. Let me. [0033] Since the remote control device to which the present invention is applied does not require a color signal, an infrared camera can be used in a dark place. [0034] For example, when the CPU 16 is connected to a network interface such as the IEEE (Institute of Electrical and Electronics Engineers) 1394 interface (hereinafter, simply referred to as the 1394 interface), the CPU 16 is connected via the network. It is possible to supply data regarding the position of the hand 20 to the device of the device and control the device. [0035] In the second embodiment of the present invention described below, the motion detection circuit 15 includes a RAM interface circuit. In addition, when the same components as those in the first embodiment described above are designated by the same reference numerals and have the same functions, their description will be omitted. [0036] The CPU 16 compares the pixel value of each macro block in the current video frame in the RAM interface circuit included in the motion detection circuit 15 with the pixel value of the corresponding macro block in the previous video frame stored in the RAM 14, and the video camera 13 compares the pixel value of the corresponding macro block. Determine if the action was performed within the field of view of. Each macroblock is composed of 16 vertical pixels and 16 horizontal pixels. [0037] Next, how the CPU 16 recognizes and locks the image of the hand 20 will be described. Here, a video frame composed of 352 horizontal pixels and 288 vertical pixels is used as an example. Since one macroblock is composed of 16 vertical and 16 horizontal pixels, this video frame has 22 vertical and 18 horizontal macroblocks. FIG. 15 is a diagram showing two consecutive video frames n and video frame n-1. MBm (n) means the mth macroblock in frame n, and MBm (n) [i, j] means the pixel located in the horizontal i-th row and the vertical j-th column in MBm (n). To do. The difference between MBm (n-1) [i, j] and MBm (n) [i, j] in two consecutive video frames n-1 and video frame n is expressed by the following equation. The CPU 16 calculates the difference for each macro block. The function ABS (x) means the absolute value of x. j = 15 i = 15 Difference = 3 3 ABS (MBm (n) [i, j]-MBm (n -1) [i, j]) j = 0 i = 0 [0038] When the difference obtained by the above equation does not reach a predetermined threshold value, the CPU 16 determines that the operation has not been performed in the macro block MBm between the video frame n-1 and the video frame n. This macroblock is called an inactive macroblock. In the case of a stationary background such as a bookshelf or sofa, the difference is 0 or a value close to 0. On the other hand, when the difference exceeds a predetermined threshold value, the CPU 16 determines that the operation has been performed in the macroblock MBm. This macroblock is called an active macroblock and is indicated by a shaded area in the figure. The CPU 16 calculates a pixel value for each corresponding macroblock in two consecutive video frames n. The CPU 16 groups adjacent active macroblocks as one area, and stores this area in the storage device or RAM 14 inside the CPU 16. The above calculation is simpler than the motion vector estimation described in the above configuration example. This arithmetic processing can be realized by software. This arithmetic processing does not require hardware. [0039] The user 18 performs a predetermined operation and displays the menu button icon 22 on the display device. The user moves the hand 20 diagonally twice, for example, between the positions 20 (a) and 20 (b), as shown in FIG. 16 and 17 are diagrams showing the entire macroblock in one video frame. For the sake of brevity, the number of macroblocks in the figure is shown to be smaller than the actual number. The shaded area indicates the active macroblock. When the user 18 starts moving the hand 20, the corresponding macroblock in the region 100 is activated. As the hand 20 moves in the direction of position 20 (b), the active region moves to the upper right (region 101). When the user 18 returns the hand 20 from the position of 20 (b) to the position of 20 (a), the active region returns to the position of region 100. When the user 18 repeats the above operation, the active region repeats the movement between the region 100 and the region 101 in accordance with the movement of the user 18's hand 20. As a result, when the active region satisfies the above conditions, the CPU 16 determines this region as an image of the hand 20 and detects the position of the hand 20. Even when other hand 20 movements are performed in the field of view of the video camera, the CPU 16 can distinguish between the predetermined movement of the hand 20 and the other movements of the hand 20 by determining the movement of the active region. [0040] When the CPU 16 detects the position of the hand 20, it subsequently supplies a command to the OSD circuit 5. The OSD circuit 5 receives this command and generates the menu button icon 22 as shown in FIG. Next, CPU 16 tracks the movement of hand 20. 18 to 21 are diagrams showing a state when the user 18 translates the hand 20 from the position of 20 (a) to the position of 20 (c). When the user 18 performs the above-mentioned operation of the hand 20, the active region detected by the CPU 16 moves from the region 100 to the region 102 as shown in FIGS. 19 and 20. The CPU 16 determines that the hand 20 has moved to the right, controls the OSD circuit 5, and moves the cursor 24 from the 30 position to the 31 position on the CRT 9. As described above, the cursor 24 follows the movement of the hand 20. The user 18 stops the cursor 24 on the desired menu button icon 22 for a certain period of time, and selects the menu button icon 22. The CPU 16 recognizes the menu button icon 22 selected by the user 18 and executes the corresponding task. [0041] If the CPU 16 loses sight of the hand 20, it controls the OSD circuit 5 to display the message "Please move your hand to the right" on the CRT9. When the user 18 follows this message, the CPU 16 subsequently controls the OSD circuit 5 and displays the message "Please move your hand up" on the CRT9. Again, when the user follows this message, the CPU 16 again detects and locks the image of the hand 20 by detecting the image that first moves to the right and then moves up. [0042] The predetermined movement of the hand 20 is not limited to the linear movement as described above, and may be other identifiable movement. Also, when selecting the menu button icon 22, user 18 can not only stop the movement of the hand 20 but also perform other identifiable actions. As a modification of the linear motion, the user 18 may move the hand 20 several times in an oblique direction, for example, from the lower left to the upper right, as shown in FIG. 28. [0043] Further, an image recognition technique may be used to detect the hand 20. This method is a more precise recognition method than the above-mentioned recognition method, and the movement of the hand 20 is also small. The user 18 either performs the action of spreading the hand 20 as shown in FIG. 22, or from the state of holding the hand 20 as shown in FIG. 23 (a), as shown in FIG. 23 (b). Perform the action of spreading the hand 20. When such an operation of the hand 20 is performed, the CPU 16 recognizes the corresponding macro block as an active macro block and detects the position of the hand 20. When the CPU 16 detects the position of the hand 20, it cuts out the image of the hand 20 by digital processing in order to prevent erroneous detection, and determines whether or not the image of the hand 20 is the image of the hand 20. [0044] As shown in FIG. 24, the CPU 16 converts the cut-out image of the hand 20 into a binary image. The most characteristic feature of the open hand 20 is that a V-shape is formed between each finger. The CPU 16 inspects all endpoint points located on the same horizontal line in a binary image, such as points p1 and q2 located on the horizontal scan line L1. In the horizontal scan line L2 located below the horizontal scan line L1, the two endpoints are approaching and eventually coincide, as shown at points p2 and q2. As described above, the CPU 16 determines whether or not the outer outline of the image has a V shape by calculating the distance between each edge point located on the same horizontal scanning line. When the CPU 16 detects one or more V-shapes, it determines that this image is the image of the hand 20. After detecting the hand 20, the CPU 16 tracks the movement of the hand 20, which is the same as the method described above. Since the image recognition of the hand 20 as described above does not require much arithmetic processing, it can be processed by software. [0045] Further, the CPU 16 may detect the hand 20 by performing image collation of the hand 20 instead of recognizing the outline of the image of the hand 20. Hereinafter, the image collation method of the hand 20 will be described with reference to FIG. 25. First, the user 18 transmits a signal to the OSD circuit 5 using a conventional infrared remote control device, and displays an image taken by the video camera 13, that is, an image of the user himself / herself on the CRT 9. Subsequently, the image 25 of the hand 20 is selected using the conventional graphical user interface. RAM 14 stores image 25. If necessary, images of a plurality of hands 20 such as a child's hand 20 and a left-handed person's hand 20 may be stored. When the user fixes the hand 20, the CPU 16 compares the image 26 in the corresponding area with the image 25 of the hand 20 stored in the RAM 14, and finds the position where both match best, as shown in FIG. To do. FIG. 27 is a diagram showing the positions where the images 25 and 26 most match. Although the images 25 and 26 do not always match exactly, the CPU 16 calculates the difference between the image 25 and the image 26 in the same manner as the calculation of the difference between the macroblocks described above. When the difference between the image 25 and the image 26 does not reach a predetermined threshold value, the CPU 16 determines that the image 26 is the hand 20 and tracks the movement of the hand 20. In addition, in order to simplify the drawing, the outline of the hand 20 is shown in FIGS. 26 and 27, but in reality, as shown in FIG. 24, a binary image is used. [0046] As mentioned above, the CPU 16 uses various methods to lock the user 18's hand 20. In the present invention, the user 18 performs a specific operation such that the CPU 16 can easily discriminate the movement and other movements of the hand 20 or distinguish the image of the hand 20 from other objects in the field of view of the video camera 13. It must be made. [0047] The moving distance of the hand 20 depends on the viewing angle of the video camera 13 and the distance between the video camera 13 and the user 18. 28 to 30 are diagrams showing a state when the user 18 moves the hand 20 in an oblique direction. As shown in FIG. 29, the viewing angle of the video camera 13 is wide or the user 18 has a wider viewing angle than when the viewing angle of the video camera 13 is narrow or the distance between the user 18 and the video camera 13 is short. When away from the camcorder 13, the movement distance of the cursor 24 on the CRT 9 becomes shorter. Further, when the viewing angle of the video camera 13 is narrow, or when the distance between the user 18 and the video camera 13 is short, as shown in FIG. 30, the moving distance of the cursor 24 corresponding to the movement of the hand 20 becomes long. In such a case, assuming that the sensitivity of the cursor 24 is constant, the former, that is, when the viewing angle of the video camera 13 is wide, or when the user 18 is away from the video camera 13, the user 18 has a hand 20. Even if you move the cursor a lot, the movement distance of the cursor 24 becomes relatively short. Similarly, when the viewing angle of the video camera 13, that is, the viewing angle of the video camera 13 is narrow, or the distance between the user 18 and the video camera 13 is short, even if the user 18 moves the hand 20 small, the moving distance of the cursor 24 is still large. become longer. [0048] In order to solve the above-mentioned problems, the CPU 16 is provided with an automatic cursor sensitivity control function. When the predetermined movement of the hand 20 in the field of view of the video camera 13 is small, the CPU 16 increases the moving distance of the cursor 24, while when the predetermined movement of the hand 20 in the field of view of the video camera 13 is large, the cursor 24 Shorten the moving distance of. For example, as shown in FIG. 29, when the predetermined operation of the hand 20 has a length of 50 pixels, the CPU 16 moves the cursor 24 by 4 pixels while the hand 20 moves by 1 pixel. That is, the CPU 16 automatically calculates the movement distance of the cursor 24 according to the movement distance of the detected hand 20. As shown in FIG. 30, when the predetermined operation of the hand 20 has a length of 200 pixels, the CPU 16 moves the cursor 24 by one pixel while the hand 20 moves by one pixel. The user 18 does not need to consider the viewing angle of the camcorder 13 or the distance from the camcorder 13, and therefore does not need to move the hand 20 significantly when moving the cursor 24 from the left edge to the right edge of the CRT 9. The automatic cursor sensitivity control as described above is performed using a software program incorporated in the CPU 16. [0049] As shown in FIG. 28, when the user performs a predetermined operation of moving the hand 20 in the diagonal direction, the CPU 16 tracks the movement of the hand 20 and moves the cursor 24 in the diagonal direction on the display screen. The CPU 16 always calculates the ratio of the diagonal distance of the video frame to the stroke distance of the hand 20. The cursor 24 is controlled in proportion to this ratio. When the user 18 always moves the hand 20 by a certain distance, the CPU 16 recognizes this distance as the movement of the largest hand 20 that needs to be detected, calculates and displays the movement distance of the corresponding cursor 24. Move the cursor 24 on the diagonal of the screen. The reference for the movement distance of the hand 20 and the movement distance of the corresponding cursor calculated as described above is also used for the subsequent movement of the hand 20. For example, if the recognized stroke of the hand 20 is 10 inches, when the CPU 16 locks the hand 20, the user 18 moves the hand 20 diagonally by 10 inches to move the cursor 24 on the CRT9 to the lower left. It can be moved from the edge to the upper right edge. Similarly, if the recognized diagonal hand 20 stroke is 20 inches, the user 18 moves the cursor 24 on the CRT9 diagonally from the lower left corner to the upper right corner by moving the hand 20 20 inches diagonally. be able to. [0050] The CPU 16 may highlight the menu button icon 22 instead of the cursor 24, such as the digital satellite system graphical user interface (DSS GUI). For example, when user 18 raises hand 20 up, CPU 16 highlights the menu button icon 22 at the top. In the present specification, the "cursor" is a television image obtained as a result of tracking the movement of the user 18 detected by the CPU 16, such as highlighting the menu button icon 22 corresponding to the movement of the hand 20 of the user 18. It shall include any changes on the display device of the vessel. [0051] The device can also be applied to a three-dimensional hand 20 motion interface. As shown in FIGS. 31 and 32, the second video camera 31 is installed on the right or left side of the user. Alternatively, it may be mounted on the ceiling. The second video camera 31 senses the movement of the hand 20 in the direction perpendicular to the display screen, as shown on the Z axis. FIG. 31 is a block diagram showing the configuration of this device. Blocks 1 to 16 are common to the components 1 to 16 shown in FIG. 1, and are omitted in FIG. 31 for the sake of brevity. The video camera 31 has the same function as the video camera 13, the RAM 32 has the same function as the RAM 14, the RAM interface 33 has the same function as the RAM interface 15, and the RAM 32 and the RAM interface 33 have the same functions. Used to detect motion on the Z axis. The video camera 31, RAM 32, and RAM interface 33 operate in the same manner as the video camera 13, RAM 14, and RAM interface 15. [0052] First, when the user 18 performs a predetermined operation such as moving the hand 20 in an oblique direction, the CPU 16 inspects the active region and detects the position of the hand 20 on the XY plane as described above. .. Further, the CPU 16 receives an image signal from the video camera 31 and detects the position of the hand 20 on the YZ plane. When the user 18 moves the hand 20 between the positions 20 (a) and 20 (b) as shown in FIG. 32, the active region is the region from region 100 as shown in FIGS. 16 and 17. Move to 101. On the other hand, the region on the YZ plane moves between the region 200 and the region 201, as shown in FIG. 33. Since these active regions move simultaneously, the CPU 16 can easily detect the position of the hand 20 on the XY and YZ planes. Further, when the user 18 moves the hand 20 in the direction perpendicular to the CRT 9, the active region moves from the region 200 to the region 202 as shown in FIG. 34. In this way, the CPU 16 detects the movement of the hand 20 on the Z axis. [0053] The three-dimensional interface of the present invention can be applied in various ways. For example, a 3D interface can also be applied to a 2D OSD. The action on the Z axis is used to select the menu button icon 22. When selecting the menu button icon 22, the user 18 first stops the hand 20 on the desired menu button icon 22. User 18 then brings hand 20 closer to CRT9 and back in place, similar to the action of pressing a button on the machine. The CPU 16 detects this operation on the Z axis as the selection of the menu button icon 22, and executes the process corresponding to the detected menu button icon 22. [0054] The operation on the Z axis can also be used when zooming. When the user 18 brings the hand 20 closer to the CRT 9, the CPU 16 sends a command to the OSD circuit 5 and the video decoder 6. Upon receiving the command, the OSD circuit 5 and the video decoder 6 magnify the image of the hand 20 on the CRT9. When user 18 moves hand 20 away from CRT9, the image of hand 20 shrinks. By using such a method, the user 18 can perform zooming with an extremely natural feeling. Further, the user 18 can select the range of the image by using the cursor 24 or move the cursor 24 according to the movement of the hand 20 to select the center of the zooming before zooming. [0055] Since the user 18 can control the cursor 24 in 3D graphics, the interface to which the present invention is applied is also suitable for computer games. When the user 18 brings the hand 20 closer to the CRT9, the cursor 24 becomes smaller and appears to be far away in 3D graphics. Therefore, the user 18 can select an object located behind an object. For example, user 18 can experience the action of searching for a document from a file holder and selecting it. In this way, by using the remote control device to which the present invention is applied, the user 18 can obtain a feeling as if he / she is actually handling an object. [0056] Further, the remote control device to which the present invention is applied can be applied not only to a digital television receiver, but also to an analog television receiver, a personal computer television telephone, and any other device including a video camera and a monitor display device. Further, the display device is not limited to the CRT, and other display devices such as a liquid crystal display device and a projector can also be used. [0057] Further, when the camcorder 13 is motor driven, the CPU 16 automatically controls pan, tilt and zoom to position the image of the hand 20 in the optimal position, usually in the center, within the field of view of the camcorder 13. Let me. [0058] [0058] Since the remote control device to which the present invention is applied does not require a color signal, an infrared camera can be used in a dark place. [0059] When the CPU 16 is connected to a network interface such as a 1394 interface, the remote control device to which the present invention is applied transmits data related to the position of the hand to another device to be connected via the network, and the device is transmitted. Can be controlled. The CPU 16 may be configured independently of the television receiver. [0060] Although the present invention has been described using preferred embodiments of the present invention, the above-described embodiments can be variously modified and modified without departing from the ideas of the present invention shown in the claims. The structures, elements, actions and equivalents corresponding to the components indicated by all functions plus means or functions plus steps in the following claims, together with the components specifically specified in other claims, have their functions. It shall include any structure, element or operation that realizes. [Simple explanation of drawings] FIG. 1 is a block diagram showing a basic configuration of a remote control device to which the present invention is applied. FIG. 2 is a diagram for explaining how a user uses a hand motion to make the remote control device shown in FIG. 1 recognize an intention to perform remote control. FIG. 3 is a diagram for explaining how a user controls the remote control device shown in FIG. 1 and moves a cursor that follows the movement of the user's hand on the display screen. FIG. 4 is a diagram showing macroblocks and motion vectors in one video frame using arrows when the remote control device shown in FIG. 1 is calculating an operation. FIG. 5 is a diagram showing snapshots of motion vectors captured at 0.5 second intervals. FIG. 6 is a diagram showing snapshots of motion vectors captured at 0.5 second intervals. FIG. 7 is a diagram showing snapshots of motion vectors captured at 0.5 second intervals. FIG. 8 is a diagram showing snapshots of motion vectors captured at 0.5 second intervals. FIG. 9 is a diagram showing a snapshot of a macroblock when the remote control device shown in FIG. 1 determines an image of a hand and locks the image. FIG. 10 is for explaining how a user makes the remote control device shown in FIG. 1 recognize the intention of performing remote control by using a hand movement different from the above-mentioned hand movement. It is a figure. 11 is a diagram showing snapshots of motion vectors detected by a macroblock and a remote control device shown in FIG. 1 at 0.5 second intervals when the hand movement shown in FIG. 10 is performed. 12 is a diagram showing snapshots of motion vectors detected by a macroblock and a remote control device shown in FIG. 1 at 0.5 second intervals when the hand movement shown in FIG. 10 is performed. FIG. 13 is a block diagram showing a basic configuration of a modified example using an MPEG encoder in an electronic device. FIG. 14 illustrates how, in a second embodiment of the present invention, a user uses hand movements to cause the remote control device shown in FIG. 1 to recognize an intention to perform remote control. It is a figure for. FIG. 15 is a diagram showing two temporally contiguous frames and corresponding macroblocks in each frame to illustrate how motion is detected in a second embodiment of the invention. is there. FIG. 16 is a diagram showing a macro block in one video frame when the remote control device in the second embodiment of the present invention performs arithmetic processing on the active region indicated by the shaded area. FIG. 17 is a diagram showing a macro block in one video frame when the remote control device in the second embodiment of the present invention performs arithmetic processing on the active region indicated by the shaded area. FIG. 18 is a diagram showing the movement of a user's hand when operating a cursor in a second embodiment of the present invention. FIG. 19 is a diagram showing a macro block when the remote control device according to the second embodiment of the present invention detects the movement of the user's hand operating the cursor on the display screen. FIG. 20 is a diagram showing a macro block when the remote control device according to the second embodiment of the present invention detects the movement of the user's hand operating the cursor on the display screen. FIG. 21. How the user controls the remote control device according to the second embodiment of the present invention, moves the cursor on the display screen so as to follow the movement of the user's hand, and displays the cursor on the display screen. It is a figure for demonstrating whether to select an icon. FIG. 22 is for explaining how a user makes a remote control device according to a second embodiment of the present invention recognize a user's hand by using a hand movement different from the above-mentioned hand movement. It is a figure of. 23 (a) and 23 (b), respectively, show the remote in a second embodiment of the present invention, how the user uses a hand movement different from the hand movement described above, respectively. It is a figure for demonstrating whether the control device is made to recognize a user's hand. FIG. 24 is a diagram for explaining how a remote control device recognizes an image of a user's hand in a third embodiment of the present invention. FIG. 25 is a diagram for explaining how a remote control device recognizes an image of a user's hand in a third embodiment of the present invention. FIG. 26 is a diagram for explaining how a remote control device recognizes an image of a user's hand in a third embodiment of the present invention. FIG. 27 is a diagram for explaining how a remote control device recognizes an image of a user's hand in a third embodiment of the present invention. FIG. 28 is a diagram for explaining how a user sets an automatic cursor sensitivity adjustment function. FIG. 29 is a diagram showing how the remote control device shown in FIG. 1 detects a user's oblique hand movement when the user is located near the television receiver. FIG. 30 is a diagram showing how the remote control device shown in FIG. 1 detects a user's oblique hand movement when the user is located away from the television receiver. FIG. 31 is a block diagram showing a basic configuration of a remote control device that detects a three-dimensional movement in a fourth embodiment of the present invention. FIG. 32 is a diagram showing an embodiment using the remote control device shown in FIG. 31. FIG. 33 is a diagram showing a macroblock and an active region thereof in one video frame when the remote control device shown in FIG. 31 performs arithmetic processing on the active region, in shaded areas. FIG. 34 is a diagram showing a macro block in one video frame when the remote control device shown in FIG. 31 performs arithmetic processing on the active region shown in the shaded area.