Communication terminal, and program for identifying information transmission source

The present invention relates to a communication terminal, and more particularly to a communication terminal that performs data communication, for example.

Conventionally, a communication terminal that performs data communication, for example, has been known in particular, and an example of this type of device is disclosed in Patent Document 1. The information communication system of this background technology is composed of a PC server, a control unit, an information unit, and a mobile terminal, and the PC server issues a command to the control unit to output data to the information unit. A plurality of optical beacons are arranged in this information unit, and each optical beacon transmits ID data including predetermined data to a mobile terminal by emitting light instead of blinking pattern of ID data that identifies itself. To do. Then, the mobile terminal can receive the plurality of ID data by photographing the plurality of ID data changed to the blinking pattern of each optical beacon with the ID recognition camera.<patcit num="1"><text>JP-A-2004-349766 [H04L 1/00, H04B 10/10, H04B 10/105, H04B 10/22, H03M 5/12]</text></patcit>

<p> However, in the information communication system in Patent Document 1, when a mobile terminal captures a plurality of optical beacons, it is not possible to narrow down (select) one optical beacon that receives ID data. In addition, a method of determining the optical beacon to receive ID data can be considered by displaying a list of optical beacons on an image like a conventional wireless LAN and displaying a GUI on a mobile terminal capable of selecting an arbitrary optical beacon. However, the following problems newly occur.</p><p> First, in the interface of a mobile terminal, since the GUI is often operated by using a plurality of operation keys, the operation of the GUI is complicated. Furthermore, in order to associate the displayed list with the optical beacon, the amount of information included in the list and the information displayed will increase, making it difficult to understand the GUI operation.</p><p> Therefore, a main object of the present invention is to provide a novel communication terminal and an information source identification program applied to a processor of such a communication terminal.</p><p> Another object of the present invention is to provide a communication terminal capable of easily identifying a visible light communication source and an information source identification program applied to a processor of such a communication terminal.</p>

<p> The present invention employs the following configuration in order to solve the above problems. The reference numerals and supplementary explanations in parentheses indicate the correspondence with the embodiments described to assist the understanding of the present invention, and do not limit the present invention in any way.</p><p> The first invention is defined by a photographing means for capturing an image including a light source for transmitting information, a display device for displaying an image captured by the photographing means, a specific means for specifying the position of a light source included in the image, and a specific means. It is a communication terminal provided with an icon display means for displaying an icon at the position of a light source.</p><p> In the first invention, the photographing means (36,38) of the communication terminal (10) captures an image (through image, through still image) including an information transmission light source (visible light communication source) existing in the field of view. Then, the captured image is displayed on a display device (26) such as an LCD monitor. The specific means (20, S11) specifies the position (coordinates) of the light source in the image displayed on the display device, and the icon display means (20, S17) displays an icon on the specified light source based on the position. indicate. As a result, the display device displays the icon corresponding to the light source.</p><p> According to the first invention, the user can easily identify the visible light communication source by displaying the icon corresponding to the light source for performing visible light communication. The icon of the present invention also includes figures, words, symbols, numbers, or any combination thereof. In addition to the icon that is linked to the program, it may be a simple display that is not linked to the program.</p><p> The second invention is subordinate to the first invention and displays related information to display related information based on the receiving means for receiving the optical signal from the light source and the optical signal transmitted from the light source set by the setting means. Provide means.</p><p> In the second invention, the receiving means including the image sensor, the camera control circuit, and the CPU receives an optical signal including index information from the light source by photographing the light source. The related information display means (20, S15, S17) displays related information indicating the type of received information (data) in the type display area (60) in the icon based on the index information included in the optical signal. .. The type of information includes text data, music data, image data, and the like.</p><p> According to the second invention, the user can easily grasp the type of information transmitted from the visible light communication source.</p><p> The third invention is dependent on the first invention or the second invention, is provided in the display device, is a touch operation detecting means for detecting a touch operation in the touch reaction region, and a touch detected by the touch operation detecting means. When the operation is an operation of selecting an icon, a setting means for setting to acquire an optical signal from a light source corresponding to the icon is further provided.</p><p> In the third invention, the touch operation detecting means (34) provided in the display device detects touch operations such as touch and slide in the touch reaction region. Then, the setting means (20, S47) is set to acquire the optical signal from the light source corresponding to the selected icon when the touch operation for selecting the displayed icon is performed.</p><p> According to the third invention, it becomes possible to arbitrarily select a light source for which information is acquired by visible light communication.</p><p> The fourth invention is subordinate to the third invention and further includes data display means for displaying data according to an optical signal transmitted from a light source set by the setting means.</p><p> In the fourth invention, the data display means (20, S49) displays the contents of the text data by the text viewer function or the like when the set visible light communication source transmits the text data.</p><p> According to the fourth invention, the user can confirm the content of the information (data) transmitted from the visible light communication source.</p><p> The fifth invention is subordinate to either the first invention or the fourth invention, and the light source emits visible light.</p><p> In the fifth invention, visible light (54a, 54b, 54c) is emitted from a light source such as a fluorescent lamp and LED lighting.</p><p> According to the fifth invention, the user can recognize the visible light existing in the field of view and photograph the light source for information transmission.</p><p> The sixth invention uses a processor of a communication terminal as an image, which comprises a photographing means (36,38) for capturing an image including a light source for transmitting information and a display device (26) for displaying the image captured by the photographing means. It is an information source specifying program that functions as a specific means (S11) for specifying the position of a included light source and an icon display means (S17) for displaying an icon at the position of a light source specified by the specific means.</p><p> In the sixth invention as well, as in the first invention, by displaying the icon corresponding to the light source for performing visible light communication, the user can easily specify the position of the visible light communication source. become.</p>

<p> According to the present invention, the user can easily identify the visible light communication source by the icon displayed on the display device.</p><p> The above-mentioned object, other object, feature and advantage of the present invention will become more apparent from the detailed description of the following examples made with reference to the drawings.</p>

With reference to FIG. 1, the communication terminal 10 includes a CPU (sometimes referred to as a processor or computer) 20, a key input device 22, and a touch panel (touch operation detecting means) 34 controlled by a touch panel control circuit 32. The CPU 20 controls a wireless communication circuit 14 corresponding to the CDMA system to output a call signal. The output call signal is transmitted from the antenna 12 and transmitted to the mobile communication network including the base station. When the other party performs an answer operation, the call ready state is established.

When the call end operation is performed by the key input device 22 or the touch panel 34 after the transition to the call enable state, the CPU 20 controls the wireless communication circuit 14 to transmit the call end signal to the other party. After transmitting the call end signal, the CPU 20 ends the call processing. Even if the call end signal is received from the other party first, the CPU 20 ends the call processing. In addition, the CPU 20 ends the call processing even when the call end signal is received from the mobile communication network regardless of the other party.

When the call signal from the other party is captured by the antenna 12 while the communication terminal 10 is activated, the wireless communication circuit 14 notifies the CPU 20 of the incoming call. The CPU 20 controls the LCD monitor 26, which is a display device, by the LCD driver 24, and displays the source information described in the incoming call notification on the LCD monitor 26. Further, the CPU 20 outputs a ring tone from a speaker for incoming call notification (not shown).

In the talkable state, the following processing is executed. The modulated audio signal (high frequency signal) sent from the other party is received by the antenna 12. The received modulated audio signal is demodulated and decoded by the wireless communication circuit 14. The received audio signal obtained by this is output from the speaker 18. The transmitted audio signal captured by the microphone 16 is encoded and modulated by the wireless communication circuit 14. As a result, the generated modulated audio signal is transmitted to the other party using the antenna 12 as described above.

The touch panel 34 is a pointing device for the user to indicate an arbitrary position on the screen of the LCD monitor 26. When the touch panel 34 is operated by pushing, sliding (stroking), or touching the upper surface of the touch panel 34 with a finger, the touch panel 34 detects the operation. Then, when the touch panel 34 detects a touch, the touch panel control circuit 32 identifies the position of the operation and outputs the coordinate data of the operated operation position to the CPU 20. That is, the user can input the direction of operation, the figure, and the like into the communication terminal 10 by pushing, sliding (rubbing), and touching the upper surface of the touch panel 34 with a finger.

Further, the touch panel 34 is a method called a capacitance method for detecting a change in capacitance between electrodes caused by a finger approaching the surface of the touch panel 34, and one or a plurality of fingers touch the touch panel 34. Detect that. Specifically, the touch panel 34 has a projection-type capacitance method that detects changes in capacitance between electrodes caused by the close contact of a finger by forming an electrode pattern on a transparent film or the like. It has been adopted. As the detection method, a surface-type capacitance method may be adopted, or a resistance film method, an ultrasonic method, an infrared method, an electromagnetic induction method, or the like may be adopted.

Here, the operation in which the user touches the upper surface of the touch panel 34 with a finger is referred to as "touch". On the other hand, the operation of releasing the finger from the touch panel 34 is called "release". Further, the operation of rubbing the surface of the touch panel 34 is referred to as "slide". Then, the coordinates indicated by the touch are referred to as "touch points", and the coordinates of the end position of the operation indicated by the release are referred to as "release points". Further, the operation in which the user touches the upper surface of the touch panel 34 and continuously releases the touch panel 34 is referred to as "touch and release". The operations performed on the touch panel 34 such as touch, release , slide, and touch and release are generally referred to as "touch operations". The operation of the touch panel 34 is not limited to the finger, and may be performed by a stick having a thin tip such as a pen. Further, a dedicated touch pen or the like may be provided for the operation. When touching with a finger, the center of gravity in the area of the finger touching the touch panel 34 is the touch point.

In addition, the communication terminal 10 can execute a camera application. The camera application stores photographic image data in focus on the subject by autofocus processing. Specifically, when an operation for executing the camera application is performed by the key input device 22 or the touch panel 34, the CPU 20 gives a command necessary for executing the camera application to the camera control circuit 36. The camera control circuit 36 controls the image sensor 38 and the focus lens 40, and converts the optical image of the viewing field acquired by the image sensor 38 into photographic image data. Then, the CPU 20 converts the photographic image data obtained from the camera control circuit 36 and focusing on the subject into compressed image data and stores it in the flash memory 28. In addition, this camera application allows you to select the size of the photographic image to be captured from WQVGA (240 x 400), VGA (640 x 480) and UXGA (1600 x 1200). Of course, the size of the photographic image to be taken by the camera application of the present invention is not limited to these, and may be another size as appropriate.

In addition, when the camera application is executed, a process of displaying a real-time moving image of the field of view (hereinafter referred to as a through image) on the LCD monitor 26 is performed. Specifically, the CPU 20 activates the image sensor driver built in the camera control circuit 36, and instructs the image sensor driver to perform an exposure operation and a charge reading operation corresponding to a designated read area.

The image sensor driver executes exposure of the imaging surface of the image sensor 38 and reading of the electric charge generated by the exposure. As a result, the raw image signal is output from the image sensor 38. The raw image signal output from the image sensor 38 is input to the camera control circuit 36. The camera control circuit 36 performs processing such as color separation, white balance adjustment, and YUV conversion on the input raw image signal to generate YUV format image data. Then, image data in YUV format is input to the CPU 20. In the YUV format image data, Y means the brightness, U means the color difference obtained by subtracting the brightness from blue, and V means the color difference obtained by subtracting the brightness from red. That is, the YUV format image data is composed of the luminance signal (Y) data, the blue color difference signal (U) data, and the red color difference signal (V) data.

This YUV format image data is temporarily stored in RAM 30 by the CPU 20. When the YUV format image data is stored in the RAM 30, the stored YUV format image data is given from the RAM 30 to the LCD driver 24 via the CPU 20. At the same time, the CPU 20 issues a thinning-out read instruction corresponding to the display position of the camera setting stored in the RAM 30 to the LCD driver 24. Then, the LCD driver 24 outputs YUV format image data to the LCD monitor 26 according to the thinning-out reading command issued from the CPU 20. Therefore, a low-resolution through image representing the field of view is reproduced on the LCD monitor 26.

The update cycle of the through image is 1000 fps, and the image sensor 38 is a CMOS image sensor capable of outputting a raw image signal at high speed.

2 (A) and 2 (B) are schematic diagrams showing the appearance of the communication terminal 10. Further, FIG. 2 (A) is a front view of the communication terminal 10, and FIG. 2 (B) is a back view of the communication terminal 10. With reference to FIG. 2 (A), the communication terminal 10 has a case C formed in a plate shape. A microphone 16 and a speaker 18 (not shown in FIG. 2A) are built in the case C. The opening OP2 leading to the built-in microphone 16 is provided on one main surface in the length direction of the case C, and the opening OP1 leading to the built-in speaker 18 is provided on the other main surface in the length direction of the case C. That is, the user listens to the sound output from the speaker 18 through the opening OP1 and inputs the sound to the microphone 16 through the opening OP2.

The key input device 22 includes three types of keys, a call key 22a, a menu key 22b, and a call end key 22c, and each key is provided on the main surface of the case C. The LCD monitor 26 is attached so that the monitor screen is exposed on the main surface of the case C. Further, a touch panel 34 is provided on the upper surface of the LCD monitor 26.

The user performs a response operation by operating the call key 22a, and performs a call end operation by operating the call end key 22c. Further, the user operates the menu key 22b to display the menu screen on the LCD monitor 26. Then, by pressing and holding the end call key 22c, the power on / off operation of the communication terminal 10 can be performed.

Further, referring to FIG. 2B, the image sensor 38 and the focus lens 40 (not shown in FIG. 2B) are built in the case C, and an opening leading to the built-in image sensor 38 and the focus lens 40. The OP3 is provided on the other surface of the case C in the longitudinal direction. That is, the user can confirm the through image including the subject by the LCD monitor 26 by pointing the opening OP3 provided on the back side of the LCD monitor 26 toward the subject. The opening OP3 may be covered with a transparent plastic cover or the like.

Here, in the present embodiment, the visible light communication source, which is a light source for transmitting information, is photographed by the camera control circuit 36 and the image sensor 38, which are photographing means, to perform visible light communication. The camera control circuit 36 and the image sensor 38 controlled by the CPU 20 function as receiving means in visible light communication.

Specifically, for a light source that emits (radiates) light with a brightness equal to or higher than a predetermined value included in the through image, it is determined whether or not it is a visible light communication source, and if it is a visible light communication source, the icon is displayed on the LCD. Display on monitor 26. Then, by selecting the displayed icon by a touch operation such as touch and release, it is possible to determine the visible light communication source for receiving the information (data).

First, a procedure for detecting a visible light communication source included in a through image will be described. Here, in order to detect a visible light communication source in the through image, a raster scan is performed from the upper left end to detect visible light having a brightness equal to or higher than a predetermined value. Further, the through image to be raster-scanned is not a moving image but an arbitrarily selected still image. Hereinafter, each still image constituting the through image will be referred to as a through still image.

FIG. 3A is an illustrated diagram showing a display example of the LCD monitor 26 that displays a through image taken by the camera application. With reference to FIG. 3 (A), the LCD monitor 26 includes a status display area 50 and a function display area 52. In the status display area 50, the radio wave reception status by the antenna 12, the remaining battery capacity of the rechargeable battery, the current date and time, and the like are displayed. Then, in the function display area 52, a through image taken by the camera application is displayed.

The through image shown in FIG. 3A includes a plurality of light sources, and each of the plurality of light sources emits visible light including an optical signal. Hereinafter, the visible light emitted by each of the plurality of light sources will be referred to as visible light 54a, visible light 54b, and visible light 54c. Further, the visible light 54a and the visible light 54b shown in FIG. 3A have substantially the same brightness. However, the brightness of the visible light 54c is lower than the brightness of the visible light 54a and the visible light 54b. The plurality of light sources are fluorescent lamps, but other lighting fixtures may be used as long as they can be used for visible light communication such as LED lighting and organic EL lighting.

In FIG. 3A, visible light 54a, visible light 54b, and visible light 54c are shown as circles for simplicity. Further, since the state display area 50, the function display area 52, and the visible light 54a-54c shown in FIG. 3 (A) are the same in the other drawings, detailed description thereof is omitted in the other drawings for the sake of simplicity. To do.

Then, a raster scan is performed on the through still image, and a light source that emits light with a brightness equal to or higher than a predetermined value is searched for. Further, when a light source that emits light with a brightness equal to or higher than a predetermined position is found by raster scanning, it is determined by centralized sampling whether or not it is a visible light communication source. In raster scan, visible light having a brightness equal to or higher than a predetermined value is searched for by determining whether or not the brightness is equal to or higher than a predetermined value based on the brightness signal data for each pixel. For example, if the brightness of the visible light 54a is equal to or higher than a predetermined position, when the search is performed up to the upper part S of the visible light 54a (see FIG. 3 (B)), intensive sampling is performed in a certain range from the upper part S of the visible light 54a. Will be.

FIG. 3 (B) is an illustrated diagram showing how intensive sampling is performed in the region A of FIG. 3 (A). With reference to FIG. 3 (B), the centralized sampling area CSA of the quadrangle is set so that the upper S indicating the current search position by the raster scan becomes the midpoint of a certain side. That is, the centralized sampling area CSA is set with the upper S as a reference. Then, the through image is updated only in the centralized sampling area CSA, and it is determined whether or not the visible light 54a is blinking at the cycle required for visible light communication.

Note that the updated through image is not displayed on the LCD monitor 26 only in the centralized sampling area CSA. The shape of the centralized sampling area CSA is a quadrangle, but it may be another figure. Further, in visible light communication, it is not necessary that all visible light is included in the centralized sampling area CSA because communication is possible if the blinking of visible light can be identified.

Then, if it blinks at the cycle required for visible light communication, the index information of the transmitted information (data) is acquired, and the center of gravity of the region in which the brightness above the predetermined position is detected in the centralized sampling region CSA. Is calculated. This is to display the icon based on the calculated coordinates of the center of gravity.

For example, in FIG. 3B, if the visible light 54a blinks at the cycle required for visible light communication, the center of the visible light 54a is calculated as the center of gravity G and stored in the RAM 30 together with the acquired index information. .. Then, in the centralized sampling area CSA, when the centralized sampling is completed, the icon ICa (see FIG. 4) is displayed based on the calculated center of gravity G and the acquired index information.

In this way, the user can recognize the visible light existing in the field of view and photograph the light source. Then, the user can recognize the visible light communication source for performing visible light communication in the through image including a plurality of light sources.

Also, when the icon ICa is drawn, raster scan is restarted from the upper S. At this time, the area coordinates of the centralized sampling area CSA are temporarily stored in the RAM 30 so that the centralized sampling area CSA is not raster-scanned. Then, when the raster scan is restarted, the latest through still image is used. That is, any thru still image initially selected to start the raster scan is updated with the latest thru still image.

FIG. 4 shows a through image displayed on the LCD monitor 26 displayed after the raster scan is completed. With reference to FIG. 4, the icon ICa is superimposed on the visible light 54a, and the icon ICb is superimposed on the visible light 54b. That is, on the LCD monitor 26, the light source that emits visible light 54a and the icon ICa are displayed in association with each other, and the light source that emits visible light 54b and the icon ICb are displayed in association with each other. When it is not necessary to distinguish between the icon ICa and the icon ICb, the icon IC is referred to as the icon IC.

Further, since the visible light 54c did not have a brightness equal to or higher than a predetermined value, the icon ICs are not displayed in an overlapping manner. However, when the light source that emits visible light 54c is a visible light communication source, when the user approaches the light source that emits visible light 54c and the brightness of the visible light 54c contained in the through image is set to a predetermined value or more. , Icon ICs will be displayed in layers.

Then, when the raster scan is completed, the raster scan is performed again on the through still image selected arbitrarily. This is because the field of view taken by the user constantly changes, so that the position where the light source included in the through image is displayed also changes. That is, this is because the display position of the icon IC to be displayed is also changed according to the change of the display position of the light source included in the through image.

Here, when a touch-and-release touch operation is performed on the icon ICa shown in FIG. 4, visible light communication is performed with a visible light communication source that emits visible light 54a corresponding to the icon ICa, and the acquired information (data). ) To switch the display of the LCD monitor 26.

FIG. 5 is an illustrated diagram showing an embodiment in which data acquired by visible light communication is displayed on the LCD monitor 26. With reference to FIG. 5, when the text data of the weather forecast is transmitted from the visible light communication source that emits visible light 54a and is touch-and-released to the icon ICa, the function display area 52 is the weather forecast. Text (character string) is displayed. Specifically, intensive sampling is performed based on the coordinates of the center of gravity G used to display the icon ICa. Then, when the text data transmitted from the visible light communication source that emits visible light 54a is received, it is temporarily stored in the RAM 30 and the temporarily stored text data is displayed on the LCD monitor 26 by the text viewer function provided in the communication terminal 10. .. This allows the user to arbitrarily select a plurality of icon ICs. Then, the user can confirm the content of the information (data) transmitted from the visible light communication source.

When the operation of terminating the text viewer function shown in FIG. 5 is performed, the LCD monitor 26 returns to the display of the through image shown in FIG.

Further, in this embodiment, the type (related information) of the icon IC to be displayed changes according to the data received by the visible light communication. Hereinafter, the types of icon ICs will be described with reference to FIGS. 6 (A)-(B).

6 (A)-(C) are schematic diagrams showing an example of image data showing the types of icon ICs. With reference to FIG. 6 (A), the alphabet "T" is displayed in the type display area 60a in the icon IC. This is an acronym for "Text" data, and the icon IC shown in Fig. 6 (A) indicates to the user that the type of data obtained by visible light communication is text data. .. Subsequently, referring to FIG. 6B, a triangle is drawn in the type display area 60b of the icon IC. This means a triangle drawn on a play key such as a music player. That is, the icon IC shown in FIG. 6B indicates to the user that the type of data obtained by visible light communication is music data. Subsequently, with reference to FIG. 6C, a symbol is drawn in the type display area 60c in the icon IC. That is, the icon IC shown in FIG. 6 (C) indicates to the user that the type of data obtained by visible light communication is image data. When it is not necessary to distinguish each of the type display areas 60a-60c, the type display area 60 is referred to.

For example, referring to FIG. 4, the alphabet "T" is displayed in the type display area 60 included in the icon ICa and the icon ICb. This indicates that each visible light communication source that emits visible light 54a and visible light 54b corresponding to the icon ICa and the icon ICb emits text data. In this way, the user can easily grasp the type of information (data) transmitted by the visible light communication source by checking the icon IC.

The icon IC may include not only three types of images but also four or more types of various types of icon images. Further, the character strings, figures and symbols displayed (drawn) in the type display area 60 shown in FIGS. 6 (A) to 6 (C) may be other character strings, figures and symbols. If the type of data obtained by visible light communication cannot be specified, an icon IC that does not include the type display area 60 is displayed.

Here, the type of the icon IC to be displayed is determined based on the visible light communication source data stored in the RAM 30. FIG. 7 is an illustrated diagram showing the data structure of the visible light communication source data. With reference to FIG. 7, the visible light communication source data is composed of a NO column, a coordinate column, and an index information column. In the "NO" column, a numerical string indicating the order of discovery by the search is stored, and "1" indicates that it is the first visible light communication source discovered. The "Coordinates" column shows the position of the visible light communication source at the display coordinates in the LCD monitor 26. That is, the calculated coordinates of the center of gravity are stored in the "coordinates" column. The origin of the display coordinates of the LCD monitor 26 and the touch position coordinates of the touch panel 34 is the upper left corner. That is, the abscissa increases from the upper left end to the upper right end, and the ordinate coordinates increase from the upper left end to the lower left end.

The "index information" column further includes an "extension" column and the like. In the "extension" column, an extension indicating the type of data transmitted from the visible light communication source is stored. For example, the extension "txt" indicating text data and the extension "wav" indicating music data are stored in the "extension" column. Further, although not shown, the extension "jpg" indicating the image data is also stored in the "extension" column.

For example, if the visible light 54a shown in FIG. 4 corresponds to "1" in the "NO" column, The coordinates of (X, Y) are the coordinates of the center of gravity G (see Fig. 3 (B)), and the index information transmitted from the visible light communication source that emits visible light 54a has the extension data of "txt". It turns out that it is included. Then, the type of the displayed icon IC is determined based on the "extension" column. That is, as shown in FIG. 4, the icon IC shown in FIG. 6 (A) is displayed based on the coordinates (X, Y) of the center of gravity G. The index information may include a file name of information (data) transmitted by a visible light communication source, metadata, and the like. Further, the extension indicating the music data is not limited to "wav", and may include "mp3" and "mid". Further, the extension indicating the image data may include "gif" and "bmp".

Here, a plurality of layers constituting the display of the LCD monitor 26 will be described. Specifically, two layers (bottom layer and top layer) are provided in an overlapping manner, the top layer is provided on the start point side (user) in the virtual space, and the bottom layer is arranged in the direction away from the start point. .. In this embodiment, a through image is displayed on the bottom layer, and an icon IC is displayed on the top layer. As a result, as shown in FIG. 4, the icon ICa can be easily drawn so as to overlap the visible light 54a. The display of the LCD monitor 26 may have three or more layers.

FIG. 8 is an illustrated diagram showing a memory map of the RAM 30. With reference to FIG. 8, the memory map 300 of the RAM 30 includes a program storage area 302 and a data storage area 304. A part of the program and data is read from the flash memory 28 all at once or partially and sequentially as needed, stored in the RAM 30, and then processed by the CPU 20 or the like.

The program storage area 302 stores a program for operating the communication terminal 10. The program for operating the communication terminal 10 is composed of a visible light communication source search program 310, a light source selection program 312, and the like.

The visible light communication source search program 310 is a program that searches for a visible light communication source by performing a raster scan on a through still image. The light source selection program 312 is a program that acquires information (data) from a visible light communication source corresponding to the icon IC when the displayed icon IC is selected.

Although not shown, the program for operating the communication terminal 10 includes a camera application program, a text viewer function program, a music player function program, an image viewer function program, and the like.

The data storage area 304 is provided with a touch buffer 320, a through image buffer 322, a raster scan buffer 324, a centralized sampling buffer 326, and a visible light communication buffer 328. Further, the data storage area 304 stores the touch coordinate map data 330, the icon data 332, and the visible light communication source data 334, and is provided with the touch flag 336.

The touch buffer 320 is a buffer for temporarily storing an input result such as a touch detected by the touch panel 34, and temporarily stores coordinate data of a touch point and a release point, for example. The through image buffer 322 is a buffer in which YUV format image data read from the camera control circuit 36 is temporarily stored (temporarily stored). The raster scan buffer 324 is a buffer for temporarily storing the through still image and the coordinates indicating the current search position when performing a raster scan on the through still image.

The centralized sampling buffer 326 is a buffer for temporarily storing the coordinates that serve as a reference for setting the centralized sampling area CSA and the coordinates (area coordinates) that indicate the area of the centralized sampling area CSA. The visible light communication buffer 328 is a buffer for temporarily storing information (data) acquired by visible light communication.

The touch coordinate map data 330 is data for associating coordinates such as touch points with respect to the touch panel 34 specified by the touch panel control circuit 32 with the display position of the LCD monitor 26. That is, the CPU 20 can associate the result of the touch operation performed on the touch panel 34 with the display of the LCD monitor 26 based on the touch coordinate map data 330. The icon data 332 is image data of the icons shown in FIGS. 6 (A)-(C).

The visible light communication source data 334 is the visible light communication source data shown in FIG. 7, and is composed of coordinate and index information. The touch flag 336 is a flag for determining whether or not the touch panel 34 is touched (touched). For example, touch flag 336 consists of a 1-bit register. When the touch flag 336 is established (on), the data value "1" is set in the register, and when the touch flag 336 is not established (off), the data value "0" is set in the register.

Although not shown, the data storage area 304 stores an image file and the like, and is also provided with other counters and flags necessary for the operation of the communication terminal 10. In addition, each flag is set to "0" in the initial state.

The CPU 20 executes a plurality of tasks in parallel, including the visible light source search process shown in FIG. 9 and the light source selection process shown in FIG. 10, under the control of a real-time OS such as Symbian or Linux.

For example, when the user performs an operation to acquire a through image, in step S1, the through image is acquired. That is, the YUV format image data output from the camera control circuit 36 is temporarily stored in the through image buffer 322. Subsequently, in step S3, a through still image is raster-scanned. That is, a through still image arbitrarily selected from the through images temporarily stored in the through image buffer 322 is temporarily stored in the raster scan buffer 324, and raster scanning is started for the through still image.

Subsequently, in step S5, it is determined whether or not the brightness equal to or higher than the predetermined value is detected. That is, it is determined by raster scan whether or not the brightness is equal to or higher than a predetermined value for each pixel of the through still image. If it is "NO" in step S5, that is, if the brightness above the predetermined value cannot be detected, the process proceeds to step S21. On the other hand, if "YES" is detected in step S5, that is, if a brightness equal to or higher than a predetermined value is detected, intensive sampling is performed in step S7. That is, as shown in FIGS. 3A and 3B, the centralized sampling area CSA is set based on the coordinates of the pixels in which the brightness equal to or higher than the predetermined value is detected, and the through image is updated in the centralized sampling area CSA. To do.

Subsequently, in step S9, it is determined whether or not the source is a visible light communication source. That is, it is determined whether or not the light source included in the set centralized sampling area CSA is blinking at the frequency for performing visible light communication. If "NO" in step S9, that is, if it is not a visible light communication source, the process proceeds to step S21. On the other hand, if "YES" in step S9, that is, if it is visible light communication, the position of the visible light communication source is calculated in step S11. For example, as shown in FIG. 3 (B), the coordinates of the center of gravity G are calculated in the centralized sampling area CSA as the position of the visible light communication source. In step S13, it is determined whether or not there is a visible light communication source nearby. That is, it is determined whether or not the coordinates of the center of gravity calculated in step S11 are included in the distance within the threshold value in each coordinate stored in the column of "coordinates" in the visible light communication source data 344.

Specifically, the distance between the coordinates of the center of gravity calculated in step S11 and each coordinate stored in the "coordinates" column in the visible light communication source data 344 is calculated by the three-square theorem. Then, it is determined whether each of the calculated distances is equal to or less than the threshold value. The CPU 20 that executes the process of step S11 functions as a specific means.

If "NO" in step S13, that is, if there is no visible light communication source nearby, index information is acquired in step S15. That is, the index information transmitted from the visible light communication source is received, and the coordinates of the center of gravity calculated in step S11 and the received index information are stored as the visible light communication source data 344. Subsequently, in step S17, an icon is displayed based on the index information. That is, in the "extension" column in the visible light communication source data 334, the last stored extension is read, and the type of the icon is determined based on the read extension.

For example, referring to Fig. 7, if the column in which "1" is stored in the column of "NO" is the last stored row, the last stored extension will be "txt", and the CPU20 will have the CPU20. Determine the type of icon IC to be the text icon IC shown in Fig. 6 (A). Furthermore, since the coordinates (X, Y) of the center of gravity G are stored in the "coordinates" column corresponding to "txt", the icon ICa is displayed on the LCD monitor 26 as shown in FIG. ..

The CPU 20 that executes the process of step S17 functions as an icon display means, and the CPU 20 that executes the processes of steps S15 and S17 functions as a related information display means.

If "YES" in step S13, that is, if there is a visible light communication source nearby, the coordinates of the nearby visible light communication source are updated in step S19, and the process proceeds to step S17. That is, the nearby visible light communication source is determined to be the visible light communication source detected by the previous raster scan. Therefore, in step S19, the coordinates of the visible light communication source are updated without acquiring the index information again. For example, if the newly calculated coordinates are (X', Y') and the distance to the coordinates (X, Y) shown in FIG. 7 is within the threshold value, the coordinates of (X, Y) are (X', Y'). , Y') coordinates are updated.

In step S21, it is determined whether or not the raster scan is completed. That is, it is determined whether or not the coordinates of the pixel for determining whether or not the brightness is equal to or higher than a predetermined value are the coordinates of the lower right end of the display coordinates of the LCD monitor 26. If NO in step S21, that is, if the raster scan has not been completed, the coordinates indicating the current search position by the raster scan are read from the raster scan buffer 324, and the process returns to step S3. Then, in step S3, the raster scan is restarted from the read coordinates. On the other hand, if YES in step S21, that is, when the raster scan is completed, the process returns to step S1.

FIG. 10 is a flow chart showing a light source selection process, which is processed in parallel with the visible light communication source search process described above. The CPU 20 determines whether or not it was touched in step S41. That is, it is determined whether or not the touch flag 336 is turned on. If "NO" in step S41, that is, if the touch flag 336 is off, the process of step S41 is repeatedly executed. On the other hand, if YES in step S41, that is, if the touch flag 336 is on, the touch point is stored in step S43. That is, the touch point detected by turning on the touch flag 336 is temporarily stored in the touch buffer 320.

Subsequently, in step S45, it is determined whether or not the icon IC has been operated. That is, it is determined whether or not the touch point is included in the display coordinates of the icon IC. If it is "NO" in step S45, that is, if the icon IC is not operated, the process proceeds to step S51. On the other hand, if YES in step S45, that is, if the icon IC is operated, data is acquired from the light source corresponding to the icon IC selected in step S47. For example, when the icon ICa shown in FIG. 4 is operated, the centralized sampling area CSA is newly set, and the information transmitted from the visible light communication source corresponding to the icon ICa is received. Then, the acquired data is temporarily stored in the visible light communication buffer 328. The display area of the icon IC may be the centralized sampling area CSA. Further, the CPU 20 that executes the process of step S47 functions as a setting means.

Subsequently, in step S49, processing according to the acquired data is performed, and the light source selection process is completed. That is, the data temporarily stored in the visible light communication buffer 328 is read out and displayed on the LCD monitor 26. For example, if the acquired (received) data is text data, the content of the acquired text data is displayed on the LCD monitor 26 by the text viewer function. If the acquired data is music data, the music data is played by the music player function. Further, if the acquired data is image data, the image data is displayed on the LCD monitor 26 by the image viewer function. The CPU 20 that executes the process of step S49 functions as a data display means.

If "NO" is determined in step S45, the process corresponding to the touch is executed in step S51. For example, the through image is enlarged or reduced with the coordinates indicated by the touch points as the center.

In this embodiment, the icon ICs are displayed every time the visible light communication source is discovered, but the icon ICs may be displayed collectively after the raster scan is completed. Specifically, step S17 in the visible light communication source search process shown in FIG. 9 may be executed after it is determined as YES in step S21. As a result, after the raster scan is completed, each icon IC is displayed based on the index information.

As can be seen from the above description, the mobile terminal 10 includes a camera control circuit 36 and an image sensor 38, and captures a through image including a visible light communication source and a through still image. Further, the through still image including the visible light communication source is displayed by the LCD monitor 26. The CPU 20 performs raster scan on the through still image, identifies the position of the visible light communication source included in the through still image, and based on the specified position, displays the icon IC corresponding to the visible light communication source on the LCD monitor 26. Display on.

As a result, the user can recognize the icon IC corresponding to the visible light communication source for performing visible light communication and can easily identify the visible light communication source.

In this embodiment, the update cycle of the through image is 1000 fps, but it may be set to 60 fps in order to reduce the processing load of the CPU 20, and the update cycle may be set to 1000 fps only when centralized sampling is performed.

Further, by providing an image sensor 38 capable of photographing infrared light, a light source that emits infrared light may be used to perform visible light communication. Further, the image sensor 38 may be a CCD image sensor instead of the CMOS image sensor.

Further, instead of the index information, header information or the like including extension information may be received. Further, if the index information including the extension information is not transmitted from the visible light communication source, the type of the icon IC may be determined from the file name including the extension.

Further, the communication system of the communication terminal 10 is not limited to the CDMA system, and a W-CDMA system, a TDMA system, a PHS system, a GSM system, or the like may be adopted. Not limited to the communication terminal 10, it may be a mobile information terminal such as a PDA (Personal Degital Assistant) equipped with a camera application. Further, visible light communication may be performed by attaching a WEB camera, a mobile camera, or the like to the communication terminal 10 that does not have the camera application and installing the software of the camera application.

<figref num="1">FIG. 1 is a block diagram showing a communication terminal of the present invention.</figref><figref num="2">FIG. 2 is an illustrated diagram showing the appearance of the communication terminal shown in FIG.</figref><figref num="3">FIG. 3 is an illustrated diagram showing an example of raster scan processing by the CPU shown in FIG.</figref><figref num="4">FIG. 4 is an illustrated diagram showing an example of position display of a through image displayed on the LCD monitor shown in FIG.</figref><figref num="5">FIG. 5 is an illustrated diagram showing an example of data displayed on the LCD monitor shown in FIG.</figref><figref num="6">FIG. 6 is an illustrated diagram showing the types of icons displayed on the LCD monitor shown in FIG.</figref><figref num="7">FIG. 7 is an illustrated diagram showing an example of the configuration of visible light communication source data stored in the RAM shown in FIG.</figref><figref num="8">FIG. 8 is an illustrated diagram showing an example of the RAM memory map shown in FIG.</figref><figref num="9">FIG. 9 is a flow chart showing the visible light communication source search process of the CPU shown in FIG.</figref><figref num="10">FIG. 10 is a flow chart showing the light source selection process of the CPU shown in FIG.</figref>

Code description

10 ... Communication terminal 20 ... CPU 22 ... Key input device 26 ... LCD monitor 30 ... RAM 34 ... touch panel 36 ... Camera control circuit 38 ... Image sensor