Optical communication device, optical communication method, and skin-capturing system
Abstract
Problem to be solved.To provide an optical communication device, an optical communication method and a skin imaging system for performing optical communication between a solid-state light emitting element such as an LED on the transmitting side and an image pickup element on the receiving side.
Solution.A lens module 10 is set in a camera 2 of a smartphone 1 to take a skin image. The lens module 10 converts the ID code into a plurality of ordered blinking patterns. The blinking pattern is set according to the frame rate of the camera 2 when shooting a moving image. The CPU 16 that controls the light emission of the LED 13 of the lens module 10 controls the extinguishing and blinking of the LED 13 according to the blinking pattern. The camera 2 captures a moving image while the LED 13 is blinking, and obtains an image for each frame. In the image, a light-dark pattern in which high brightness (bright) and low brightness (low) repeat along the vertical direction of the image is obtained in response to the blinking of the LED 13. This light / dark pattern is converted into the above-mentioned ID code. [Selection diagram] Fig. 7

Term
Projected expiry 28 December 2032.
- Priority and filed
- Published
- Today
- Projected expiry
9 claims: 3 independent, 6 dependent
- 1設定されたフレームレートで動画を撮像する撮像素子、および当該撮像素子に撮像された動画の各フレームの画像データを解析する画像解析手段を有する受信装置と、 点滅可能な固体発光素子、および当該固体発光素子を前記フレームレートで1フレームまたは数フレームが撮像される所定時間内毎に、前記フレームレートでの1ライン分の時間以上の時間間隔の点灯および消灯からなる複数の点滅パターンで点灯と消灯とを行わせる発光制御手段を有する送信装置とを備え、 前記送信装置の前記発光制御手段は、各点滅パターンとデジタル信号の各値とを関連付けて記憶した点滅パターン記憶手段と、当該点滅パターン記憶手段に記憶された前記点滅パターンと前記値との関連に基づいて、デジタル信号の各値を前記固体発光素子の前記点滅パターンに順次変換する送信変換手段とを備え、 かつ、前記発光制御手段が前記送信変換手段により変換された点滅パターンに基づいて前記固体発光素子を点滅させ、 前記受信装置の前記画像解析手段は、前記画像データの解析結果として、前記固体発光素子の前記点滅パターンの点滅により、撮像された前記画像データの垂直方向に沿って現れる輝度の高低からなる明暗パターンを求め、 前記受信装置は、前記画像解析手段により求められる前記明暗パターンと前記点滅パターンとの相関関係に基づいて、各明暗パターンとデジタル信号の各値とを関連付けて記憶した明暗パターン記憶手段と、当該明暗パターン記憶手段に記憶された前記明暗パターンと前記値との関連に基づいて、前記画像解析手段により順次解析される明暗パターンをデジタル信号の各値に変換する受信変換手段とを備えることを特徴とする光通信装置。
- 2前記送信装置の前記発光制御手段は、前記撮像素子が出力する映像信号の同期信号に対して非同期で、前記固体発光素子の点灯および消灯を制御し、 前記送信装置の前記点滅パターン記憶手段に記憶される各点滅パターンは、前記所定時間内に含まれる点灯と消灯との繰り返し回数により規定され、 前記受信装置の前記明暗パターン記憶手段に記憶される各明暗パターンは、撮像された前記画像データの垂直方向に現れる輝度の高低により表される水平方向に沿った明るい帯および/または暗い帯の本数により規定されていることを特徴とする請求項1に記載の光通信装置。
- 3前記受信装置は、前記撮像素子から出力される映像信号の同期信号に同期して同期用信号を出力する同期出力手段を備え、 前記送信装置は、前記同期出力手段から送信される同期用信号が入力される同期入力手段を備え、 前記送信装置の前記発光制御手段は、前記同期入力手段に入力された同期用信号により、前記撮像素子で撮像される各フレームおよび各フレームの各ラインに同期して前記固体発光素子の点灯および消灯を制御し、 前記送信装置の前記点滅パターン記憶手段に記憶される各点滅パターンは、点灯と消灯に加えて点灯の時間幅および消灯の時間幅により規定され、 前記受信装置の前記明暗パターン記憶手段に記憶される各明暗パターンは、撮像された前記画像データの垂直方向に現れる輝度の高低により表される水平方向に沿った前記明るい帯および前記暗い帯に加えて前記明るい帯の幅と前記暗い帯の幅とにより規定されていることを特徴とする請求項1に記載の光通信装置。
- 4前記送信装置の前記固体発光素子を各色毎に点灯と消灯が制御可能な3色LEDとし、 前記送信装置の前記発光制御手段は、前記3色LEDの各色毎に独立した点灯と消灯からなる点滅パターンによりデジタル信号を送信し、 前記送信装置の前記点滅パターン記憶手段に記憶される各点滅パターンは、前記3色LEDの各色の点灯と消灯により規定され、 前記受信装置は、カラーの撮像素子を有し、 前記受信装置の前記画像解析手段は、撮像された前記画像データの前記3色LEDの各色に対応する垂直方向に現れる明暗のパターンを求め、 前記受信装置の前記明暗パターン記憶手段に記憶される各明暗パターンは、撮像された前記画像データの垂直方向に現れ、かつ、前記3色LEDの発光色に対応する各色毎の輝度により表される水平方向に沿った明るい帯および暗い帯に規定されていることを特徴とする請求項1から請求項3のいずれか1項に記載の光通信装置。
- 5設定されたフレームレートで動画を撮像する撮像素子、および当該撮像素子に撮像された画像データを解析する画像解析手段を有する受信装置と、 点滅可能な固体発光素子、および当該固体発光素子を前記フレームレートで1フレームまたは数フレームが撮像される所定時間内毎に、前記フレームレートでの1ライン分の時間以上の時間間隔の点灯および消灯からなる複数の点滅パターンで点灯と消灯とを行わせる発光制御手段を有する送信装置とを備える光通信装置で行われる光通信方法であって、 前記送信装置の前記発光制御手段は、各点滅パターンとデジタル信号の各値との関連に基づいて、デジタル信号を順次前記点滅パターンに変換するとともに、当該点滅パターンに基づいて、前記固体発光素子の点灯と消灯を制御し、 前記受信装置の前記撮像素子は、前記固定発光素子が前記点滅パターンで点灯または消灯している状況で撮像素子により動画を撮像し、 撮像された前記動画の画像データを解析して、前記画像データの垂直方向に沿って現れる輝度の高低からなる明暗バターンを求め、 前記受信装置は、求められた前記明暗パターンと前記点滅パターンとの相関関係に基づいた当該明暗パターンと前記デジタル信号の各値との関連に基づいて、順次求められる明暗パターンを前記デジタル信号の各値に変換することを特徴とする光通信方法。
- 6前記送信装置の前記発光制御手段は、前記撮像素子が出力する映像信号の同期信号に対して非同期で、前記固体発光素子の点灯および消灯を制御し、 前記送信装置は、各点滅パターンを、前記所定時間内に含まれる点灯と消灯との繰り返し回数により規定し、 前記受信装置は、各明暗パターンを、撮像された前記画像データの垂直方向に現れる輝度の高低により表される水平方向に沿った明るい帯および/または暗い帯の本数により規定することを特徴とする請求項5に記載の光通信方法。
- 7前記受信装置は、前記撮像素子から出力される映像信号の同期信号に同期して同期用信号を出力し、 前記送信装置は、前記受信装置から送信される同期用信号を受信し、 前記送信装置の前記発光制御手段は、受信した同期用信号により、前記撮像素子で撮像される各フレームおよび各フレームの各ラインに同期して前記固体発光素子の点灯および消灯を制御し、 前記送信装置は、各点滅パターンを、点灯と消灯に加えて点灯の時間幅および消灯の時間幅により規定し、 前記受信装置は、各明暗パターンを、撮像された前記画像データの垂直方向に現れる輝度の高低により表される水平方向に沿った前記明るい帯および前記暗い帯に加えて前記明るい帯の幅と前記暗い帯の幅とにより規定することを特徴とする請求項5に記載の光通信方法。
- 8前記送信装置の前記固体発光素子を各色毎に点灯と消灯が制御可能な3色LEDとし、 前記送信装置の前記発光制御手段は、前記3色LEDの各色毎に独立した点灯と消灯からなる点滅パターンによりデジタル信号を送信し、 前記送信装置は、各点滅パターンを、前記3色LEDの各色の点灯と消灯により規定し、 前記受信装置は、カラーの撮像素子を有し、 前記受信装置の前記画像解析手段は、撮像された前記画像データの前記3色LEDの各色に対応する垂直方向に現れる明暗パターンを求め、 前記受信装置は、各明暗パターンを撮像された前記画像データの垂直方向に現れ、かつ、前記3色LEDの発光色に対応する各色毎の輝度により表される水平方向に沿った明るい帯および暗い帯により規定していることを特徴とする請求項5から請求項7のいずれか1項に記載の光通信方法。
- 9前記請求項1から請求項4のいずれかに記載の光通信装置を備える肌撮像システムであって、 前記撮像素子を有するデジタルカメラを備え、かつ、前記撮像素子を制御するとともに、前記撮像素子に撮像された動画データの各フレームの画像データを解析する画像解析手段と、前記画像解析手段により求められる前記明暗パターンと前記点滅パターンとの相関関係に基づいて、各明暗パターンとデジタル信号の各値とを関連付けて記憶した明暗パターン記憶手段と、当該明暗パターン記憶手段に記憶された前記明暗パターンと前記値との関連に基づいて、前記画像解析手段により順次解析される明暗パターンをデジタル信号の各値に変換する受信変換手段として機能する制御手段を備える携帯型電子機器と、 前記固体発光素子としての撮影照明用LEDと、肌画像の撮影に用いられるコンバージョンレンズと、前記照明用LEDの発光を制御する前記発光制御手段とを備えるレンズモジュールとからなり、 前記発光制御手段が、各点滅パターンとデジタル信号の各値とを関連付けて記憶した点滅パターン記憶手段と、前記点滅パターン記憶手段に記憶された前記点滅パターンと前記値との関連に基づいて、デジタル信号の各値を前記固体発光素子の前記点滅パターンに順次変換する送信変換手段として機能することを特徴とする肌撮像システム。
Independent claims9
91 paragraphs, as filed
The present invention relates to an optical communication device using a solid-state light emitting element and an image pickup element, an optical communication method, and a skin imaging system.
Conventionally, conversion lenses that can be attached to portable camera-equipped mobile electronic devices such as camera-equipped mobile phones (including smartphones) and camera-equipped tablet-type electronic devices have been known (see, for example, Patent Document 1). By attaching the conversion lens to the lens (master lens) on the side of the electronic device, the focal length can be changed to the wide-angle side or the telephoto side even if the lens of the camera of the portable electronic device is a single focus lens. In some cases, a lens having a close-up function (macro imaging function) is used as a conversion lens to enable macro imaging.
Further, a skin camera for magnifying and imaging the skin of a face and analyzing the condition of the skin is known. For example, a mobile phone camera has a function as a skin camera, for example, a close-up shot of a subject enlarged. A conversion lens that adds a function to perform is also known. In this case, for example, by sending the captured skin image to a company that analyzes the skin image by e-mail or uploading it to the server of the website, the skin analysis result may be sent back by e-mail. It will be easily possible.
When a skin image captured from a smartphone equipped with a conversion lens as described above is sent to a predetermined server (for example, a server of a cosmetics-related website), a service is provided to send back the analysis result of the skin image. In some cases, for example, it is conceivable to provide a service to a customer who has purchased a conversion lens for free or at a low price.
In this case, for example, a conversion lens with the above-mentioned service and a fake conversion lens at almost the same level are sold at a low price, and the customer who purchased this fake conversion lens receives the above-mentioned service. Can be considered.
In this case, the sales of the conversion lens may decrease, or the operating cost may increase by analyzing the skin image captured by the fake conversion lens. Therefore, an authentication method that can distinguish between the skin image transmitted by the purchaser of the regular conversion lens and the skin image transmitted by the purchaser of the fake conversion lens is required.
Further, in a state where the above-mentioned service is established, for example, when a cosmetics company sells a conversion lens as a set with a specific cosmetic for skin care, in addition to the above-mentioned service, the cosmetics company sells as a set. It is conceivable that you want to obtain only the skin image of the conversion lens of.
In this case as well, it is necessary to distinguish between the skin image of the customer who purchased the conversion lens as a single item and the skin image of the customer who purchased the conversion lens as a set with the cosmetics. Distinguishing between skin images taken with a regular conversion lens and skin images taken with a non-genuine conversion lens, conversion lenses associated with a specific company (business operator) by the above-mentioned set sales, etc., and individual sales, etc. As a method of distinguishing conversion lenses that are not associated with a specific company, it is conceivable to give identification information such as a serial number and an ID number to each conversion lens. When distinguishing the conversion lens between single item sales and set sales, it is necessary to set the identification information so that it can be sieved into two.
When such identification information is used, it is necessary to transmit the identification information together with the skin image to a server that analyzes the skin image or the like. In this case, it is conceivable that the customer inputs the serial number described in the conversion lens or the instruction manual attached to the conversion lens from the smartphone and transmits the skin image.
<p><patcit num="1"><text>Japanese Unexamined Patent Publication No. 2012-8283</text></patcit></p>
<p> By the way, conventionally, it has not been performed to attach identification information to a conversion lens and transmit the identification information together with the captured image data. When the customer (user) inputs the identification information to the smartphone as described above, the user is dissatisfied with the time and effort required to input the identification information, or the wrong identification information is registered due to an input error, or due to an input error. There is a risk that the identification information cannot be registered, or that the identification information is analyzed because the identification information is leaked through the user or a manual or the like in which the identification information is described.</p><p> Therefore, the smartphone automatically recognizes the identification information individually given to the conversion lens without any special operation by the user, and automatically transfers the identification information to the server of the website that provides the above-mentioned service. It is required to send. That is, when the skin image data is transmitted from the smartphone, the identification information is required to be automatically transmitted or read from the conversion lens side to the smartphone side. In this case, it is preferable that the smartphone has a configuration in which the identification information given to the conversion lens can be obtained at the lowest possible cost.</p><p> The present invention has been made in view of the above circumstances, and provides an optical communication device, an optical communication method, and a skin imaging system suitable for obtaining identification information of a conversion lens in a portable electronic device such as a smartphone. With the goal.</p>
<p> In order to solve the above problems, the optical communication device of the present invention is an image pickup device that captures a moving image at a set frame rate, and an image analysis means that analyzes image data of each frame of the moving image captured by the image pickup device. With a receiver that has The blinkable solid-state light emitting element and the solid-state light emitting element are turned on and off at a time interval equal to or longer than one line at the frame rate every predetermined time during which one frame or several frames are imaged at the frame rate. It is equipped with a transmission device having a light emission control means for turning on and off in a plurality of blinking patterns consisting of the above. The light emission control means of the transmission device relates to a blinking pattern storage means that stores each blinking pattern and each value of a digital signal in association with each other, and the blinking pattern and the value stored in the blinking pattern storage means. Based on this, a transmission conversion means for sequentially converting each value of the digital signal into the blinking pattern of the solid-state light emitting element is provided. In addition, the light emission control means blinks the solid-state light emitting element based on the blinking pattern converted by the transmission conversion means. As a result of analyzing the image data, the image analysis means of the receiving device has a light-dark pattern consisting of high and low brightness appearing along the vertical direction of the imaged image data due to the blinking of the blinking pattern of the solid-state light emitting element. Seeking, The receiving device includes a light-dark pattern storage means that stores each light-dark pattern and each value of a digital signal in association with each other based on the correlation between the light-dark pattern and the blinking pattern obtained by the image analysis means, and the light-dark pattern. It is characterized by comprising a reception conversion means for converting a light / dark pattern sequentially analyzed by the image analysis means into each value of a digital signal based on the relationship between the light / dark pattern stored in the pattern storage means and the value. To do.</p><p> According to such a configuration, the transmitting device converts each value of the digital signal into a corresponding blinking pattern. While the solid-state light emitting element emits light in each blinking pattern in the transmitting device in this way, the image pickup element of the receiving device captures a moving image in a state where the light emission of the solid-state light emitting element is captured. At this time, if the light of the solid-state light emitting element is included in the image to be captured, it is not necessary to directly image the solid-state light emitting element.</p><p> When the image sensor takes an image in a state where the solid-state light emitting element is blinking in each blinking pattern as described above, the captured image data depends on the brightness corresponding to the blinking of the solid-state light emitting element along the vertical direction. The change in brightness is imaged. The light-dark pattern as the change of light-dark corresponds to the above-mentioned blinking pattern, and if each value of the digital signal is assigned to the blinking pattern, each value can be read out from the light-dark pattern corresponding to each blinking pattern. Each value here is, for example, more than 1-bit data of 0 and 1, for example, preferably a value of 2 bits or more, and for example, each blinking pattern is 3 bits (0 to 7) or 4 bits. It may correspond to the value (0 to 15) of.</p><p> As a result, a solid-state light emitting element is arranged on the transmitting side, and an image sensor capable of taking a moving image is arranged on the receiving side to enable optical communication. In this case, depending on whether it is bright or dark for each frame of the moving image, it is possible to send and receive a digital signal in each frame, for example, with a number of bits of 2 bits or more, instead of sending and receiving 1-bit data for each frame. Therefore, high-speed communication becomes possible. It should be noted that the image sensor is known to read one frame at a time and one to read one frame one line at a time when reading the accumulated charge from each pixel. In the present invention, one line is known. It is necessary to use an image sensor that reads the charge of each pixel one by one. In the present invention, it is preferable to use, for example, a C-MOS sensor that reads line by line as the image sensor. The present invention can also be applied to a configuration in which the charge is read by several lines such as two lines each.</p><p> In the above configuration of the present invention, the light emission control means of the transmission device controls the lighting and extinguishing of the solid light emitting element asynchronously with respect to the synchronization signal of the video signal output by the image pickup device. Each blinking pattern stored in the blinking pattern storage means of the transmitting device is defined by the number of repetitions of lighting and extinguishing included in the predetermined time. Each light-dark pattern stored in the light-dark pattern storage means of the receiving device is the number of bright bands and / or dark bands along the horizontal direction represented by the magnitude of the brightness appearing in the vertical direction of the captured image data. It is preferable that it is specified by.</p><p> According to such a configuration, it is possible to transmit and receive a digital signal without synchronizing with the synchronization signal of the image sensor, and it is not necessary to provide a configuration for transmitting a synchronization signal from the receiving side to the transmitting side, which is costly. Can be reduced, and the structure of the optical communication device can be simplified. In this case, the timing of the frame of the image sensor and each line of the frame cannot be matched with the timing of turning on and off the solid-state light emitting element. Therefore, for example, the solid-state light emitting element is turned on by the imaging time of each frame based on the frame rate. And control the extinguishing.</p><p> In this case, for example, two frames are set as one set, and for example, a blinking pattern in which turning off and on is repeated three times in one frame time, a blinking pattern repeated five times, a blinking pattern that is chestnutized seven times, etc. The solid-state light emitting element is controlled by such a blinking pattern.</p><p> On the other hand, in the moving image data, light and dark stripes corresponding to the repeated turning on and off of the blinking pattern of the solid-state light emitting element described above are formed in the image data at least every other frame of the moving image data. By counting the number of bright bands and the number of dark bands of the stripes, it is possible to read the light / dark pattern of the image data corresponding to the blinking pattern of the solid-state light emitting element, whereby the digital signal is transmitted from the transmitting side to the receiving side. It will be done.</p><p> That is, data can be transmitted and received asynchronously without causing the solid-state light emitting element to emit light in synchronization with the synchronization signal of the image sensor, and a structure for transmitting and receiving a synchronization signal from the receiving side to the transmitting side is required. It is possible to enable optical communication using a digital camera at low cost. In this method, basically, the frequency of repeating blinking (brightness and darkness) within a predetermined time changes. For example, when the number of repetitions of blinking within a predetermined time is small, the frequency becomes low, and when the number of repetitions is large, the frequency becomes low. It becomes a high frequency. When representing a digital signal with this, the simplest case can be 0 for low frequencies and 1 for high frequencies. This converts frequency changes into digital Shingo, which is frequency shift keying (FSK). Using keying), the number of repetitions of blinking within a predetermined time is converted into a digital signal by changing the frequency of the frequency. Therefore, in the present invention, the transmitting side converts the digital signal into the high and low frequencies of the repetition of turning on and off the LED per unit time (predetermined time), and this is imaged by the image sensor on the receiving side and imaged. In the image for each frame, the high and low frequencies of repeating light and dark appearing in response to the above-mentioned lighting and extinguishing of the LED are converted into a digital signal.</p><p> Further, in the above configuration of the present invention, The receiving device includes a synchronous output means that outputs a synchronization signal in synchronization with a synchronization signal of a video signal output from the image sensor. The transmitting device includes a synchronization input means into which a synchronization signal transmitted from the synchronization output means is input. The light emission control means of the transmission device turns on and off the solid light emitting element in synchronization with each frame imaged by the image pickup device and each line of each frame by the synchronization signal input to the synchronization input means. Control and Each blinking pattern stored in the blinking pattern of the transmitting device is defined by a lighting time width and an extinguishing time width in addition to lighting and extinguishing. Each light-dark pattern stored in the light-dark pattern storage means of the receiving device is in addition to the bright band and the dark band along the horizontal direction represented by the magnitude of the brightness appearing in the vertical direction of the captured image data. It is preferable that the width of the bright band and the width of the dark band are defined.</p><p> According to such a configuration, the transmitting device side can control the lighting and extinguishing of the solid-state light emitting element corresponding to each frame of the image pickup element of the receiving device and each line of each frame, as in the case of the above-mentioned asynchronous case. In addition, the value is not expressed by the number of bright and dark bands reflected in the captured image data, but can be expressed by, for example, the width of each band. For example, like a barcode, the width of a bright band is divided into a narrow band and a wide band, and the dark band is divided into a narrow band and a wide band. It is possible to represent a 1-bit value for each band. That is, communication is possible for the number of bits that are repeatedly turned on and off within the time of one frame, and it is possible to send and receive information with a large number of bits in one frame. As a result, the communication speed can be increased. The classification of the width of the band is not limited to such two types of narrow width and wide width, and the number of types of band width may be more than 2.</p><p> Further, in the above configuration of the present invention, the solid-state light emitting element of the transmitting device is a three-color LED whose lighting and extinguishing can be controlled for each color. The light emission control means of the transmission device transmits a digital signal by a blinking pattern consisting of lighting and extinguishing independently for each color of the three-color LED. Each blinking pattern stored in the blinking pattern storage means of the transmitting device is defined by turning on and off each color of the three-color LED. The receiving device has a color image sensor and has a color image sensor. The image analysis means of the receiving device obtains a pattern of light and darkness appearing in the vertical direction corresponding to each color of the three-color LED of the captured image data. Each light-dark pattern stored in the light-dark pattern storage means of the receiving device appears in the vertical direction of the captured image data and is represented by the brightness of each color corresponding to the emission color of the three-color LED. It is preferably defined as a light band and a dark band along the horizontal direction.</p><p> According to such a configuration, the solid-state light emitting element is a three-color LED that can independently control the lighting and extinguishing of each color, and the image sensor is made to correspond to the color, so that the information is basically tripled. Can be sent and received. Information may be sent individually in a blinking pattern for each color, or data of 3 bits may be sent by combining lighting and extinguishing of three colors.</p><p> Further, the optical communication method of the present invention includes an image pickup device that captures a moving image at a set frame rate, and a receiving device having an image analysis means that analyzes the image data captured by the image pickup device. A blinkable solid-state light emitting element, and the solid-state light emitting element is turned on and off at a time interval equal to or longer than one line at the frame rate every predetermined time during which one frame or several frames are imaged at the frame rate. It is an optical communication method performed by an optical communication device including a transmission device having a light emission control means for turning on and off in a plurality of blinking patterns. The light emission control means of the transmission device sequentially converts a digital signal into the blinking pattern based on the relationship between each blinking pattern and each value of the digital signal, and based on the blinking pattern, of the solid-state light emitting element. Controls on and off, The image sensor of the receiving device captures a moving image with the image sensor in a situation where the fixed light emitting element is turned on or off in the blinking pattern. The image data of the captured moving image is analyzed to obtain a light / dark pattern consisting of high and low brightness appearing along the vertical direction of the image data. The receiving device sequentially obtains a light-dark pattern for each of the digital signals based on the relationship between the light-dark pattern and each value of the digital signal based on the correlation between the obtained light-dark pattern and the blinking pattern. It is characterized by converting to a value.</p><p> In the above configuration of the present invention, the light emission control means of the transmission device controls the lighting and extinguishing of the solid light emitting element asynchronously with respect to the synchronization signal of the video signal output by the image pickup device. The transmitting device defines each blinking pattern by the number of repetitions of lighting and extinguishing included in the predetermined time. It is preferable that the receiving device defines each light / dark pattern by the number of bright bands and / or dark bands along the horizontal direction represented by the magnitude of the brightness appearing in the vertical direction of the captured image data.</p><p> Further, in the above configuration of the present invention, the receiving device outputs a synchronization signal in synchronization with the synchronization signal of the video signal output from the image sensor. The transmitting device receives the synchronization signal transmitted from the receiving device, and receives the synchronization signal. The light emission control means of the transmission device controls the lighting and extinguishing of the solid-state light emitting element in synchronization with each frame imaged by the image pickup element and each line of each frame by the received synchronization signal. In the transmitter, each blinking pattern is defined by a lighting time width and a lighting time width in addition to lighting and extinguishing. In addition to the bright band and the dark band along the horizontal direction represented by the height of the brightness appearing in the vertical direction of the captured image data, the receiving device sets each light and dark pattern with the width of the bright band and the width of the bright band. It is preferable to specify by the width of the dark band.</p><p> Further, in the above configuration of the present invention, the solid-state light emitting element of the transmitting device is a three-color LED whose lighting and extinguishing can be controlled for each color. The light emission control means of the transmission device transmits a digital signal by a blinking pattern consisting of lighting and extinguishing independently for each color of the three-color LED. The transmitter defines each blinking pattern by turning on and off each color of the three-color LED. The receiving device has a color image sensor and has a color image sensor. The image analysis means of the receiving device obtains a light / dark pattern that appears in the vertical direction corresponding to each color of the three-color LED of the captured image data. The receiving device appears in the vertical direction of the image data in which each light and dark pattern is captured, and has a bright band and dark along the horizontal direction represented by the brightness of each color corresponding to the emission color of the three-color LED. It is preferable to specify by a band.</p><p> In an optical communication method such as these, it is possible to obtain the same effects as those of the corresponding optical communication device described above.</p><p> Further, the skin imaging system of the present invention includes the above-mentioned optical communication device, includes a digital camera having the image pickup element, controls the image pickup element, and has a moving image captured by the image pickup element. Based on the correlation between the image analysis means that analyzes the image data of each frame of the data and the light / dark pattern and the blinking pattern obtained by the image analysis means, each light / dark pattern and each value of the digital signal are associated with each other. Based on the stored light-dark pattern storage means and the relationship between the light-dark pattern stored in the light-dark pattern storage means and the value, the light-dark pattern sequentially analyzed by the image analysis means is converted into each value of a digital signal. A portable electronic device equipped with a control means that functions as a reception conversion means, and The lens module includes an LED for photographing illumination as the solid-state light emitting element, a conversion lens used for photographing a skin image, and the emission control means for controlling the emission of the illumination LED. Digitally based on the blinking pattern storage means stored by the light emission control means in association with each blinking pattern and each value of the digital signal, and the relationship between the blinking pattern and the value stored in the blinking pattern storage means. It is characterized in that it functions as a transmission conversion means for sequentially converting each value of a signal into the blinking pattern of the solid-state light emitting element. According to such a configuration, it is possible to obtain the same effect as that of the above-mentioned optical communication device.</p><p> The skin image system of the present invention includes a portable electronic device including a camera and a conversion lens that is detachably attached to the position of the master lens of the camera of the portable electronic device and for close-up photography of the skin by the camera. It is a skin photography system that is equipped A tubular lens housing that supports the conversion lens and An opening formed at the tip of the lens housing to face the skin to be photographed, and A code provided on the peripheral edge of the opening, showing identification information that makes the conversion lens identifiable on a surface facing the camera, and having an image recognizable code. The code is photographed when the skin facing the opening is photographed by the camera, and the identification information can be read from the code photographed together with the skin.</p><p> According to such a configuration, for example, a bar code or a two-dimensional bar code that can recognize an image as a code including identification information for identifying the conversion lens is provided on the peripheral edge of the opening, and the code is simultaneously provided with the skin. By taking a picture, the skin and the code are taken as one image, and the code for identifying the conversion lens can be read from the conversion lens to the portable electronic device side. As a result, even if there is no data communication means between the conversion lens and the portable electronic device, the identification information is transmitted from the conversion lens side to the portable electronic device side. In addition, each skin image always includes an image having a code that can identify the conversion lens used, so that the conversion lens used can be identified at any time.</p>
<p> According to the present invention, by providing a solid-state light emitting element in the transmitting device and a digital camera in the receiving device, optical communication becomes possible, and a camera of a portable electronic device such as a smartphone is used as the digital camera on the receiving side. It becomes possible to communicate.</p>
<figref num="1">It is a front view which shows the skin imaging system as an optical communication apparatus which concerns on 1st Embodiment of this invention.</figref><figref num="2">It is a front view which shows the lens module as a transmission device of the skin imaging system.</figref><figref num="3">It is sectional drawing which shows the lens module.</figref><figref num="4">It is sectional drawing which shows the lens housing part of the lens module.</figref><figref num="5">It is sectional drawing which shows the lens housing part of the lens module.</figref><figref num="6">It is a top view which shows the LED substrate of the lens module.</figref><figref num="7">It is a block diagram which shows the said skin imaging system.</figref><figref num="8">It is a figure for demonstrating the optical communication method in the skin imaging system.</figref><figref num="9">It is a block diagram which shows the skin imaging system as an optical communication apparatus which concerns on 2nd Embodiment of this invention.</figref><figref num="10">It is a figure for demonstrating the experiment of the analysis method of the captured image in the optical communication apparatus which concerns on Example of this invention.</figref><figref num="11">It is a figure for demonstrating the experiment of the analysis method of the image taken with the optical communication apparatus.</figref><figref num="12">It is a figure for demonstrating the experiment of the analysis method of the captured image in the said optical communication apparatus.</figref><figref num="13">It is sectional drawing which shows the said lens housing part of 3rd Embodiment.</figref><figref num="14">It is a figure which shows the skin image and the bar code image image | photographed by the skin image system of 3rd Embodiment.</figref>
Hereinafter, the first embodiment of the present invention will be described with reference to the drawings. As shown in FIGS. 1 to 7, the skin imaging system of this embodiment is a smartphone 1 (FIGS. 1 and FIG. 7) as a portable electronic device provided with a digital camera (camera 2 shown in FIG. 2 including an imaging element). Two lenses 11a and 11b (shown in Fig. 3 to Fig. 5 and Fig. 7) that are attached to the camera 2 of the smartphone 1 to magnify and take a close-up shot of the skin 24 (shown in Fig. 7). It consists of a conversion lens 11 and a lens module 10 equipped with LEDs 12 and 13 (shown in FIGS. 3 to 7) for shooting illumination.
In FIGS. 1 and 2, the lens module 10 is a front portion of the housing 10a excluding the portion of the lens housing 20 described later that includes the barrel 20a (shown in FIGS. 3 to 5) that supports the conversion lens 11. Is shown without showing the circuit board 16b on which the internal battery 17, the power switch 19, and the electronic circuit unit 16a are mounted. Further, the smartphone 1 and the lens module 10 constitute an optical communication device, the smartphone 1 is a receiving device, and the lens module 10 is a transmitting device.
The lens module 10 includes a housing 10a, and the housing 10a is formed in a flat box shape except for the lens housing 20, and is on the opposite side of the front surface provided with the display (not shown) of the smartphone 1. The conversion lens 11 is arranged so as to overlap the lens portion of the camera 2 on the rear surface.
The housing 10a is fixed to the smartphone 1 by, for example, a clip type or a band type. In the clip type, for example, the lens module 10 is provided with a clip member (not shown) for sandwiching the smartphone 1 between the lens module 10 and the lens module 10. That is, the smartphone 1 is sandwiched between the lens module 10 and the clip member fixed to the lens module 10 by an elastic force.
In this case, the lens module 10 can be moved vertically and horizontally with respect to the back surface of the smartphone 1 within an allowable range, and can correspond to the arrangement of the cameras 2 of each model of the smartphone 1. In the band type, for example, a band (not shown) that expands and contracts such as a rubber band is attached to the lens module like a wristwatch band, and the smartphone 1 is inserted into the band to play the lens module 10 on the smartphone 1. It is abbreviated, and in this case as well, the conversion lens 11 can be moved vertically and horizontally with respect to the back surface of the smartphone 1.
The lens module 10 includes the conversion lenses 11 and LEDs 12 and 13 described above, an LED drive circuit 14 for driving the LED 12 (shown in FIG. 7), and an LED drive circuit 15 for driving the LED 13 (shown in FIG. 7). Further, the lens module 10 includes a lens housing 20 in which the conversion lens 11 is housed as shown in FIGS. 4 and 5. The lens housing 20 is provided with a barrel 20a that supports the conversion lens 11 on the base end side (the side that is attached to the camera 2), and the tip surface is a contact portion 20b that comes into contact with the skin 24 when shooting the skin 24. The contact portion 20b is in contact with the skin 24 to block outside light. Further, a rectangular (substantially square) opening 20c is formed in the contact portion 20b, and a portion of the skin 24 facing the opening 20c is photographed.
As shown in FIGS. 3 to 5, the barrel 20a supports the conversion lens 11 including the lens 11a and the lens 11b, and the base end portion of the lens housing 20 (the end portion on the side mounted on the camera 2). It is fixed inside. A second polarizing plate 22, which will be described later, is provided at the base end of the barrel 20a. Further, in the lens housing 20, an LED substrate 12a (shown in FIGS. 3 to 6) provided with LEDs 12 and 13 for irradiating illumination light for photography toward the opening 20c of the contact portion 20b is supported. ing.
The LED 12.13 is provided on the LED substrate 12a. In addition, as will be described later, there is one LED 13 for photographing the texture of the skin 24, and two LEDs 12 for photographing the stains on the skin 24. Further, the two LEDs 12 of the LED substrate 12a are covered with the first polarizing plate 21 described later. The LED drive circuits 14 and 15 are also provided on the LED board 12a. The LED drive circuits 14 and 15 may be provided on the circuit board 16b.
As shown in FIGS. 3 to 5, the LED substrate 12a is arranged so that the LEDs 12 and 13 directly illuminate the skin 24 facing the opening 20c of the lens housing 20, but is reflected by the half mirror. By doing so, the light of the LEDs 12 and 13 may be applied to the skin 24 facing the opening 20c.
Further, the lens module 10 includes an electronic circuit unit 16a including a CPU 16 as a light emission control means for controlling lighting and extinguishing of LEDs 12 and 13 via LED drive circuits 14 and 15. A one-chip microcomputer is mounted on the electronic circuit unit 16a, and the one-chip microcomputer is provided with a CPU 16 (shown in FIG. 7) and storage means such as ROM and RAM.
Further, the lens module 10 includes a power supply circuit 18 (shown in FIG. 7) having a battery 17 for supplying power to the LED drive circuits 14, 15 and the CPU 16 and the like on a circuit board 16b fixed to the housing 10a, and a power supply. It has a switch 19. Although a disk-shaped button battery is shown as the battery 17, the battery may be, for example, a AAA battery.
Further, the above-mentioned two LEDs 12 and 13 have slightly different emission colors, and the LED 12 is used for imaging a stain on the skin 24, and the emission color is substantially white. On the other hand, the LED 13 is used for imaging the texture of the skin 24, and has a strong yellow color with respect to the emission color of the LED 12 and is close to the skin color.
Further, the first polarizing plate 21 is provided on the LED 12 for spot photography, and the second polarizing plate 22 is provided between the conversion lens 11 and the camera 2. This is for photographing the spots slightly below the surface of the skin 24, and with these polarizing plates 21 and 22, the reflected light inside the skin 24 when the spots of the skin 24 are photographed by the LED 12 is compared with the skin 24. It greatly reduces the reflected light on the surface and makes it easier to photograph stains. The polarization directions of the first polarizing plate 21 and the second polarizing plate 22 are orthogonal to each other. The light of the LED 12 passes through the first polarizing plate 21 and is polarized, and this polarized light is maintained even if it is reflected by the skin 24, and it becomes difficult to pass through the second polarizing plate 22. On the other hand, the light reflected inside the skin 24 loses its polarized light and passes through the second polarizing plate 22. It is not necessary to provide the polarizing plates 21 and 22.
In such a lens module 10, under the control of the CPU 16 of the electronic circuit unit 16a, the LED 13 is emitted (lit) and the LED 12 is left off when the texture is photographed, and the LED 12 is emitted and the LED 13 is emitted when the stain is photographed. Leave it off.
As is well known, the smartphone 1 has a function as a mobile phone and can make a call using a wireless line. In addition, the smartphone 1 has a display (not shown), can be connected to the Internet via a wireless line, can send and receive e-mails via the Internet, and can browse websites with a browser as an application (app). It has become. Further, for example, a file can be uploaded to a server such as a website, and a file can be downloaded from a server such as a website.
Further, the smartphone 1 is provided with a control device (control means: illustrated in FIG. 7) 3 having a CPU, ROM, RAM, etc., and can execute an application as a program. Further, a flash memory 4 as a storage device is connected to the control device 3, and it is possible to store downloaded applications, music files, moving image files, still image files, and other applications (programs).
Further, the control device 3 can control the camera 2 to take a picture (imaging), and can save the captured image data in the flash memory 4. In addition, image data can be analyzed and processed by an application for image processing and image analysis. The camera 2 has an image sensor and can shoot still images and moving images. For example, the camera 2 shoots an image at a set frame rate such as 30 fps.
In, for example, a CMOS (Complementary Metal Oxide Semiconductor) image sensor as an image sensor of the camera 2, the sensors of each pixel are arranged side by side in a matrix.
In other words, the sensors of each pixel are arranged for each line which is arranged in the horizontal direction and in the vertical direction. When reading the charge accumulated in the sensor of each pixel corresponding to the amount of incident light, for example, the charge of each sensor on the line is read from the upper line in the vertical direction toward the lower line. Be done. The read charge will be output as a video signal.
In this case, the amount of electric charge read from the sensor of each pixel corresponding to the synchronization signal is sequentially output as a signal. This synchronization signal is an image by the Vsync signal (vertical synchronization signal) output in synchronization with the Pixel Clock signal (individual synchronization signal) corresponding to the output of electric charge from each sensor along the line in order. The output of the minute charge is started.
When this Vsync signal is output from the image sensor, the first Hsync signal (horizontal synchronization signal: horizontal synchronization signal) is output, and the first (top) horizontal row (horizontal first) in the vertical (vertical) direction is output. From the sensors in the upper row), each Pixel Clock signal is sequentially output in one direction (for example, from left to right), corresponding to each sensor, and a signal based on the charge indicating the amount of received light is output. ..
Also, based on the next Hsync signal, it moves to the sensor on the horizontal line one level below, and again, from the left side, the signal corresponding to each sensor is output along one direction for each Pixel Clock signal. Will be done. After that, each Hsync signal shifts to a line along the horizontal direction one level below, and the signal corresponding to the above-mentioned charge is output. Then, when the output of the signal corresponding to the last line is completed, the video signal for one image (one frame) is output, and again, based on the Vsync signal and the Hsync signal, the next The signal corresponding to the first line of the frame is output. By arranging the signals corresponding to each sensor so that the video signal shifts to the next line for each of the above-mentioned Hsync signals and shifts to the next frame for each Vsync signal, it is possible to display a moving image.
The above-mentioned lens module 10 shown in FIG. 7 functions as a transmission device of an optical communication device, and a CPU 16 as a light emission control means blinks an LED 13 which is a solid-state light emitting element in a blinking pattern for communication. There is. (Since the polarizing plate is not in the optical system, this is used.) In the following description, the CPU 16 controls the light emission of the LED 13 when performing optical communication.
Further, the smartphone 1 shown in FIG. 7 is the receiving side of the optical communication device, and the mounted camera 2 functions as the receiving device.
If the transmitter knows the timing of the Vsync signal and Hsync signal of the sensor chip, that is, if it can be synchronized with the synchronization signal, the LED starts blinking at the timing of the Vsync signal for one frame of time. (For example, 1/15 second in the case of 15 fps) After that, the blinking for one frame ends. This blinking is when the light-off time = the lighting time. Here, the lens module 10 on the transmitting side is set to 15 fps (set to 1/15 second for one frame), and the smartphone 1 on the receiving side is set to 15 fps by the application program. It is assumed that there is. In this case, the sensor obtains a one-frame image with a striped pattern corresponding to the blinking of the LED.
For example, consider the case of 8M with 3200 pixels in width, 2400 pixels in height, and a frame rate of 15 fps. When the initial state is off and the on and off are repeated 5 times in 1/15 second, the image of 1 frame has bright linear stripes from the top and dark linear stripes below it, and this combination is 5 An image that is repeated many times is obtained, and the width of each stripe is 240 pixels (2400/10). Therefore, if 10 types of images having different widths of the stripes are prepared, the receiving side can recognize the numerical values of 0 to 10 corresponding to the stripes.
Here, in the above example, the frame rate is set to 15 FPS, which is the same as that of the TV receiver, but it can be changed as appropriate. Since the image is actually cut out, recognition is performed using only the central portion of the sensor size. However, for the sake of simplicity, the entire sensor will be described below.
First, the lens module 10 as a transmitter will be described. As described above, the CPU 16 is a one-chip microcomputer and has a ROM. A blinking pattern is stored in this ROM as a blinking pattern storage means. In the case of a predetermined time for two frames at the above-mentioned frame rate, for example, 15 fps, a plurality of blinking patterns in which turning on and off are repeated in substantially the same time width within 2/15 seconds are stored.
If the lighting and extinguishing of LED 13 can be controlled by the blinking pattern consisting of lighting and extinguishing in synchronization with the Hsync signal and Vsync signal at the time of imaging of the camera 2 described above, for example, one type of blinking from 10 types of blinking patterns for each frame. By blinking in a pattern, it is possible to transmit one of the ten types of blinking patterns for each frame.
However, here, since there is no means to output the synchronization signal output from the camera 2 of the smartphone 1 as the receiving device to the lens module 10 as the transmitting device, the time width of the entire blinking pattern of the LED 13 is set when the camera 2 is imaged. It can only be adjusted to the frame rate of.
In this case, except when the blinking pattern of the time width of 2 frames is accidentally synchronized, in the imaging with the camera 2, the blinking start of LED 13 in one type of blinking pattern is the frame (1st frame). It will start from the middle, and in the next frame (2nd frame), the blinking of LED13 with one type of blinking pattern is imaged over 1 frame, and in the middle of the next frame (3rd frame). The blinking of LED 13 in one type of blinking pattern will end.
Therefore, in the second frame, the blinking of the LED 13 in the same blinking pattern is imaged. When the LED 13 blinks in such a blinking pattern over 2 frames continuously for 2 frames each, the 3rd frame described above becomes the 1st frame of the next blinking pattern and corresponds to the 2nd frame. The frames to be used are every other frame in a row.
In every other frame, which is the second frame, the blinking pattern does not change in the middle of the frame, so that the number of bright bands (dark bands) corresponding to the blinking pattern can be read. In every other frame corresponding to the first frame and the third frame, the blinking pattern changes in the middle unless a plurality of the same blinking patterns are continuous.
Even if the switching timing of each blinking pattern of LED 13 and the switching timing of each frame of camera 2 happen to be synchronized, the blinking pattern is switched every two frames, so the blinking pattern is changed every other frame. It is possible to read. Also, when the LED 13 is lit (off) in each line of each frame, it is synchronized with the synchronization signal of the camera 2, especially the Hsync signal, so the lighting starts from the middle of the line corresponding to the lighting start and corresponds to the lighting end. It will be turned off in the middle of the line, but if each lighting time and each turning off time of LED 13 blinking is an integral multiple of the time for one line in the frame, a bright band on the line orthogonal to the frame line Alternatively, the width indicated by the number of lines in the dark band is the same even if the horizontal position of the line orthogonal to the line is different.
If the lighting time and extinguishing time of the blinking pattern are longer than one line, preferably two lines or more, the time does not necessarily have to be an integral multiple of one line, but the above-mentioned When the LED 13 can be blinked in synchronization with the signal of, the lighting time and the extinguishing time are preferably an integral multiple of the time for one line.
As this blinking pattern, for example, a blinking pattern in which the light is turned off 7 times and a lighting time is 6 times within a predetermined time for 2 frames and a blinking pattern in which the light is turned off 11 times are turned on. A blinking pattern of 10 times, a blinking pattern of 15 times off, a blinking pattern of 14 times on, a blinking pattern of 19 times off, and 18 times on. Expressed only by the blinking pattern and the number of times of lighting (the number of lightings), it is assumed that the blinking pattern is 22 times, 26 times, 30 times, 34 times, 38 times, and 42 times.
Then, for each blinking pattern, for example, 0 for 6 lightings, 1 for 10 lightings, 2 for 14 lightings, 3 for 18 lightings, 4 for 22 lightings, 5 for 26 lightings, and 30 lightings. Is associated with 6, the number of lights 34 is 7, the number of lights 38 is associated with the start bit, and the number of lights 42 is associated with the stop bit.
The combination of these blinking patterns and numerical values is stored in the ROM of CPU 16 as a data table. The numerical value corresponds to 4-bit information, and the combination of two blinking patterns is 8-bit, and the combination of four blinking patterns is 16-bit data. This makes it possible to transmit a digital signal.
The blinking pattern described above is an example, and may be a combination of other blinking patterns. Here, in order to prevent errors, each blinking pattern is set so that there is a difference of 4 or more in each lighting count, but the difference may be small or large.
Further, an ID code (serial number) is stored in the ROM of the CPU 16, and this ID code is read out and converted into a blinking pattern based on the above-mentioned data table. For example, the ID code is stored as an 8-bit ASCII code, and each ASCII code is converted into two blinking patterns representing data of 4 bits each, for example, by the CPU 16 as a transmission conversion means. Become. Based on the row of the converted blinking patterns, the CPU 16 controls the lighting and extinguishing of the LED 13, and causes the light emission along the row of the blinking patterns.
The ID code may be converted into a blinking pattern first and stored in the ROM in the state of being converted into the blinking pattern described above. When the power switch 19 is turned on, the CPU 16 starts operation and starts controlling the lighting and extinguishing of the LED in the blinking pattern corresponding to the above-mentioned ID code. Since it may not be accurately recognized by the receiving side, it is preferable that the ID transmission by the lighting control is set to be performed a plurality of times.
When the power switch 19 of the lens module 10 is turned on, the CPU 16 controls the light emission of the LED 13 based on the blinking pattern converted from the above-mentioned ID code based on the program. At this time, the ID code is converted into a sequence of a plurality of blinking patterns in which the order is determined.
The CPU 16 controls the lighting and extinguishing of the LED 13 by repeating the lighting and extinguishing indicated by each blinking pattern based on the sequence of blinking patterns. The camera 2 of the smartphone 1 captures the blinking pattern consisting of the lighting and extinguishing of the LED 13.
Next, the smartphone 1 which is a receiving device will be described. As shown in FIG. 7, the smartphone 1 includes a control device 3 capable of controlling the camera 2 and the camera 2 and analyzing an image captured by the image sensor of the camera 2. In the image sensor, based on the above-mentioned synchronization signal, an image of one frame of pixels is started for each Vsync signal, and one line of pixels is photographed for each Hsync signal. The CMOS sensor reads the accumulated charge corresponding to the amount of light incident on the sensor of each pixel for each line. Here, for example, when the amount of light (brightness) input to the image sensor changes significantly when capturing an image for one frame, the electric charge accumulated in the sensor of each line will be different. As described above, when the charge of the sensor is read for each Hsync signal, for example, one line at a time from the upper line to the lower line, if the bright state and the dark state are repeated, the video signal in the bright state is obtained. There is a line to be output and a line to which a video signal in a dark state is output. As a result, when looking at the image data for one frame captured, it is possible to capture a striped image in which the bright band portion along the horizontal direction and the dark band portion are repeated along the vertical direction. It will be possible.
Therefore, when the above-mentioned LED 13 is blinking in the above-mentioned blinking pattern, for example, when a skin or a predetermined screen is photographed, a light / dark pattern appears along the vertical direction of the image corresponding to the blinking pattern. .. That is, an image in which bright bands and dark bands are alternately arranged along the horizontal direction is captured.
Here, since the LED 13 is not synchronized with the synchronization signal of the image sensor, it cannot emit light in a blinking pattern for each frame, and blinks in the same blinking pattern for a predetermined time for two frames. Therefore, as will be described later, image data of a moving image having a perfect light / dark pattern corresponding to a predetermined blinking pattern can be obtained every other frame.
The control device 3 can analyze and extract this image data to obtain a light-dark pattern in which high-luminance portions and low-luminance portions for one frame are alternately arranged. From this light-dark pattern, the number of repetitions of light and dark (the number of bright bands and dark bands) is calculated. The number of repetitions of this light-dark pattern has a one-to-one correspondence with the blinking pattern of the LED 13 when an image is taken.
The flash memory 4 as the light / dark pattern storage means of the smartphone 1 stores a light / dark pattern data table corresponding to the above-mentioned blinking pattern data table as data for the optical communication image program of the skin imaging application. ..
In this light-dark pattern data table, the light-dark pattern is read from the image of one frame while the blinking pattern is for two frames, so the corresponding light-dark pattern corresponds to the number of repetitions of turning off and turning on the blinking pattern. Has halved the number of repetitions of bright (high brightness) and dark (low brightness).
Therefore, in the data table stored in the flash memory 4, for each light-dark pattern as the number of repetitions of light and dark (for example, as the number of bright bands), for example, the number of bright (high-brightness) bands is 0. 1 in 5 bright bands, 2 in 7 bright bands, 3 in 9 bright bands, 4 in 11 bright bands, 5 in 13 bright bands, 6 in 15 bright bands, bright bands The number of lines 17 is 7, the number of bright bands 19 is associated with the start bit, and the number of bright bands 21 is associated with the stop bit.
As a result, 4 bits are assigned to each of the captured image data and can be converted into a digital signal, and the control device 3 as the image analysis means and the reception conversion means reads out the digital signal from the image data. This digital signal is an ID code, and the ID code of the lens module 10 is read out.
The smartphone 1 can send the ID code together with the captured skin image to the server that analyzes the skin image via the Internet. On the server side, this ID code can be used to identify whether the lens module 10 is a genuine product, whether the lens module 10 is for a campaign (collaboration) with a specific cosmetics company (manufacturer or sales company), or whether it is a normal product. It will be possible.
An optical communication method in a skin imaging system as such an optical communication device will be described with reference to FIG. When the skin imaging application is started on the smartphone 1, the imaging of the moving image consisting of the image 34 for each frame is started. At this time, it is necessary to turn on the power switch 19 of the lens module 10.
So, for example, when you start the skin image shooting application on the smartphone 1 side, a message to the effect that "Please turn on the power switch 19 of the lens module 10" is voiced from the smartphone 1 before starting the video recording. It will be done on display.
In this case, it is preferable that the lens housing 20 is not affected by external light. For example, the tip of the lens housing 20 is pressed against the skin 24 or, for example, a dedicated screen, or the tip is covered with a dedicated lid. It is preferable that it is. Therefore, it is preferable that the smartphone outputs, for example, a message instructing the tip of the lens housing 20 to come into contact with the imaged portion of the skin after the message of turning on the power switch 19 described above.
In the camera 2, the image sensor starts imaging after the instruction to turn on the power of the lens module 10 described above, and in this state, the LED 13 of the lens module 10 is set every two frames based on the blinking pattern converted as described above. It will be turned off and on repeatedly.
When the power switch 19 of the lens module 10 is turned on in this way, the skin 24 is imaged by the image sensor of the camera 2 under the condition that the LED 13 blinks according to the blinking pattern.
Here, as described above, when the LED 13 blinks in a blinking pattern for two frames of time in an asynchronous state with respect to the synchronization signal of the camera 2, the light-dark pattern which is the blinking pattern captured by the camera 2 is The previous blinking pattern ends in the middle of the captured frame of the camera 2, and the next blinking pattern starts. Therefore, in this frame, the two blinking patterns are mixed.
In the next frame, an image corresponding to one blinking pattern is taken. Further, in the next frame pattern, one blinking pattern ends in the middle and the next blinking pattern is started, so that the two blinking patterns are mixed. Since such a situation is repeated until the imaging of all the blinking patterns is completed, frames having a light / dark pattern corresponding to one blinking pattern are generated every other frame.
The control device 3 of the smartphone 1 as a receiving device first recognizes the blinking pattern associated with the start bit. Therefore, for example, the image data of every other frame from the frame in which the start bit is recognized may be analyzed, or all the frames are sequentially analyzed to obtain a frame corresponding to one blinking pattern. You may find them one by one.
After the control device 3 recognizes the image 31 which is the number of repetitions of light and dark as the start bit, the image 34 of every other frame includes a light and dark pattern which is converted into an electronic signal (data). A plurality of images 32a, 32b, 32c ... Further, in the lens module 10 as a transmission device, a blinking pattern serving as the above-mentioned start bit is transmitted at the start of data transmission due to blinking of the LED 13, and a stop bit is transmitted at the end of data transmission.
Therefore, when the control device 3 analyzes the image 34 for each frame of the moving image and recognizes the image 33 having the light / dark pattern serving as the stop bit, the control device 3 ends the shooting of the moving image of the camera 2.
Next, as a skin imaging system, for example, with the LED 13 lit, the camera 2 of the smartphone 1 captures a still image 35 for analyzing the texture of the skin. Next, the CPU 16 of the lens module 10 turns on the LED 12 and captures a still image 36 for analyzing skin spots. Then, upload the captured still image data to the server of the company that analyzes the skin.
In such a skin imaging system as an optical communication device, it is possible to transmit data from the lens module 10 as a transmitting device to the imaging element of the camera 2 which is a digital camera. Therefore, for a portable electronic device having a camera 2 such as a smartphone 1, wireless data communication with a device provided with a solid-state light emitting element such as an LED and a microcomputer for controlling the lighting and extinguishing of the solid-state light emitting element. Becomes possible.
Therefore, the ID code of the conversion lens 11 (lens module 10) as described above can be transmitted to the smartphone 1 without a general communication device.
In addition, since data can be received by capturing a moving image with a built-in camera such as a smartphone 1 or tablet, the camera captures a location that is less affected by external light and where a transmitter is installed. Then, it is possible to build a system in which data is input.
Next, a second embodiment of the present invention will be described. As shown in FIG. 9, in the skin imaging system of the second embodiment, the smartphone 1 is provided with a flash 5 for the camera 2 composed of LEDs. This flash 5 is provided in a general smartphone 1 and can be turned on during normal shooting with the camera 2, but the lighting time can be controlled by the control device 3. There is.
Further, the lens module 10 is provided with a PD (photodiode) 5 facing the LED which is the flash 5 of the smartphone 1, detects the light emission of the flash 5 as a synchronous output means, and outputs the detection signal to the CPU 16. It has become like.
As the synchronization output means, the flash 5 causes the control device 3 to lightly output a vertical synchronization signal as a synchronization signal of the video signal output from the camera 2 and a synchronization signal indicating the output timing of the horizontal synchronization signal. Be controlled. That is, when the camera 2 starts shooting a moving image, it emits light in accordance with the timing of the output synchronization signal. This light emission is the timing of turning on and off the LED 13 controlled by the CPU 16 as the light emission control means in the lens module 10.
The difference between the vertical synchronization signal and the horizontal synchronization signal is, for example, a pattern having a short emission time and a pattern having a long emission time, a pattern having a low emission brightness and a pattern having a high emission brightness, or a pattern in which the flash 5 is made to emit a pulse. Distinguish by differences in patterns.
In the CPU 16, for example, when the above-mentioned skin imaging application is activated and the flash 5 is turned on by the skin imaging application, this is detected by the PD25 and a detection signal is input to the CPU. When the light emission of the detected flash 5 is a vertical synchronization signal, it starts lighting and extinguishing in a blinking pattern for one frame, and switches on and off at the timing of the horizontal synchronization signal.
In this case, for example, when the lighting and extinguishing of 100 lines are alternately performed, the lighting of the LED 13 is started by the input of the light emission of the flash 5 as the synchronization signal corresponding to the horizontal synchronization signal. Then, when the synchronization signal corresponding to the 100th horizontal synchronization signal is input, the LED 13 is turned off. Further, when the 100th horizontal synchronization signal is input, the lighting can be repeated. In addition, when a vertical synchronization signal is input, the lighting and extinguishing of the LED 13 in the next blinking pattern is controlled in the stored blinking pattern sequence.
This makes it possible to send and receive data with one blinking pattern for each frame instead of every other frame. In addition, since high-precision control is possible, the width of the bright band and the dark band in the light-dark pattern can be recognized with high accuracy. Therefore, for example, a narrow bright band and a dark band in the light-dark pattern such as a barcode can be used. By applying 0 and assigning 1 to the wide bright band and dark band that are more than twice the width of the narrow band so that light and dark are arranged alternately, 1 bit of data is used for each band. Can be in the assigned state.
In this case, 0 is assigned to lighting and extinguishing with a narrow time width on the blinking pattern side, and 1 is assigned to lighting and extinguishing with a time width that is more than twice the time width in the narrow time width. Also in this case, the time width of each extinguishing light is defined by the number of lines in one frame because the vertical synchronization signal and the horizontal synchronization signal are synchronized as described above.
Here, when there is a delay in the synchronization signal, that is, when the synchronization signal is received and the LED 12 is turned on or off, the synchronization signal is delayed with respect to the imaging timing of one line. May be advanced by the delay time. For example, even if the camera 2 starts shooting a moving image on the smartphone 1 side and the synchronization signal is output and the timing of the synchronization signal is obtained, the flash 5 as the synchronization signal is emitted earlier than the actual synchronization signal. Good. According to such an optical communication device, the communication accuracy can be improved and the communication speed can be increased.
<p> An experiment was conducted to read digital data from the captured image in optical communication using the light emission of LED 13 and the imaging by the image sensor of camera 2. In this experiment, a still image was taken instead of a moving image in a state where the LED 13 was repeatedly turned on and off in the same time width, that is, in a state where the LED 13 was blinking in a predetermined blinking pattern and no outside light was introduced. An experiment was conducted to see if the light / dark pattern corresponding to the blinking pattern could be read from the captured image data.</p><p> In FIG. 10, reference numeral 41 indicates an captured image, reference numeral 42 indicates a graph of luminance data along a line in the Y direction as the vertical direction of the image 41, and reference numeral 43 indicates a graph. It is a graph of the luminance data which shade-corrected the luminance data shown in 42.</p><p> Image 41 is a one-frame image taken by a smartphone camera, and 5 to 6 striped patterns can be confirmed, but it is clear due to the high brightness of the central part and the integral action of the C-MOS sensor described later. It is not a striped pattern. Graph 42 shows the brightness change of the line portion along the Y direction in the substantially center of the X direction of the image 41, the vertical axis shows the position along the Y direction on the image 41, and the horizontal axis shows the brightness. Is shown.</p><p> Here, due to the integral action of the C-MOS sensor, the contrast between the light when the LED 13 is turned on and the dark when the LED is turned off is low as shown in the image 41 and the graph 42. Further, in the shooting of the skin image, since the data of the peripheral portion around the central portion of the image data is not used, the peripheral portion of the image 41 is dimly lit by the LED 13, and as shown in Graph 42, The brightness of the central part of image 41 is high, and the brightness of the upper and lower parts is low.</p><p> Such a bias in brightness can be corrected by shading correction that corrects the brightness distribution of the entire image so as to be uniform. Graph 43 is a shading-corrected version of the brightness shown in Graph 42, and the difference (contrast) between the brightness of the bright part and the brightness of the dark part becomes clear, and the boundary between the high-brightness part and the low-brightness part becomes clear. Therefore, the number of repetitions of light and dark can be easily recognized.</p><p> Next, an experiment was conducted in which the difference from the background was used instead of the shading correction. In FIG. 11, the image 44 is an LED modulated image taken by blinking the LED 13, and the image 45 is a background image taken with the LED 13 turned off, for example. In this example, the image is taken as a moving image instead of a still image, and the frame rate is set to 15 fps. Here, an image is shown in a state where the field of view of the camera 2 is squarely limited by the lens housing 20 described above. The entire images 44 and 45 correspond to the shooting range of the image sensor.</p><p> The bright square part in the center of this range is the part of the skin that is photographed with the field of view limited by the lens housing 20. That is, an opening 20C is provided on the tip side of the lens housing 20, and the skin facing the opening 20c is photographed. Here, in this example, the entire imaging range of the image sensor has 3096 lines (pixels) in the vertical direction, but the skin portion actually photographed has 1600 lines (pixels) in the vertical direction. The number of pixels in the horizontal direction of the image of the skin portion is also set to be 1600 pixels.</p><p> Taking the difference between the LED modulated image 44 captured with the LED 13 blinking in a predetermined blinking pattern (the number of repetitions of the predetermined lighting and extinguishing) and the background image 45 captured with the LED turned off, the image is taken. It becomes 46. The brightness along the line in the Y direction of this image 46 is shown in Graph 46a. The upper line (wave) of the graph 46a shows the change in the vertical brightness of the image 46 from which the difference is taken, and the lower side shows the wave of the fundamental frequency among the waves of the change in the brightness. .. Further, on the horizontal axis which is the time axis in the graph 46a, the unit is the time for one line of the image sensor, that is, the time which is the interval between the horizontal synchronization signals, and the time is indicated by the number of lines.</p><p> In this example, the frequency is set to 590 Hz when the on / off blinking pattern of the LED 13 is set to the wavelength of a set of lighting and extinguishing time widths per second. That is, it is a state in which lighting and extinguishing are repeated 590 times per second. On the other hand, since the line of one image captured by the image sensor is 3096 and the frame rate of the camera 2 is 15 fps, the number of lines from which charges can be read per second is 3096 × 15.</p><p> Here, when the above-mentioned LED modulation frequency is 590 Hz and the number of lines from which charges are read per second is 3096 × 15, the wavelength that is the sum of the time width of one high brightness and the time width of low brightness is , About 79 lines.</p><p> Graph 46b shows the result of performing a fast Fourier transform (FFT) as a frequency analysis on the wave as the Y-direction luminance change shown in graph 46a. In this graph 46b, the vertical axis is the luminance and the horizontal axis is the frequency (the reciprocal of the wavelength indicated by the number of lines). Further, in the graph 46b, the upper side is the result of frequency analysis on the (corrected) brightness wave read from the actual image 46, and the lower side is the result of frequency analysis on the above-mentioned fundamental wave.</p><p> In graph 46b, the reciprocal of the frequency of the wave consisting of the change in brightness read from image 46 is the wavelength, but the frequency (frequency of the fundamental wave) obtained by frequency analysis was 0.0127 (about 1/79). The reciprocal of this frequency indicates the wavelength and the number of lines, which is 79. This corresponds to a wavelength that is the sum of the lighting time width and the extinguishing time width of the wave, which consists of repeating the lighting and extinguishing of the LED 13 calculated as described above.</p><p> Therefore, the wavelength (frequency) of the wave consisting of the repetition of turning on and off in one frame of the LED 12 can be read from the image data captured by the image sensor of the camera 2.</p><p> Therefore, data can be transmitted / received by a change in the number of repetitions of turning on and off the LED 13 within a predetermined time, that is, a change in the frequency of the wave consisting of repeating turning on and off of the LED 13. In other words, frequency shift keying (FSK) enables data communication.</p><p> Next, FIG. 12 shows the experimental results when the LED 13 was actually turned on and off at different frequencies.</p><p> The experimental method is the same as in FIG. 11, and the frame rate and the vertical resolution of the image sensor (image captured by) are also the same as the values shown in FIG. 11, except that the LED modulation frequency is different. It has become. Here, we examined the case where the wavelength of the wave of repeated lighting and extinguishing of LED 13 is 150 lines and the case where the same wavelength is 30 lines. The upper image 47 was captured when the number of lines was 150, and the lower image 48 was captured when the number of lines was 30. Graph 49 shows the result of correcting the brightness change along the Y-direction line of image 47 by the difference from the above-mentioned background image, and the brightness change along the Y-direction line of image 48 is the above-mentioned background image. The result corrected by the difference between and is shown in Graph 50.</p><p> The result of frequency analysis of the wave shown in graph 49 by FFT is shown in graph 51, and the result of frequency analysis of the wave shown in graph 50 by FFT is shown in fluff 52. As shown in Graph 49 when the wavelength is 150 and Graph 50 when the wavelength is 30, the corrected luminance change can sufficiently confirm the high luminance portion and the low luminance portion, and can be sufficiently confirmed. The difference in wavelength, that is, the difference in frequency can be sufficiently confirmed between Graph 49 and Graph 50.</p><p> In addition, as a result of frequency analysis, when the wavelength of the wave consisting of repeated lighting and extinguishing of LED13 is 150, the analyzed frequency is the reciprocal of the wavelength, and the image sensor captures the wave of repeated lighting and extinguishing of LED13. It is shown that the frequency can be read from the image 47 taken in. In Graph 51, in addition to the obtained fundamental frequency, relatively large harmonics that are thought to be caused by FFT analysis are observed.</p><p> In addition, the analyzed frequency when the number of wavelengths is 30, is the reciprocal of the wavelength of the wave of repeated lighting and extinguishing of LED13, and from image 48 taken by the image sensor of the repeatedly wave of lighting and extinguishing of LED13. It is shown that the frequency is readable. In Graph 52, FFT analysis was performed by setting the FFT so that frequencies other than the fundamental wave and harmonics were not overlapped.</p><p> As a result, the frequency of the wave (the number of repetitions per unit time) due to the repetition of turning on and off the LED 13 can be read from the image captured under the illumination of the LED 13, and by frequency shift keying between the LED and the image sensor. It was shown that data communication by light is possible.</p><p> Next, a third embodiment of the present invention will be described. In the third embodiment, identification information is provided around the opening 20c of the lens housing 20 within the shooting range of the camera 2 with an optically readable code (a code readable by image recognition) such as a so-called barcode. It is a thing. Here, this code is referred to as a bar code, but the bar code is not limited to a general bar code, and may be various bar codes (including a two-dimensional bar code) or may be optically read. It does not have to be a so-called barcode as long as it is a possible code.</p><p> In this embodiment, a barcode portion 20d is provided around the opening 20c of the contact portion 20b of the lens housing 20, and the back surface of the barcode portion 20d (the surface facing the conversion lens 11 and the camera 2) is provided. Bar code 20e is provided. The barcode 20e may be attached as a sticker or may be printed on the barcode portion 20d, for example. Further, the barcode portion 20d may be integrally molded with the lens housing 20 or may be retrofitted to the lens housing 20.</p><p> As shown in FIG. 14, the barcode 20e, which is the identification information, is formed around the skin image 63 taken in a rectangular shape (square shape) corresponding to the opening 20c in the image 61 in which the skin is taken, for example. The barcode image 20f (the image of the barcode 20e taken) is displayed in a rectangular frame shape. That is, there is a barcode image 20f around the skin image 63 of the image 61 in which the skin is photographed. One unit of barcode may be displayed on one side of the rectangle, and the same one may be provided on four sides so that the reading side can recognize the clearest one.</p><p> This makes it possible to read the identification information from the barcode image 20f of the image 61 in which the skin is photographed. The identification information may be read by the smartphone 1 or by the server to which the skin image 63 is sent.</p><p> In FIG. 14, the circular portion indicates the shooting range 62 of the conversion lens 11, but the portion protruding from the image 61 is a portion that is not imaged. Further, the position of the barcode 20e is preferably in the optical axis direction of the conversion lens 11 and in the direction orthogonal to the optical axis, as close as possible to the skin exposed from the opening 20c.</p><p> According to such a configuration, in the captured image 61, the barcode image 20f is included around the skin image 63, and the captured image 61 includes the identification information by the barcode 20e and the skin image 63. It is possible to identify the conversion lens 11 (lens module 10) used for shooting the skin image 63.</p><p> Further, the barcode 20e is composed of, for example, black and white, and this barcode may be used for color correction such as white balance. Further, in the third embodiment, the barcode 20e is arranged on the portion of the lens housing 20 that is imaged together with the skin, but a cap is detachably provided on the tip side (contact portion 20b) of the lens housing 20. Originally, an optically readable code such as various barcodes may be provided on the back surface of the cap (the surface facing the conversion lens 11 when attached).</p><p> At this time, the barcode may be attached to the back surface of the cap as a sticker or printed on the back surface of the cap. Further, since the barcode can be provided in the rectangular portion corresponding to the opening 20c, for example, various two-dimensional barcodes can be preferably used.</p><p> Further, in the above-described embodiment, the LED 13 for lighting for shooting is used to transmit information by the blinking pattern, but the LED for transmitting information according to the blinking pattern may be provided separately from the LED 13 for lighting.</p><p> Further, in the above-described embodiment, in each type of blinking pattern, the time width of the high-luminance portion is substantially the same, the time width of the low-luminance portion is the same, and the time width of the high-luminance portion is the same. And the time width of the low-luminance portion are the same, but may be different.</p><p> For example, the time width of the high-luminance portion is set to multiple types such as the reference width, twice the reference width, and three times the reference width, and the time width of the low-luminance portion is also the reference width and the reference width. There may be a plurality of types such as twice the width and three times the reference width. In this case, it is possible to allocate 1 bit of data to each bar, as in the case of a barcode consisting of, for example, narrow white and black bars and white and black bars that are more than twice as wide. Further, by increasing the types of widths, for example, it is possible to allocate data such as 2 bits, 3 bits, and 4 bits to one bar (band).</p>
1 Smartphone (portable electronic device, receiver) 2 Camera (image sensor) 3 Control device (control means, image analysis means, reception conversion means) 4 Flash memory (light and dark pattern storage means) 5 Flash (synchronous output means) 10 Lens module (transmitter) 11 conversion lens 12 LED (solid-state light emitting element, LED for shooting lighting) 13 LED (solid-state light emitting element, LED for shooting lighting) 16 CPU (light emission control means, blinking pattern storage means, transmission conversion means) 25 PD (Synchronous input means)
15 sheets
Sheet 1 Sheet 2 Sheet 3 Sheet 4 Sheet 5 Sheet 6 Sheet 7 Sheet 8 Sheet 9 Sheet 10 Sheet 11 Sheet 12 Sheet 13 Sheet 14 Sheet 15
Every citation, both waysCites: the store holds 3 of 4
| Document | Relation | Office | Cited during |
|---|---|---|---|
| JP2020521178A | Cited by | Japan | Search report |
| JP2017512401A | Cited by | Japan | Search report |
| CN112822411A | Cited by | China | Search report |
| WO2019190077A1 | Cited by | World Intellectual Property Organization (WIPO) | International search |
| KR20180091589A | Cited by | Republic of Korea | Search report |
| US11391938B2 | Cited by | United States of America | Applicant |
| JP2017512401A | Cited by | Japan | Search report |
| CN113839733A | Cited by | China | Search report |
| JP2003179556A | Cites | Japan | Search report |
| JP2010160286A | Cites | Japan | Search report |
| JP2011254317A | Cites | Japan | Search report |
| CHRISTOS DANAKIS ET AL.: "Using a CMOS Camera Sensor for Visible Light Communication", 3RD IEEE WORKSHOP ON OPTICAL WIRELESS COMMUNICATIONS (OWC'12), JPN6016044016, 2012, pages 1244 - 1248, XP032341559, ISSN: 0003440632, DOI: 10.1109/GLOCOMW.2012.6477759 | Non-patent | – | Search report |
2 members in 1 office
Priority claims2
| Document | Office | Kind | Date |
|---|---|---|---|
| 2012288861 | Japan | A | |
| JP20120288861 | – | – | – |
Members2
| Document | Office | Kind | |
|---|---|---|---|
| JP2014131237AThis record | Japan | A | |
| JP6059012B2 | Japan | B2 |
12 legal events, as the office reported them to INPADOC
Over the term
Point at a mark for the eventEvents
| Event | Code | |
|---|---|---|
| Cancellation because of no payment of annual feesLAPS | LAPS | |
| Written notification of registration of transferJAPANESE INTERMEDIATE CODE: R350R350 | R350 | |
| Written request for registration of change of domicileJAPANESE INTERMEDIATE CODE: R313531S531 | S531 | |
| Written request for registration of change of nameJAPANESE INTERMEDIATE CODE: R313533S533 | S533 | |
| Certificate of patent or registration of utility modelJAPANESE INTERMEDIATE CODE: R150R150 | R150 | |
| First payment of annual fees (during grant procedure)JAPANESE INTERMEDIATE CODE: A61A61 | A61 | |
| Written decision to grant a patent or to grant a registration (utility model)JAPANESE INTERMEDIATE CODE: A01A01 | A01 | |
| Decision of grant or rejection writtenTRDD | TRDD | |
| Written amendmentJAPANESE INTERMEDIATE CODE: A523A521 | A521 | |
| Notification of reasons for refusalJAPANESE INTERMEDIATE CODE: A131A131 | A131 | |
| Report on retrievalJAPANESE INTERMEDIATE CODE: A971007A977 | A977 | |
| Written request for application examinationJAPANESE INTERMEDIATE CODE: A621A621 | A621 |
Numbers
- Publication
- 2014131237
- Publication, DOCDB
- 2014131237
- Publication, EPODOC
- JP2014131237
- Application
- 288861
- Application, DOCDB
- 2012288861
- Application, EPODOC
- JP20120288861
Titles2
- Japanese
- 光通信装置、光通信方法および肌撮像システム
- English
- Optical communication device, optical communication method and skin imaging system
Classification
- IPC, 4
- H04B10 116
- G02B15 10
- G02B21 06
- G02B21 36