Recording audio and electronic images

This invention relates to the field of electronic still cameras and more particularly to electronic still cameras which have an integral printer capable of printing audio and visual information.

Systems for providing audio recording and playback in association with individual electronic still prints are known in the art. Examples of such systems are disclosed in US-A-3,439,598; US-A-4,270,853; US-A-4,270,854; US-A-4,905,029; and US-A-5,276,472. In the '598 patent, a belt driven recording needle inscribes audio information in spiral grooves on the back layer of a multilaminate slide film. This requires special film and film processing that would allow for removal of the recording film laminate and its attachment to the slide frame for playback by needle-based playback apparatus. In the '854 patent, sound is recorded on an instant print by placing the print, after it has been ejected, into an auxiliary slot in the camera and then proceeding to record the audio on a magnetic strip on the print border. With this system, audio can only be recorded after the picture has been captured and only on an instant print. Also, the only way disclosed for playback is with the camera. The '853 patent discloses a similar apparatus for an instant print camera for recording audio on a magnetic strip in the margin of the instant print paper within a film pack. The '029 patent discloses a microphone and tape recording mechanism to record sound in the camera which is then recorded onto a separate magnetic strip by means of separate recording/playback apparatus. The strip may then be adhesively applied to the print or to the album page adjacent the print and the sound reproduced by means of a special playback apparatus with a reciprocating playback head which is placed against the strip. This arrangement requires a separate tape recording mechanism in the camera. The '472 patent discloses the recording of digital audio information on a magnetic layer formed on a film. The digital audio information is in the form of a barcode.

Additionally, there are three known commercially available systems for combining sound with still photographs. One is the "Talking Picture Frame" sold by Talking Pictures, Inc. which utilizes a special picture frame having a voice recording IC to record the sound in RAM chips in the frame. When the frame is lifted, a microswitch activates the IC to playback the recorded sound.

Another system is the "Mini Box Comm" sold by Foto-Fonics, Inc. which utilizes a separate record/playback box to record sound on an adhesively backed strip adhered to the print. The print is inserted into the box for recording and playback and the print is in motion during both modes of operation. The third system is the Mavica electronic camera sold by Sony Corporation which is an electronic camera as opposed to a electronic film camera. Images are acquired by electronic sensors for recording on 2.5" floppy disc memories for subsequent readout by electronic visual display means. Sound is recorded on the floppy disc for subsequent playback along with the recorded image.

A problem with the photographic film systems described above is that the magnetic strips, if kept integral with the prints, are limited to use with instant print cameras. In the '854 patent, the sound must be written in a separate step involving the camera user which is a uncontrolled variable in providing the benefit as described. In the case of the system described in US-A-4,905,029, the sound is recorded on separate strips that may become lost or not easily associated with the prints when they are returned from the photofinisher. In the case of the picture frame the sound is recorded after the print is returned from the photofinisher thus losing the benefit of sound recorded at the time the picture is taken.

It is an object of the invention to provide audio and visual information on a receiver produced by a printer in an electronic camera.

This object is achieved by a method of recording audio and image data on a receiver comprising the steps of: <ul id="ul0001" list-style="none" compact="compact"><li>a) capturing audio information and converting it to digital signals;</li><li>b) capturing an electronic image which corresponds to the captured audio information;</li><li>c) recording the audio information on the receiver by applying a nonvisible ink to the receiver; and</li><li>d) recording the electronic image by applying visible ink to the receiver. <ul id="ul0002" list-style="none" compact="compact"><li>FIG. 1 is a schematic illustration of an electronic camera with an integral printer for recording audio and visual information according to the present invention;</li><li>FIG. 2 is a schematic block diagram of audio recording and reproducing circuits useful in the camera of FIG. 1;</li><li>FIG. 3 is a schematic block diagram of control circuitry for operating the image sensor of the electronic camera;</li><li>FIGS. 4a, 4b, and 4c are illustrations of a receiver illustrating different locations of the barcode;</li><li>FIG. 5 is a simplified diagram of hand-held apparatus for reading audio digital signals recorded on the receiver of FIGS. 4a, 4b, or 4c; and</li><li>FIG. 6 is a schematic diagram of playback circuits useful in the hand-held apparatus of FIG. 5.</li></ul></li></ul>

Referring to FIG. 1, an electronic camera 16 with an integral printer 16a is shown which is specifically adapted to capture an electronic image by an image sensor 14 (see FIG. 3) and print such an image. The integral printer 16a may, for example, be an inkjet printer which is adapted to record images but also print a barcode format on receiver P such as shown in FIGS. 4a, 4b, and 4c. Electronic camera 16 is provided with a built-in audio transducer, for example microphone 17, a shutter button 15, and a miniature speaker 20. The shutter button 15 can take many forms known in the art, such as the two position switch which can, in the first position, provide an auto focus activation and, in the second position, activate the camera shutter (not shown) to capture an image. The electronic camera 16 is provided with a viewfinder 21 and will be understood to include the conventional elements found in electronic cameras. A receiver or print "P" is produced by the integral printer 16a. A liquid crystal display (LCD) 18 is mounted on the integral printer 16a. After a user views the image on the LCD 18 and finds it acceptable, the user actuates the integral printer 16a by depressing a print button 16b.

Referring to FIG. 2, there is shown an expanded block diagram of the circuits and components included in the audio recording and reproducing circuits of electronic camera 16. Buttons or switches 22 permit a user to provide inputs into a system microcontroller 23 which condition the system microcontroller 23 to initiate and control the various operating functions of the electronic camera 16. Among these functions are the sound recording and playback functions of the present invention. Moreover, the system microcontroller 23 also responds to the camera shutter button 15 to cause the capture of an image on an image sensor 14 as is well known in the art. Microphone 17 and speaker 20 are coupled to analog amplifier and data processing circuit 24 to input and play back the audio. A sound processor integrated circuit (IC) 25 serves to convert analog signals input from microphone 17 into coded digital information suitable for storage in a digital memory 26 and for converting the digital information into analog signals suitable for playback through speaker 20. Sound processor IC 25 may be a Texas Instruments TMS3477 and digital memory 26 may be a random access memory (RAM) such as a Hitachi HM628128. One of the functions of digital memory 26 is to serve as a temporary storage location for the sound data associated with an individual captured image. Also shown is system bus 80 which shown simplistically for clarity carries both address and data to all attached devices.

Referring to FIG. 3, a schematic block diagram of the control circuitry for operating the image sensor 14 with the system microcontroller 23 is shown. Conditioned by the system microcontroller 23, timing generator 100 activates the logical sequencing of timing signals used to control the devices necessary to electronically capture an image. In order to operate the image sensor 14 which typically can be a Charge Coupled Device (CCD), CCD drivers 102 receive signals from timing generator 100 which cause the generation of appropriate clocking signals that control the image integration time and data clocking of the image sensor 14. As image data is clocked out of image sensor 14, it is subject to the analog signal processing 112 which can include a conventional correlated double sample and hold circuit which stabilizes the image data against known references before conversion to representative image data digital signals. The image data digital signals are then sent to digital memory 26 (which stores the electronic image) via system bus 80 as controlled by address generator 104. To display the captured image on LCD 18, address generator 104 working in conjunction with timing generator 100 cause the image data digital signals to be processed by digital signal processor 110. Digital signal processor 110 is known to subs ample the image data digital signals to a resolution suitable for display on LCD 18. The subsampled image data digital signals are then further processed by digital-to-analog (D/A) converter 108 into analog signals which when received by display interface electronics 42 cause the image data signals to be displayed by LCD 18. Display interface electronics 42, for example, contain the necessary elements to obtain a signal compatible with LCD 18 which usually takes the NTSC format well known in the art.

Referring back to FIG. 2 when it is desired to record sound in association with taking a picture, the camera user selects sound recording via a user input selector switch 22 that causes the system microcontroller 23 to set the digital memory 26 to the "write" mode and then enables the analog amplifier and data processing circuit 24 for audio recording. Assuming the user desires to have image related audio, the user talks into the camera microphone 17 to identify the scene with appropriate information such as date, time, f-stop, shutter speed, picture taking location, people in the scene, or aims the microphone to record live sounds from the scene. The data processing circuit 24 and sound processor IC 25 convert the incoming analog signal to coded digital data which is then stored in the digital memory 26. Audio may be recorded and stored in digital memory 26 in this manner before, during or after the picture-taking event

Once having stored the audio in digital memory 26, the stored audio information can be reviewed by playing the stored information via speaker 20 in the camera. To do this, the user selects the " review" mode by means of a user input switch 22 which causes the microcontroller 23 to set the digital memory 26 to the "read" mode thereby enabling the sound processor IC 25 and the analog amplifier and data processing circuit 24 to play back audio through speaker 20 If the recorded sound is not satisfactory, the user can easily change it by simply repeating the recording process described above. It is thus apparent that the use of this digital memory 26 allows quick audio review and, if necessary, re-recording.

The electronic camera 16 operates in two different modes. In the first mode, after the shutter button 15 is depressed, an image is captured and displayed on LCD 18. The user then decides if he or she is going to record audio information by depressing the appropriate switch 22. The audio information is stored in digital memory 26. Thereafter, the user causes the receiver P to be made by depressing print button 16b which provides a signal to the printer controller 28. The printer controller 28 is also adapted to receive the captured digital image from the image sensor 14 and to apply such digital image to the integral printer 16a.

In the second mode, the user desires to simultaneously capture audio information and record a picture. This is accomplished by conditioning the camera system microcontroller 23 by an appropriate input from one of the switches 22. In this case, the audio information will begin to be recorded directly onto a receiver when the shutter button 15 is activated to a first position. The image is captured upon activation of shutter button 15 to a second position. The total available recording time is fixed, for example, at 10 seconds. The audio recording and direct printing occurs for the 10 seconds simultaneously following the activation of shutter button 15.

As has been described above, the integral printer 16a can be an inkjet printer which can record digital images and barcode data corresponding to audio information. The system microcontroller 23 shown in FIG. 2 can cause various different formats to be used which will be discussed in connection with FIGS. 4a, 4b, and 4c. The barcode information is recorded by the inkjet printer as an ink responsive to light in the infrared (IR) portion of the spectrum. Nonvisible IR responsive inks can be used in accordance with the present invention. For examples of dyes that can be dispersed in a liquid medium and are responsive to light in the infrared portion of the spectrum and can be used by an inkjet printer, see US-A-5,691,114, columns 9 and 10.

Turning now to FIGS. 4a, 4b, and 4c which are illustrations of a receiver illustrating different locations of the barcode. FIG. 4a depicts a situation where the barcode 45 is provided along the margin or one of the edges of the receiver P. As shown in FIGS. 4b and 4c, the barcode 45 is recorded in tracks over the entire image. In FIG. 4b, the barcode 45 corresponding to the audio information was first recorded prior to the recording of the image. In FIG. 4c, the barcode 45 was recorded after the image was recorded.

Having recorded digital audio signals on the receiver as described above, the sound subsequently may be reproduced by means of a portable, hand-held playback device 50 shown in FIG. 5 having a form factor about the same as a credit card and only slightly thicker. Such a device comprises a button battery power source 51, a memory storage IC 52, an optical IR emitter/detector pair 53, a sound data decoder IC 54, a planar speaker 55, analog drive circuits 56, and digital control logic unit 57.

Referring to FIG. 6, functional block diagram of circuitry for the device of FIG. 5 comprises user input buttons 58 coupled to digital control logic unit 57 which controls the operation of sound data decoder IC 54 and an analog drive circuits 56 which includes speaker amplifier 56a and analog drive and amplitude comparator unit 56b. The latter unit is coupled to the optical IR emitter/detector pair 53 which operates to sense the recorded barcode audio digital signals on the receiver (see FIGS. 4a, b, and c) and to produce a digital output signal which is temporarily stored in memory storage IC 52. To play back the audio from the recorded audio digital signals on the print, the user enables the hand held reader by pressing an "on" switch of user input buttons 58 which causes digital control logic unit 57 to set memory storage IC 52 to the "store" mode and also to enable an LED (not shown) to indicate the "power on" status. Next the user scans the optical emitter of optical IR emitter/detector pair 53 over the encoded audio information shown as a barcode pattern. Preferably the data is encoded in a self-clocking format such as is described in US-A-4,876,697 or in US-A-4,954, 825, the disclosures of which are incorporated herein by reference, in order to make the reader scanspeed tolerant. While scanning, the unit decodes the printed pattern back into digital data and stores it in the local memory. Once scanned, the receiver is held in a comfortable manner to be observed while the audio player input switch is activated. The digital control logic unit 57 then proceeds to set the memory storage IC 52 to read, and to enable the sound data decoder IC 54 to convert the digital signal data to an audio signal which is passed via the amplifier to the speaker. A particular advantage of such an arrangement is that the user can replay the audio as many times as desired without re-scanning the print, since the data is retained in the digital memory storage IC 52 until power is removed or until the user clears the memory via the user input buttons 58 and digital control logic unit 57.

Other features of the invention are included below.

The method wherein the recorded audio digital signals are in the form of a barcode recorded in the same area of the receiver as is the recorded visual image.

The method wherein the barcode is recorded in a plurality of spaced tracks on the receiver.

The method wherein the barcode is recorded near at least one edge of the receiver.

In an electronic camera which includes a printer for recording audio and image data on a receiver comprising the steps of: <ul id="ul0003" list-style="none" compact="compact"><li>a) means for capturing audio information and converting it to digital signals;</li><li>b) means for capturing an electronic image which corresponds to the captured audio information; and</li><li>c) control means responsive to the audio digital signals and the captured electronic image for causing the printer to record audio information on the receiver by applying an IR responsive ink to the receiver and record the electronic image by applying visible ink to the receiver.</li></ul>

The electronic camera wherein a plurality of colored inks are recorded on the receiver which correspond to the electronic image.

The electronic camera wherein the recorded audio digital signals are in the form of a barcode recorded in the same area of the receiver as is the recorded visual image.

The electronic camera wherein the barcode is recorded in a plurality of spaced tracks on the receiver.

The electronic camera wherein the barcode is recorded near at least one edge of the receiver.

The electronic camera wherein the control means first causes the audio digital signals to be applied to the receiver and thereafter causes the visible ink to be applied to the receiver.

The electronic camera wherein the control means first causes the visible ink to be applied to the receiver where a user can preview it and then causes the audio digital signals to be applied to the receiver.

PARTS LIST

<dl id="dl0001" compact="compact"><dt>P</dt><dd>receiver or print</dd><dt>14</dt><dd>image sensor</dd><dt>15</dt><dd>shutter button</dd><dt>16</dt><dd>electronic camera</dd><dt>16a</dt><dd>integral printer</dd><dt>16b</dt><dd>print button</dd><dt>17</dt><dd>microphone</dd><dt>18</dt><dd>liquid crystal display</dd><dt>20</dt><dd>speaker</dd><dt>21</dt><dd>viewfinder</dd><dt>22</dt><dd>buttons or switches</dd><dt>23</dt><dd>microcontroller</dd><dt>24</dt><dd>data processing circuit</dd><dt>25</dt><dd>sound processor integrated circuit</dd><dt>26</dt><dd>digital memory</dd><dt>28</dt><dd>printer controller</dd><dt>42</dt><dd>display interface electronics</dd><dt>45</dt><dd>barcode</dd><dt>50</dt><dd>hand-held playback device</dd><dt>51</dt><dd>button battery power source</dd><dt>52</dt><dd>memory storage IC</dd><dt>53</dt><dd>optical IR emitter/detector pair</dd><dt>54</dt><dd>sound data decoder IC</dd><dt>55</dt><dd>planar speaker</dd><dt>56</dt><dd>analog drive circuits</dd><dt>56a</dt><dd>speaker amplifier</dd><dt>56b</dt><dd>amplitude comparator unit</dd><dt>57</dt><dd>digital control logic unit</dd><dt>58</dt><dd>user input buttons</dd><dt>80</dt><dd>system bus</dd><dt>100</dt><dd>timing generator</dd><dt>102</dt><dd>CCD drivers</dd><dt>104</dt><dd>address generator</dd><dt>108</dt><dd>digital-to-analog (D/A) converter</dd><dt>110</dt><dd>digital signal processor</dd><dt>112</dt><dd>analog signal processing</dd></dl>