Soc structure of video codec-embedded image sensor and method of driving image sensor using the same
Claim Score by NHIP
Abstract
A system on chip (SoC) structure of a video codec-embedded image sensor and a method of driving an image sensor using the same are provided. The image sensor includes a first domain including a codec processing unit processing image data using a video codec and a core processor, a second domain including an image sensor pixel and an analog-to-digital converter (ADC), a third domain including an image signal processor (ISP) performing signal processing on the image data obtained from the second domain, a fourth domain including a formatter converting a data format of the data generated in the first to third domains to the outside, an a clock generation unit providing system clocks to the first to fourth domains, respectively.

Term
Projected expiry 31 October 2031.
- Priority
- Filed
- Published
- Today
- Projected expiry
20 claims: 3 independent, 17 dependent
- 1An image sensor comprising:a second domain configured to generate a digital signal image data from a light signal representing an image;a first domain configured to perform codec-processing on the image data using a video codec;a third domain configured to perform signal-processing on the image data from the second domain;a fourth domain configured to convert data format of the data generated in the first, second, and third domains and output the converted data;and a clock generation unit configured to provide a system clock to each of the first, second, third, and fourth domains through separate clock supply lines, wherein the first domain is capable of controlling a data flow between the first, second, third, and fourth domains.
- 11Broadest claimClaim Score 89, very broad(NHIP)A method of driving an image sensor comprising a demarcated domain configured to control the image sensor and clock supply lines connected to the demarcated domain, the method comprising:performing an idle operation comprising stopping the operation of the image sensor;and obtaining an image data using the image sensor.
- 20An image sensor comprising:a plurality of domains, each comprising a device configured to process an image;and a clock generation unit configured to provide system clocks to the plurality of domains through separate clock supply lines, wherein one of the domains comprises: a codec processing unit for processing image data using a video codec;and a core processor, and wherein the clock generation unit provides a system clock at a speed corresponding to an operation speed of the device in each of the plurality of domains.
Independent claims3
96 paragraphs in 5 sections, as filed
CROSS-REFERENCE TO RELATED APPLICATIONS
p-0002This application claims priority to Korean Patent Application No. 10-2010-0108305 filed on Nov. 2, 2010 in the Korean Intellectual Property Office, the disclosure of which is incorporated herein by reference in its entirety.
BACKGROUND OF THE INVENTION
p-00031. Field of the Invention
p-0004The present invention relates generally to a system on chip (SoC) structure of a video codec-embedded image sensor and a method of driving an image sensor using the same, and more particularly to a structure for implementing an image sensor including a codec which can support video encoding and decoding, as an SoC, and a method for driving an image sensor using the structure.
p-00052. Description of the Related Art
p-0006A typical system on chip (SoC) type CMOS image sensor (CIS) merely supports image signal processing and the so-called “3A” (Auto White Balance, Auto Exposure, Auto Focus) and pixel readout functions.
p-0007However, as the H.264 and MPEG-4 encoding and decoding functions are implemented in mobile terminals and automobile applications, implementing a typical SoC type CIS described above in a video system is problematic because, to do so, a CIS chip would be required in addition to a video codec chip for encoding the CIS output image data into a commercial video format such as H.264/MPEG-4.
p-0008An additional chip in a video system imposes unfavorable constraints such as lengthened development time, additional costs, technical difficulties of modularization, etc.
SUMMARY OF THE INVENTION
p-0009An aspect of the present invention provides a system on chip (SoC) structure of a video codec-embedded image sensor and a method of driving an image sensor using the same, in particular, provides a structure for integrating a video codec in an image sensor and a method for driving the image sensor implemented to have the structure.
p-0010According to an aspect of the present invention, there is provided a video codec-embedded image sensor including: a first domain codec-processing image data using a video codec and controlling an overall operation of the image sensor; a second domain receiving an image signal as a light signal and generating image data as a digital electrical signal; a third domain performing signal-processing on the image data obtained from the second domain; a fourth domain converting a data format of the data generated in the first to third domains so as to be output to the outside, and outputting the data to the outside; and a clock generation unit providing a system clock to each of the first to fourth domains through each of separate clock supply lines, wherein the first domain controls a data flow among the first to fourth domains.
p-0011The first domain may include: a codec processing unit encoding or decoding the image data according to the video codec; and a core processor controlling the operations of the first to fourth domains.
p-0012The clock generation unit may generate a first clock having a period changing according to an operational state of the first domain, and generate a second clock for the second domain, a third clock for the third domain, and a fourth clock for the fourth domain, and a first clock supply line supplying the first clock to the first domain, a second clock supply line supplying the second clock to the second domain, a third clock supply line supplying the third clock to the third domain, and a fourth clock supply line supplying the fourth clock to the fourth domain may be separated.
p-0013The operational state of the first domain may include an idle state in which the first clock is not generated when the image sensor is not in use, a low speed operational state in which the first clock is generated with a period longer than a reference period when codec encoding or decoding is not performed, a normal operational state in which the first clock is generated at the reference period when the codec processing unit performs codec encoding or decoding, and a high speed operational state in which the first clock is generated with a period shorter than the reference period when the codec processing unit and the core processor performs codec encoding or decoding.
p-0014According to another aspect of the present invention, there is provided a method for driving an image sensor which processes image data using a video codec and includes separate domains controlling the overall operation of the image sensor and clock supply lines separately connected to the domains, including: an idle operation of stopping the operation of the image sensor; and an image data obtaining operation of obtaining image data using the image sensor.
p-0015The method may further include: a codec processing operation of processing the image data obtained in the data obtaining operation using the video codec.
p-0016According to another aspect of the present invention, there is provided an image sensor including: a plurality of domains each including a device mounted therein and processing an image; and a clock generation unit providing system clocks to the plurality of domains through separate clock supply lines, wherein one of the domains includes a codec processing unit for processing image data using a video codec, and a core processor, and the clock generation unit provides a system clock at a speed corresponding to an operation speed of the device mounted in each of the plurality of domains, by domain.
BRIEF DESCRIPTION OF THE DRAWINGS
p-0017The above and other aspects, features and other advantages of the present invention will be more clearly understood from the following detailed description taken in conjunction with the accompanying drawings, in which:
p-0018<figref idrefs="DRAWINGS">FIG. 1</figref> is a schematic block diagram showing function blocks of a video codec-embedded image sensor demarcated by operation clock;
p-0019<figref idrefs="DRAWINGS">FIG. 2A</figref> is a view showing a flow of image data obtained in a second domain;
p-0020<figref idrefs="DRAWINGS">FIG. 2B</figref> is a view showing a flow of image data signal-processed in a third domain;
p-0021<figref idrefs="DRAWINGS">FIG. 2C</figref> is a view showing a flow of image data which are currently being processed by codec or have already been processed by codec in a first domain;
p-0022<figref idrefs="DRAWINGS">FIG. 3</figref> is a flow chart illustrating an operational process of a method for driving a video codec-embedded image sensor according to an embodiment of the present invention; and
p-0023<figref idrefs="DRAWINGS">FIG. 4</figref> is a view showing the relationship between respective operational operations of the method for driving the video codec-embedded image sensor according to an embodiment of the present invention and the speed of clocks provided to the first domain.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT
p-0024Embodiments of the present invention will now be described in detail with reference to the accompanying drawings. The invention may, however, be embodied in many different forms and should not be construed as being limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.
p-0025In the drawings, the shapes and dimensions of elements may be exaggerated for clarity, and the same reference numerals will be used throughout to designate the same or like components.
p-0026Throughout the specification and claims, unless explicitly described to the contrary, the word “comprise” and variations such as “comprises” or “comprising”, will be understood to imply the inclusion of stated elements but not the exclusion of any other elements.
p-0027The problems that may arise in a two-chip video system are briefly described below.
p-0028The two chips in a video system according to H.264/MPEG-4 refer to a CIS sensor chip and a backend processor chip. The CIS sensor chip produces a CIS image, and the backend processor chip video encodes the received CIS image and stores and outputs the encoded image signal to a display device. There are certain problems associated with modularizing these two chips into a single module such as interface and firmware mismatch between the CIS sensor and the backend processor. Also, increasing the module area to mount the two chips increases the overall fabrication cost of the module, where the high cost would act to degrade the marketability of the fabricated module.
p-0029Furthermore, the CIS system and the codec processing module system have different operational schemes, and, due to this and other differences, simply integrating the CIS and codec processing modules into a single chip does not provide a solution to many problems related to power controlling, interfacing, etc.
p-0030Thus, it is required to have an SoC structure that allows such heterogeneous systems to coexist in a single chip and provides a way for controlling the heterogeneous systems in the single chip structure.
p-0031The function blocks configured to perform both CIS and codec processing in a single chip and the operations between the function blocks are described below according to an embodiment of the present invention.
p-0032The function blocks integrated in one chip to perform both CIS and codec processing include: a core processor, a pixel read-out circuit, an image signal processor (ISP), and a codec processing unit. Also, it may be possible to utilize a formatter outputting image data to the outside and a direct memory access (DMA) and a bus to transmit data between the function blocks.
p-0033If necessary, a storage device storing image data may be provided.
p-0034The pixel read-out circuit includes an image sensor pixel such as a CMOS or the like. An image data obtained from each image sensor pixel is outputted through the pixel read-out circuit.
p-0035The output data from the pixel read-out circuit may be transferred to the ISP. The ISP may perform the image signal processing on the image data (such as the “3A” for Auto White Balance, Auto Exposure, and Auto Focus) and output the image signals.
p-0036It is possible that the image data outputted from the pixel read-out circuit may be transferred directly to the codec processing unit. Alternatively, it is also possible that the image data outputted form the pixel read-out circuit may be processed by the ISP prior to being transferred to the codec processing unit. The codec processing unit may encode the received image data using a video codec. Also, the codec processing unit may decode encoded image data inputted from the outside.
p-0037The image outputted from the pixel read-out circuit may be signal-processed directly by the ISP or may be processed by the codec processing unit and outputted through a formatter, and may be stored in a storage device.
p-0038In the process of transferring the image data, a direct memory access (DMA) and a bus may be used.
p-0039The function blocks integrated in the SoC configured for the CIS and codec processing mode as well as the operations and transmission data between the respective functional blocks have been described above. Hereinafter, an SoC structure related to the operations of and data transmission between the function blocks within the SoC and a method for driving the same according to an embodiment of the present invention will be described.
p-0040<figref idrefs="DRAWINGS">FIG. 1</figref> is a schematic block diagram showing the function blocks of a video codec-embedded image sensor demarcated by an operation clock according to an embodiment of the present invention.
p-0041With reference to <figref idrefs="DRAWINGS">FIG. 1</figref>, the video codec-embedded image sensor according to an embodiment of the present invention may include first to fourth domains <b>100</b> to <b>400</b> and a clock generation unit <b>500</b>.
p-0042The first domain <b>100</b> may process image data using a video codec and control the operation of the overall image sensor. The first domain <b>100</b> may include a codec processing unit <b>120</b> encoding or decoding image data according to a video codec and a core processor <b>110</b> controlling the first to fourth domains <b>100</b> to <b>400</b>.
p-0043Also, the first domain <b>100</b> may further include a DMA <b>140</b> and a bus <b>150</b> for a data transmission between the first to fourth domains and a data transmission between the units in the first domain. If necessary, the first domain <b>100</b> may further include a memory <b>130</b> storing data. Also, if necessary, the first domain <b>100</b> may further include one or more controllers (not shown) for storing data in an external storage device.
p-0044The second to fourth domains <b>200</b> to <b>400</b> may be connected to the bus <b>150</b>.
p-0045Also, the codec processing unit <b>120</b> and the core processor <b>110</b> of the first domain <b>100</b> may be connected to the bus <b>150</b>, and the memory <b>130</b> of the first domain <b>100</b> may be further connected to the bus <b>150</b>.
p-0046The core processor <b>110</b> may control function blocks within the first to fourth domains <b>100</b> to <b>400</b>. Also, the core processor <b>110</b> may support video codec encoding and decoding in a software-wise manner (i.e., by software).
p-0047The codec processing unit <b>120</b> includes a hardware core processor processing image data using a video codec. The codec processing unit <b>120</b> may encode image data and decode encoded image data.
p-0048The DMA <b>140</b> is a device responsible for handling a data transmission between two different function blocks. The DMA <b>140</b> may read data from a first device and transfer the read data to a second device. Also, for a data transmission between a plurality of devices, the DMA <b>140</b> may read temporarily stored data from the first data and temporarily store the read data in the second device. Thus, the first and/or second device may further include a storage unit for temporarily storing input/output data. The storage unit may be implemented to be first input first output (FIFO).
p-0049The second domain <b>200</b> includes an image sensor pixel <b>210</b> and an analog-to-digital converter (ADC) <b>220</b>, and may further include a pixel read-out circuit (e.g., a CDS, a PGA, or the like) for outputting image data from the image sensor pixel <b>210</b>.
p-0050The image sensor pixel <b>210</b> may include optical sensing pixels such as a CCD, a CMOS, or the like. The image sensor pixel <b>210</b> converts a light signal into an electrical signal.
p-0051The ADC <b>220</b> converts an analog signal outputted from the image sensor pixel <b>210</b> or the pixel read-out circuit into a digital signal.
p-0052Also, the second domain may further include a storage unit <b>230</b> for supporting a data transmission using the DMA <b>140</b>. The storage unit <b>230</b> may be connected to the bus <b>150</b>.
p-0053The third domain <b>300</b> includes an image signal processor (ISP) <b>310</b> performing signal processing on image data obtained from the second domain <b>200</b>. Also, the third domain <b>300</b> may further include a storage unit <b>320</b> supporting a data transmission using the DMA <b>140</b>. The storage unit <b>320</b> may be connected to the bus <b>150</b>.
p-0054The fourth domain <b>400</b> includes a formatter <b>410</b> converting a data format in order to output data generated in the first to third domains <b>100</b> to <b>300</b> to the outside. The fourth domain <b>400</b> may further include a storage unit <b>420</b> supporting a data transmission using the DMA <b>140</b>. The storage unit <b>420</b> may be connected to the bus <b>150</b>.
p-0055The clock generation unit <b>500</b> may provide a system clock to each of the first to fourth domains <b>100</b> to <b>400</b> through a separated clock supply line.
p-0056The clock generation unit <b>500</b> generates first to fourth clocks clk<b>1</b> to clk<b>4</b>, and the first to fourth clocks clk<b>1</b> to clk<b>4</b> are provided to the first to fourth domains <b>100</b> to <b>400</b>, respectively. Preferably, a first clock supply line providing the first clock clk<b>1</b> to the first domain <b>100</b>, a second clock supply line providing the second clock clk<b>2</b> to the second domain <b>200</b>, a third clock supply line providing the third clock clk<b>3</b> to the third domain <b>300</b>, and a fourth clock supply line providing the fourth clock clk<b>4</b> to the fourth domain <b>400</b> may be separated.
p-0057Whereas the periods of the second to fourth clocks clk<b>2</b> to clk<b>4</b> may be pre-set, the period of the first clock clk<b>1</b> may be variable and can be changed according to the operational states of the first domain <b>100</b>.
p-0058The operational states of the first domain <b>100</b> include an idle state, a low speed operational state, a normal operational state, and a high speed operational state. An idle state refers to a state in which the first clock clk<b>1</b> is not generated when the image sensor is not in use. The low speed operational state refers to a state in which the first clock clk<b>1</b> is generated with a period longer than a reference period when codec encoding or decoding is not performed. The normal operational state refers to a state in which the first clock clk<b>1</b> is generated at the reference period when the codec processing unit performs codec encoding or decoding. The high speed operational state refers to a state in which the first clock clk<b>1</b> is generated with a period shorter than the reference period when the codec processing unit and the core processor performs codec encoding or decoding. Table 1 below shows the operational states of the first domain according to an operation scenario of the first domain.
p-0059<tables id="TABLE-US-00001" num="00001"><table frame="none" colsep="0" rowsep="0"><tgroup align="left" colsep="0" rowsep="0" cols="3"><colspec colname="offset" colwidth="28pt" align="left" /><colspec colname="1" colwidth="112pt" align="left" /><colspec colname="2" colwidth="77pt" align="left" /><thead><row><entry /><entry namest="offset" nameend="2" rowsep="1">TABLE 1</entry></row><row><entry /><entry namest="offset" nameend="2" align="center" rowsep="1" /></row><row><entry /><entry>Operation scenario</entry><entry>First clock</entry></row><row><entry /><entry namest="offset" nameend="2" align="center" rowsep="1" /></row></thead><tbody valign="top"><row><entry /><entry>HW codec + SW codec</entry><entry>High speed</entry></row><row><entry /><entry>HW codec</entry><entry>Normal</entry></row><row><entry /><entry>HW/SW codec in standby</entry><entry>Low speed</entry></row><row><entry /><entry>CIS in standby/ready</entry><entry>Clock off</entry></row><row><entry /><entry namest="offset" nameend="2" align="center" rowsep="1" /></row></tbody></tgroup></table></tables>
p-0060The first to fourth clocks clk<b>1</b> to clk<b>4</b> generated in the clock generation unit <b>500</b> may not be synchronized clock signals, and their clock periods may be different from each other. Thus, the first to fourth domains <b>100</b> to <b>400</b> may operate asynchronously.
p-0061Thus, the first to fourth domains <b>100</b> to <b>400</b> may independently operate and their power may be independently controlled. Also, interfacing between the first to fourth domains <b>100</b> to <b>400</b> may be resolved using an asynchronous data transmission scheme using the DMA or the like.
p-0062<figref idrefs="DRAWINGS">FIGS. 2A to 2C</figref> are views showing a data flow of the video codec-embedded image sensor according to an embodiment of the present invention.
p-0063<figref idrefs="DRAWINGS">FIG. 2A</figref> shows a processing flow of an image data produced in the second domain <b>200</b> according to an embodiment of the present invention.
p-0064With reference to <figref idrefs="DRAWINGS">FIG. 2A</figref>, the DMA <b>140</b> is configured to transfer an image data obtained by the image sensor pixel <b>210</b> of the second domain <b>200</b> to any one or more of the other domains such as the first, third, and fourth domains <b>100</b>, <b>300</b>, and <b>400</b>.
p-0065The image data in the image sensor pixel <b>210</b> of the second domain <b>200</b> may be outputted to the ADC <b>200</b> by a pixel read-out circuit (e.g., the CDS or the PGA). The ADC <b>220</b> is configured to convert the obtained image data into a digital signal and store the converted digital signal in the storage unit <b>230</b>.
p-0066The DMA <b>140</b> is capable of temporarily reading the image data stored in the storage unit <b>230</b> of the second domain <b>200</b> through the bus <b>150</b> and temporarily storing the read image data. The DMA <b>140</b> may store the temporarily stored image data through the bus <b>150</b> in one or more of storage media such as the storage unit <b>320</b> of the third domain <b>300</b>, the storage unit <b>420</b> of the fourth domain <b>400</b>, and the memory <b>130</b> of the first domain <b>100</b>.
p-0067<figref idrefs="DRAWINGS">FIG. 2B</figref> shows a process flow of an image data signal-processed in the third domain <b>300</b> according to an embodiment of the present invention.
p-0068With reference to <figref idrefs="DRAWINGS">FIG. 2B</figref>, the image data signal-processed in the ISP <b>310</b> may be transferred to other domains such as to the first or fourth domain <b>100</b> or <b>400</b> by the DMA <b>140</b>.
p-0069The ISP <b>310</b> of the third domain <b>300</b> may read the image data stored in the storage unit <b>320</b> and process the read data. In general, the ISP unit <b>310</b> of the third domain <b>300</b> may perform image signal processing functions such as one or more of the “3A” functions for Auto White Balance, Auto Exposure, and Auto Focus. The ISP <b>310</b> may store the signal-processed image data again in the storage unit <b>320</b>.
p-0070The DMA <b>140</b> may read the image data stored in the storage unit <b>320</b> of the third domain <b>300</b> through the bus <b>150</b> and temporarily store the read image data therein. The DMA <b>140</b> may store the temporarily stored image data in the storage unit <b>420</b> of the fourth domain <b>400</b> and the memory <b>130</b> of the first domain through the bus <b>150</b>.
p-0071<figref idrefs="DRAWINGS">FIG. 2C</figref> a process flow example of image data that is being or was processed by a codec in the first domain according to an embodiment of the present invention.
p-0072With reference to <figref idrefs="DRAWINGS">FIG. 2C</figref>, the codec processing unit <b>120</b> may access the memory <b>130</b> to codec-process the image data stored in the memory <b>130</b> and to store the codec-processed image data in the memory <b>130</b>. The codec-processed image data may be transferred to the fourth domain <b>400</b>. Or, when needed, the core processor <b>100</b> may perform the operation along with the codec processing unit <b>120</b>.
p-0073The codec processing unit <b>120</b> is configured to read the image data stored in the memory <b>130</b>, encode the read image data according to a video codec, or decode the encoded image data stored in the memory <b>130</b>. The core processor <b>110</b> may read the image data stored in the memory <b>130</b> and encode the read image data by software according to the video codec or decode the encoded image data stored in the memory <b>130</b> by software.
p-0074When needed, the DMA <b>140</b> may control an image transmission between the core processor <b>110</b>, the codec processing unit <b>120</b>, and the memory <b>130</b>. The DMA <b>140</b> may read the image data stored in the memory <b>130</b> and transfer the read image data to the core processor <b>110</b> or the codec processing unit <b>120</b>, and the DMA <b>140</b> may also read the image data processed in the processor <b>110</b> or the codec processing unit <b>120</b> and store the read image data in the memory <b>130</b>.
p-0075Also, the DMA <b>140</b> may read the image data processed in the processor <b>110</b> or the codec processing unit <b>120</b> or the codec-processed data stored in the memory <b>130</b> and store the read data in the storage unit <b>410</b> of the fourth domain <b>400</b>.
p-0076The fourth domain <b>400</b> converts the data format of the image data stored in the storage unit <b>420</b> using the formatter <b>410</b> and outputs the format-converted image data to the outside.
p-0077The core processor <b>110</b> may process the audio codec. Thus, the core processor <b>110</b> can support video and audio coding by, for example, a software to reproduce a multimedia file. Also, a hardware accelerator may be connected to the bus <b>150</b> to perform channel coding on the image data or the like. Thus, it should be readily understood that the functions can be easily added according to usage according to an embodiment of the present invention.
p-0078<figref idrefs="DRAWINGS">FIG. 3</figref> is a flow chart illustrating the process of driving a video codec-embedded image sensor according to an embodiment of the present invention;
p-0079The method for driving a video codec-embedded image sensor according to an embodiment of the present invention may use an image sensor having a demarcated domain processing image data using a video codec and controlling an overall operation of the image sensor and separated clock supply lines connected to the domain.
p-0080Referring to <figref idrefs="DRAWINGS">FIG. 3</figref>, a method of driving a video codec-embedded image sensor according to an embodiment of the present invention may include, inter alia: an idle operation S<b>10</b>, an image data obtaining operation S<b>20</b>, a codec processing operation S<b>30</b>, and an output operation S<b>40</b>.
p-0081In the idle operation S<b>10</b>, the image sensor wait for an image sensing command or may not begin the operation.
p-0082When an image sensing command is received, the process proceeds to the image data obtaining operation S<b>20</b> to obtain an image by operating the second domain of the image sensor. In the pixel data outputting operation S<b>21</b>, the image signal as a light signal in the image sensor pixel <b>210</b> is converted into an electrical signal, and the converted image signals are output as image data.
p-0083When an ISP performing command is received, an image signal processing operation S<b>22</b> is performed. In the image signal processing operation S<b>22</b>, image signal processing such as 3A or the like may be performed.
p-0084When there is no ISP performing command, the codec processing operation S<b>30</b> or the output operation S<b>40</b> may be performed.
p-0085Also, the method of driving a video codec-embedded image sensor may use an image sensor, in which the demarcated domain includes a codec processing unit for encoding or decoding image data according to a video codec, a core processor for controlling the operation of the image sensor, and a data transmission unit for controlling and performing a data transmission within the image sensor.
p-0086When an encoding command is received, the codec processing operation S<b>30</b> is performed by determining whether a high speed encoding processing is to be performed. When a high speed encoding processing is determined to be performed, a high speed processing operation S<b>32</b> is performed, but when a high speed encoding processing is determined as not to be performed, a low speed processing operation S<b>31</b> is performed. In the low speed processing operation S<b>31</b>, the codec processing unit <b>120</b> may only be operated to encode the image data by, for example, hardware or in a hardware-wise manner, but, in the high speed processing operation S<b>32</b>, the core processor <b>110</b> as well as the codec processing unit <b>120</b> encode the image data by, for example, software.
p-0087When there is no encoding command, the output operation S<b>40</b> may be performed.
p-0088In the output operation S<b>40</b>, the image data is converted according to an output format and then outputted to an external device.
p-0089<figref idrefs="DRAWINGS">FIG. 4</figref> shows the clock speeds provided to the first domain with respect to the operations performed for driving the video codec-embedded image sensor according to an embodiment of the present invention.
p-0090With reference to <figref idrefs="DRAWINGS">FIG. 4</figref>, the clocks provided to the first domain <b>100</b> may have different speeds according to the different operational stages.
p-0091In an idle state S<b>10</b>, a driving clock may not be provided so as to stop the operation of the first domain <b>100</b> of the image sensor in order to prevent power consumption (e.g., clk<b>1</b>). Also, according to an embodiment of the present invention, a portion or the entire amount of the power provided to the domains <b>100</b> to <b>400</b> of the image sensor may be cut off.
p-0092In an image data obtaining operation S<b>20</b>, the first domain <b>100</b> of the image sensor may serve to control the operations of the second or third domain <b>200</b>, <b>300</b> and a data flow, so a low speed clock may be supplied to the first domain <b>100</b>. Also, the second or third clock clk<b>2</b>, clk<b>3</b> with a pre-set period may be provided to the second or third domain <b>200</b>, <b>300</b>.
p-0093In a low speed processing operation S<b>31</b> of the codec processing operation S<b>30</b>, a normal clock is provided to the first domain <b>100</b> of the image sensor. In a high speed processing operation S<b>32</b> of the codec processing operation S<b>30</b>, a high speed clock may be provided to the first domain <b>100</b> of the image sensor. This is because, in the case of high speed processing operation S<b>32</b>, both the codec processing unit <b>120</b> and the core processor <b>110</b> perform the codec processing; thus, a faster data transmission and faster operation of function blocks are required.
p-0094As set forth above, the SoC structure of a video codec-embedded image sensor and the method of driving an image sensor using the same according to various embodiments of the invention are advantageous in that it would be possible to sense an image using a single chip set, and the sensed image data can be encoded into a commercial compression format such as H.264/MPEG-4 or the like with relative ease.
p-0095In the SoC structure of a video codec-embedded image sensor and the method of driving an image sensor using the same according to various embodiments of the invention, the single chip implementation of the image sensor and the codec processing unit provides effective solutions to the system operational difficulties related to, for example, the mismatching firmwares of the image sensor and the codec processing unit that are implemented as two separate chips.
p-0096In the SoC structure of a video codec-embedded image sensor and the method of driving an image sensor using the same according to various embodiments of the present invention, the single chip implementation of the image sensor and the codec processing unit leads to reduced mounting area and thus can reduce the manufacturing cost and enhance the marketability of the manufactured product.
p-0097While the present invention has been shown and described in connection with the embodiments, it will be apparent to those skilled in the art that modifications and variations can be made without departing from the spirit and scope of the invention as defined by the appended claims.
Contents5
7 sheets
Sheet 1 Sheet 2 Sheet 3 Sheet 4 Sheet 5 Sheet 6 Sheet 7
Every citation, both ways
| Document | Relation | Office | Cited during |
|---|---|---|---|
| US2016267349A1 | Cited by | United States of America | Pre-grant |
| CN112995557A | Cited by | China | Search report |
| US10055672B2 | Cited by | United States of America | Applicant |
| US2016267349A1 | Cited by | United States of America | Search report |
| CN118301307A | Cited by | China | Search report |
| US10268886B2 | Cited by | United States of America | Applicant |
| US10103869B2 | Cited by | United States of America | Search report |
| CN109040839A | Cited by | China | Search report |
| US2003048361A1 | Cites | United States of America | Pre-grant |
| US2009238479A1 | Cites | United States of America | Pre-grant |
| US8106804B2 | Cites | United States of America | Pre-grant |
| USRE39963E | Cites | United States of America | Pre-grant |
2 members in 2 offices; this record represents the family
Priority claims1
| Document | Office | Kind | Date |
|---|---|---|---|
| 20100108305 | Republic of Korea | A |
Members2
| Document | Office | Kind | |
|---|---|---|---|
| US2012105680A1 | United States of America | A1 | |
| KR20120047357A | Republic of Korea | A |
24 transactions on the USPTO file
Abandoned after 1 non-final rejection.
- Non-final rejections
- 1
- Final rejections
- 0
- RCEs
- 0
- Appeals
- 0
Over time
Point at a mark for the transactionTransactions
| Event | Code | |
|---|---|---|
| Email NotificationEML_NTR | EML_NTR | |
| Mail Abandonment for Failure to Respond to Office ActionAbandonedMABN2 | MABN2 | |
| Aband. for Failure to Respond to O. A.AbandonedABN2 | ABN2 | |
| Electronic ReviewELC_RVW | ELC_RVW | |
| Email NotificationEML_NTF | EML_NTF | |
| Mail Non-Final RejectionNon-final rejectionMCTNF | MCTNF | |
| Non-Final RejectionNon-final rejectionCTNF | CTNF | |
| Information Disclosure Statement consideredIDSC | IDSC | |
| Case Docketed to Examiner in GAUDOCK | DOCK | |
| Email NotificationEML_NTR | EML_NTR | |
| PG-Pub Issue NotificationPG-ISSUE | PG-ISSUE | |
| Information Disclosure Statement (IDS) FiledM844 | M844 | |
| Information Disclosure Statement (IDS) FiledWIDS | WIDS | |
| Case Docketed to Examiner in GAUDOCK | DOCK | |
| Application Dispatched from OIPEOIPE | OIPE | |
| Application Is Now CompleteCOMP | COMP | |
| Email NotificationEML_NTR | EML_NTR | |
| Filing ReceiptFLRCPT.O | FLRCPT.O | |
| Sent to Classification ContractorPGPC | PGPC | |
| Cleared by OIPE CSRL194 | L194 | |
| Request from applicant for the USPTO to retrieve the Priority DocumentPDREQUST | PDREQUST | |
| Applicants have given acceptable permission for participating foreignAPPERMS | APPERMS | |
| IFW Scan & PACR Auto Security ReviewSCAN | SCAN | |
| Initial Exam Team nnIEXX | IEXX |
2 legal events, as the office reported them to INPADOC
Over the term
Point at a mark for the eventEvents
| Event | Code | |
|---|---|---|
| Information on status: application discontinuationABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTIONSTCB | STCB | |
| AssignmentAS | AS |
Numbers
- Publication
- 20120105680
- Application
- 13285164
Titles
- English
- SOC STRUCTURE OF VIDEO CODEC-EMBEDDED IMAGE SENSOR AND METHOD OF DRIVING IMAGE SENSOR USING THE SAME
Classification
- CPC, 3
- H04N19/61
- H04N25/00
- H04N19/42
- IPC, 1
- H04N25 00