Adaptive surrounding view monitoring apparatus and method thereof
Claim Score by NHIP
Abstract
An adaptive surrounding view monitoring apparatus and a method thereof are disclosed, in which the apparatus uses a steerable camera set to take horizontal images while sending the horizontal images to a control unit where they are combined into an image of 180-degree or 360-degree surrounding view. It is noted that the surrounding view image maybe an image of front surrounding view, an image of rear surrounding view or an image of ambient surrounding view relating to a vehicle, and using that, visual blind spot of a driver driving the vehicle can be eliminated and the field of vision is widened, and thereby, the probability of accident occurring is reduced.

Term
6.1 yearsto projected expiry
Projected expiry 12 October 2032, counted from filing; an application has no term until it is granted.
- Priority
- Filed
- Published
- Today
- Projected expiry
20 claims: 2 independent, 18 dependent
- 1Broadest claimClaim Score 89, very broad(NHIP)An adaptive surrounding view monitoring apparatus, comprising:a steerable camera set;a control unit, electrically connected to the steerable camera set;a display unit, electrically connected to the control unit;and a mode switch, electrically connected to the control unit.
- 9An adaptive surrounding view monitoring method, comprising the steps of:making an evaluation to determine whether a moving speed of a vehicle is smaller than a threshold value;if so, enabling a control unit to be situated in an activation state before proceeding to the next step;making an evaluation to determine whether the vehicle is issuing a reverse (R) gear signal;if so, processing an operation for identifying in which mode a mode switch is situated;processing an operation for outing a front surrounding view image relating to the front of the vehicle to a display unit for displaying, if the mode switch is identified to be situated in a front surrounding view mode;processing an operation for outing a rear surrounding view image relating to the rear of the vehicle to a display unit for displaying, if the mode switch is identified to be situated in a rear surrounding view mode;and processing an operation for outing an ambient surrounding view image relating to the ambient environment of the vehicle to a display unit for displaying, if the mode switch is identified to be situated in an ambient surrounding view mode.
Independent claims2
42 paragraphs in 6 sections, as filed
CROSS-REFERENCE TO RELATED APPLICATION
0001This non-provisional application claims priority under 35 U.S.C. §119(a) on Patent Application No. 100111504 filed in Taiwan (R.O.C.) on Apr. 1, 2011, the entire contents of which are hereby incorporated by reference.
FIELD
0002The present disclosure relates to an adaptive surrounding view monitoring apparatus and method thereof, and more particularly, to a monitoring apparatus capable of using a steerable camera set for taking a series of images relating to ambient environment of a vehicle in a horizontal manner while sending the series of images to a control unit to be combined into an image of surrounding view so as to be provided to a driver of the vehicle. Consequently, not only visual blind spots of the driver can be eliminated, but also the field of vision is widened, and thereby, the probability of accident occurring is reduced.
BACKGROUND
0003Pillars are the vertical supports of the passenger compartment of a vehicle, that are known respectively as the A, B, C or D-pillar moving in profile view from the front to rear. Such pillar nomenclature derived from viewing a vehicle in profile can be used as reference points for the passenger compartment of a vehicle. Among which, the A-pillar of a vehicle is the first pillar of the passenger compartment, usually is the one arranged at a position between side-view mirrors and the windshield. The B-pillar of a vehicle is the second pillar of the passenger compartment, after the A-Pillar. The B-pillar, usually used to house the hinges for any rear doors, is the one arranged at a position between the front seats and the rear seats. The C-pillar generally is rearmost pillar supporting the back window and the rear part of a vehicle roof, that is arranged at a position corresponding to the headrest of the rear seat.
0004For a driver driving a vehicle, there can be plenty of blind spots existed surrounding the vehicle that cannot be directly observed or through either the rear-view or side-view mirrors by the driver while at the controls due to the obstruction of the vehicle's body structure. Consequently, the driver will have to turn one's head or even change one's body position while negotiating a turn, backing up or changing lane so as to be free from the obstruction of the vehicle's body structure, such as the A-pillars, or the limited field-of-view of its rear-view and side-view mirrors. However, such driving behaviors may increase the risk of collision.
0005It is noted that humans have an almost 210-degree forward-facing horizontal field of view. However, the range of visual abilities is not uniform across a field of view. For humans that are not moving, the ability to perceive shape and motion clearly only covers about 70 degrees of the field of view. However, when one is riding on a moving vehicle, the slower the vehicle is moving, the larger the field of view will be, and vice versa. Thus, since a driver's field of view is decreasing with the increasing of driving speed and the driver's field of view can further be restricted by a vehicle's body structure, such as the A-pillars, while driving the vehicle, there can be plenty of blind spots existed surrounding the vehicle and thus any driving condition happening in the adjacent lanes of the vehicle that fall into these blind spots may not be visible and awared by the driver.
0006In response to the aforesaid situations, there are already many commercial equipments that can help eliminating blind spots by providing bird-view images showing ambient environment of a vehicle or images respectively capturing the front, left, right and rear view of the vehicle. Nevertheless, those equipments still have the following shortcomings: <ul id="ul0001" list-style="none"><li id="ul0001-0001" num="0000"><ul id="ul0002" list-style="none"><li id="ul0002-0001" num="0007">1. For bird-view image, objects shown in a bird-view image, especially those non-ground objects, will be distorted and thus might not be easily identified.</li><li id="ul0002-0002" num="0008">2. It is difficult to form a seamless stitching image using a plurality of bird-view images capturing respectively different ambient areas of a vehicle, since there can be ambient areas that are not included in the plural bird-view images.</li><li id="ul0002-0003" num="0009">3. For images with overlapping field-of-view, ultrasonic radars are generally being used for detecting the actual situation in that overlapping area.</li><li id="ul0002-0004" num="0010">4. Those equipments can only display one image of a single direction or simultaneously two images of two different directions, but are not able to display images of adjacent directions, resulting that there are still blind spots existed.</li></ul></li></ul>
0011To sum up, the equipments that are current available not only fail to provide distortion-free images that can include all the ambient areas of a vehicle without blind spots, but also fail to display images of adjacent directions simultaneously in addition to their having to use additional devices other than those used for generating images.
SUMMARY
0012The object of the present disclosure is to provide an adaptive surrounding view monitoring apparatus and a method thereof, in which the apparatus uses a steerable camera set to take images while sending the images to an image process unit where they are distorted, aligned and mosaicked into a panoramic image of surrounding view, to be used for assisting a driver to clearly observe the ambient environment of a vehicle at control without any blind spot, and thereby, not only the probability of accident occurring is reduced, but also the prior art shortcomings of failing to display images of adjacent directions simultaneously in addition to their having to use additional devices other than those used for generating images can be prevented.
0013To achieve the above object, the present disclosure provides an adaptive surrounding view monitoring apparatus, comprising: a steerable camera set; a control unit, connected to the steerable camera set; a display unit, electrically connected to the control unit; and a mode switch, electrically connected to the control unit.
0014Moreover, the present disclosure further provides an adaptive surrounding view monitoring method, comprising the steps of: <ul id="ul0003" list-style="none"><li id="ul0003-0001" num="0000"><ul id="ul0004" list-style="none"><li id="ul0004-0001" num="0015">making an evaluation to determine whether a moving speed of a vehicle is smaller than a threshold value; if so, enabling a control unit to be situated in an activation state; otherwise, enabling the control unit to be situated in a standby state;</li><li id="ul0004-0002" num="0016">making an evaluation to determine whether the vehicle is issuing a reverse (R) gear signal; if so, processing an operation for identifying in which mode a mode switch is situated;</li><li id="ul0004-0003" num="0017">processing an operation for outing a front surrounding view image relating to the front of the vehicle to a display unit for displaying, if the mode switch is identified to be situated in a front surrounding view mode;</li><li id="ul0004-0004" num="0018">processing an operation for outing a rear surrounding view image relating to the rear of the vehicle to a display unit for displaying, if the mode switch is identified to be situated in a rear surrounding view mode; and</li><li id="ul0004-0005" num="0019">processing an operation for outing an ambient surrounding view image relating to the ambient environment of the vehicle to a display unit for displaying, if the mode switch is identified to be situated in an ambient surrounding view mode.</li></ul></li></ul>
0020To sum up, the adaptive surrounding view monitoring apparatus and the method thereof that are provided in the present disclosure have the following advantages: <ul id="ul0005" list-style="none"><li id="ul0005-0001" num="0000"><ul id="ul0006" list-style="none"><li id="ul0006-0001" num="0021">1. As the steerable camera set is designed to capture images in a horizontal manner, those non-ground objects can be shown without distortion and thus can be easily identified.</li><li id="ul0006-0002" num="0022">2. Using the apparatus and method of the present disclosure, an image of 180-degree or 360-degree surrounding view relating to the ambient environment of a vehicle can be obtained, and thereby, the field of view of the driver at control of the vehicle can be widened without any blind spot.</li><li id="ul0006-0003" num="0023">3. Since all the blind spots of the vehicle are eliminated using the surrounding view image, there is no need to have additional devices other than those used for generating images to be installed in the monitoring apparatus for detecting ambient environment of the vehicle.</li><li id="ul0006-0004" num="0024">4. Since the surrounding view image includes images of adjacent directions or images of multiple directions, not only the field of view is widened, but also the blind spots are eliminated.</li></ul></li></ul>
0025Further scope of applicability of the present application will become more apparent from the detailed description given hereinafter. However, it should be understood that the detailed description and specific examples, while indicating exemplary embodiments of the disclosure, are given by way of illustration only, since various changes and modifications within the spirit and scope of the disclosure will become apparent to those skilled in the art from this detailed description.
BRIEF DESCRIPTION OF THE DRAWINGS
0026The present disclosure will become more fully understood from the detailed description given herein below and the accompanying drawings which are given by way of illustration only, and thus are not limitative of the present disclosure and wherein:
0027<figref idref="DRAWINGS">FIG. 1</figref> is a schematic diagram showing an adaptive surrounding view monitoring apparatus according to the present disclosure.
0028<figref idref="DRAWINGS">FIG. 2</figref> is a schematic diagram showing a steerable camera set that is mounted on a vehicle according to an embodiment of the present disclosure.
0029<figref idref="DRAWINGS">FIG. 3</figref> is a schematic diagram showing a steerable camera set that is mounted on a vehicle according to another embodiment of the present disclosure.
0030<figref idref="DRAWINGS">FIG. 4</figref>, being composed of <figref idref="DRAWINGS">FIG. 4A</figref> and <figref idref="DRAWINGS">FIG. 4B</figref>, is a flow chat showing steps performed in an adaptive surrounding view monitoring method according to the present disclosure.
0031<figref idref="DRAWINGS">FIG. 5</figref> is a schematic diagram showing an image of front surrounding view relating to a vehicle of the present disclosure.
0032<figref idref="DRAWINGS">FIG. 6</figref> is a schematic diagram showing an image of rear surrounding view relating to a vehicle of the present disclosure.
0033<figref idref="DRAWINGS">FIG. 7</figref> is a schematic diagram showing an image of ambient surrounding view relating to a vehicle of the present disclosure.
DESCRIPTION OF THE EXEMPLARY EMBODIMENTS
0034For your esteemed members of reviewing committee to further understand and recognize the fulfilled functions and structural characteristics of the disclosure, several exemplary embodiments cooperating with detailed description are presented as the follows.
0035Please refer to <figref idref="DRAWINGS">FIG. 1</figref>, which is a schematic diagram showing an adaptive surrounding view monitoring apparatus according to the present disclosure. As shown in <figref idref="DRAWINGS">FIG. 1</figref>, the adaptive surrounding view monitoring apparatus comprises: a steerable camera set <b>1</b>, a control unit <b>2</b>, a display <b>3</b> and a mode switch <b>4</b>.
0036In <figref idref="DRAWINGS">FIG. 1</figref>, the steerable camera set <b>1</b> includes a plurality of steerable cameras, such as the four cameras <b>10</b>, <b>11</b>, <b>12</b>, <b>13</b>, whereas each steerable camera is comprised of: a micro process unit, a motor module and an image sensor, such as the micro process units <b>100</b>, <b>110</b>, <b>120</b>, <b>130</b>, the motor modules <b>101</b>, <b>111</b>, <b>121</b>, <b>131</b> and the image sensors <b>102</b>, <b>112</b>, <b>122</b>, <b>132</b> respectively for the four steerable cameras <b>10</b>, <b>11</b>, <b>12</b>, <b>13</b>. Moreover, the motor modules <b>101</b>, <b>111</b>, <b>121</b>, <b>131</b> are electrically connected to the image sensors <b>102</b>, <b>112</b>, <b>122</b>, <b>132</b> in respective for enabling the image sensors <b>102</b>, <b>112</b>, <b>122</b>, <b>132</b> to be steered and orientated toward their respective angles as required. It is noted that the horizontal field of view of each steerable camera is between 40 degrees and 210 degrees; and for clarity, the cameras included in the steerable camera set are divided into four groups, which are a first steerable camera group, a second steerable camera group, a third steerable camera group and a fourth steerable camera group.
0037In this embodiment, the control unit <b>2</b> is further configured with a camera control element <b>20</b>, an image input element <b>21</b>, a signal processing element <b>22</b>, an image processing element <b>23</b>, a memory element <b>24</b>, an image output element <b>25</b> and an image database <b>26</b>
0038The camera control element <b>20</b> is electrically connected to the micro process units <b>100</b>, <b>110</b>, <b>120</b>, <b>130</b> of the four cameras <b>10</b>, <b>11</b>, <b>12</b>, <b>13</b> in respective for controlling the steerable cameras <b>10</b>, <b>11</b>, <b>12</b>, <b>13</b>, and simultaneously the camera control element <b>20</b> is electrically connected to the image processing element <b>23</b>, whereas the four steerable cameras <b>10</b>, <b>11</b>, <b>12</b>, <b>13</b> are respectively being included in the first steerable camera group, the second steerable camera group, the third steerable camera group and the fourth steerable camera group.
0039The image input element <b>21</b> is electrically connected to the image sensor of each steerable camera, i.e. the four images sensors <b>102</b>, <b>112</b>, <b>122</b>, <b>132</b>, in the steerable camera set <b>1</b> where the analog signals from those image sensors are converted into digital signals, and simultaneously the image input element <b>21</b> is electrically connected to the image processing element <b>23</b>.
0040The signal processing element <b>22</b> is provided for processing a signal transmitted from the vehicle, such as a left-turn signal, a right-turn signal, a signal relating to the moving speed of the vehicle, a reverse (R) gear signal, a non-reverse (R) gear signal, a mode switch signal, and so on. As shown in <figref idref="DRAWINGS">FIG. 1</figref>, the signal processing element <b>22</b> is electrically connected to the image processing element <b>23</b> for transmitting the vehicle signal to the image processing element <b>23</b> for enabling the image processing element <b>23</b> to issue a command correspondingly.
0041The image processing element <b>23</b> is used for distorting, aligning and mosaicking into a panoramic image of surrounding view and also for device activation control. It is noted that the image mosaicking method for the image processing element <b>23</b> had already been disclosed in TW Pat. Appl. No, 098145942, and thus will not be described further herein.
0042The memory element <b>24</b>, being used for data registering or buffering, is electrically connected to the image processing element <b>23</b>.
0043The image output element <b>25</b>, being used for converting digital signals into analog signals, is electrically connected to the image processing element <b>23</b>.
0044Moreover, the image database <b>26</b> is also electrically connected to the image processing element <b>23</b> so as to be used for storing images that are processed by the image processing element <b>23</b>.
0045In addition, the display unit <b>3</b> is electrically connected to the image processing element <b>23</b> for image displaying; and the mode switch <b>4</b>, being embedded with a front surrounding view mode <b>40</b>, a rear surrounding view mode <b>41</b> and an ambient surrounding view mode <b>42</b>, is also electrically connected to the image processing element <b>23</b>. Moreover, all the aforesaid electrical connections are respectively being enabled by a means selected from the group consisting of: a wired means and a wireless means.
0046Please refer to <figref idref="DRAWINGS">FIG. 2</figref>, which is a schematic diagram showing a steerable camera set that is mounted on a vehicle according to an embodiment of the present disclosure. As shown in <figref idref="DRAWINGS">FIG. 2</figref>, there is a steerable camera set <b>1</b> mounted on a vehicle <b>5</b>, whereas the vehicle <b>5</b> is configured with a front <b>53</b>, a rear <b>54</b>, two A-pillars <b>50</b>, two B-pillars <b>51</b>, a C-pillars <b>52</b> and two rear-view mirrors <b>55</b> in a manner that the two A-pillars <b>50</b> are disposed respectively at the two sides of the vehicle <b>5</b>, which are the same to the two B-pillars <b>51</b> and the two C-pillars <b>52</b>, while the two rear-view mirrors <b>55</b> are arranged at positions respectively corresponding to the two A-pillars <b>50</b>.
0047In <figref idref="DRAWINGS">FIG. 2</figref>, the steerable camera <b>10</b> is arranged at the middle of the front <b>53</b> of the vehicle <b>5</b>, and thus is being grouped in the first steerable camera group; the steerable camera <b>11</b> is arranged at the left side of the vehicle <b>5</b> at a position between the left A-pillar <b>50</b> and the left C-pillar <b>52</b> or at a position corresponding to the left B-pillar <b>51</b>, and thus is being grouped in the second steerable camera group; the steerable camera <b>12</b> is arranged at the right side of the vehicle <b>5</b> at a position between the right A-pillar <b>50</b> and the right C-pillar <b>52</b> or at a position corresponding to the right B-pillar <b>51</b>, and thus is being grouped in the third steerable camera group; and the steerable camera <b>13</b> is arranged at the middle of the rear <b>54</b> of the vehicle <b>5</b>, and thus is being grouped in the fourth steerable camera group.
0048Please refer to <figref idref="DRAWINGS">FIG. 3</figref> is a schematic diagram showing a steerable camera set that is mounted on a vehicle according to another embodiment of the present disclosure. In <figref idref="DRAWINGS">FIG. 3</figref>, the cameras <b>10</b>, <b>13</b>, that are grouped respectively in the first steerable camera group and the fourth steerable camera group, are being disposed at positions the same as those shown in <figref idref="DRAWINGS">FIG. 2</figref>, while the two cameras <b>11</b>, <b>13</b>, that are grouped respectively in the second steerable camera group and the third steerable camera group, are disposed at positions corresponding to the two rear-view mirror <b>55</b> in respectively.
0049Moreover, the control unit <b>2</b>, the display unit <b>3</b> and the mode switch <b>4</b> can be arranged inside the frame of the vehicle <b>5</b>, in which the mode switch <b>4</b> and the display unit <b>3</b> should be arranged at positions proximate to a driver of the vehicle <b>5</b> so as to facilitating the driver to easily view the images displayed on the display unit <b>3</b> and also to change to the required image mode through the control of the mode switch <b>4</b>.
0050The foregoing description only describes the components for an adaptive surrounding view monitoring apparatus and how the adaptive surrounding view monitoring apparatus is constructed using the aforesaid components. The description provided hereinafter will relates to an adaptive surrounding view monitoring method. Please refer to <figref idref="DRAWINGS">FIG. 4</figref>, which is composed of <figref idref="DRAWINGS">FIG. 4A</figref> and <figref idref="DRAWINGS">FIG. 4B</figref>, is a flow chat showing steps performed in an adaptive surrounding view monitoring method according to the present disclosure.
0051In <figref idref="DRAWINGS">FIG. 4</figref>, the adaptive surrounding view monitoring method comprises the steps of: <ul id="ul0007" list-style="none"><li id="ul0007-0001" num="0000"><ul id="ul0008" list-style="none"><li id="ul0008-0001" num="0052">step <b>60</b>: making an evaluation to determine whether a moving speed of a vehicle is smaller than a threshold value, whereas the threshold value is a defined value relating to the moving speed of the vehicle that is adjustable; if the moving speed is smaller than the threshold value, the flow proceeds to step <b>601</b> for enabling a control unit <b>2</b> to be situated in an activation state and the proceeds to step <b>61</b>; otherwise, the flow proceeds to step <b>600</b> for enabling the control unit <b>2</b> to be situated in a standby state;</li><li id="ul0008-0002" num="0053">step <b>61</b>: making an evaluation to determine whether the vehicle is issuing a reverse (R) gear signal; if so, the flow proceeds to step <b>610</b> where an operation is processed for identifying in which mode a mode switch <b>4</b> is situated, and thereafter, if the mode switch <b>4</b> is identified to be situated in the front surrounding view mode <b>40</b>, the flow will proceed to step <b>611</b>, if the mode switch <b>4</b> is identified to be situated in the rear surrounding view mode <b>41</b>, the flow will proceed to step <b>612</b>, and if the mode switch <b>4</b> is identified to be situated in the ambient surrounding view mode <b>42</b>, the flow will proceed to step <b>612</b>; otherwise, the flow proceeds to step <b>62</b>;</li><li id="ul0008-0003" num="0054">step <b>611</b>: when the mode switch <b>4</b> is situated in the front surrounding view mode <b>40</b>, enabling a camera control element <b>20</b> to control the micro process units <b>100</b>, <b>110</b>, <b>120</b> and motor modules <b>101</b>, <b>111</b>, <b>121</b> in cameras <b>10</b>, <b>11</b>, <b>12</b> belonging to a first steerable camera group, a second steerable camera group and a third steerable camera group, for enabling the image sensors <b>102</b>, <b>112</b>, <b>122</b> of the cameras <b>10</b>, <b>11</b>, <b>12</b> in the first steerable camera group, the second steerable camera group and the third steerable camera group to be orientated adequately for capturing images, while providing the captured images to an image processing element <b>23</b> through an image input element <b>21</b> where they are distorted, aligned and mosaicked into the image of 180-degree surrounding view to the front of the vehicle <b>5</b> to the display unit <b>3</b> through an image output element <b>25</b>, as shown in <figref idref="DRAWINGS">FIG. 5</figref>;</li><li id="ul0008-0004" num="0055">step <b>612</b>: when the mode switch <b>4</b> is situated in the rear surrounding view mode <b>41</b>, enabling the camera control element <b>20</b> to control the cameras <b>11</b>, <b>12</b>, <b>13</b> belonging to the second steerable camera group and the third steerable camera group and the fourth steerable camera group, for enabling the image sensors <b>112</b>, <b>122</b>, <b>132</b> of the cameras <b>11</b>, <b>12</b>, <b>13</b> in the first steerable camera group, the second steerable camera group and the third steerable camera group to be orientated adequately for capturing images, while providing the captured images to an image processing element <b>23</b> through an image input element <b>21</b> where they are distorted, aligned and mosaicked into the image of 180-degree surrounding view to the rear of the vehicle <b>5</b> to the display unit <b>3</b> through an image output element <b>25</b>, as shown in <figref idref="DRAWINGS">FIG. 6</figref>;</li><li id="ul0008-0005" num="0056">Step <b>613</b>: when the mode switch <b>4</b> is situated in the ambient surrounding view mode <b>42</b>, enabling the camera control element <b>20</b> to control the cameras <b>10</b>, <b>11</b>, <b>12</b>, <b>13</b> belonging to a first steerable camera group, a second steerable camera group, a third steerable camera group and a fourth steerable camera group, for enabling the image sensors <b>102</b>, <b>112</b>, <b>122</b>, <b>132</b> of the cameras <b>10</b>, <b>11</b>, <b>12</b>, <b>13</b> in the first steerable camera group, the second steerable camera group, the third steerable camera group and the fourth steerable camera group to be orientated adequately for capturing images, while providing the captured images to an image processing element <b>23</b> through an image input element <b>21</b> where they are distorted, aligned and mosaicked into the image of 360-degree surrounding view to the ambient environment of the vehicle <b>5</b> to the display unit <b>3</b> through an image output element <b>25</b>, as shown in <figref idref="DRAWINGS">FIG. 7</figref>;</li><li id="ul0008-0006" num="0057">step <b>62</b>: determining whether the vehicle <b>5</b> is issuing a signal selected from the group consisting of: a left-turn signal and a right-turn signal, if there in no such left-turn signal or right-turn signal being issued, the flow will proceeds to step <b>624</b>; otherwise, the flow proceeds to step <b>620</b>;</li><li id="ul0008-0007" num="0058">step <b>620</b>: performing the operation for identifying in which mode a mode switch <b>4</b> is situated, and thereafter, if the mode switch <b>4</b> is identified to be situated in the front surrounding view mode <b>40</b>, the flow will proceed to step <b>621</b>, if the mode switch <b>4</b> is identified to be situated in the rear surrounding view mode <b>41</b>, the flow will proceed to step <b>622</b>, and if the mode switch <b>4</b> is identified to be situated in the ambient surrounding view mode <b>42</b>, the flow will proceed to step <b>623</b>;</li><li id="ul0008-0008" num="0059">step <b>621</b>: when the mode switch <b>4</b> is situated in the front surrounding view mode <b>40</b>, enabling a camera control element <b>20</b> to control the cameras <b>10</b>, <b>11</b>, <b>12</b> belonging to a first steerable camera group, a second steerable camera group and a third steerable camera group, for enabling the image sensors <b>102</b>, <b>112</b>, <b>122</b> of the cameras <b>10</b>, <b>11</b>, <b>12</b> to be orientated adequately for capturing images, while providing the captured images to an image processing element <b>23</b> through an image input element <b>21</b> where they are distorted, aligned and mosaicked into the image of 180-degree surrounding view to the front of the vehicle <b>5</b> to the display unit <b>3</b> through an image output element <b>25</b>, as shown in <figref idref="DRAWINGS">FIG. 5</figref>;</li><li id="ul0008-0009" num="0060">step <b>622</b>: when the mode switch <b>4</b> is situated in the rear surrounding view mode <b>41</b>, enabling the camera control element <b>20</b> to control the cameras <b>11</b>, <b>12</b>, <b>13</b> belonging to the second steerable camera group and the third steerable camera group and the fourth steerable camera group, for enabling the image sensors <b>112</b>, <b>122</b>, <b>132</b> of the cameras <b>11</b>, <b>12</b>, <b>13</b> in the first steerable camera group, the second steerable camera group and the third steerable camera group to be orientated adequately for capturing images, while providing the captured images to an image processing element <b>23</b> through an image input element <b>21</b> where they are distorted, aligned and mosaicked into the image of 180-degree surrounding view to the rear of the vehicle <b>5</b> to the display unit <b>3</b> through an image output element <b>25</b>, as shown in <figref idref="DRAWINGS">FIG. 6</figref>;</li><li id="ul0008-0010" num="0061">step <b>623</b>: when the mode switch <b>4</b> is situated in the ambient surrounding view mode <b>42</b>, enabling the camera control element <b>20</b> to control the cameras <b>10</b>, <b>11</b>, <b>12</b>, <b>13</b> belonging to a first steerable camera group, a second steerable camera group, a third steerable camera group and a fourth steerable camera group, for enabling the image sensors <b>102</b>, <b>112</b>, <b>122</b>, <b>132</b> of the cameras <b>10</b>, <b>11</b>, <b>12</b>, <b>13</b> in the first steerable camera group, the second steerable camera group, the third steerable camera group and the fourth steerable camera group to be orientated adequately for capturing images, while providing the captured images to an image processing element <b>23</b> through an image input element <b>21</b> where they are distorted, aligned and mosaicked into the image of 360-degree surrounding view to the ambient environment of the vehicle <b>5</b> to the display unit <b>3</b> through an image output element <b>25</b>, as shown in <figref idref="DRAWINGS">FIG. 7</figref>;</li><li id="ul0008-0011" num="0062">step <b>624</b>: performing the operation for identifying in which mode a mode switch <b>4</b> is situated, and thereafter, if the mode switch <b>4</b> is identified to be situated in the front surrounding view mode <b>40</b>, the flow will proceed to step <b>625</b>, if the mode switch <b>4</b> is identified to be situated in the rear surrounding view mode <b>41</b>, the flow will proceed to step <b>626</b>, and if the mode switch <b>4</b> is identified to be situated in the ambient surrounding view mode <b>42</b>, the flow will proceed to step <b>627</b>;</li><li id="ul0008-0012" num="0063">step <b>625</b>: when the mode switch <b>4</b> is situated in the front surrounding view mode <b>40</b>, enabling a camera control element <b>20</b> to control the cameras <b>10</b>, <b>11</b>, <b>12</b> belonging to a first steerable camera group, a second steerable camera group and a third steerable camera group, for enabling the image sensors <b>102</b>, <b>112</b>, <b>122</b> of the cameras <b>10</b>, <b>11</b>, <b>12</b> to be orientated adequately for capturing images, while providing the captured images to an image processing element <b>23</b> through an image input element <b>21</b> where they are distorted, aligned and mosaicked into the image of 180-degree surrounding view to the front of the vehicle <b>5</b> to the display unit <b>3</b> through an image output element <b>25</b>, as shown in <figref idref="DRAWINGS">FIG. 5</figref>;</li><li id="ul0008-0013" num="0064">step <b>626</b>: when the mode switch <b>4</b> is situated in the rear surrounding view mode <b>41</b>, enabling the camera control element <b>20</b> to control the cameras <b>11</b>, <b>12</b>, <b>13</b> belonging to the second steerable camera group and the third steerable camera group and the fourth steerable camera group, for enabling the image sensors <b>112</b>, <b>122</b>, <b>132</b> of the cameras <b>11</b>, <b>12</b>, <b>13</b> in the first steerable camera group, the second steerable camera group and the third steerable camera group to be orientated adequately for capturing images, while providing the captured images to an image processing element <b>23</b> through an image input element <b>21</b> where they are distorted, aligned and mosaicked into the image of 180-degree surrounding view to the rear of the vehicle <b>5</b> to the display unit <b>3</b> through an image output element <b>25</b>, as shown in <figref idref="DRAWINGS">FIG. 6</figref>; and</li><li id="ul0008-0014" num="0065">step <b>627</b>: when the mode switch <b>4</b> is situated in the ambient surrounding view mode <b>42</b>, enabling the camera control element <b>20</b> to control the cameras <b>10</b>, <b>11</b>, <b>12</b>, <b>13</b> belonging to a first steerable camera group, a second steerable camera group, a third steerable camera group and a fourth steerable camera group, for enabling the image sensors <b>102</b>, <b>112</b>, <b>122</b>, <b>132</b> of the cameras <b>10</b>, <b>11</b>, <b>12</b>, <b>13</b> in the first steerable camera group, the second steerable camera group, the third steerable camera group and the fourth steerable camera group to be orientated adequately for capturing images, while providing the captured images to an image processing element <b>23</b> through an image input element <b>21</b> where they are distorted, aligned and mosaicked into the image of 360-degree surrounding view to the ambient environment of the vehicle.</li></ul></li></ul>
0066To sum up, the adaptive surrounding view monitoring apparatus and method of the present disclosure are capable of using a steerable camera set <b>1</b> to take a series of images relating to ambient environment of a vehicle <b>5</b> for overcoming the problem of image distortion that is commonly seen in the conventional bird-view images.
0067Moreover, by the use of the image processing element <b>23</b> for processing images while mosaicking the same seamlessly so as to be displayed either an image of 180-degree front or rear surrounding view, or an image of 360-degree ambient surrounding view on the display unit <b>3</b> without any blind spot. Consequently, as the resulting image of 180-degree front or rear surrounding view, or image of 360-degree ambient surrounding view can help eliminating all the blind spots existed surrounding the vehicle due to the obstruction of the vehicle's body structure, no additional device is required for detecting the ambient environment of the vehicle.
0068Furthermore, the driver of the vehicle is able to selected between the front surrounding view mode <b>40</b>, the rear surrounding view mode <b>41</b>, and the ambient surrounding view mode <b>42</b> using the mode switch <b>4</b>, so as to consequently direct the steerable camera set <b>1</b> to be orientated adequately for capturing images to be used in the generating of required image of surrounding view.
0069With respect to the above description then, it is to be realized that the optimum dimensional relationships for the parts of the disclosure, to include variations in size, materials, shape, form, function and manner of operation, assembly and use, are deemed readily apparent and obvious to one skilled in the art, and all equivalent relationships to those illustrated in the drawings and described in the specification are intended to be encompassed by the present disclosure.
Contents6
7 sheets
Sheet 1 Sheet 2 Sheet 3 Sheet 4 Sheet 5 Sheet 6 Sheet 7
Every citation, both ways
| Document | Relation | Office | Cited during |
|---|---|---|---|
| US11126865B2 | Cited by | United States of America | Search report |
| US10109116B2 | Cited by | United States of America | Search report |
| EP3117406A4 | Cited by | European Patent Office (EPO) | Search report |
| US9387813B1 | Cited by | United States of America | Search report |
| US2017024933A1 | Cited by | United States of America | Search report |
| WO2024013585A1 | Cited by | World Intellectual Property Organization (WIPO) | International search |
| US9811741B2 | Cited by | United States of America | Search report |
| US2015015710A1 | Cited by | United States of America | Pre-grant |
| US2012105642A1 | Cited by | United States of America | Pre-grant |
| EP3235685A1 | Cited by | European Patent Office (EPO) | Search report |
| US10609340B2 | Cited by | United States of America | Applicant |
| US10318823B2 | Cited by | United States of America | Search report |
| US10464482B2 | Cited by | United States of America | Search report |
| US10589676B2 | Cited by | United States of America | Applicant |
| US11124118B2 | Cited by | United States of America | Applicant |
| CN105774654A | Cited by | China | Search report |
| US2015103159A1 | Cited by | United States of America | Pre-grant |
| US2014211007A1 | Cited by | United States of America | Pre-grant |
| US2023001922A1 | Cited by | United States of America | Search report |
| EP4202885A4 | Cited by | European Patent Office (EPO) | Search report |
| US9701258B2 | Cited by | United States of America | Search report |
| US11042996B2 | Cited by | United States of America | Search report |
| US9854209B2 | Cited by | United States of America | Search report |
| US10650254B2 | Cited by | United States of America | Applicant |
| US11170227B2 | Cited by | United States of America | Applicant |
| US10065574B2 | Cited by | United States of America | Applicant |
| US2017039782A1 | Cited by | United States of America | Pre-grant |
| US2014267688A1 | Cited by | United States of America | Pre-grant |
| US2003080877A1 | Cites | United States of America | Pre-grant |
| US2003090570A1 | Cites | United States of America | Pre-grant |
| US2003137586A1 | Cites | United States of America | Pre-grant |
| US2005174429A1 | Cites | United States of America | Pre-grant |
| US2005225636A1 | Cites | United States of America | Pre-grant |
| US2005240342A1 | Cites | United States of America | Pre-grant |
| US2006119472A1 | Cites | United States of America | Pre-grant |
| US2006215020A1 | Cites | United States of America | Pre-grant |
| US2007299584A1 | Cites | United States of America | Pre-grant |
| US2008055411A1 | Cites | United States of America | Pre-grant |
| US2009085913A1 | Cites | United States of America | Pre-grant |
| US2009143967A1 | Cites | United States of America | Pre-grant |
| US2010066518A1 | Cites | United States of America | Pre-grant |
| US2010220189A1 | Cites | United States of America | Pre-grant |
| US2011106380A1 | Cites | United States of America | Pre-grant |
| US2011181406A1 | Cites | United States of America | Pre-grant |
| US2012158256A1 | Cites | United States of America | Pre-grant |
| US2013155236A1 | Cites | United States of America | Pre-grant |
| US5680123A | Cites | United States of America | Pre-grant |
| US6535242B1 | Cites | United States of America | Pre-grant |
| US7171027B2 | Cites | United States of America | Pre-grant |
5 priority claims, no other members on record
Priority claims5
| Document | Office | Kind | Date |
|---|---|---|---|
| 100111504 | Taiwan Province of China | A | |
| 100111504 | Taiwan Province of China | A | |
| 100111504 | Taiwan Province of China | – | |
| 100111504 | – | – | – |
| TW20110111504 | – | – | – |
51 transactions on the USPTO file
Allowed after 1 non-final rejection.
- Non-final rejections
- 1
- Final rejections
- 0
- RCEs
- 0
- Appeals
- 0
Over time
Point at a mark for the transactionTransactions
| Event | Code | |
|---|---|---|
| Payment of Maintenance Fee, 8th Year, Large EntityM1552 | M1552 | |
| Payment of Maintenance Fee, 4th Year, Large EntityM1551 | M1551 | |
| Recordation of Patent Grant MailedPGM/ | PGM/ | |
| Patent Issue Date Used in PTA CalculationAllowedPTAC | PTAC | |
| Email NotificationEML_NTR | EML_NTR | |
| Issue Notification MailedAllowedWPIR | WPIR | |
| Dispatch to FDCD1935 | D1935 | |
| Application Is Considered Ready for IssuePILS | PILS | |
| Issue Fee Payment VerifiedN084 | N084 | |
| Issue Fee Payment ReceivedIFEE | IFEE | |
| Correspondence Address ChangeC.ADB | C.ADB | |
| Electronic ReviewELC_RVW | ELC_RVW | |
| Email NotificationEML_NTF | EML_NTF | |
| Mail Notice of AllowanceAllowedMN/=. | MN/=. | |
| Notice of Allowance Data Verification CompletedAllowedN/=. | N/=. | |
| Reasons for AllowanceEX.R | EX.R | |
| Examiner's Amendment CommunicationEX.A | EX.A | |
| Information Disclosure Statement consideredIDSC | IDSC | |
| Interview Summary - Examiner Initiated - TelephonicEXET | EXET | |
| Interview Summary - Examiner InitiatedEXIE | EXIE | |
| Electronic Information Disclosure StatementEIDS. | EIDS. | |
| Information Disclosure Statement (IDS) FiledWIDS | WIDS | |
| Date Forwarded to ExaminerFWDX | FWDX | |
| Response after Non-Final ActionA... | A... | |
| Electronic ReviewELC_RVW | ELC_RVW | |
| Email NotificationEML_NTF | EML_NTF | |
| Mail Non-Final RejectionNon-final rejectionMCTNF | MCTNF | |
| Non-Final RejectionNon-final rejectionCTNF | CTNF | |
| Information Disclosure Statement consideredIDSC | IDSC | |
| Electronic Information Disclosure StatementEIDS. | EIDS. | |
| Information Disclosure Statement (IDS) FiledWIDS | WIDS | |
| Case Docketed to Examiner in GAUDOCK | DOCK | |
| Email NotificationEML_NTR | EML_NTR | |
| PG-Pub Issue NotificationPG-ISSUE | PG-ISSUE | |
| Case Docketed to Examiner in GAUDOCK | DOCK | |
| Case Docketed to Examiner in GAUDOCK | DOCK | |
| Case Docketed to Examiner in GAUDOCK | DOCK | |
| Case Docketed to Examiner in GAUDOCK | DOCK | |
| Case Docketed to Examiner in GAUDOCK | DOCK | |
| Application Dispatched from OIPEOIPE | OIPE | |
| Application Is Now CompleteCOMP | COMP | |
| Email NotificationEML_NTR | EML_NTR | |
| Filing ReceiptFLRCPT.O | FLRCPT.O | |
| Request for Foreign Priority (Priority Papers May Be Included)RQPR | RQPR | |
| Sent to Classification ContractorPGPC | PGPC | |
| Cleared by OIPE CSRL194 | L194 | |
| Information Disclosure Statement consideredIDSC | IDSC | |
| Electronic Information Disclosure StatementEIDS. | EIDS. | |
| Information Disclosure Statement (IDS) FiledWIDS | WIDS | |
| IFW Scan & PACR Auto Security ReviewSCAN | SCAN | |
| Initial Exam Team nnIEXX | IEXX |
6 legal events, as the office reported them to INPADOC
Over the term
Point at a mark for the eventEvents
| Event | Code | |
|---|---|---|
| Maintenance fee paymentMAFP | MAFP | |
| Maintenance fee paymentMAFP | MAFP | |
| Fee payment procedurePAYER NUMBER DE-ASSIGNED (ORIGINAL EVENT CODE: RMPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITYFEPP | FEPP | |
| Fee payment procedurePAYOR NUMBER ASSIGNED (ORIGINAL EVENT CODE: ASPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITYFEPP | FEPP | |
| Information on status: patent grantGrantedPATENTED CASESTCF | STCF | |
| AssignmentAS | AS |
Numbers
- Publication
- 20120249791
- Publication, DOCDB
- 2012249791
- Publication, EPODOC
- US2012249791
- Application
- 13157966
- Application, DOCDB
- 201113157966
- Application, EPODOC
- US201113157966
Titles
- English
- ADAPTIVE SURROUNDING VIEW MONITORING APPARATUS AND METHOD THEREOF
Patent term adjustment
- A delay
- +467 daysthe office missed an examination deadline
- B delay
- +84 dayspendency past three years
- Applicant delay
- −61 days
- Net adjustment
- 490 days
Classification
- CPC, 5
- H04N7/181
- B60R2300/605
- B60R2300/802
- B60R1/23
- B60R1/28
- IPC, 1
- H04N7 18
- USPC, 2
- 348148000
- 348E07085