Device for monitoring around vehicle

Description: TECHNICAL FIELD [Detailed description of the invention]

【0001】

[Technical field to which the invention belongs]

The present invention relates to a vehicle peripheral monitoring device that assists a driver by capturing image information around the vehicle, which is a blind spot for the driver, using an imaging device, performing image processing, and displaying the image information on the display device.

【0002】

[Conventional technology]

Due to the structure of the vehicle, it is inevitable that the vehicle body itself blocks the driver's line of sight and creates a blind spot. However, in order to avoid obstacles and contact with passers-by and other vehicles, it is preferable to reduce such blind spots as much as possible.

【0003】

As a method of reducing the blind spot, an under mirror or the like found in large vehicles is known. Further, as disclosed in Japanese Patent Application Laid-Open No. 11-334470 as a method of covering a wide blind spot range, an image of the blind spot portion is acquired by using an imaging device installed in the vehicle body, and this is subjected to predetermined image processing. There is known a device that displays the image on the display device in the vehicle after the application. By using such a device, it is possible to effectively reduce the blind spot portion and enhance the safety at the time of parking.

【0004】

In addition, there is known a device that acquires image information around a vehicle using a plurality of image pickup devices and then processes the images to convert them into a composite image viewed from the same viewpoint (WO00 / 07373 International Publication). ). According to this device, the driver can easily visually grasp the information around the vehicle.

【0005】

[Problems to be Solved by the Invention]

However, in this device, image information acquired from a plurality of imaging devices is converted into a viewpoint and combined to generate a composite image. Therefore, if there is a three-dimensional object in an imaging region common to the plurality of imaging devices, this is a viewpoint. It causes an error in conversion, and the object may be blurred or disappear when combined. If such a three-dimensional object is in the vicinity of the course of the vehicle, there is a risk of collision or contact as the vehicle advances, so it is rather necessary to display information.

【0006】

Therefore, it is an object of the present invention to provide a vehicle peripheral monitoring device capable of making the driver surely recognize the situation around the vehicle.

【0007】

[Means for solving problems]

In order to solve the above problems, the vehicle peripheral monitoring device according to the present invention is (1) an image pickup device that is installed at a plurality of locations of the vehicle and captures image information of the outside world, and (2) is captured by these image pickup devices. In a vehicle peripheral monitoring device including a control device for processing an image and (3) a display device for displaying an image processed by the control device, (4) a three-dimensional object detecting means for detecting a three-dimensional object existing around the vehicle. Further, the control device generates a composite image of the surroundings of the vehicle from a predetermined viewpoint based on the images captured by a plurality of imaging devices, and the position of the three-dimensional object detected by the three-dimensional object detecting means. It is characterized in that the boundary region at the time of compositing an image is set based on the information.

【0008】

According to the present invention, the final boundary region for compositing the overlapping regions of the images obtained by each imaging device is set based on the detected position information of the three-dimensional object. It is possible to display the existence and position of a three-dimensional object in the display image. For example, based on the position information of the three-dimensional object, the composition process may be performed so that the composite image becomes discontinuous near the position of the three-dimensional object or at a position away from the three-dimensional object. If the composite processing is performed so that the composite image becomes discontinuous at a position away from the three-dimensional object, the image of the three-dimensional object can be appropriately displayed, and the driver can easily recognize the three-dimensional object. On the other hand, if the compositing process is performed so that the composite image becomes discontinuous in the vicinity of the three-dimensional object, the driver can estimate the position of the three-dimensional object from this discontinuous portion. For example, viewpoint conversion and α blending can be used for this synthesis process.

【0009】

The vehicle peripheral monitoring device according to the present invention is a vehicle peripheral monitoring device provided with (1) to (3), and the control device is a composition of the vehicle peripheral view from a predetermined viewpoint based on images captured by a plurality of imaging devices. It may generate an image, and may be characterized in that the boundary region at the time of compositing an image is periodically changed.

【0010】

By periodically changing the boundary region during synthesis, the disappearance and blurring of the three-dimensional object due to the synthesis do not occur continuously during the cycle, and the driver can easily recognize the existence and position of the three-dimensional object. It becomes possible.

【0011】

Further, the vehicle peripheral monitoring device according to the present invention further includes (4') traveling direction predicting means for predicting the traveling direction of the vehicle from the vehicle behavior in the vehicle peripheral monitoring device including (1) to (3). , The control device generates a composite image of the surroundings of the vehicle from a predetermined viewpoint based on the images captured by these a plurality of imaging devices, and synthesizes the composite image based on the traveling direction predicted by the traveling direction predicting means. It may be characterized in that a boundary region at the time of image composition is set.

【0012】

By setting the boundary area at the time of compositing the composite image based on the traveling direction, the existence of the three-dimensional object existing in the traveling direction and its position can be displayed with priority, and the three-dimensional object with a possibility of contact / collision can be displayed. Can be accurately recognized by the driver, and contact and collision between the vehicle and a three-dimensional object in the blind spot can be effectively prevented.

【0013】

Alternatively, the vehicle peripheral monitoring device according to the present invention is a vehicle peripheral monitoring device including (1) to (3), in which (4 ") a course setting for setting an appropriate vehicle course from a target movement position set by the driver. Further provided with means, the control device generates a composite image of the surroundings of the vehicle from a predetermined viewpoint based on the images captured by these a plurality of imaging devices, and the course set by the course setting means. It may be characterized in that the boundary region at the time of compositing the composite image is set based on.

【0014】

In this case, if the driver sets the course first, the existence of the three-dimensional object existing on the target course and its position can be displayed with priority, and when the driver goes on the target course, contact / collision occurs. It is possible to make the driver accurately recognize a possible three-dimensional object, and it is possible to effectively prevent contact and collision between the vehicle and the three-dimensional object in the blind spot.

【0015】

Here, when the traveling direction predicted by the predicting means or the course direction set by the course setting means is other than straight, the control device may use the predicted traveling direction or the vehicle body traveling direction on the set target course direction side. It is preferable to set the position away from the target course direction. With this setting, it is possible to accurately display a three-dimensional object that may cause entrainment when traveling while steering.

【0016】

BEST MODE FOR CARRYING OUT THE INVENTION

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In order to facilitate understanding of the description, the same reference number is assigned to the same component in each drawing as much as possible, and duplicate description is omitted.

【0017】

FIG. 1 is a block diagram showing a configuration of a vehicle peripheral monitoring device according to the present invention. The control device of the vehicle peripheral monitoring device of this embodiment includes a driving support ECU 10 that controls the entire vehicle and an image processing ECU 11 that processes an image signal. Image signals of the cameras 20a to 20d attached to a plurality of locations of the vehicle, for example, four locations described later, are input to the image processing ECU 11. On the other hand, the driving support ECU 10 is connected to the touch panel 31, the display device 32, and the alarm 33, which are input / output devices installed in the driver's seat, as well as the system switch 41, the steering sensor 42, and the shift position sensor 43. The output signals of the wheel speed sensors 44FR, FL, RR, RL, and clearance sonar 45 are input.

【0018】

FIG. 2 is a diagram for explaining a state in which the cameras 20a to 20d are mounted on a vehicle, and FIG. 3 is a diagram for explaining a region (hereinafter, referred to as an imaging region) to be imaged by each of the cameras 20a to 20d.

【0019】

In the present embodiment, the cameras 20a to 20d are wide-angle cameras, one on each of the front, left, rear, and right sides of the vehicle 1. Here, a camera with a horizontal angle of view of 175 degrees and a vertical angle of view of 89 degrees is used. The arrangement of cameras is not limited to one at each location, and two cameras with a horizontal angle of view of about 90 degrees may be arranged at each location in combination. Further, it may be arranged at a corner portion of the vehicle 1, or may be arranged at both the corner portion and the substantially central portion of the front / rear and sides.

【0020】

If the camera arrangement as shown in FIG. 2 is adopted, those imaging areas will be as shown in FIG. 3 when viewed from above the vehicle 1. As is clear from FIG. 3, in order to acquire image information up to the immediate vicinity of the vehicle 1, it is necessary to use cameras with as wide a horizontal angle of view as possible for each of the cameras 20a to 20d. On the other hand, when a camera having such a wide horizontal angle of view is used, an imaging region that overlaps between adjacent cameras (for example, the front camera 20a and the left camera 20d) is generated. In the viewpoint transformation described later, the processing of this overlapping imaging region becomes a problem.

【0021】

The vehicle peripheral monitoring device of the present embodiment generates and displays an image as if viewed from above the vehicle 1 as shown in FIG. 3 from the images acquired by the cameras 20a to 20d, thereby displaying the driver. It is a device that presents obstacles, lanes, etc. in the vicinity of the vehicle or on its path as visual information. This viewpoint conversion process is performed by the image processing ECU 11.

【0022】

Before explaining the specific operation of this embodiment, the principle of this viewpoint conversion will be briefly described with reference to FIG. In Figure 4, the actual camera viewpoint is p_c, P the virtual viewpoint to be converted_iAnd. Each focal length is f_c, F_iThen, each image is a plane P that is separated from the viewpoint by the focal length._c, P_iIt is a projection image formed on the points intersecting at. For example, a plane projection P of any point on each object_cThe image point projected above is the line connecting the point and the viewpoint and the plane P._cIs the intersection of.

【0023】

Here, the image point X in the image acquired by the camera_cThe virtual image plane P_iX above_iConsider the case of converting to. This X_cThe object imaged in is actually the ground plane P_wIf it exists above, its plane P_wTop position X_wIs the viewpoint p_cAnd plane P_cTop point X_cPlane P by extending the line connecting_wIt becomes a point that intersects with. And this X_wAnd virtual viewpoint p_iLine connecting with and virtual screen P_iThe intersection with is the virtual screen P_iImage point X up_cConverted position X_iIs.

【0024】

The image processing ECU 11 further generates images of arbitrary viewpoints, for example, the surroundings of the vehicle from above the vehicle 1 by viewpoint conversion from the images acquired by each camera, and these and the vehicle stored in the memory or the like. By synthesizing the image seen from that position of 1, the image as if it was taken from above the vehicle 1 is combined and displayed on the display device 32.

【0025】

Here, the position of the actual object in the depth direction of the screen cannot be determined from the image information acquired by the actual camera. Therefore the object is X_wNot X_wAnd viewpoint p_cFor example X on the line connecting_w'Projection screen P even if present in position_cSame X above_cIt will be located on top. Therefore, the virtual screen P_iAbove, originally X_i'The image to be projected at the position is X_wSame as X_iWill be projected on. This is a factor that causes the three-dimensional object to be blurred or disappear when the viewpoint is changed.

【0026】

For example, consider the case where the pole 5 exists as an obstacle to the right rear of the vehicle 1 as shown in FIG. When the viewpoint is changed from the image acquired by the right-side camera 20d to the image seen from above the vehicle 1, the image of the pole 5 is originally shown in the shaded area 5A as shown in FIG. 6 (a). It is converted as an image of the area extending diagonally backward from the position of pole 5 of. On the other hand, when the viewpoint is changed from the image acquired by the rear camera 20c to the image seen from above of the vehicle 1, the image of the pole 5 is shown in the shaded area 5B as shown in FIG. 6 (b). It is converted as an image of the area extending diagonally from the original position of the pole 5.

【0027】

When so-called α-blending is used for the entire overlapping area as in the conventional method as a method for processing the overlapping area when superimposing these to generate a display image, both cameras 20c and 20d are used as shown in FIG. 6 (c). Images 5A and 5B converted from pole 5 are projected in the overlapping area 7 of the imaging area, respectively, and since the brightness is halved, both are displayed faintly, so that they are displayed as blurry images. Become. Conventionally, this has a drawback that it is difficult for the driver to recognize a three-dimensional object.

【0028】

The present embodiment is characterized in that a synthesis process is performed so that the driver can easily recognize the three-dimensional object in the displayed image. Hereinafter, each of the four processing methods having different synthesis processing will be described.

【0029】

(First processing method) In this first processing method, the clearance sonar 45 is used to detect the position of a three-dimensional object in the vicinity of the vehicle. The operation ECU 10 sets an optimum composition position based on the detected position information of the three-dimensional object, and instructs the image processing ECU 11 to perform image composition.

【0030】

For example, α-blending is performed only in a part of the overlapping regions including the detected three-dimensional object, and the compositing process is performed using each overlapping viewpoint conversion image. In the other areas of the overlapping area, composition is performed using only the viewpoint conversion image obtained from the image acquired by the camera on the area side of the three-dimensional object.

【0031】

FIG. 7A is a diagram showing an example of a display result when the state shown in FIG. 5 is displayed by using this first processing method. In this processing method, the boundary 71 for image composition is set at a position away from the three-dimensional object based on the position information of the three-dimensional object 5. Then, the result of viewpoint conversion of the image acquired by 20d is displayed for the region 7d on the front side of the boundary 71, and the result of viewpoint conversion of the image acquired by the camera 20c is displayed for the region 7c behind the boundary 71 as it is. As a result, the disappearance and blurring of the three-dimensional object in the displayed image are prevented, and the driver can easily recognize the existence of the three-dimensional object from the discontinuity of the images of the regions 7c and 7d at the boundary 71. Further, if the position of the three-dimensional object is aligned and displayed in the display image based on the detected position information, the visibility is improved.

【0032】

FIG. 7B is a diagram showing the display display result by the modification method of the first processing method. In the area 71'including a three-dimensional object, the images acquired by the cameras 20c and 20d are converted from the viewpoint, and the results are superimposed and displayed by α blending. On the other hand, for the regions 7c and 7d that sandwich the region 71', the results of viewpoint conversion of the images acquired by the cameras 20c and 20d, respectively, are displayed as they are. As a result, the three-dimensional object disappears and is displayed except for the part near the ground, but the driver can easily find the three-dimensional object at the 5x position due to the discontinuity of the boundary between 71'and 7c and 7d. You can recognize that. Further, if the position of the three-dimensional object is aligned and displayed based on the detected position information, the visibility is improved.

【0033】

(Second processing method) This processing method makes it easy to visually recognize a three-dimensional object by periodically switching the boundary region at the time of synthesis. Here, a processing method in which the boundary region has no width and is moved periodically within the overlapping region will be described. However, the boundary region is provided with a width of 1/2 or less of the overlapping region, and α blending is performed within the boundary region. You may perform the synthesis processing by the above.

【0034】

FIG. 8 shows the time change of the display screen by this processing method. At some point, it is assumed that the boundary position 72 of the image composition is located near the limit position of the angle of view of the rear camera 20c as shown in FIG. 8 (a). In this state, the image obtained by viewpoint conversion from the image acquired by the camera 20c is used in most of the overlapping area 7c behind the boundary position 72, and the camera is used in the area 7d in front of the boundary position 72. The image obtained by viewpoint conversion from the image acquired in 20d is used. In this case, the pole 5 shown in FIG. 5 is displayed as a region 5c extending to the right with respect to the vehicle 1 within the region 7c.

【0035】

When the boundary position 72 is rotated around the point 73 and moved backward and set to the position shown in FIG. 8 (b), the region 7c behind the boundary position 72 is the same as in FIG. 8 (a). The pole 5 is displayed as the area 5c extending to the right with respect to the vehicle 1. However, since the image position of the pole 5 is originally different in the region 7d beyond the boundary position 72 of the extension, the image is divided at this boundary position 72, and it is displayed as if it disappeared as an image after that.

【0036】

Further, when the boundary position 72 is rotationally moved around the point 73 and moved backward, as shown in FIG. 8 (c), this time, in the front region 7d portion, as the region 5d where the pole 5 extends rearward. It is displayed, the image is divided at the boundary position 72 part, and it is displayed in a disappeared form in the area 7c beyond that.

【0037】

When the boundary position 72 is positioned near the limit position of the angle of view of the right-side camera 20d as shown in Fig. 8 (d), only the area 7d is displayed in the overlapping area, and as a result, the pole 5 extends backward. It will be displayed as area 5d.

【0038】

By repeating this, for lines drawn on the ground, the displayed image does not change even if the boundary position 72 moves, but for three-dimensional objects and objects existing higher than the ground, The displayed image changes according to the movement of the boundary position 72. Therefore, the driver can easily recognize a three-dimensional object or the like by this change.

【0039】

In FIG. 8, this boundary position 72 can be moved in the order of (a) (b) (c) (d) (a) ..., (a) (b) ( You may go like c) (d) (c) ... Further, instead of scanning the entire overlapping region 72, a plurality of locations may be set as boundary positions, and these may be periodically switched.

【0040】

In both cases of switching and scanning, it is preferable to set the time of one cycle until returning to the original boundary position to about 1 to several seconds. In addition, the driver can arbitrarily set the time of one cycle of image switching, the time of one cycle is variable based on the vehicle speed, and the time of one cycle is automatically adjusted so that the faster the vehicle speed, the shorter the time of one cycle. May be good.

【0041】

(Third processing method) This processing method predicts the traveling direction of the vehicle and sets the boundary position at the time of synthesis according to the predicted traveling direction. FIG. 9 shows a flowchart of control of this process.

【0042】

The control shown in FIG. 9 is mainly executed by the driving assistance ECU 10, and this control routine is repeatedly executed at predetermined intervals from the time when the ignition key of the vehicle is turned on to the time when the ignition key of the vehicle is turned off.

【0043】

As shown in FIG. 9, in step S1, the steering angle and vehicle speed are read by reading the output values of the steering sensor 42 and the wheel speed sensor 44. Then, in step S2, the traveling direction of the vehicle is predicted from this information. In step S3, an appropriate boundary position is set from the traveling direction predicted in this way. In step S4, the image processing ECU 11 synthesizes the viewpoint conversion image of the image obtained by the cameras 20a to 20d based on the set boundary position, and displays the obtained image on the display device 32 in step S5.

【0044】

10 (a) and 10 (b) are diagrams showing an example of the image thus obtained. In both cases, as shown in Fig. 5, the vehicle retreats from the state where the three-dimensional pole 5 is located behind the right rear of the vehicle 1, and Fig. 10 (a) retreats while turning right toward the pole 5. Figure 10 (b) shows the case of retreating while turning left to the opposite side of pole 5.

【0045】

As shown in Fig. 10 (a), when reversing while turning right toward the pole 5, the rear right side of the vehicle may come into contact with the pole 5 closest to it. First, in step S2, the course of each wheel is predicted. In this case, it is the right rear wheel RR that enters the overlapping area 7 most, and its course is 8 in the figure._RRPredicted as shown in. For the driver, the course of this right rear wheel RR 8_RRSince it is desirable to accurately display the above obstacles, etc., the boundary position is set so as to display the image information on the camera 20c side that has acquired the image information on the path. That is, in step S3, the boundary position 74 is set to the position farthest from this predicted course (camera 20d side), and in step S4, an image as shown in FIG. 10A is synthesized.

【0046】

In the display image, the pole 5 which is a three-dimensional object is displayed as a region 5c extending toward the vehicle side at the boundary position 74 side. On the other hand, the course of the right rear wheel RR 8_RRAs for the above, the viewpoint conversion image generated based on the image obtained on the camera 20c side is displayed, so the course 8_RREven if there is an obstacle on the top, the position information can be displayed accurately, and the driver can perform appropriate steering to avoid the obstacle.

【0047】

On the contrary, as shown in Fig. 10 (b), when reversing while turning left to the side opposite to the pole 5 side, the right front part of the vehicle may come into contact with the pole 5 closest to the pole 5 this time. First, in step S2, the course of each wheel is predicted in the same manner. In this case, it is the right front wheel FR that enters the overlapping area 7 most, and its course is 8 in the figure._FRPredicted as shown in. For the driver, the course of this right front wheel FR 8_FRSince it is desirable to accurately display the above obstacles, etc., the boundary position is set so as to display the image information on the camera 20d side that has acquired the image information on the path. That is, in step S3, the boundary position 74 is set to the position farthest from this predicted course (on the camera 20c side), and in step S4, an image as shown in FIG. 10B is synthesized.

【0048】

In the display image, the pole 5 which is a three-dimensional object is displayed as a region 5d extending to the rear of the vehicle on the boundary position 74 side. On the other hand, the course of the right front wheel FR 8_FRAs for the above, the viewpoint conversion image generated based on the image obtained on the camera 20d side is displayed, so the course 8_FREven if there is an obstacle on the top, the position information can be displayed accurately, and the driver can perform appropriate steering to avoid the obstacle.

【0049】

By setting the boundary position according to the predicted course of the vehicle 1 in this way, it is possible to effectively reduce the blind spot in the region where entrainment is likely to occur, especially when steering is involved. In addition, the position of the three-dimensional object can be easily visually recognized.

【0050】

Here, an example in which the boundary at the time of synthesis does not have a width has been described, but it may be set as a boundary region having a constant width, and processing such as α blending may be performed within the boundary region.

【0051】

(Fourth processing method) This processing method sets the boundary position at the time of synthesis according to the target course of the vehicle set by the driver. In the following, a case where parallel parking is supported will be described as an example. FIG. 11 is a diagram for explaining the relationship between the parking position and the vehicle before starting parallel parking, and FIG. 12 is a diagram showing an example of a setting screen for the target parking position.

【0052】

A case where the own vehicle 1 is parked in the space 9a in the space 9 between the already parked vehicles 1a and 1b as shown in FIG. 11 will be described. Here, the space 9b between the space 9a and the vehicle 1a in front of the space 9a is a space for performing a parking operation for parking the vehicle 1 in the space 9a.

【0053】

The driver sets the parking space 9a after stopping the vehicle 1 to the right front of the space 9a where the vehicle 1 is to be parked. Specifically, while actually displaying the parking target position on the display device 32 provided with the touch panel 31 as shown in FIG. 12, it is necessary for the parking space 9a and the front and rear vehicles 1a and 1b to have a margin and parking operation. The entire space 9 including the space 9b is combined and displayed by the image processing ECU 11. Then, the driver sets the target parking position by moving the space 9 using the cursor keys displayed on the touch panel 31. From this target parking position 9a and the current vehicle position, the driving support ECU 10 sets a target course 90 (here, shown as a movement locus of the center of gravity of the vehicle) at the time of parking operation as shown in FIG.

【0054】

FIG. 14 shows a flowchart of control of the synthesis process, and this control is mainly executed by the driving support ECU 10 from the setting of the target parking position to the actual completion of the parking operation (FIG. 14). It is repeatedly executed at predetermined intervals (until it reaches the end of the target course 90 shown in 13.).

【0055】

As shown in FIG. 14, in step S11, the steering angle, vehicle speed, etc. are read by reading the output values of the steering sensor 42, the wheel speed sensor 44, and the shift position sensor 43. Then, in step S12, the position and posture of the vehicle 1 are calculated from this information. In step S13, it is determined whether or not the predicted position and posture of the vehicle 1 are located on the set target course 90. If the deviation from the target course 90 is large, the process proceeds to step S14 and the target course 90 is reset. When a driving operation such as turning back is required due to a large deviation, an appropriate driving operation is displayed on the display device 32 and the driver is alerted by the alarm 33.

【0056】

If it is determined in step S13 that the deviation from the target course 90 is small, or after resetting the target course 90 in step S14, the process proceeds to step S15, which is appropriate according to the position and posture of the vehicle on the course. Set the boundary position. In step S16, the image processing ECU 11 synthesizes the viewpoint conversion image of the image obtained by the cameras 20a to 20d based on the set boundary position, and displays the obtained image on the display device 32 in step S17.

【0057】

Specifically, to explain the above parallel parking as an example, as shown in FIG. 15, when the parallel parking operation is started, that is, when the vehicle center of gravity is at point A on the course 90, the rudder is turned to the left. It is necessary to perform a backward operation. In this case, the image information on the left side is important on the rear left side of the vehicle, and the image information on the rear side is important on the rear right side. Therefore, the rear camera 20c is in the overlapping area of the left side camera 20b and the rear camera 20c on the rear left side of the vehicle. Set the boundary position 75L to the side so that the image acquired in step 1 has priority. On the other hand, in the overlapping area between the rear camera 20c and the right side camera 20d on the rear right side of the vehicle, the boundary position 75R is set to the rear side so that the image acquired by the right side camera 20d has priority.

【0058】

When the vehicle 1 retreats further and reaches the position B as shown in FIG. 16, it is necessary to return the rudder and turn further to the right to retreat. In this case, the rear left image information is important on the rear left side of the vehicle, and the lateral image information is important on the rear right side. Therefore, the left side in the overlapping region of the left side camera 20b and the rear camera 20c on the rear left side of the vehicle. Set the boundary position 75L to the rear side so that the image acquired by the camera 20b has priority. On the other hand, in the overlapping area between the rear camera 20c and the right side camera 20d on the rear right side of the vehicle, the boundary position 75R is set to the side so that the image acquired by the rear camera 20c has priority.

【0059】

When the vehicle 1 further retreats and reaches the position C as shown in FIG. 17, it is necessary to return the rudder and perform a straight retreat operation. In this case, since the image information on the left and right sides of the vehicle is not important, the boundary positions 75L and 75R are set on the side so that the image acquired by the rear camera 20c is prioritized in any overlapping area. To do.

【0060】

By setting the boundary position at the time of image composition according to the position and posture of the vehicle on the course as described above, it is possible to effectively reduce the blind spot in the region where entrainment is likely to occur, especially when steering is involved. is there. In addition, the position of a three-dimensional object can be easily visually recognized.

【0061】

Here, an example in which the boundary at the time of synthesis does not have a width has been described, but it may be set as a boundary region having a constant width, and processing such as α blending may be performed within the boundary region. Further, although the case of parallel parking has been described here as an example, it can also be applied to driving operations such as garage entry and cruise control. Further, it can be suitably applied to a parking support system by automatic steering in which steering is automated and the driving operation is performed by the driver, and an automatic parking system in which steering and driving are completely automated.

【0062】

Two or more of the above four processing methods may be combined and used by switching according to the driving operation. Further, in the above description, a composite example of converting an image acquired by each camera into an image viewed from the same virtual viewpoint has been described, but it may be converted as a so-called panoramic image or a developed image.

【0063】

[Effect of the invention]

As described above, according to the present invention, in the vehicle peripheral monitoring device that displays the situation around the vehicle in an easy-to-see manner for the driver by viewpoint conversion and image composition based on the images acquired by a plurality of cameras, a three-dimensional object is displayed. By setting the boundary position at the time of composition according to the detected and detected position, the predicted course, or the set target course, the disappearance and blurring of the three-dimensional object can be prevented, and important image information can be obtained. It becomes possible to effectively provide the driver.

【0064】

Alternatively, according to the present invention, in the vehicle peripheral monitoring device, by periodically changing or switching the boundary position at the time of composition, it is easy for the driver to visually recognize a three-dimensional object, and important image information can be effectively obtained by the driver. It will be possible to provide to.

[Simple explanation of drawings]

[Figure 1]

It is a block diagram which shows the structure of the vehicle peripheral monitoring apparatus which concerns on this invention.

[Figure 2]

It is a figure explaining the mounting state of the cameras 20a to 20d in a vehicle.

[Fig. 3]

It is a figure explaining the imaging region to image with each camera 20a ~ 20d.

[Fig. 4]

It is a figure explaining the principle of viewpoint transformation.

[Fig. 5]

It is a perspective view which shows the relationship between the vehicle 1 and the pole 5 which exists in the right rear part.

[Fig. 6]

It is a figure explaining the conventional image composition example.

[Fig. 7]

It is a figure which shows an example of the display result at the time of display using the 1st processing method.

[Fig. 8]

It is a figure which shows an example of the display result at the time of display using the 2nd processing method.

[Fig. 9]

It is a flowchart of the control of the third processing method.

[Fig. 10]

It is a figure which shows an example of the display result at the time of display using the 3rd processing method.

[Fig. 11]

It is a figure explaining the relationship between a parking position and a vehicle before parallel parking is started.

[Fig. 12]

It is a figure which shows the setting screen example of the target parking position.

[Fig. 13]

It is a figure explaining the set target course.

[Fig. 14]

It is the flowchart of the control of the 4th processing method.

[Fig. 15]

It is a figure which shows an example of the display result at the time of display using the 4th processing method.

[Fig. 16]

It is a figure which shows the continuation of FIG.

[Fig. 17]

It is a figure which shows the continuation of FIG.

[Explanation of symbols]

1 ... vehicle, 5 ... three-dimensional object (pole), 10 ... driving support ECU, 11 ... image processing ECU, 20 ... camera, 31 ... touch panel, 32 ... display device ,