JP5004923B2 - Vehicle driving support device - Google Patents

Description

  The present invention relates to a vehicle driving support apparatus that displays an image captured by an imaging unit mounted on a vehicle and allows a driver to recognize an object existing around the vehicle.

  Conventionally, by displaying an image in front of the vehicle captured by an infrared camera mounted on the vehicle on a display unit or the like of the navigation system, a pedestrian or the like existing around the vehicle is displayed to the driver when driving at night. 2. Description of the Related Art A driving support apparatus for a vehicle that is recognized is known (see, for example, Patent Document 1).

Further, since the sensitivity characteristics of the image sensor of the camera change depending on the temperature or the like, for example, it is known to perform calibration for periodically resetting the sensitivity of the image sensor in an infrared camera (for example, Patent Document 2). reference). Calibration is performed by closing the camera shutter and blocking light incident on the image sensor, and setting the sensitivity of the image sensor so that the brightness of the image captured by the image sensor in this state is constant. .
JP 2001-6096 A US Pat. No. 6,929,410B2

  As described above, when calibration of the infrared camera is executed while the driver is driving the vehicle while checking the image ahead of the vehicle displayed on the display unit, the shutter is closed and the infrared camera is used. Imaging cannot be performed. In this case, the image on the display unit that has been changed according to the running of the vehicle is not suddenly updated (freeze state), which may cause the driver to feel uncomfortable.

  As described above, the situation in which imaging by the infrared camera becomes impossible occurs not only when the calibration is executed, but also when, for example, the synchronization signal is lost to the infrared camera.

  SUMMARY OF THE INVENTION An object of the present invention is to provide a vehicle driving support device that alleviates a driver from feeling uncomfortable due to a freeze of a display image when imaging by an imaging means becomes impossible.

The present invention related images around a vehicle taken by the imaging means mounted on a vehicle, the improvement of the driving support apparatus for a vehicle to be displayed on the display means mounted on the vehicle,
An image memory for holding data of an image captured by the imaging means;
Object image extraction means for extracting an image portion of the object from the image picked up by the image pickup means, with the vehicle traveling in front of the vehicle as the object;
A relative speed calculating means for calculating a relative speed between an object in real space corresponding to the image portion extracted by the object image extracting means and the vehicle;
When imaging by the imaging unit is disabled, imaging by the imaging unit is disabled based on past image data held in the image memory and the relative speed calculated by the relative speed calculating unit. Pseudo image display means for generating a pseudo image representing an estimated change in the image portion of the object during a long period and displaying the pseudo image on the display means.

And the 1st aspect of the present invention is:
Vehicle speed detecting means for detecting the traveling speed of the vehicle,
The pseudo image display means is a first image obtained by enlarging or reducing the image portion of the object based on the relative speed calculated by the relative speed calculation means among the past image data held in the image memory. The estimated image is synthesized with the second estimated image obtained by enlarging or reducing the change in the background image other than the image portion of the object based on the traveling speed of the vehicle detected by the vehicle speed detecting unit, Characterized by generating images

  According to the present invention, when imaging by the imaging unit becomes impossible, the pseudo image display unit calculates the past image data held in the image memory and the relative speed calculation unit. Based on the relative speed, a pseudo image is generated by estimating and representing a change in the image portion of the object during a period in which imaging by the imaging unit is impossible. In this way, when the pseudo image representing the change in the image portion of the object based on the relative speed between the vehicle and the object is not captured by the imaging unit, Furthermore, it approximates the image picked up by the image pickup means. Therefore, when imaging by the imaging unit becomes impossible, the pseudo image display unit generates the pseudo image and displays the pseudo image on the display unit. The image displayed on the display means can be changed and visually recognized by the driver. For this reason, it is possible to alleviate the driver from feeling uncomfortable due to the freeze of the display image while imaging by the imaging unit is impossible.

Further, according to the present invention, the size of the image portion of the object in the image captured by the imaging unit is enlarged or reduced according to the relative speed between the object and the vehicle. On the other hand, the background image other than the image portion of the object is enlarged or reduced according to the traveling speed of the vehicle. As described above, the method of changing the size differs between the image portion of the object and the other background images.

  Therefore, the pseudo image display means generates the first estimated image enlarged or reduced based on the relative speed between the object and the vehicle for the image portion of the object, and the image of the object For the background image other than the portion, the second estimated image enlarged or reduced based on the traveling speed of the vehicle is generated, and the first estimated image and the second estimated image are combined to generate the second estimated image. Generate a pseudo image. Thereby, the pseudo image can be made closer to the image that is captured when the imaging unit can capture the image, which can further alleviate the driver from feeling uncomfortable.

  In addition, vehicle behavior detection means for detecting at least one of a change in the horizontal direction and a change in the vertical direction of the vehicle as the behavior of the vehicle is provided, and the pseudo image display means includes the first estimation. Changing the position of the first estimated image in the composite image of the image and the second estimated image based on the behavior of the vehicle detected by the vehicle behavior detecting means to generate the pseudo image It is characterized by.

  According to the present invention, the position of the first estimated image in the synthesized image of the first estimated image and the second estimated image according to the behavior of the vehicle detected by the vehicle behavior detecting means. Is changed according to the behavior of the vehicle, and the pseudo image is brought closer to the actual image captured when the imaging unit can capture the image, thereby further mitigating the driver from feeling uncomfortable. be able to.

  In addition, the pseudo image display unit is configured to detect the position of the second estimated image in the combined image of the first estimated image and the second estimated image by the vehicle behavior detecting unit. The pseudo image is generated based on the change.

  According to the present invention, the position of the second estimated image in the composite image of the first estimated image and the second estimated image according to the behavior of the vehicle detected by the vehicle behavior detecting means. Is changed according to the behavior of the vehicle, and the pseudo image is brought closer to the actual image captured when the imaging unit can capture the image, thereby further mitigating the driver from feeling uncomfortable. be able to.

The second aspect of the present invention includes a calibration unit that calibrates the imaging unit with a shutter provided in the imaging unit being closed, and the pseudo image display unit includes: The pseudo image is generated and displayed on the display unit when imaging by the imaging unit becomes impossible due to execution of calibration.

  According to the present invention, the pseudo image is generated by the pseudo image display unit and displayed on the display unit while the calibration unit is calibrating the calibration unit. Therefore, during the calibration of the image pickup means, the image displayed on the display means is in a frozen state, which can alleviate the driver from feeling uncomfortable.

  Embodiments of the present invention will be described with reference to FIGS. FIG. 1 is a configuration diagram of a vehicle driving support apparatus according to the present invention. The vehicle driving support apparatus according to the present invention is configured using an image processing unit 1. The image processing unit 1 includes an infrared camera 2R, 2L that can detect far infrared rays (corresponding to the imaging means of the present invention), and a yaw rate sensor 3 that detects the yaw rate of the vehicle (corresponding to the vehicle behavior detecting means of the present invention). A vehicle speed sensor 4 for detecting the traveling speed of the vehicle (corresponding to the vehicle speed detecting means of the present invention), a brake sensor 5 for detecting the amount of brake operation by the driver, and a steering angle for detecting the steering angle of the steering wheel by the driver. A sensor 6 (corresponding to the vehicle behavior detecting means of the present invention), a pitch angle sensor 7 (corresponding to the vehicle behavior detecting means of the present invention) for detecting the pitch angle of the vehicle, a speaker 8 for alerting by voice, And the display for displaying the image obtained by the infrared cameras 2R and 2L and making the driver visually recognize an object with high possibility of contact Location 9 (corresponding to the display unit of the present invention) is connected.

  Referring to FIG. 2, infrared cameras 2R and 2L are disposed at the front portion of vehicle 10 at a substantially target position with respect to the center portion of vehicle 10 in the vehicle width direction, and two infrared cameras 2R and 2L are arranged. The optical axes are parallel to each other, and the infrared cameras 2R and 2L are fixed at the same height from the road surface. The infrared cameras 2R and 2L have a characteristic that the output level increases (the luminance increases) as the temperature of the imaged object increases. The infrared camera 2R is provided with a shutter 30R for blocking light entering the image sensor, and similarly, the infrared camera 2L is provided with a shutter 30L. Further, the display device 7 is provided so that the screen 7 a is displayed at a front position on the driver side of the front window of the vehicle 10.

  Referring to FIG. 1, an image processing unit 1 is an electronic unit composed of a microcomputer (not shown) and converts analog video signals output from infrared cameras 2R and 2L into digital data. The microcomputer has a function of performing various arithmetic processes by the microcomputer on the image ahead of the vehicle captured in the image memory 28.

  Then, by causing the microcomputer to execute a vehicle driving support program, the microcomputer extracts an image portion of the object in real space from the image captured by the infrared camera 2R. 20. Corresponding image extracting means 21 for extracting an image portion having a correlation with the image portion of the object from the image captured by the infrared camera 2L, the image portion of the object and the corresponding image extracted by the object extracting means 20 Parallax calculating means 22 for calculating the parallax with the image portion extracted by the extracting means 21; distance calculating means 23 for calculating the distance between the object and the vehicle 10 based on the parallax calculated by the parallax calculating means 22; When the relative speed calculating means 24 for calculating the relative speed between the vehicle 10 and the infrared camera 2R, 2L is disabled. Pseudo image display means 25 for generating a similar image and displaying it on the display device 9, calibration means 26 for calibrating the infrared cameras 2R and 2L, and contact determination for determining the possibility of contact between the vehicle 10 and the object It functions as the means 27.

  Then, the object extraction means 20, the relative speed calculation means 24, the pseudo image display means 25, the calibration means 26, the image memory 28, the yaw rate sensor 3, the vehicle speed sensor, which are configured by the image processing unit 1 in this way. 4, the steering angle sensor 6, and the pitch angle sensor 7 constitute a vehicle driving support device of the present invention.

  Here, since the sensitivity characteristics of the imaging devices of the infrared cameras 2R and 2L change depending on the temperature or the like, it is possible to obtain a better image by performing calibration that periodically resets the sensitivity of the imaging device. Then, the calibration unit 26 closes the shutters 30R and 30L of the infrared cameras 2R and 2L so that light incident on the image sensor is blocked, and the image sensor so that the brightness of the image captured in this state is constant. Set the sensitivity of and perform calibration.

  In addition, the calibration is performed not only on the sensitivity characteristics of the image sensor but also on the pan angle error of the infrared cameras 2R and 2L (left and right stereo cameras), for example, by using an object whose distance from the infrared cameras 2R and 2L is known. This can be done by a standard correction or the like.

  As described above, since the calibration is performed with the shutters of the infrared cameras 2R and 2L being closed, it is impossible to image the front of the vehicle 10 with the infrared cameras 2R and 2L during the calibration. In addition, when the input / output of the video synchronization signal between the image processing unit 1 and the infrared cameras 2R and 2L is interrupted, it becomes impossible to image the front of the vehicle 10 by the infrared cameras 2R and 2L.

  Therefore, the image processing unit 1 determines whether or not imaging by the infrared cameras 2R and 2L is possible. When the image processing unit 1 is capable of imaging with the infrared cameras 2R and 2L, the image processing unit 1 performs monitoring processing around the vehicle 10 based on the images captured by the infrared cameras 2R and 2L according to the flowchart shown in FIG. Execute.

  On the other hand, when imaging by the infrared cameras 2R and 2L is impossible, the image processing unit executes a process of generating a pseudo image and displaying it on the display device 9 according to the flowcharts shown in FIGS.

  First, the monitoring process around the vehicle 10 by the image processing unit 1 will be described with reference to the flowchart shown in FIG.

  In STEP 1, the image processing unit 1 inputs analog signals of infrared images output from the infrared cameras 2R and 2L, and in STEP 2, a grayscale image obtained by digitizing the analog signals by A / D conversion is input to the image memory 28. Hold.

  In STEP1 to STEP2, a grayscale image (hereinafter referred to as a right image) obtained by the infrared camera 2R and a grayscale image (hereinafter referred to as a left image) obtained by the infrared camera 2L are acquired. Since the right image and the left image have a shift (parallax) in the horizontal position of the image portion of the same object, the distance from the vehicle 10 to the object in real space can be calculated based on this parallax. it can.

  In subsequent STEP 3, the image processing unit 1 binarizes the right image as a reference image (pixels with luminance equal to or higher than a threshold value are set to “1 (white)”, and pixels smaller than the threshold value are set to “0 (black)”. To generate a binary image. The next STEP 4 to STEP 6 are processes by the object extraction means 20.

  In STEP 4, the object extraction means 20 converts the image portion of each white area included in the binary image into run length data (data of white pixel lines continuous in the x (horizontal) direction of the binary image). Further, the object extraction unit 20 labels a line having a portion overlapping in the y (vertical) direction of the binary image as one image part in STEP 5, and in STEP 6, the labeled image part is an image candidate of the monitoring object. Extract as

  In the next STEP 7, the image processing unit 1 calculates the gravity center G, area S, and circumscribing aspect ratio ASPECT of each image candidate. Note that a specific calculation method is described in detail in the above-mentioned Japanese Patent Application Laid-Open No. 2001-6096, and the description thereof is omitted here. Then, the image processing unit 1 executes the following STEP 8 to STEP 9 and STEP 20 to STEP 22 in parallel.

  In STEP 8, the image processing unit 1 performs identity determination on image portions extracted from binary images based on images taken by the infrared cameras 2R and 2L at predetermined sampling periods, and images of the same object The time-series data of the position of the image portion determined to be (centroid position) is stored in the memory (time tracking). In STEP 9, the image processing unit 1 reads the vehicle speed VCAR detected by the vehicle speed sensor 4 and the yaw rate YR detected by the yaw rate sensor 3, and calculates the turning angle θr of the vehicle 10 by time integration of the yaw rate YR. To do.

  Further, STEP 20 to STEP 21 are processes by the corresponding image extraction means 21. The corresponding image extracting unit 21 selects one of the candidate images of the monitoring target extracted by the target extracting unit 20 in STEP 20 and selects the corresponding search image a (selected from the grayscale image of the right image. Image of the entire region surrounded by the circumscribed rectangle of the candidate image). In subsequent STEP 21, the corresponding image extracting means 20 sets a search area for searching for an image corresponding to the search image from the grayscale image of the left image, and executes a correlation calculation with the search image to extract the corresponding image.

  The subsequent STEP 22 is processing by the parallax calculation means 22. The parallax calculation means 22 calculates the difference between the centroid position of the search image and the centroid position of the corresponding image as the parallax dx, and proceeds to STEP 10.

  In STEP 10, the image processing unit 1 executes “distance calculation processing” in which the distance z between the object in the real space corresponding to the search image and the corresponding image and the vehicle 10 is calculated by the following equation (1).

  Here, z: distance between the object and the vehicle 10, f: focal length of the infrared cameras 2R, 2L, p: pixel pitch of the infrared cameras 2R, 2L, D: baseline length of the infrared cameras 2R, 2L, dx: parallax.

  Subsequent STEP 11 to STEP 15 and STEP 30 are processes by the contact determination means 27. In STEP 11, the contact determination means 27 converts the coordinates (x, y) of the search image 30a and the distance z between the object calculated in STEP 10 and the vehicle 10 into real space coordinates (X, Y, Z). Then, the coordinates of the position of the object in the real space corresponding to the search image 10a are calculated. As shown in FIG. 2, the real space coordinates (X, Y, Z) are set such that the midpoint position of the attachment positions of the infrared cameras 2R, 2L is the origin 0, X is the vehicle width direction of the vehicle 10, and Y Is set in the vertical direction, and Z is set in the front direction of the vehicle 10. The image coordinates are set such that the center of the image is the origin, the horizontal direction is x, and the vertical direction is y.

  In the next STEP 12, the contact determination unit 27 performs a turning angle correction for correcting a positional deviation on the image due to the turning of the vehicle 10. Further, in STEP 13, the contact determination means 27 determines the object and the vehicle from the time series data of the real space position of the same object after turning angle correction obtained from a plurality of images captured within a predetermined monitoring period. A relative movement vector with respect to 10 is calculated.

  The specific calculation method of the real space coordinates (X, Y, Z) and the movement vector of the object is described in detail in the above-mentioned Japanese Patent Application Laid-Open No. 2001-6096, and will not be described here.

  Next, in STEP 14, the contact determination unit 27 determines the possibility of contact between the vehicle 10 and the object, and executes “attention determination processing” for determining whether or not it is necessary to call attention. When it is determined by the “attention call determination process” that it is necessary to call attention, the process branches to STEP 30 to output a warning sound by the buzzer 8 and display a warning on the display device 9. On the other hand, when it is determined by the “attention calling determination process” that it is not necessary to call attention, the process returns to STEP 1 and the image processing unit 1 does not call attention.

  In the “attention determination process”, the image processing unit 1 may check whether the target object is in contact with the host vehicle 10 within a margin time, whether the target object is within the approach determination area set around the host vehicle, Determining whether a target object may enter the proximity determination area from outside the proximity determination area and come into contact with the host vehicle 10, whether the target object is a pedestrian, whether the target object is an artificial structure, etc. Then, it is determined whether or not it is necessary to call attention.

  Note that the specific processing content of the “attention determination processing” is described in detail as “alarm determination processing” in the aforementioned Japanese Patent Laid-Open No. 2001-6069, and thus description thereof is omitted here. In addition, as the alert display, for example, in the image in front of the vehicle 10 displayed on the display device 9, a display that emphasizes the image portion of the object by increasing the brightness is performed.

  Next, processing for generating a pseudo image by the image processing unit 1 and displaying it on the display device 9 will be described according to the flowcharts shown in FIGS.

  The image processing unit 1 reads the latest image data held in the image memory 28 in STEP40. The image memory 28 may hold image data for several tens of frames going back from the present to the past, and when the image is taken by the infrared cameras 2R and 2L, the latest image data is overwritten and 1 is overwritten. You may make it hold | maintain the image data for a frame.

  In subsequent STEP 41, the image processing unit 1 detects the traveling speed VCAR of the vehicle 10 detected by the vehicle speed sensor 4, the yaw rate YR of the vehicle 10 detected by the yaw rate sensor 3, the steering hole of the vehicle 10 detected by the steering angle sensor. The operation angle SA and the pitch angle PR of the vehicle 10 detected by the pitch angle sensor 7 are input.

  In subsequent STEP 42, the image processing unit 1 reads out the data of the real space position (lateral position X, height Y, distance Z) of the object calculated in STEP 11 of FIG. 3 described above from the image data read in STEP 40. In step 43, the frame counter Fct is set to zero. The frame counter Fct is for managing the number of frames of the pseudo image to be displayed.

  In the next STEP 44, the image processing unit 1 determines whether or not the object is a vehicle traveling in front of the vehicle 10 (hereinafter referred to as a forward vehicle). The image processing unit 1 determines that the target object is a forward vehicle when the real space position (lateral position X, height Y, distance Z) of the target object is within a preset determination reference range. When the object is a forward vehicle, the processing of STEP 45 to STEP 48 is executed, and when the object is not a forward vehicle, the process branches to STEP 49.

  As another criterion for determining whether or not the vehicle is a forward vehicle, for example, when the vehicle is recognized as a vehicle in the vehicle exclusion process in pedestrian detection, the object is determined to be a forward vehicle. Also good.

  In STEP 45, the image processing unit 1 increments the frame counter Fct, and in STEP 46, when there are a plurality of objects determined to be the preceding vehicle, the object having the closest distance from the vehicle 10 is extracted. The next STEP 47 is processing by the relative speed calculation means 24. The relative speed calculation means 24 calculates the relative speed between the vehicle 10 and the object from the travel speed of the vehicle 10 and the distance change amount of the object per time. To do.

  Subsequent STEP 48 to STEP 52 and STEP 60 in FIG. 5 are processes by the pseudo image display means 25. The pseudo image display means 25 sets the enlargement / reduction rate α for the image portion of the object in STEP 48, and the enlargement / reduction rate α when the relative speed is positive (a state where the object approaches the vehicle 10) (1 < α), and when the relative speed is negative (the object is moving away from the vehicle 10), the enlargement / reduction ratio α is set to reduction (α <1).

  In the next STEP 49, the pseudo image display means 25 normally defines a distance (front gaze distance) at which most drivers look forward when driving as a fixed distance Zc, and the fixed distance Zc and the vehicle The enlargement / reduction ratio β is determined from the traveling speed of the vehicle. If the vehicle 10 is traveling forward, the enlargement / reduction ratio β is an enlargement setting (1 <β).

  In the subsequent STEP 50 of FIG. 5, the pseudo image display means 25 determines whether or not the image position needs to be corrected based on the behavior of the vehicle 10 based on the vehicle speed VCAR, yaw rate YR, steering angle SA, and pitch angle PA input in STEP 41 of FIG. Judging. If correction of the image position is necessary, the process proceeds to STEP 51. If correction of the image position is not necessary, the process branches to STEP 60.

  Referring to FIG. 6, the pseudo image display means 25 is an image obtained by enlarging or reducing the image portion (OBJ1) of the object of the image data Im_0 held in the image memory 28 at the enlargement / reduction ratio α in STEP 51 (present invention). And an image (corresponding to the second estimated image of the present invention) obtained by enlarging the background image other than the image portion (OBJ1) of the object with the enlargement / reduction ratio β. Im_1 is generated.

  Further, when a change in the left-right direction of the vehicle 10 is observed from the yaw rate YR and the steering angle SA, the pseudo image display means 25 generates a composite image according to the magnitude of the change in the left-right direction. Correction for moving the range Dm0 displayed on the display device 9 of Im_1 in the left-right direction is performed. For example, referring to FIG. 7, when a change in the right direction (right turn) is observed, the range Dm0 displayed on the display device 9 of the composite image Im_1 is determined according to the magnitude of the change in the right direction. The range Dm1 moved by Δh is set as a pseudo image displayed on the display device 9.

  In addition, when a change in the vertical direction of the vehicle 10 is observed from the pitch angle PA, the pseudo image display unit 25 determines the composite image according to the magnitude of the change in the vertical direction. Correction for moving the range Dm0 displayed on the display device 9 of Im_1 in the vertical direction is performed. For example, referring to FIG. 7, when an upward change (uphill) is observed, the range Dm0 displayed on the display device 9 of the composite image Im_1 is set to Δv corresponding to the magnitude of the upward change. A range Dm2 that has been moved by a distance is taken as a pseudo image to be displayed on the display device 9.

  The pseudo image display means 25 holds the pseudo image data generated in this way in the image memory 28 and displays the pseudo image on the display device 9. On the other hand, when it is determined that the correction based on the vehicle behavior is not necessary, the composite image Im_0 using the above-described enlargement / reduction magnifications α and β is generated in STEP 60, and the position of the composite image Im_0 is not corrected, and the composite image Im_0 becomes a pseudo image as it is.

  Then, the pseudo image display means 25 generates and updates a pseudo image by repeatedly executing the processing of STEP 50 to STEP 51 and STEP 60 until the frame counter Fct becomes equal to or larger than the preset number N of frames in subsequent STEP 52.

  As a result, even when the infrared camera 2R, 2L cannot be imaged, as in the case where the calibration of the infrared camera 2R, 2L is performed by the calibration unit 26, the display device 9 responds to the passage of time. A changing pseudo image is displayed. Therefore, the image displayed on the display device 9 is in a frozen state, which can alleviate the driver from feeling uncomfortable.

  Next, when the frame counter Fct is greater than or equal to N in STEP 52, the process proceeds to STEP 53, and the image processing unit 1 determines whether or not the state has returned to the state in which the video synchronization signal has been input. When the video synchronization signal is restored, the image processing unit 1 executes the monitoring process according to the flowchart of FIG. 3 because the infrared cameras 2R and 2L can capture images.

  In STEP 53, when the video synchronization signal has not been restored, it is considered that the infrared cameras 2R and 2L have failed or the video signal input unit of the image processing unit 1 has failed. Therefore, in this case, the process proceeds to STEP 54 and the image processing unit 1 displays an error on the display device 9.

  In the present embodiment, the pseudo image is generated by enlarging or reducing the image portion of the object with the enlargement / reduction ratio α and enlarging the background image other than the image portion of the object with the enlargement factor β. The image portion of the object and the background image may be enlarged or reduced at the same enlargement / reduction ratio.

  In the present embodiment, a change in the behavior of the vehicle is detected by the yaw rate sensor, the steering angle sensor, and the pitch angle sensor, and the position of the object and the background image in the pseudo image are corrected according to the behavior of the vehicle. However, even when this process is not performed, the effects of the present invention can be obtained.

  Further, in the present embodiment, the infrared cameras 2R and 2L are used as the imaging means of the present invention, but a visible camera that captures a visible image may be used.

  In addition, the present invention can be applied not only to the configuration of a stereo camera using two infrared cameras 2R and 2L but also to a configuration using a monocular camera. In this case, the distance from the host vehicle to the object is calculated based on the change in the size of the image portion of the unified object between the images captured at at least two different times, the traveling speed of the host vehicle, and the like. can do.

  Further, in the present embodiment, the calibration unit 26 performs the calibration by closing the shutters of the infrared cameras 2R and 2L. However, even when the calibration is not performed, the imaging by the imaging unit is performed for some reason. If there is a possibility that it becomes impossible, the effect of the present invention can be obtained.

The block diagram of the driving assistance device of the vehicle of this invention. Explanatory drawing of the attachment aspect to the vehicle of the driving assistance device of the vehicle shown in FIG. The flowchart which showed the process sequence in the image processing unit in case the imaging by an infrared camera is possible. The flowchart which showed the process sequence in the image processing unit when imaging with an infrared camera is impossible. The flowchart which showed the process sequence in the image processing unit when imaging with an infrared camera is impossible. Explanatory drawing of the process which produces | generates a pseudo image based on the image data hold | maintained at the image memory. Explanatory drawing of the process which produces | generates a pseudo image based on the behavior of a vehicle.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Image processing unit, 2R, 2L ... Infrared camera, 3 ... Yaw rate sensor, 4 ... Vehicle speed sensor, 5 ... Brake sensor, 6 ... Steering angle sensor, 7 ... Pitch angle sensor, 8 ... Speaker, 9 ... Display apparatus, 20 ... object extraction means, 21 ... corresponding image extraction means, 22 ... parallax calculation means, 23 ... distance calculation means, 24 ... relative speed calculation means, 25 ... pseudo image display means, 26 ... calibration means, 27 ... contact determination means 28 ... Image memory

Claims (4)

In a driving support apparatus for a vehicle that displays an image around the vehicle imaged by an imaging unit mounted on the vehicle on a display unit mounted on the vehicle,
An image memory for holding data of an image captured by the imaging means;
Object image extraction means for extracting an image portion of the object from the image picked up by the image pickup means, with the vehicle traveling in front of the vehicle as the object ;
A relative speed calculating means for calculating a relative speed between an object in real space corresponding to the image portion extracted by the object image extracting means and the vehicle;
When imaging by the imaging unit is disabled, imaging by the imaging unit is disabled based on past image data held in the image memory and the relative speed calculated by the relative speed calculating unit. Pseudo image display means for generating a pseudo image representing an estimated change of the image portion of the object in a certain period, and displaying the pseudo image on the display means ;
Vehicle speed detecting means for detecting the traveling speed of the vehicle,
The pseudo image display means is a first image obtained by enlarging or reducing the image portion of the object based on the relative speed calculated by the relative speed calculation means among the past image data held in the image memory. The estimated image is synthesized with the second estimated image obtained by enlarging or reducing the change in the background image other than the image portion of the object based on the traveling speed of the vehicle detected by the vehicle speed detecting unit, A vehicle driving support device for generating an image . The vehicle driving support device according to claim 1,
Vehicle behavior detection means for detecting at least one of a change in the horizontal direction and a change in the vertical direction of the vehicle as the behavior of the vehicle;
The pseudo image display means is configured to determine a position of the first estimated image in a composite image of the first estimated image and the second estimated image based on the behavior of the vehicle detected by the vehicle behavior detecting means. The vehicle driving support device is characterized in that the pseudo image is generated. The vehicle driving support device according to claim 2 ,
Said pseudo image displaying means based on the position of the second estimated image in the composite image of the first estimated image and said second estimation image, the behavior of the vehicle detected by the vehicle behavior detection means The vehicle driving support device is characterized in that the pseudo image is generated. In a driving support apparatus for a vehicle that displays an image around the vehicle imaged by an imaging unit mounted on the vehicle on a display unit mounted on the vehicle,
An image memory for holding data of an image captured by the imaging means;
Object image extraction means for extracting an image portion of the object from the image picked up by the image pickup means, with the vehicle traveling in front of the vehicle as the object;
A relative speed calculating means for calculating a relative speed between an object in real space corresponding to the image portion extracted by the object image extracting means and the vehicle;
When imaging by the imaging unit is disabled, imaging by the imaging unit is disabled based on past image data held in the image memory and the relative speed calculated by the relative speed calculating unit. Pseudo image display means for generating a pseudo image representing an estimated change of the image portion of the object in a certain period, and displaying the pseudo image on the display means;
A calibration means for calibrating the imaging means, with the shutter provided in the imaging means being closed,
The pseudo image display means generates the pseudo image and displays the pseudo image on the display means when imaging by the imaging means becomes impossible due to the calibration of the imaging means. Support device.