System and method for detecting objects in an autonomous vehicle

TECHNICAL FIELD

The present disclosure relates to vehicles controlled by automated driving systems, particularly those configured to automatically control vehicle steering, acceleration, and braking during a drive cycle without human intervention.

INTRODUCTION

The operation of modern vehicles is becoming more automated, i.e. able to provide driving control with less and less driver intervention. Vehicle automation has been categorized into numerical levels ranging from Zero, corresponding to no automation with full human control, to Five, corresponding to full automation with no human control. Various automated driver-assistance systems, such as cruise control, adaptive cruise control, and parking assistance systems correspond to lower automation levels, while true “driverless” vehicles correspond to higher automation levels.

SUMMARY

An automotive vehicle according to the present disclosure includes a cabin, an optical sensor arranged to capture images of the cabin, a light control system operable to modify an intensity or direction of incident light in the cabin, and a controller. The controller is configured to automatically, in response to an object detection request, control the optical sensor to capture a first image of the cabin with a first incident light configuration, control the light control system to modify an intensity or direction of incident light in the cabin to a second incident light configuration, control the optical sensor to capture a second image of the cabin with the second incident light configuration, detect a change in a shadow between the first image and the second image, infer the presence of an object in the cabin based on the change in shadow, and perform a corrective action based on the presence of the object.

In an exemplary embodiment, the light control system comprises an interior light disposed in the cabin, and the controller is configured to control the light control system by selectively turning the interior light on or off.

In an exemplary embodiment, the cabin is provided with at least one window, the light control system comprises a window shade having an open position and a closed position, and the controller is configured to control the light control system by selectively moving the window shade between the open position and the closed position.

In an exemplary embodiment, the light control system comprises at least one actuator configured to control vehicle steering, acceleration, or braking, and the controller is configured to control the light control system by selectively controlling the actuator to move the vehicle between a first orientation with respect to an ambient light source and a second orientation with respect to an ambient light source.

In an exemplary embodiment, the object detection request is based on an image difference between a reference cabin image captured before a drive cycle and a final cabin image captured after a drive cycle.

In an exemplary embodiment, the corrective action comprises signaling an alert to a user.

A method of detecting an object according to the present disclosure includes providing a detection space with an optical sensor arranged to capture images of the detection space, a light control system operable to modify an intensity or direction of incident light in the detection space, and at least one controller in communication with the optical sensor and the light control system. The method also includes automatically controlling the optical sensor, via the at least one controller, to capture a first image of the detection space with a first incident light configuration. The method additionally includes automatically controlling the light control system, via the at least one controller, to modify an intensity or direction of incident light in the cabin to a second incident light configuration, distinct from the first incident light configuration. The method further includes automatically controlling the optical sensor, via the at least one controller, to capture a second image of the detection space with the second incident light configuration, automatically detecting a change in a shadow between the first image and the second image via the at least one controller, and automatically signaling an alert, via the at least one controller, based on the detected change in shadow.

In an exemplary embodiment, the detection space is provided with at least one window, the light control system comprises a window system operable to modify incident light through the window, and automatically controlling the light control system comprises controlling the window system to modify incident light.

In an exemplary embodiment, automatically detecting a change in shadow comprises performing an image difference algorithm on the first image and the second image.

In an exemplary embodiment, the light control system comprises an interior light disposed in the detection space, and automatically controlling the light control system comprises selectively turning the interior light on or off.

In an exemplary embodiment, the detection space comprises a cabin of an automotive vehicle. In such embodiments, the light control system may comprise at least one actuator configured to control vehicle steering, acceleration, or braking, and automatically controlling the light control system may comprise selectively controlling the actuator to move the vehicle between a first orientation with respect to an ambient light source and a second orientation with respect to the ambient light source.

A detection system for an automotive vehicle according to the present disclosure includes an optical sensor arranged to capture images of a portion of the vehicle, a light control system operable to modify an intensity or direction of incident light on the portion of the vehicle, and a controller. The controller is configured to control the optical sensor to capture a reference image of the portion of the vehicle prior to a drive cycle, control the optical sensor to capture a first image of the portion of the vehicle subsequent the drive cycle, control the light control system to modify an intensity or direction of incident light in the cabin to a second incident light configuration in response to the first image differing from the reference image, control the optical sensor to capture a second image of the cabin with the second incident light configuration, and automatically signal an alert in response to the second image differing from the first image.

Embodiments according to the present disclosure provide a number of advantages. For example, the present disclosure provides a system and method for automatically determining that an object has been left behind in a vehicle, and for taking corrective action when such a determination is made.

The above and other advantages and features of the present disclosure will be apparent from the following detailed description of the preferred embodiments when taken in connection with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

<figref idref="DRAWINGS">FIG. 1</figref> is a schematic diagram of a communication system including an autonomously controlled vehicle according to an embodiment of the present disclosure;

<figref idref="DRAWINGS">FIG. 2</figref> is a schematic block diagram of an automated driving system (ADS) for a vehicle according to an embodiment of the present disclosure;

<figref idref="DRAWINGS">FIG. 3</figref> is a flowchart representation of a method of controlling a vehicle according to an embodiment of the present disclosure; and

<figref idref="DRAWINGS">FIGS. 4A-4D</figref> are representations of images of a vehicle cabin according to an embodiment of the present disclosure.

DETAILED DESCRIPTION

Embodiments of the present disclosure are described herein. It is to be understood, however, that the disclosed embodiments are merely examples and other embodiments can take various and alternative forms. The figures are not necessarily to scale; some features could be exaggerated or minimized to show details of particular components. Therefore, specific structural and functional details disclosed herein are not to be interpreted as limiting, but are merely representative. The various features illustrated and described with reference to any one of the figures can be combined with features illustrated in one or more other figures to produce embodiments that are not explicitly illustrated or described. The combinations of features illustrated provide representative embodiments for typical applications. Various combinations and modifications of the features consistent with the teachings of this disclosure, however, could be desired for particular applications or implementations.

<figref idref="DRAWINGS">FIG. 1</figref> schematically illustrates an operating environment that comprises a mobile vehicle communication and control system <b>10</b> for a motor vehicle <b>12</b>. The communication and control system <b>10</b> for the vehicle <b>12</b> generally includes one or more wireless carrier systems <b>60</b>, a land communications network <b>62</b>, a computer <b>64</b>, a mobile device <b>57</b> such as a smart phone, and a remote access center <b>78</b>.

The vehicle <b>12</b>, shown schematically in <figref idref="DRAWINGS">FIG. 1</figref>, is depicted in the illustrated embodiment as a passenger car, but it should be appreciated that any other vehicle including motorcycles, trucks, sport utility vehicles (SUVs), recreational vehicles (RVs), marine vessels, aircraft, etc., can also be used. The vehicle <b>12</b> includes a propulsion system <b>13</b>, which may in various embodiments include an internal combustion engine, an electric machine such as a traction motor, and/or a fuel cell propulsion system.

The vehicle <b>12</b> also includes a transmission <b>14</b> configured to transmit power from the propulsion system <b>13</b> to a plurality of vehicle wheels <b>15</b> according to selectable speed ratios. According to various embodiments, the transmission <b>14</b> may include a step-ratio automatic transmission, a continuously-variable transmission, or other appropriate transmission. The vehicle <b>12</b> additionally includes wheel brakes <b>17</b> configured to provide braking torque to the vehicle wheels <b>15</b>. The wheel brakes <b>17</b> may, in various embodiments, include friction brakes, a regenerative braking system such as an electric machine, and/or other appropriate braking systems.

The vehicle <b>12</b> additionally includes a steering system <b>16</b>. While depicted as including a steering wheel for illustrative purposes, in some embodiments contemplated within the scope of the present disclosure, the steering system <b>16</b> may not include a steering wheel.

The vehicle <b>12</b> includes a wireless communications system <b>28</b> configured to wirelessly communicate with other vehicles (“V2V”) and/or infrastructure (“V2I”). In an exemplary embodiment, the wireless communication system <b>28</b> is configured to communicate via a dedicated short-range communications (DSRC) channel. DSRC channels refer to one-way or two-way short-range to medium-range wireless communication channels specifically designed for automotive use and a corresponding set of protocols and standards. However, wireless communications systems configured to communicate via additional or alternate wireless communications standards, such as IEEE 802.11 and cellular data communication, are also considered within the scope of the present disclosure.

The propulsion system <b>13</b>, transmission <b>14</b>, steering system <b>16</b>, and wheel brakes <b>17</b> are in communication with or under the control of at least one controller <b>22</b>. While depicted as a single unit for illustrative purposes, the controller <b>22</b> may additionally include one or more other controllers, collectively referred to as a “controller.” The controller <b>22</b> may include a microprocessor or central processing unit (CPU) in communication with various types of computer readable storage devices or media. Computer readable storage devices or media may include volatile and nonvolatile storage in read-only memory (ROM), random-access memory (RAM), and keep-alive memory (KAM), for example. KAM is a persistent or non-volatile memory that may be used to store various operating variables while the CPU is powered down. Computer-readable storage devices or media may be implemented using any of a number of known memory devices such as PROMs (programmable read-only memory), EPROMs (electrically PROM), EEPROMs (electrically erasable PROM), flash memory, or any other electric, magnetic, optical, or combination memory devices capable of storing data, some of which represent executable instructions, used by the controller <b>22</b> in controlling the vehicle.

The controller <b>22</b> includes an automated driving system (ADS) <b>24</b> for automatically controlling various actuators in the vehicle. In an exemplary embodiment, the ADS <b>24</b> is a so-called Level Three automation system. A Level Three system indicates “Conditional Automation”, referring to the driving mode-specific performance by an automated driving system of all aspects of the dynamic driving task with the expectation that the human driver will respond appropriately to a request to intervene.

Other embodiments according to the present disclosure may be implemented in conjunction with so-called Level One or Level Two automation systems. A Level One system indicates “driver assistance”, referring to the driving mode-specific execution by a driver assistance system of either steering or acceleration using information about the driving environment and with the expectation that the human driver perform all remaining aspects of the dynamic driving task. A Level Two system indicates “Partial Automation”, referring to the driving mode-specific execution by one or more driver assistance systems of both steering and acceleration using information about the driving environment and with the expectation that the human driver perform all remaining aspects of the dynamic driving task.

Still other embodiments according to the present disclosure may also be implemented in conjunction with so-called Level Four or Level Five automation systems. A Level Four system indicates “high automation”, referring to the driving mode-specific performance by an automated driving system of all aspects of the dynamic driving task, even if a human driver does not respond appropriately to a request to intervene. A Level Five system indicates “full automation”, referring to the full-time performance by an automated driving system of all aspects of the dynamic driving task under all roadway and environmental conditions that can be managed by a human driver.

In an exemplary embodiment, the ADS <b>24</b> is configured to control the propulsion system <b>13</b>, transmission <b>14</b>, steering system <b>16</b>, and wheel brakes <b>17</b> to control vehicle acceleration, steering, and braking, respectively, without human intervention via a plurality of actuators <b>30</b> in response to inputs from a plurality of sensors <b>26</b>, which may include GPS, RADAR, LIDAR, optical cameras, thermal cameras, ultrasonic sensors, and/or additional sensors as appropriate.

<figref idref="DRAWINGS">FIG. 1</figref> illustrates several networked devices that can communicate with the wireless communication system <b>28</b> of the vehicle <b>12</b>. One of the networked devices that can communicate with the vehicle <b>12</b> via the wireless communication system <b>28</b> is the mobile device <b>57</b>. The mobile device <b>57</b> can include computer processing capability, a transceiver capable of communicating using a short-range wireless protocol, and a visual smart phone display <b>59</b>. The computer processing capability includes a microprocessor in the form of a programmable device that includes one or more instructions stored in an internal memory structure and applied to receive binary input to create binary output. In some embodiments, the mobile device <b>57</b> includes a GPS module capable of receiving GPS satellite signals and generating GPS coordinates based on those signals. In other embodiments, the mobile device <b>57</b> includes cellular communications functionality such that the mobile device <b>57</b> carries out voice and/or data communications over the wireless carrier system <b>60</b> using one or more cellular communications protocols, as are discussed herein. The visual smart phone display <b>59</b> may also include a touch-screen graphical user interface.

The wireless carrier system <b>60</b> is preferably a cellular telephone system that includes a plurality of cell towers <b>70</b> (only one shown), one or more mobile switching centers (MSCs) <b>72</b>, as well as any other networking components required to connect the wireless carrier system <b>60</b> with the land communications network <b>62</b>. Each cell tower <b>70</b> includes sending and receiving antennas and a base station, with the base stations from different cell towers being connected to the MSC <b>72</b> either directly or via intermediary equipment such as a base station controller. The wireless carrier system <b>60</b> can implement any suitable communications technology, including for example, analog technologies such as AMPS, or digital technologies such as CDMA (e.g., CDMA2000) or GSM/GPRS. Other cell tower/base station/MSC arrangements are possible and could be used with the wireless carrier system <b>60</b>. For example, the base station and cell tower could be co-located at the same site or they could be remotely located from one another, each base station could be responsible for a single cell tower or a single base station could service various cell towers, or various base stations could be coupled to a single MSC, to name but a few of the possible arrangements.

Apart from using the wireless carrier system <b>60</b>, a second wireless carrier system in the form of satellite communication can be used to provide uni-directional or bi-directional communication with the vehicle <b>12</b>. This can be done using one or more communication satellites <b>66</b> and an uplink transmitting station <b>67</b>. Uni-directional communication can include, for example, satellite radio services, wherein programming content (news, music, etc.) is received by the transmitting station <b>67</b>, packaged for upload, and then sent to the satellite <b>66</b>, which broadcasts the programming to subscribers. Bi-directional communication can include, for example, satellite telephony services using the satellite <b>66</b> to relay telephone communications between the vehicle <b>12</b> and the station <b>67</b>. The satellite telephony can be utilized either in addition to or in lieu of the wireless carrier system <b>60</b>.

The land network <b>62</b> may be a conventional land-based telecommunications network connected to one or more landline telephones and connects the wireless carrier system <b>60</b> to the remote access center <b>78</b>. For example, the land network <b>62</b> may include a public switched telephone network (PSTN) such as that used to provide hardwired telephony, packet-switched data communications, and the Internet infrastructure. One or more segments of the land network <b>62</b> could be implemented through the use of a standard wired network, a fiber or other optical network, a cable network, power lines, other wireless networks such as wireless local area networks (WLANs), or networks providing broadband wireless access (BWA), or any combination thereof. Furthermore, the remote access center <b>78</b> need not be connected via land network <b>62</b>, but could include wireless telephony equipment so that it can communicate directly with a wireless network, such as the wireless carrier system <b>60</b>.

While shown in <figref idref="DRAWINGS">FIG. 1</figref> as a single device, the computer <b>64</b> may include a number of computers accessible via a private or public network such as the Internet. Each computer <b>64</b> can be used for one or more purposes. In an exemplary embodiment, the computer <b>64</b> may be configured as a web server accessible by the vehicle <b>12</b> via the wireless communication system <b>28</b> and the wireless carrier <b>60</b>. Other computers <b>64</b> can include, for example: a service center computer where diagnostic information and other vehicle data can be uploaded from the vehicle via the wireless communication system <b>28</b> or a third party repository to or from which vehicle data or other information is provided, whether by communicating with the vehicle <b>12</b>, the remote access center <b>78</b>, the mobile device <b>57</b>, or some combination of these. The computer <b>64</b> can maintain a searchable database and database management system that permits entry, removal, and modification of data as well as the receipt of requests to locate data within the database. The computer <b>64</b> can also be used for providing Internet connectivity such as DNS services or as a network address server that uses DHCP or other suitable protocol to assign an IP address to the vehicle <b>12</b>. The computer <b>64</b> may be in communication with at least one supplemental vehicle in addition to the vehicle <b>12</b>. The vehicle <b>12</b> and any supplemental vehicles may be collectively referred to as a fleet.

As shown in <figref idref="DRAWINGS">FIG. 2</figref>, the ADS <b>24</b> includes multiple distinct control systems, including at least a perception system <b>32</b> for determining the presence, location, classification, and path of detected features or objects in the vicinity of the vehicle. The perception system <b>32</b> is configured to receive inputs from a variety of sensors, such as the sensors <b>26</b> illustrated in <figref idref="DRAWINGS">FIG. 1</figref>, and synthesize and process the sensor inputs to generate parameters used as inputs for other control algorithms of the ADS <b>24</b>.

The perception system <b>32</b> includes a sensor fusion and preprocessing module <b>34</b> that processes and synthesizes sensor data <b>27</b> from the variety of sensors <b>26</b>. The sensor fusion and preprocessing module <b>34</b> performs calibration of the sensor data <b>27</b>, including, but not limited to, LIDAR to LIDAR calibration, camera to LIDAR calibration, LIDAR to chassis calibration, and LIDAR beam intensity calibration. The sensor fusion and preprocessing module <b>34</b> outputs preprocessed sensor output <b>35</b>.

A classification and segmentation module <b>36</b> receives the preprocessed sensor output <b>35</b> and performs object classification, image classification, traffic light classification, object segmentation, ground segmentation, and object tracking processes. Object classification includes, but is not limited to, identifying and classifying objects in the surrounding environment including identification and classification of traffic signals and signs, RADAR fusion and tracking to account for the sensor's placement and field of view (FOV), and false positive rejection via LIDAR fusion to eliminate the many false positives that exist in an urban environment, such as, for example, manhole covers, bridges, overhead trees or light poles, and other obstacles with a high RADAR cross section but which do not affect the ability of the vehicle to travel along its path. Additional object classification and tracking processes performed by the classification and segmentation model <b>36</b> include, but are not limited to, freespace detection and high level tracking that fuses data from RADAR tracks, LIDAR segmentation, LIDAR classification, image classification, object shape fit models, semantic information, motion prediction, raster maps, static obstacle maps, and other sources to produce high quality object tracks. The classification and segmentation module <b>36</b> additionally performs traffic control device classification and traffic control device fusion with lane association and traffic control device behavior models. The classification and segmentation module <b>36</b> generates an object classification and segmentation output <b>37</b> that includes object identification information.

A localization and mapping module <b>40</b> uses the object classification and segmentation output <b>37</b> to calculate parameters including, but not limited to, estimates of the position and orientation of vehicle <b>12</b> in both typical and challenging driving scenarios. These challenging driving scenarios include, but are not limited to, dynamic environments with many cars (e.g., dense traffic), environments with large scale obstructions (e.g., roadwork or construction sites), hills, multi-lane roads, single lane roads, a variety of road markings and buildings or lack thereof (e.g., residential vs. business districts), and bridges and overpasses (both above and below a current road segment of the vehicle).

The localization and mapping module <b>40</b> also incorporates new data collected as a result of expanded map areas obtained via onboard mapping functions performed by the vehicle <b>12</b> during operation and mapping data “pushed” to the vehicle <b>12</b> via the wireless communication system <b>28</b>. The localization and mapping module <b>40</b> updates previous map data with the new information (e.g., new lane markings, new building structures, addition or removal of constructions zones, etc.) while leaving unaffected map regions unmodified. Examples of map data that may be generated or updated include, but are not limited to, yield line categorization, lane boundary generation, lane connection, classification of minor and major roads, classification of left and right turns, and intersection lane creation. The localization and mapping module <b>40</b> generates a localization and mapping output <b>41</b> that includes the position and orientation of the vehicle <b>12</b> with respect to detected obstacles and road features.

A vehicle odometry module <b>46</b> receives data <b>27</b> from the vehicle sensors <b>26</b> and generates a vehicle odometry output <b>47</b> which includes, for example, vehicle heading and velocity information. An absolute positioning module <b>42</b> receives the localization and mapping output <b>41</b> and the vehicle odometry information <b>47</b> and generates a vehicle location output <b>43</b> that is used in separate calculations as discussed below.

An object prediction module <b>38</b> uses the object classification and segmentation output <b>37</b> to generate parameters including, but not limited to, a location of a detected obstacle relative to the vehicle, a predicted path of the detected obstacle relative to the vehicle, and a location and orientation of traffic lanes relative to the vehicle. Data on the predicted path of objects (including pedestrians, surrounding vehicles, and other moving objects) is output as an object prediction output <b>39</b> and is used in separate calculations as discussed below.

The ADS <b>24</b> also includes an observation module <b>44</b> and an interpretation module <b>48</b>. The observation module <b>44</b> generates an observation output <b>45</b> received by the interpretation module <b>48</b>. The observation module <b>44</b> and the interpretation module <b>48</b> allow access by the remote access center <b>78</b>. The interpretation module <b>48</b> generates an interpreted output <b>49</b> that includes additional input provided by the remote access center <b>78</b>, if any.

A path planning module <b>50</b> processes and synthesizes the object prediction output <b>39</b>, the interpreted output <b>49</b>, and additional routing information <b>79</b> received from an online database or the remote access center <b>78</b> to determine a vehicle path to be followed to maintain the vehicle on the desired route while obeying traffic laws and avoiding any detected obstacles. The path planning module <b>50</b> employs algorithms configured to avoid any detected obstacles in the vicinity of the vehicle, maintain the vehicle in a current traffic lane, and maintain the vehicle on the desired route. The path planning module <b>50</b> outputs the vehicle path information as path planning output <b>51</b>. The path planning output <b>51</b> includes a commanded vehicle path based on the vehicle route, vehicle location relative to the route, location and orientation of traffic lanes, and the presence and path of any detected obstacles.

A first control module <b>52</b> processes and synthesizes the path planning output <b>51</b> and the vehicle location output <b>43</b> to generate a first control output <b>53</b>. The first control module <b>52</b> also incorporates the routing information <b>79</b> provided by the remote access center <b>78</b> in the case of a remote take-over mode of operation of the vehicle.

A vehicle control module <b>54</b> receives the first control output <b>53</b> as well as velocity and heading information <b>47</b> received from vehicle odometry <b>46</b> and generates vehicle control output <b>55</b>. The vehicle control output <b>55</b> includes a set of actuator commands to achieve the commanded path from the vehicle control module <b>54</b>, including, but not limited to, a steering command, a shift command, a throttle command, and a brake command.

The vehicle control output <b>55</b> is communicated to actuators <b>30</b>. In an exemplary embodiment, the actuators <b>30</b> include a steering control, a shifter control, a throttle control, and a brake control. The steering control may, for example, control a steering system <b>16</b> as illustrated in <figref idref="DRAWINGS">FIG. 1</figref>. The shifter control may, for example, control a transmission <b>14</b> as illustrated in <figref idref="DRAWINGS">FIG. 1</figref>. The throttle control may, for example, control a propulsion system <b>13</b> as illustrated in <figref idref="DRAWINGS">FIG. 1</figref>. The brake control may, for example, control wheel brakes <b>17</b> as illustrated in <figref idref="DRAWINGS">FIG. 1</figref>.

In an autonomous vehicle, and particularly in an autonomous vehicle which may be shared by a plurality of passengers, it may be difficult to determine whether an object has been left behind by a passenger. Whereas in conventional human-driven vehicles, a driver may inspect the cabin to ensure that no objects have been forgotten, autonomous vehicles may not have a human driver.

Referring now to <figref idref="DRAWINGS">FIGS. 3 and 4A-4D</figref>, a method of controlling a vehicle according to the present disclosure is illustrated. The method begins at block <b>100</b> in <figref idref="DRAWINGS">FIG. 3</figref>.

A reference image of an occupant cabin is captured, as illustrated at block <b>102</b>. The reference image refers to an image of the interior of the vehicle which is captured before initiating a drive cycle, e.g. with no occupants in the cabin. An illustrative occupant cabin <b>80</b> of the vehicle <b>12</b>′ is illustrated in <figref idref="DRAWINGS">FIG. 4A</figref>. Among the sensors <b>26</b> is one or more optical cameras <b>26</b>′ arranged to capture images of the occupant cabin <b>80</b>. In an exemplary embodiment, the reference image may be captured by the optical camera <b>26</b>′ based on a command from the controller <b>22</b>. The command may be generated, for example, based on a determination that no occupants are currently in the vehicle <b>12</b>.

The occupant cabin <b>80</b> is additionally provided with at least one light control system <b>82</b>. The light control system <b>82</b> is in communication with or under control of the controller <b>22</b>. The light control system <b>82</b> is operable to modify a light characteristic inside the vehicle, e.g. to modify an intensity or orientation of incident light in the occupant cabin <b>80</b>. As used here, incident light refers to light falling on surfaces in the occupant from one or more interior or exterior light sources. In a first exemplary embodiment, the light control system <b>82</b> includes one or more interior lights disposed in the occupant cabin <b>80</b>, e.g. a dome light. In a second exemplary embodiment, the light control system <b>82</b> includes one or more actuators, e.g. of the actuators <b>30</b>, configured to selectively open or close shades associated with one or more vehicle windows or sunroofs. In a third exemplary embodiment, the light control system <b>82</b> includes one or more electrochromic windows configured to selectively increase opacity to decrease transmitted light from the exterior of the vehicle. Other embodiments may include other light control systems or any combination of the above.

In some embodiments, capturing the reference image of the occupant cabin may include controlling the light control system <b>82</b> to provide a predefined reference light setting prior to capturing the image, e.g. by closing all available window shades and turning on all interior lights of the occupant cabin <b>80</b>.

Subsequent capturing the reference image, a drive cycle is performed, e.g. by picking up one or more passengers and conveying the passengers to their destinations, as illustrated at block <b>104</b>. During such drive cycles, passengers may leave objects in the vehicle, e.g. the object <b>84</b> illustrated in <figref idref="DRAWINGS">FIG. 4B</figref>.

A first post-ride image of the occupant cabin is captured, as illustrated at block <b>106</b>. The first post-ride image may be captured by the optical camera <b>26</b>′ based on a command from the controller <b>22</b>. The command may be generated, for example, based on a determination that no occupants are currently in the vehicle <b>12</b>. The first post-ride image has a first incident light configuration. In some embodiments, capturing the first post-ride image of the occupant cabin may include controlling the light control system <b>82</b> to replicate the predefined reference light setting. In such embodiments, the first incident light configuration therefore corresponds to the reference light setting. In other embodiments, the first incident light configuration corresponds to ambient light, e.g. from external light sources such as the sun or street lights. Based on light sources of the first incident light configuration, the object <b>84</b> will cast a first shadow <b>86</b> in the illustration of <figref idref="DRAWINGS">FIG. 4C</figref>.

An image difference between the first post-ride image and the reference image is determined, as illustrated at block <b>108</b>. This may be performed by any suitable image difference algorithm.

A determination is made of whether the image difference indicates a potential object left behind in the cabin <b>80</b>, as illustrated at operation <b>110</b>. In an exemplary embodiment, a determination may be made that a potential object is left behind in response to the image difference between the first post-ride image and the reference image exceeding a threshold in a region of the occupant cabin <b>80</b>. However, it may be difficult to ascertain, based on a two-dimensional image, whether the difference is due to an object left behind, or due to a stain or discoloration on upholstery or carpet in the cabin <b>80</b>.

If the determination of operation <b>110</b> is negative, i.e. the image difference does not indicate a potential object, then the algorithm ends at block <b>124</b>. The algorithm may then repeat for a subsequent drive cycle. In an exemplary embodiment, the first post-ride image may be re-used as a reference image for a subsequent drive cycle.

If the determination of operation <b>110</b> is positive, then one or more vehicle systems are controlled to change incident light orientation and/or intensity, as illustrated at block <b>112</b>. A second incident light configuration, which is distinct from the first incident light configuration, is thereby generated. In an exemplary embodiment, the light control system <b>82</b> is controlled to change light orientation and/or intensity, e.g. by turning on or off interior lights, opening or closing a window or shade, or modifying opacity of one or more electrochromic windows. In another exemplary embodiment, vehicle actuators <b>30</b> are controlled by the ADS <b>24</b> to reposition the vehicle <b>12</b> in a different orientation relative to ambient light sources, and thereby change the orientation of incident light in the cabin <b>80</b>. Based on light sources of the first incident light configuration, the object <b>84</b> will cast a second shadow <b>86</b>′ in the illustration of <figref idref="DRAWINGS">FIG. 4D</figref>.

A subsequent post-ride image of the occupant cabin is captured, as illustrated at block <b>114</b>. The subsequent post-ride image may be captured by the optical camera <b>26</b>′ based on a command from the controller <b>22</b>. An image difference between the subsequent post-ride image and the first post-ride image is determined, as illustrated at block <b>116</b>. This may be performed by any suitable image difference algorithm.

A determination is made of whether the image difference indicates an object left behind in the cabin <b>80</b>, as illustrated at operation <b>118</b>. In an exemplary embodiment, a determination may be made that an object is left behind in response to the image difference between the subsequent post-ride image and the first post-ride image exceeding a threshold. Such a difference based on the change in incident light is indicative of a three-dimensional object casting a shadow, as illustrated in <figref idref="DRAWINGS">FIG. 4D</figref>, rather than a stain or discoloration.

In response to the determination of operation <b>118</b> being negative, i.e. the image difference does not indicate an object, then the presence of a stain in the occupant cabin may be inferred and corrective action is taken, as illustrated at block <b>120</b>. In an exemplary embodiment, this includes automatically controlling the vehicle <b>12</b>, via the ADS <b>24</b>, to a service facility for cleaning. Other corrective action may also be taken. The algorithm then terminates at block <b>124</b>.

In response to the determination of operation <b>120</b> being positive, i.e. the image difference does indicate an object, then the presence of an object in the occupant cabin may be inferred and corrective action is taken, as illustrated at block <b>122</b>. In an exemplary embodiment, this includes automatically communicating an alert to one or more recent passengers, e.g. by communicating a notification to the passenger's mobile device, honking a vehicle horn, flashing vehicle lights, or any other suitable means of communicating an alert. The algorithm then terminates at block <b>124</b>.

Variations of the above are, of course, possible. As an example, one or more further post-ride images may be captured with other incident light configurations to provide additional precision about the location, shape, size, or other characteristics of objects left behind in the vehicle. As a further example, a size of the object may be calculated based on a length of the shadow cast by the object and knowledge of the location of the light source used to create the shadow. Moreover, similar methods to those discussed above may be implemented to detect objects in other detection spaces, e.g. in vehicle trunks, or indeed in non-automotive settings.

As may be seen the present disclosure provides a system and method for automatically determining that an object has been left behind in a vehicle, and for taking corrective action when such a determination is made.

While exemplary embodiments are described above, it is not intended that these embodiments describe all possible forms encompassed by the claims. The words used in the specification are words of description rather than limitation, and it is understood that various changes can be made without departing from the spirit and scope of the disclosure. As previously described, the features of various embodiments can be combined to form further exemplary aspects of the present disclosure that may not be explicitly described or illustrated. While various embodiments could have been described as providing advantages or being preferred over other embodiments or prior art implementations with respect to one or more desired characteristics, those of ordinary skill in the art recognize that one or more features or characteristics can be compromised to achieve desired overall system attributes, which depend on the specific application and implementation. These attributes can include, but are not limited to cost, strength, durability, life cycle cost, marketability, appearance, packaging, size, serviceability, weight, manufacturability, ease of assembly, etc. As such, embodiments described as less desirable than other embodiments or prior art implementations with respect to one or more characteristics are not outside the scope of the disclosure and can be desirable for particular applications.