Motion sickness reduction for in-vehicle displays

BACKGROUND

0001Vehicles are increasingly being equipped with display devices for various purposes. As examples, the display devices may output information relating to the vehicle's condition, current or past trips, or may be used for entertainment purposes. In general, the displays form a portion of so-called “infotainment” systems of the vehicle. With the growth of self-driving technology, the integration display technology in vehicle cabins is likely to increase, because passengers will be less encumbered by navigation and vehicle-operation duties. Examples of display devices that are integrated into some vehicles or may be integrated into vehicles manufactured in the future include console screens, virtual reality (VR) displays, augmented reality (AR) displays, in-cabin monitors, and displays that can be overlayed on windows, windshields, etc.

SUMMARY

0002Vehicle occupants' use and enjoyment of the video data output by such displays are often diminished due to disruptions in the smoothness of the vehicle's movement. For instance, passengers might experience symptoms of motion sickness (e.g., car sickness) due to jittering or other micro- or macro-movements of the display devices, which are caused by the vehicle hitting bumps or potholes in the road, sharp turns, and other aberrations from smooth linear movement of the vehicle.

0003This disclosure describes system configurations and techniques that leverage a vehicle's in-built or after-market sensor hardware to improve the passenger experience with respect to consuming video data output by in-vehicle displays. Examples of sensor technology that are or can be integrated into vehicles include LiDAR (Light Detection and Ranging), radar, cameras (e.g., still or video cameras), infrared imaging technology, accelerometers, gyroscopes, etc. In many cases, one or more of these sensors form a part of self-driving infrastructure and/or connected vehicle capabilities that are becoming more common with the evolution of vehicle technology.

0004As such, the system configurations of this disclosure, in many cases, leverage sensor technology that is integrated or added after-market to a vehicle for self-driving or connected vehicle purposes, to implement measures that potentially mitigate occupant motion sickness stemming from the use of displays. In various examples, the systems of this disclosure use data generated by the sensors to generate a movement prediction model for the immediate future. In turn, the systems of this disclosure preemptively adjust the video output of the displays to compensate for micro-movements and/or macro-movements that are imminent, according to the movement prediction model. By adjusting the video output(s) to compensate for these movements when or just before these movements occur, the systems of this disclosure maintain at least some level of congruity between the passenger's visual perception of the video output and the physical movements experienced by the passenger, thereby mitigating or potentially eliminating one of the common causes of motion sickness caused by watching video in a moving vehicle.

0005In one example, this disclosure describes a method includes storing, by processing circuitry, a predictive movement model to a memory. The method further includes receiving, by the processing circuitry, road condition information, and detecting, by the processing circuitry, a discrepancy between the received road condition information and a portion of the predictive movement model. The method further includes updating, by the processing circuitry, the predictive movement model based on the received road condition information to correct the discrepancy, and controlling, by the processing circuitry, one or more display devices in communication with the processing circuitry to adjust video data output by the one or more display devices based on the updated predictive movement model.

0006In another example, this disclosure describes a computing system that includes a memory and processing circuitry in communication with the memory. The memory is configured to store a predictive movement model. The processing circuitry is configured to receive road condition information, to detect a discrepancy between the received road condition information and a portion of the predictive movement model, and to update the predictive movement model based on the received road condition information to correct the discrepancy. The processing circuitry is further configured to control one or more display devices in communication with the processing circuitry to adjust video data output by the one or more display devices based on the updated predictive movement model.

0007In another example, this disclosure describes an apparatus that includes means for storing a predictive movement model, and means for receiving road condition information. The apparatus further includes means for detecting a discrepancy between the received road condition information and a portion of the predictive movement model, means for updating the predictive movement model based on the received road condition information to correct the discrepancy, and means for controlling one or more display devices to adjust video data output by the one or more display devices based on the updated predictive movement model.

0008In another example, this disclosure describes a non-transitory computer-readable medium encoded with instructions. The instructions, when executed, cause processing circuitry of a computing system to store a predictive movement model, to receive road condition information, to detect a discrepancy between the received road condition information and a portion of the predictive movement model, to update the predictive movement model based on the received road condition information to correct the discrepancy, and to control one or more display devices to adjust video data output by the one or more display devices based on the updated predictive movement model.

0009The systems of this techniques described in this disclosure provide technical improvements over currently-available technology. As one example, the movement prediction model-based compensation techniques of this disclosure improve the precision of the video output, particularly in the context of the moving vantage point from where the video is viewed, inside of a moving vehicle. As another example, the systems of this disclosure may reduce computing resource usage in some instances of video output correction by utilizing previously-collected data of the currently-navigated road, instead of generating a new movement prediction model. The previously-collected data may be available locally if collected by the same vehicle, or from a cloud-based resource in a crowdsourcing-based implementation.

0010The details of one or more embodiments of the invention are set forth in the accompanying drawings and the description below. Other features, objects, and advantages of the invention will be apparent from the description and drawings, and from the claims.

BRIEF DESCRIPTION OF DRAWINGS

0011<figref idref="DRAWINGS">FIG. 1</figref> is a conceptual diagram illustrating an example operating environment of the techniques of this disclosure.

0012<figref idref="DRAWINGS">FIG. 2</figref> is a block diagram illustrating an example system of this disclosure, in which a server device communicates via a wireless network with multiple automobiles.

0013<figref idref="DRAWINGS">FIGS. 3A-3C</figref> are conceptual diagrams illustrating different examples of a cabin of an automobile of this disclosure.

0014<figref idref="DRAWINGS">FIG. 4</figref> is a block diagram illustrating an example apparatus configured to perform the techniques of this disclosure.

0015<figref idref="DRAWINGS">FIG. 5</figref> is a flowchart illustrating an example process that a computing system may perform, in accordance with one example of the disclosure.

DETAILED DESCRIPTION

0016<figref idref="DRAWINGS">FIG. 1</figref> is a conceptual diagram illustrating an example operating environment of the techniques of this disclosure. As one example, the operating environment of the techniques of this disclosure is automobile <b>10</b>. In one example of the disclosure, automobile <b>10</b> may include components configured to perform various movement prediction model-based display adjustments of this disclosure. Such techniques mitigate or eliminate motion sickness caused by the movements of automobile <b>10</b> altering the user experience of viewing the video output by in-vehicle display device(s) of automobile <b>10</b>. In the example of <figref idref="DRAWINGS">FIG. 1</figref>, automobile <b>10</b> may include sensor hardware <b>12</b> and a computing system <b>14</b>. For ease of illustration, sensor hardware <b>12</b> is illustrated in <figref idref="DRAWINGS">FIG. 1</figref> as a single unit. However, sensor hardware <b>12</b> may incorporate one or more of the various sensors described herein, including any combination of one or multiple LiDAR detectors, radar detectors, still cameras, moving picture cameras, infrared detectors, accelerometers, gyroscopes, and/or others. While the systems of this disclosure are described with reference to automotive applications (as implemented in automobile <b>10</b>), it should be understood that the systems of this disclosure may also be implemented in other contexts.

0017Automobile <b>10</b> may be any type of passenger vehicle. Sensor hardware <b>12</b> may include one or both of original equipment manufacturer (OEM) parts that are integrated into automobile <b>10</b> at the time of factory manufacture, and/or after-market parts that are added to automobile <b>10</b> as a modification subsequently to factory manufacture. Sensor hardware <b>12</b> may be mounted to automobile <b>10</b> or may be integrated in or carried by structure of automobile <b>10</b>, such as bumpers, sides, windshields, or the like.

0018Automobile <b>10</b> also includes or otherwise has operable access to computing system <b>14</b>. Computing system <b>14</b> is described herein as being an onboard computing system integrated into automobile <b>10</b>. Computing system <b>14</b> may include various components, which are not called out individually in <figref idref="DRAWINGS">FIG. 1</figref> for ease of illustration. For instance, computing system <b>14</b> may include processing circuitry and one or more memory devices. Examples of the processing circuitry of computing system <b>14</b> include, but are not limited to, one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), fixed function circuitry, programmable processing circuitry, various combinations of fixed function with programmable processing circuitry, or other equivalent integrated circuitry or discrete logic circuitry. The processing circuitry of computing system <b>14</b> may be the central processing unit (CPU) of automobile <b>10</b>. Some components of computing system <b>10</b> may be specialized hardware, such as integrated and/or discrete logic circuitry that provide specific functionalities, and optionally, that implement parallel processing capabilities with respect to the processing circuitry of computing system <b>14</b>.

0019The processing circuitry of computing system <b>14</b> may execute various types of applications, such as various occupant experience related applications including climate control interfacing applications, entertainment and/or infotainment applications, cellular phone interfaces (e.g., as implemented using Bluetooth® links), stock trackers, vehicle functionality interfacing applications, web or directory browsers, or other applications that enhance the occupant experience within the confines of automobile <b>10</b>. Additionally, the processing circuitry of computing system <b>14</b> may also process data received from sensor hardware <b>12</b>, in order to implement various self-driving capabilities and/or connected vehicle capabilities of automobile <b>10</b>.

0020For example, the processing circuitry of computing system <b>14</b> may use data received from sensor hardware <b>12</b> to provide information to the driver/passengers of automobile <b>10</b> about road conditions or speed, and/or to generate autonomous-driving instructions to be stored and/or relayed to engine components of automobile <b>10</b>. The processing circuitry of computing system <b>14</b> may also generate road condition data, or vehicle location data (e.g., GPS coordinates) to be transmitted directly to other vehicles or to a cloud-based administrative system, to implement various connected vehicle capabilities.

0021The memory devices of computing system <b>14</b> may store instructions for execution of one or more of the applications described, as well as instructions to be translated and related to engine components of automobile <b>10</b> in machine-interpretable formats. The memory devices of computing system <b>14</b> may also store information that is to be transmitted to other vehicles or uploaded to a server in a connected or synchronous vehicle scenario. In various examples, the memory devices of computing system <b>14</b> may include a command buffer to which the processing circuitry of computing system <b>14</b> stores information.

0022The memory devices described herein may include, be, or be part of the total memory for automobile <b>10</b>. The memory devices of computing system <b>14</b> may include one or more computer-readable storage media. Examples of memory devices that computing system <b>14</b> can incorporate include, but are not limited to, a random access memory (RAM), an electrically erasable programmable read-only memory (EEPROM), flash memory, or other medium that can be used to carry or store desired program code in the form of instructions and/or data structures and that can be accessed by a computer or one or more processors (e.g., the processing circuitry described above).

0023In some aspects, the memory devices of computing system <b>14</b> may store instructions that cause the processing circuitry of computing system <b>14</b> to perform the functions ascribed in this disclosure to the processing circuitry. Accordingly, at least one of the memory devices may represent a computer-readable storage medium having instructions stored thereon that, when executed, cause one or more processors (e.g., the processing circuitry) to perform various functions. For instance, at least one of the memory devices is a non-transitory storage medium. The term “non-transitory” indicates that the storage medium is not embodied in a carrier wave or a propagated signal. However, the term “non-transitory” should not be interpreted to mean that the memory devices are non-movable or that the stored contents are static. As one example, at least one of the memory devices described herein can be removed from automobile <b>10</b>, and moved to another device. As another example, memory, substantially similar to one or more of the above-described memory devices, may be inserted into one or more receiving ports of automobile <b>10</b>. In certain examples, a non-transitory storage medium may store data that can, over time, change (e.g., in RAM).

0024Sensor hardware <b>12</b> may include a laser emitter that is configured to emit laser pulses, to implement LiDAR-based capabilities. In these examples, sensor hardware <b>12</b> further includes a receiver to receive laser light reflected off objects near LiDAR components of sensor hardware <b>12</b>. To implement LiDAR-based detection, sensor hardware <b>12</b> measures distance to an object by illuminating the object with pulsed laser light and measuring the reflected pulses. Differences in return times and wavelengths of the reflected pulses can then be used to determine topographical aberrations, such as road curvature, altitude differences, etc.

0025Sensor hardware <b>12</b> may include a global positioning sensor (GPS) or similar sensor to determine the physical location of the sensor and objects sensed from the reflected laser light. Sensor hardware <b>12</b> may be further configured to detect additional information, such as speed, acceleration, angular velocity, orientation, etc. using gyroscope hardware, accelerometer hardware, etc. Computing system <b>14</b> may also implement image-processing capabilities in instances where computing system <b>14</b> invokes camera hardware of sensor hardware <b>12</b> to capture image data. For instance, computing system <b>14</b> may perform image-processing to determine aberrations in road conditions that lie ahead in the path of automobile <b>10</b>.

0026Computing system <b>14</b> may leverage sensor hardware <b>12</b> (which includes one or both of sensor technology that is integrated or added after-market to automobile <b>10</b> for self-driving or connected vehicle purposes), to implement measures that potentially mitigate occupant motion sickness stemming from the use of in-vehicle displays. In various examples, computing system <b>14</b> uses data generated by sensor hardware <b>12</b> to generate a movement prediction model for the immediate future. Computing system <b>14</b> may, in turn, use the movement prediction model to preemptively adjust the video output of the in-vehicle displays to compensate for imminent micro-movements and/or imminent macro-movements of the in-vehicle displays automobile <b>10</b>, according to the movement prediction model. By adjusting the video output(s) to compensate for these movements when or just before these movements of the in-vehicle displays occur, computing system <b>14</b> may maintain at least some level of congruity between the passenger's visual perception of the video output and the physical movements experienced by the passenger, thereby mitigating or potentially eliminating one of the common causes of motion sickness caused by watching video in a moving vehicle (automobile <b>10</b> in this case).

0027In accordance with system configurations of this disclosure, computing system <b>14</b> may use data received from sensor hardware <b>12</b> to generate a movement prediction model that provides details (or approximate predictions thereof) with respect to the near future of the present journey of automobile <b>10</b>. In some instances, such as when computing system <b>14</b> is generating a new movement prediction model without leveraging past-collected or crowdsourced road condition information, computing system <b>14</b> may generate the movement prediction model within a sensing range of sensor hardware <b>12</b>. For instance, computing system <b>14</b> may generate the model to reflect travel conditions from the present location of automobile <b>10</b> to a point that is within the LiDAR, radar, or camera range of sensor hardware <b>12</b>.

0028Another value that computing system <b>14</b> uses as a parameter or argument in generating the movement prediction model is the speed at which automobile <b>10</b> is currently traveling. Although described herein with respect to the scalar quantity of speed, it will be appreciated that in various examples, computing system <b>14</b> may also use the vector quantity of velocity, the scalar quantity of acceleration, etc. as measured by one or more accelerometers and/or gyroscopes included in sensor hardware <b>12</b>. Using the speed at which automobile <b>10</b> is traveling toward a detected road condition, computing system <b>14</b> may generate the movement prediction model to include a predicted length of time until the detected road condition causes an aberration (e.g., micromovement or macromovement) of in-vehicle display(s) of automobile <b>10</b>.

0029Based on the movement prediction model's predicted length of time after which automobile <b>10</b> will encounter an aberration in road conditions, and based on the nature of the road condition that will be encountered, computing system <b>14</b> may formulate the display compensation measure to be implemented. That is, computing system <b>14</b> may determine the way in which the video data is to be adjusted for compensating the aberration so that from the perspective of the viewer the video data does not shift. Computing system <b>14</b> may also determine the time at which to implement the video adjustment. For instance, if computing system <b>14</b> determines that the detected road condition causes an upward jerk in the movement of automobile <b>10</b> (e.g., as may be caused by a speed bump), computing system <b>14</b> may formulate a movement of the video output to remain congruent with the viewer's orientation at the time before automobile <b>10</b> jerks upward upon encountering the aberration in road conditions. As one scenario, if automobile <b>10</b> is traveling at a speed of 36 km/h (10 m/s) and sensor hardware <b>12</b> detects a pothole 10 meters away, the passenger(s) of automobile <b>10</b> will experience the jolt of hitting the pothole after 1 second. By compensating for the aberrations in the movement of automobile <b>10</b> in this way, computing system <b>14</b> implements the techniques of this disclosure to reduce or eliminate one possible cause of motion sickness associated with watching an in-vehicle display while traveling in automobile <b>10</b>.

0030According to various aspects of this disclosure, computing system <b>14</b> may implement the predictive model generation techniques described above using aspects of machine learning. In various examples, computing system <b>14</b> may implement machine learning locally (as described below with respect to <figref idref="DRAWINGS">FIG. 4</figref>) and/or may use machine learning-based information obtained from a remote device (as described below with respect to <figref idref="DRAWINGS">FIG. 2</figref>). In some examples, computing system <b>14</b> may update previously-generated or previously-accessed predictive movement models based on various factors, such as whether the information collected by sensor hardware <b>12</b> is congruent with the already-available movement model, or whether the corrective measures implemented with respect to the video output were effective. For instance, computing system <b>14</b> may elicit user feedback from one or more passengers of automobile <b>10</b> on whether the video output by the in-vehicle display devices caused motion sickness or were not compensated for shake or jitter to the passengers' satisfaction.

0031In examples where computing system <b>14</b> stores previously-generated or pre-loaded predictive movement models locally at automobile <b>10</b>, computing system <b>14</b> may implement the machine learning aspects of this disclosure to update (e.g., to correct) the locally-stored predictive movement model based on the road condition data that was most recently collected by sensor hardware <b>12</b>. In examples in which computing system <b>14</b> obtains predictive movement models from a remote device, such as a cloud-based server, computing system <b>14</b> may communicate the updates (e.g., model corrections) to the server. In this way, computing system <b>14</b> may be configured to participate (using identification of automobile <b>10</b>) in a crowdsourcing-based road condition data collection system administered by a cloud-based server system.

0032<figref idref="DRAWINGS">FIG. 2</figref> is a block diagram illustrating an example system <b>20</b> of this disclosure, in which a server device <b>22</b> communicates via a wireless network <b>16</b> with multiple automobiles <b>10</b>A-<b>10</b>N (“automobiles <b>10</b>”). Automobile <b>10</b> of <figref idref="DRAWINGS">FIG. 1</figref> represents any one of automobiles <b>10</b> illustrated in <figref idref="DRAWINGS">FIG. 2</figref>. Each of automobiles <b>10</b> includes respective sensor hardware <b>12</b>A-<b>12</b>N (“sensor hardware <b>12</b>”). System <b>20</b> of <figref idref="DRAWINGS">FIG. 2</figref> represents an example in which the techniques of this disclosure are implemented in a crowdsourcing-oriented manner. Server device <b>22</b> facilitates the crowdsourcing-based implementations of the techniques of this disclosure with respect to a discrete physical area denoted by traffic zone <b>18</b>. In the example of <figref idref="DRAWINGS">FIG. 2</figref>, server device <b>22</b> may represent a portion or the entirety of a “cloud-based” system for road condition-based display adjustments to mitigate motion sickness. That is, server device <b>22</b> is configured to receive and store road condition information, and communicate portions of information to one or more of automobiles <b>10</b>. In some examples, server device <b>22</b> may also implement various machine learning functionalities to tune or update existing road condition information for traffic zone <b>18</b>. Optionally, server device <b>22</b> may also be configured to implement machine learning functionalities to tune or update predictive movement models for any of automobiles <b>10</b>.

0033Server device <b>22</b> implements various aspects of this disclosure to gather, or crowdsource, road condition information from vehicles traveling through traffic zone <b>18</b>, and to disseminate road condition and (optionally) movement prediction model information to vehicles traveling through traffic zone <b>18</b>. For instance, server device <b>22</b> uses communication unit <b>24</b> to receive information via over wireless network <b>16</b>. It will be appreciated that communication unit <b>24</b> may equip server device <b>22</b> with an either a direct interface or a transitive interface to wireless network <b>16</b>. In cases where communication unit <b>24</b> represents a direct interface to wireless network <b>16</b>, communication unit <b>24</b> may include, be, or be part of various wireless communication hardware, including, but not limited to, one or more of Bluetooth®, 3G, 4G, 5G, or WiFi® radios. In cases where communication unit <b>24</b> represents a first link in a transitive interface to wireless network <b>16</b>, communication unit <b>24</b> may represent wired communication hardware, wireless communication hardware (or some combination thereof), such as any one or any combination of a network interface card (e.g, an Ethernet card and/or a WiFi® dongle), USB hardware, an optical transceiver, a radio frequency transceiver, Bluetooth®, 3G, 4G, 5G, or WiFi® radios, and so on. Wireless network <b>16</b> may also enable the illustrated devices to communicate GPS and/or dGPS, such as location information of one or more of automobiles <b>10</b>.

0034While communication unit <b>24</b> is illustrated as a single, standalone component of server device <b>22</b>, it will be appreciated that, in various implementations, communication unit <b>24</b> may form multiple components, whether linked directly or indirectly. Moreover, portions of communication unit <b>24</b> may be integrated with other components of server device <b>22</b>. At any rate, communication unit <b>24</b> represents network hardware that enables server device <b>22</b> to reformat data (e.g., by packetizing or depacketizing) for communication purposes, and to signal and/or receive data in various formats over wireless network <b>16</b>.

0035Wireless network <b>16</b> may comprise aspects of the Internet or another public network. While not explicitly shown in <figref idref="DRAWINGS">FIG. 2</figref> for ease of illustration purposes, wireless network <b>16</b> may incorporate network architecture comprising various intermediate devices that communicatively link server device <b>22</b> to one or more of automobiles <b>10</b>. Examples of such devices include wireless communication devices such as cellular telephone transmitters and receivers, WiFi® radios, GPS transmitters, etc. Moreover, it will be appreciated that while wireless network <b>16</b> delivers data to automobiles <b>10</b> and collects data from automobiles <b>10</b> using wireless “last mile” components, certain aspects of wireless network <b>16</b> may also incorporate tangibly-connected devices, such as various types of intermediate-stage routers.

0036Communication unit <b>24</b> of server device <b>22</b> is communicatively coupled to processing circuitry <b>26</b> of server device <b>22</b>. Processing circuitry <b>26</b> may be formed in one or more microprocessors, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), digital signal processors (DSPs), processing circuitry (including fixed function circuitry and/or programmable processing circuitry), or other equivalent integrated logic circuitry or discrete logic circuitry. As shown in <figref idref="DRAWINGS">FIG. 2</figref>, processing circuitry <b>26</b> is communicatively coupled to system memory <b>32</b> of server device <b>22</b>.

0037System memory <b>32</b>, in some examples, are described as a computer-readable storage medium and/or as one or more computer-readable storage devices. In some examples, system memory <b>32</b> may include, be, or be part of temporary memory, meaning that a primary purpose of system memory <b>32</b> is not long-term storage. System memory <b>32</b>, in some examples, is described as a volatile memory, meaning that system memory <b>32</b> do not maintain stored contents when the computer is turned off. Examples of volatile memories include random access memories (RAM), dynamic random access memories (DRAM), static random access memories (SRAM), and other forms of volatile memories known in the art.

0038In some examples, system memory <b>32</b> are used to store program instructions for execution by processing circuitry <b>26</b>. System memory <b>32</b>, in one example, are used by logic, software, or applications implemented at server device <b>22</b> to temporarily store information during program execution. System memory <b>32</b>, in some examples, also include one or more computer-readable storage media. Examples of such computer-readable storage media may include a non-transitory computer-readable storage medium, and various computer-readable storage devices. System memory <b>32</b> may be configured to store larger amounts of information than volatile memory. System memory <b>32</b> may further be configured for long-term storage of information. In some examples, system memory <b>32</b> include non-volatile storage elements. Examples of such non-volatile storage elements include magnetic hard discs, optical discs, floppy discs, flash memories, or forms of electrically programmable memories (EPROM) or electrically erasable and programmable (EEPROM) memories.

0039Automobiles <b>10</b> represent vehicles configured to automate one or more tasks associated with vehicle operation. In examples where automobiles <b>10</b> are capable of automating some, if not all of the tasks associated with vehicle operation except for providing input related to destination selection. It will be appreciated that automobiles <b>10</b> are capable of automating various tasks, although not every vehicle of automobiles <b>10</b> may implement automation of each function at all times. That is, in some instances, one or more of automobiles <b>10</b> may disable the automation of certain tasks, e.g., based on a user input to instigate such a disabling of one or more operation tasks.

0040Automobiles <b>10</b> are assumed in the description below as passenger cars, although aspects of this disclosure may apply to any type of vehicle capable of conveying one or more occupants and operating autonomously, such as buses, recreational vehicles (RVs), semi-trailer trucks, tractors or other types of farm equipment, trains, motorcycles, personal transport vehicles, and so on. Each of automobiles <b>10</b> is equipped with communication logic and interface hardware, by which each of each of automobiles <b>10</b> may send and receive data over wireless network <b>16</b>. Each of automobiles <b>10</b> is also equipped with respective sensor hardware <b>12</b>, which enables each of automobiles <b>10</b> to determine road conditions in the surroundings of the respective automobile <b>10</b>.

0041One or more of automobiles <b>10</b> may collect road condition data in traffic zone <b>18</b>, and transmit or “upload” the road condition data to server device <b>22</b>, via wireless network <b>16</b>. For instance, communication unit <b>24</b> may receive data packets from one or more of automobiles <b>10</b>. Communication unit <b>24</b> may decapsulate the packets to obtain respective payload information of the packets. In turn, communication unit <b>24</b> may forward the payloads to processing circuitry <b>26</b>.

0042Processing circuitry <b>26</b> may implement further processing of the payload data of the packets received from automobiles <b>10</b>. For instance, processing circuitry <b>26</b> may determine whether or not a particular payload is applicable to road data collected from within traffic zone <b>18</b>, or from another traffic zone with respect to which server device <b>22</b> is configured to store road condition information. Additionally, processing circuitry <b>26</b> may store portions of decapsulated, processed payloads to system memory <b>32</b>. More specifically, processing circuitry <b>26</b> may store the selected portions of the processed payloads to road condition heuristics buffer <b>34</b>, which is implemented in system memory <b>32</b>.

0043Processing circuitry <b>26</b> of server device <b>22</b> implements various techniques of this disclosure to gather and store road condition information within traffic zone <b>18</b>. In turn, processing circuitry <b>26</b> may invoke communication unit <b>24</b> to transmit portions of the road condition data stored in road condition heuristics buffer to one or more of automobiles <b>10</b>. That is, one or more of automobiles <b>10</b> may obtain, from server device <b>22</b>, crowdsourced road condition information about traffic zone <b>18</b>. As examples, any of automobiles <b>10</b> may obtain road condition information previously uploaded by others of automobiles <b>10</b>, and/or by the same automobile <b>10</b>, and/or by other vehicles that have previously traveled in traffic zone <b>18</b>.

0044In turn, the respective automobile <b>10</b> that acquires the road condition data from server device <b>22</b> over wireless network <b>16</b> may use the road condition data to perform in-vehicle display adjustment measures of this disclosure. In these examples, one or more of automobiles <b>10</b> leverage crowdsourced road condition information for traffic zone <b>18</b>, thereby alleviating or sometimes eliminating the need to use respective sensor hardware <b>12</b> to collect road condition data during the journey. In some examples, automobiles <b>10</b> may tune the road condition data stored to road condition heuristics buffer <b>34</b> by uploading more up-to-date road condition information, if the latest collected road condition data is incongruent with the data available from road condition heuristics buffer <b>34</b>. Processing circuitry <b>26</b> may update the information stored to road condition heuristics buffer <b>34</b> based on the most-recently received road condition data from traffic zone <b>18</b>. In this way, the crowdsourcing-based techniques of this disclosure enable server device <b>22</b> to maintain current road condition information with respect to traffic zone <b>18</b> by leveraging data uploaded by automobiles <b>10</b>.

0045In some examples, server device <b>22</b> may perform certain aspects of generating display adjustments described above with respect to automobile <b>10</b>. Processing circuitry <b>26</b> may invoke model generation unit <b>28</b>, which may use location information of one or more of automobiles <b>10</b> (as determined using GPS coordinates) to determine the location of road conditions that the respective automobiles <b>10</b> may use to adjust displayed video data to improve the user experience of the passengers. In these examples, server device <b>22</b> may provide road condition data of traffic zone <b>18</b> (or portions thereof) to automobiles <b>10</b> before automobiles <b>10</b> encounter the road conditions. In turn, automobiles <b>10</b> may use the road condition data received from server device <b>22</b> to generate video adjustment measures to implement at the time of or before encountering the road conditions.

0046In this way, server device <b>22</b> may leverage the data stored to road condition heuristics buffer <b>34</b> (e.g., data gathered from past trips) to enable automobiles to generate the movement prediction model using already-available data in some areas. These aspects of the system configurations of this disclosure thereby reduce the resource consumption at automobiles <b>10</b> that would otherwise be caused by dynamic model generation throughout the entire duration of every trip. Model generation unit <b>28</b> is shown in <figref idref="DRAWINGS">FIG. 2</figref> using dashed lines to illustrate the optional nature of server device <b>22</b> being configured to perform the functionalities ascribed above to model generation unit <b>28</b>. Model generation unit <b>28</b> may use the most up-to-date information stored to road condition heuristics buffer <b>34</b> to generate the predictive movement model, thereby leveraging the crowdsourcing-based and/or machine learning-based aspects of this disclosure.

0047<figref idref="DRAWINGS">FIGS. 3A-3C</figref> are conceptual diagrams illustrating different examples of a cabin <b>30</b> (called out as cabin <b>30</b>A, <b>30</b>B, and <b>30</b>C, respectively) of automobile <b>10</b>. Each of <figref idref="DRAWINGS">FIGS. 3A-3C</figref> illustrates a use-case scenario in which computing system <b>14</b> may implement motion sickness alleviation (e.g., machine-learning or crowdsourcing-based motion sickness alleviation) techniques of this disclosure. For example, computing system <b>14</b> or server device <b>22</b> may update a predictive movement model based on data received sensor hardware <b>12</b>. In turn, computing system <b>14</b> (whether based on a locally-updated model or an updated model received from server device <b>22</b>) may provide video data adjustments to one or more of in-vehicle displays <b>33</b>A-<b>33</b>C. Cabin <b>30</b> is illustrated in <figref idref="DRAWINGS">FIGS. 3A-3C</figref> without passengers being seated in cabin <b>30</b>, for ease of illustration purposes only.

0048<figref idref="DRAWINGS">FIG. 3A</figref> illustrates an example in which the in-vehicle display <b>33</b>A represents a VR/AR overlay on a windshield of cabin <b>30</b>A of automobile <b>10</b>. In the example of <figref idref="DRAWINGS">FIG. 3A</figref>, in-vehicle display <b>33</b>A presents traffic conditions, such as traffic conditions at a later stage of a present trip that automobile <b>10</b> is traveling on, such as a railway crossing stop. In some examples, the VR/AR overlays may be implemented on windows of cabin <b>30</b>A, either instead of or in addition to the illustrated VR/AR overlay on the windshield.

0049<figref idref="DRAWINGS">FIG. 3B</figref> illustrates an example in which the in-vehicle display <b>33</b>B represents a portion an in-cabin infotainment or entertainment system of cabin <b>30</b>B of automobile <b>10</b>. In the example of <figref idref="DRAWINGS">FIG. 3B</figref>, in-vehicle display <b>33</b>A presents entertainment content, such as a movie or a television show.

0050<figref idref="DRAWINGS">FIG. 3C</figref> illustrates an example in which the in-vehicle display <b>33</b>C represents a portable device brought into cabin <b>30</b>C of automobile <b>10</b> by a passenger (not shown). In the example of <figref idref="DRAWINGS">FIG. 3C</figref>, in-vehicle display <b>33</b>C presents entertainment content. In some examples, the portable device (e.g., tablet computer, smartphone, etc.) that includes in-vehicle display <b>33</b>C may communicate with computing system <b>14</b> via wired or wireless means, such as USB, Bluetooth®, or others. In any of the scenarios illustrated in <figref idref="DRAWINGS">FIGS. 3A-3C</figref>, computing system <b>14</b> may leverage predictive movement information generated using machine learning-based techniques of this disclosure to adjust the image or video data output by in-vehicle displays <b>30</b>, to alleviate or potentially eliminate stimuli that cause motion sickness in passengers of automobile <b>10</b>.

0051<figref idref="DRAWINGS">FIG. 4</figref> is a block diagram illustrating an example apparatus configured to perform the techniques of this disclosure. In particular, <figref idref="DRAWINGS">FIG. 4</figref> shows an portions of computing system <b>14</b> of <figref idref="DRAWINGS">FIG. 1</figref> in more detail. Again, in some examples, computing system <b>14</b> may be part of automobile <b>10</b>. However, in other examples, computing system <b>14</b> may be a stand-alone system or may be integrated into other devices for use in other applications which may benefit from pose estimation. It will be understood that, in some examples, various functionalities attributed to server device <b>22</b> in the description of <figref idref="DRAWINGS">FIG. 2</figref> above may be implemented by computing system <b>14</b>. Again, various functionalities of this disclosure may be implemented locally at automobile <b>10</b> (e.g., by computing system <b>14</b>) or remotely (e.g., by server device <b>22</b> of <figref idref="DRAWINGS">FIG. 2</figref>).

0052In the example of <figref idref="DRAWINGS">FIG. 4</figref>, predictive movement model generation unit <b>42</b> includes a pre-processing unit <b>44</b>, a machine learning unit <b>46</b>, and a post-processing unit <b>52</b>. Predictive movement model generation unit <b>42</b> is configured to receive road condition information from sensor hardware <b>12</b>. Pre-processing unit <b>44</b> is configured to make the unstructured raw input (i.e., road conditions and/or the speed at which automobile <b>10</b> is traveling) into structuralized data that can be processed by other components of predictive movement model generation unit <b>42</b> and/or of computing system <b>14</b>.

0053Pre-processing unit <b>44</b> may be configured to provide the structuralized data to machine learning unit <b>46</b>. Machine learning unit <b>46</b> may implement various forms of machine learning technology, including, but not limited to, artificial neural networks, deep learning, support vector machine technology, Bayesian networks, etc. Using the structuralized data obtained from pre-processing unit <b>44</b>, machine learning unit <b>46</b> may perform comparison operations with respect to predictive model <b>48</b>. If machine learning unit <b>46</b> detects a discrepancy between any of the structuralized data received from pre-processing unit <b>44</b> and the road conditions reflected in predictive model <b>48</b>, machine learning unit <b>46</b> may update the data of predictive model <b>48</b> to incorporate the more up-to-date road condition information of traffic zone <b>18</b>. In this way, machine learning unit <b>46</b> implements dynamic model generation or model updating operations of this disclosure to use and to share updates to obsolete road condition information at traffic zone <b>18</b>.

0054Post-processing unit <b>52</b> may obtain the updated version of predictive model <b>48</b>, and convert the data of predictive model <b>48</b> into final output. For example, post-processing unit <b>52</b> may be configured to translate predictive model <b>48</b> into one or more machine-readable formats. In various examples, predictive movement model generation unit <b>42</b> may provide the output generated by post-processing unit <b>52</b> to one or more display operation applications <b>56</b>. Display operation applications <b>56</b> may implement various techniques of this disclosure to adjust the video output of in-cabin displays <b>30</b> to compensate for jitter, shakes, or other micro/macro movements of automobile <b>10</b> due to aberrations in road conditions.

0055In some examples, one or more autonomous driving applications <b>58</b> may use the predictive model information received from post-processing unit <b>52</b> to determine autonomous driving actions to be taken, based on road conditions. Autonomous driving applications <b>58</b> are illustrated in <figref idref="DRAWINGS">FIG. 4</figref> using a dashed-line border to indicate that the use of predictive movement models of this disclosure need not necessarily be used for autonomous driving operations, and indeed, may be implemented both in vehicles that have and in vehicles that do not have autonomous driving capabilities.

0056Computing system <b>14</b> includes processing circuitry <b>62</b> in communication with memory <b>64</b>. Processing circuitry <b>62</b> may be implemented as fixed-function processing circuitry, programmable processing circuitry, or any combination thereof. Fixed-function circuitry refers to circuits that provide particular functionality and are preset on the operations that can be performed. Programmable processing circuitry refers to circuits that can programmed to perform various tasks and provide flexible functionality in the operations that can be performed. For instance, programmable processing circuitry may represent hardware that executes software or firmware that cause programmable circuits to operate in the manner defined by instructions of the software or firmware. Fixed-function circuitry may execute software instructions (e.g., to receive parameters or output parameters), but the types of operations that the fixed-function processing circuits perform are generally immutable. In some examples, one or more of the units may be distinct circuit blocks (fixed-function or programmable), and in some examples, the one or more units may be integrated circuits.

0057Processing circuitry <b>62</b> may be configured to execute a set of instructions in predictive movement model generation unit <b>42</b> to perform various techniques of this disclosure. The instructions that define predictive movement model generation unit <b>42</b> may be stored in memory <b>64</b>. In some examples, the instructions that define predictive movement model generation unit <b>42</b> may be downloaded to the memory <b>64</b> over a wired or wireless network.

0058In some examples, memory <b>64</b> may be a temporary memory, meaning that a primary purpose of memory <b>64</b> is not long-term storage. Memory <b>64</b> may be configured for short-term storage of information as volatile memory and therefore not retain stored contents if powered off. Examples of volatile memories include random access memories (RAM), dynamic random-access memories (DRAM), static random-access memories (SRAM), and other forms of volatile memories known in the art.

0059Memory <b>64</b> may include one or more non-transitory computer-readable storage mediums. Memory <b>64</b> may be configured to store larger amounts of information than typically stored by volatile memory. Memory <b>64</b> may further be configured for long-term storage of information as non-volatile memory space and retain information after power on/off cycles. Examples of non-volatile memories include magnetic hard discs, optical discs, flash memories, or forms of electrically programmable memories (EPROM) or electrically erasable and programmable (EEPROM) memories. Memory <b>64</b> may store program instructions (e.g., predictive movement model <b>42</b>) and/or information (e.g., predictive model <b>48</b>) that, when executed, cause processing circuitry <b>62</b> to perform the techniques of this disclosure.

0060Processing circuitry <b>62</b> may store the most up-to-date version of predictive model <b>48</b> to road condition heuristics store <b>66</b>. In various examples, road condition heuristics store <b>66</b> may be positioned within automobile <b>10</b>, or may be a remote store, such as in the case of road condition heuristics buffer <b>34</b> of <figref idref="DRAWINGS">FIG. 2</figref>. In either case, processing circuitry <b>62</b> may use communications hardware, such as an communications bus (if road condition heuristics store <b>66</b> is positioned within automobile <b>10</b>) or a network interface (e.g., interface hardware), such as 3G, 4G, 5G, WiFi®, etc. in cases where road condition heuristics store <b>66</b> is positioned at a remote location.

0061<figref idref="DRAWINGS">FIG. 5</figref> is a flowchart illustrating an example process <b>70</b> that computing system <b>54</b> may perform, in accordance with one example of the disclosure. One or more processors, such as processing circuitry <b>62</b> of computing system <b>14</b> may be configured to perform the techniques shown in <figref idref="DRAWINGS">FIG. 5</figref>. As described above, in some examples, computing system <b>14</b> may be part of automobile <b>10</b>. In this example, automobile <b>10</b> may be configured to use the predictive movement model information produced by computing system <b>14</b> to adjust video outputs of in-vehicle display devices. However, the techniques of this disclosure are not so limited. Computing system <b>14</b> may be configured to perform the techniques of <figref idref="DRAWINGS">FIG. 5</figref> for autonomous driving decisions or the like.

0062In one example of the disclosure, computing system <b>14</b> may receive road condition information from sensor hardware <b>12</b> (<b>72</b>). In turn, computing system <b>14</b> may further obtain a previously-generated predictive movement model with respect to the present location of automobile <b>10</b> (<b>74</b>). In various examples, computing system <b>14</b> may obtain the predictive movement model from a local storage device, such as memory <b>64</b>, or from a remote device, such as from server device <b>22</b>. In various examples, computing system <b>14</b> may control one or more display devices (e.g., one or more of in-vehicle displays <b>33</b>) in communication with processing circuitry of computing system <b>14</b> to adjust video data output by the one or more display devices based on the updated predictive movement model.

0063In turn, computing system <b>14</b> may detect a discrepancy between the road condition information from sensor hardware <b>12</b> and the predictive movement model (<b>76</b>). For example, computing system <b>14</b> may invoke machine learning unit <b>46</b> to perform the comparison operations used to detect the discrepancy. Computing system <b>14</b>, such as by invoking machine learning unit <b>46</b>, may update the predictive movement model using the road condition information received from sensor hardware <b>12</b> (<b>78</b>). That is, computing system <b>14</b> may update the predictive movement model in response to the detected discrepancy between the dynamically-collected road condition information and the predictive movement model.

0064In turn, computing system <b>14</b> may store the updated predictive movement model (<b>82</b>). In various examples, computing system <b>14</b> may store the updated predictive movement model locally (e.g. to memory <b>64</b>) or to a remote location (e.g., by transmitting the updated predictive movement model to server device <b>22</b>). As shown in <figref idref="DRAWINGS">FIG. 5</figref>, computing system <b>14</b> may implement process <b>70</b> in an iterative manner (returning from step <b>82</b> to step <b>72</b>), such as by iteratively performing process <b>70</b> at different locations on a journey of automobile <b>10</b>.

0065Example 1: A computing system comprising a memory configured to store a predictive movement model; and processing circuitry in communication with the memory, the processing circuitry configured to: receive road condition information; detect a discrepancy between the received road condition information and a portion of the predictive movement model; update the predictive movement model based on the received road condition information to correct the discrepancy; and control one or more display devices in communication with the processing circuitry to adjust video data output by the one or more display devices based on the updated predictive movement model.

0066Example 2: The computing system of Example 1, wherein to update the predictive movement model, the processing circuitry is configured to implement machine learning.

0067Example 3: The computing system of either Example 1 or Example 2, wherein to control the one or more display devices to adjust the video data, the processing circuitry is configured to: determine a time of a physical movement of the one or more display devices; and control the one or more display devices to adjust the video data output at or before the time of the physical movement.

0068Example 4: The computing system of any of Examples 1-3, wherein the processing circuitry is in communication with sensor hardware, and wherein to receive the road condition information, the processing circuitry is configured to receive the road condition information from the sensor hardware.

0069Example 5: The computing system of Example 4, wherein the received road condition information comprises road condition information associated with a present location of a vehicle to which the sensor hardware is coupled.

0070Example 6: The computing system of Example 5, wherein the received road condition information associated with the present location of the vehicle indicates an aberration in a road on which the vehicle is traveling.

0071Example 7: The computing system of Example 6, wherein the processing circuitry is in communication with one or more display devices, and wherein the processing circuitry is further configured to control the one or more display devices to adjust video data output by the one or more display devices to reduce motion sickness caused by movement of the one or more display devices due to the aberration in the road on which the vehicle is traveling.

0072Example 8: The computing system of any of Examples 4-7, wherein the sensor hardware comprises one or more of a LiDAR device, a radar device, a gyroscope, or an accelerometer.

0073Example 9: The computing system of any of Examples 1-8, wherein the one or more display devices comprise one or more of a display of an in-vehicle infotainment system, a display device configured to output a virtual reality overlay, a display device configured to output an augmented reality overlay, or a portable device that is communicatively coupled to the computing system via interface hardware of a vehicle that includes the computing system.

0074Example 10: An apparatus comprising: means for storing a predictive movement model; means for receiving road condition information; means for detecting a discrepancy between the received road condition information and a portion of the predictive movement model; means for updating the predictive movement model based on the received road condition information to correct the discrepancy; and means for controlling one or more display devices to adjust video data output by the one or more display devices based on the updated predictive movement model.

0075Example 11: A method comprising: storing, by processing circuitry, a predictive movement model to a memory; receiving, by the processing circuitry, road condition information; detecting, by the processing circuitry, a discrepancy between the received road condition information and a portion of the predictive movement model; updating, by the processing circuitry, the predictive movement model based on the received road condition information to correct the discrepancy; and controlling, by the processing circuitry, one or more display devices in communication with the processing circuitry to adjust video data output by the one or more display devices based on the updated predictive movement model.

0076Example 12: The method of Example 11, wherein updating the predictive movement model comprises updating, by the processing circuitry, the predictive movement model according to machine learning.

0077Example 13: The method of either Example 11 or Example 12, wherein controlling the one or more display devices to adjust the video data comprises: determining, by the processing circuitry, a time of a physical movement of the one or more display devices; and controlling, by the processing circuitry, the one or more display devices to adjust the video data output at or before the time of the physical movement.

0078Example 14: The method of any of Examples 11-13, wherein the received road condition information comprises road condition information associated with a present location of a vehicle.

0079Example 15: The method of any of Examples 11-14, wherein the received road condition information associated with the present location of the vehicle indicates an aberration in a road on which the vehicle is traveling.

0080Example 16: A non-transitory computer-readable storage medium encoded with instructions that, when executed, cause processing circuitry of a computing system to: store a predictive movement model; receive road condition information; detect a discrepancy between the received road condition information and a portion of the predictive movement model; update the predictive movement model based on the received road condition information to correct the discrepancy; and control one or more display devices to adjust video data output by the one or more display devices based on the updated predictive movement model.

0081Example 17: The non-transitory computer-readable storage medium of Example 16, wherein the instructions that, when executed, cause the processing circuitry to update the predictive movement model comprise instructions that, when executed, cause the processing circuitry to update the predictive movement model according to machine learning.

0082Example 18: The non-transitory computer-readable storage medium of either Example 16 or Example 17, wherein the instructions that, when executed, cause the processing circuitry to control the one or more display devices to adjust the video data comprise instructions that, when executed, cause the processing circuitry to: determine a time of a physical movement of the one or more display devices; and control the one or more display devices to adjust the video data output at or before the time of the physical movement.

0083Example 19: The non-transitory computer-readable storage medium of any of Examples 16-18, wherein the received road condition information comprises road condition information associated with a present location of a vehicle.

0084Example 20: The non-transitory computer-readable storage medium of Example 19, wherein the received road condition information associated with the present location of the vehicle indicates an aberration in a road on which the vehicle is traveling.

0085It is to be recognized that depending on the example, certain acts or events of any of the techniques described herein can be performed in a different sequence, may be added, merged, or left out altogether (e.g., not all described acts or events are necessary for the practice of the techniques). Moreover, in certain examples, acts or events may be performed concurrently, e.g., through multi-threaded processing, interrupt processing, or multiple processors, rather than sequentially.

0086In one or more examples, the functions described may be implemented in hardware, software, firmware, or any combination thereof If implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium and executed by a hardware-based processing unit. Computer-readable media may include computer-readable storage media, which corresponds to a tangible medium such as data storage media, or communication media including any medium that facilitates transfer of a computer program from one place to another, e.g., according to a communication protocol. In this manner, computer-readable media generally may correspond to (1) tangible computer-readable storage media which is non-transitory or (2) a communication medium such as a signal or carrier wave. Data storage media may be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, code and/or data structures for implementation of the techniques described in this disclosure. A computer program product may include a computer-readable medium.

0087By way of example, and not limitation, such computer-readable data storage media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage, or other magnetic storage devices, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. Also, any connection is properly termed a computer-readable medium. For example, if instructions are transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. It should be understood, however, that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transitory media, but are instead directed to non-transitory, tangible storage media. Combinations of the above should also be included within the scope of computer-readable media.

0088Instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), complex programmable logic devices (CPLDs), or other equivalent integrated or discrete logic circuitry. Accordingly, the term “processor,” as used herein may refer to any of the foregoing structure or any other structure suitable for implementation of the techniques described herein. Also, the techniques could be fully implemented in one or more circuits or logic elements.

0089The techniques of this disclosure may be implemented in a wide variety of devices or apparatuses, including an integrated circuit (IC) or a set of ICs (e.g., a chip set). Various components, modules, or units are described in this disclosure to emphasize functional aspects of devices configured to perform the disclosed techniques, but do not necessarily require realization by different hardware units.

0090Various examples of the invention have been described. These and other examples are within the scope of the following claims.