System for recording and real-time transmission of in-flight of aircraft cockpit to ground services

SYSTEM FOR RECORDING AND REAL-TIME TRANSMISSION OF IN-FLIGHT OF AIRCRAFT COCKPIT TO GROUND SERVICES

BACKGROUND OF THE INVENTION

1. Field of the Invention

[0001] A system, method and device for aircraft monitoring and flight deck or cockpit recording, and more particularly, a system that may be implemented to ascertain real-time information about events taking place within monitored areas of an aircraft.

2. Brief Description of the Related Art

[0002] The current systems for aircraft monitoring and flight deck or cockpit recording are inadequate in many respects. For example, if an airliner crashes, the only way to retrieve reliable data as to the cause of the crash or mishap is to retrieve the so-called black-box flight recorder. Many smaller aircraft do not have such flight recorders, making reconstructing the events leading up to a mishap difficult for investigators, insurance companies and safety regulators. In addition, sometimes it is extremely difficult to recover an aircraft flight recorder, and sometimes they are never recovered at all. For example, Air France Flight 447 went into the Atlantic Ocean and it was nearly two years before the flight recorders were recovered. By way of another example, Malaysian Airlines Flight 370 presumably crashed into the Indian Ocean, and more than two years later, still no wreckage or flight recorders have yet been recovered. In fact there are presently on average of over 400 aircraft incidents per year resulting in one or more deaths, many of them in smaller aircraft with no flight recorders at all.

[0003] Even when a flight recorder is recovered, it does not necessarily provide the entire information set as to the activities leading up to the event or mishap. For example, many flight recorders record the voices of the pilots in the cockpit, but they do not record the image of the pilots, what they are doing, nor even an indication of whether they are present on the flight deck or even conscious, such as the situation which happened with Helios Airways Flight 522. In that flight, the entire occupancy was incapacitated by decompression in the aircraft at altitude, and military jets had to be scrambled to verify visually what had happened, since there were no communications from the aircraft, and no means to obtain a picture of the flight deck to determine the crew’s condition.

[0004] Furthermore, many crashes are the result of pilot error or misunderstanding of instruments or flight conditions. In such situations, it is possible and even likely that an independent flight expert or artificial intelligence flight system could be of material assistance in preventing a mishap or crash, if a means to insure that in an emergency flight situation, such resources were made available to the pilots. Even experienced professional pilots can make simple mistakes that could be corrected quickly with objective independent suggestions. For example, Air France Flight 447 spent over 3 minutes in a nose-up stall, entirely under the control of the pilots, with multiple indicators on the instrument panel indicating the elements of a stall condition, until it crashed into the ocean in a decent from 38,000 feet. The stall and ultimate crash could have easily been prevented had at some point in those 3 minutes, the pilots simply recognized the state of the aircraft as indicated by some of their instruments, pointed the nose down, and regained sufficient air flow over the wings to induce the aircraft into flight again.

[0005] In other situations, where there is possible breach of the flight deck by hijackers, often little is understood about an aircraft situation unless the hijackers choose to communicate. For example, in the September 11, 2001 attacks on the US, a significant period of time passed before ground controllers realized the gravity of the situation, since the attackers were not communicating anything over air traffic control frequencies. In fact several individuals knew the exact situation in the aircraft before airliner executives or air traffic control officials knew, based on passenger-to-ground calls. Still today, there is no system for ground controllers to independently view the status of an airliner flight deck.

[0006] A similar situation exists with respect to private aircraft. Private aircraft have many more accidents per year than airliners, and are less equipped to record incidents or provide for assistance in the case of in-flight emergencies.

[0007] The problems cited above are exacerbated by the fact that installing anything into an aircraft that might obtain readings from instruments or other aircraft-installed systems requires lengthy and expensive certification processes. Installing anything in an aircraft which in any way connects to instruments or existing aircraft systems typically violates the aircraft’s airworthiness certificate.

[0008] What is needed is a system which is completely independent of an aircraft’s existing avionics and systems, but which can nevertheless monitor the major elements affecting an aircraft and its operation.

SUMMARY OF THE INVENTION

[0009] A system, method and device are provided for monitoring an aircraft. The system, method and device are configured to monitor and obtain information from the major elements affecting an aircraft and its operation. The system is configured to obtain the information independently of and without being tied into the existing avionics and aircraft operational and guidance components.

[0010] The system, method and devices obtain information about the aircraft, which may include, for example, but not be limited to, information obtained by monitoring the aircraft’s avionics and instruments, both audio and video of the cockpit in order to determine the activities of the pilots, including their presence, as well as sounds in the cockpit, such as audible alerts, pilots or others’ conversations in the cockpit, and view through the cockpit windows to provide some indication of outside conditions or weather in which the aircraft is operating.

[0011] In addition, the present system, method and device, rather than merely recording the monitored information, such as, for example, the aforementioned elements, to a recording device (which has to be found intact and undamaged in order to ascertain its contents), the system may transmit the aforementioned elements wirelessly to a location, facility, airline operations center or other suitable ground-system during flight. According to some embodiments, the transmission may occur selectively when activated under certain conditions in flight or by pilots.

[0012] In addition, according to some embodiments, the system is configured to communicate with the pilot of the aircraft when warranted, which preferably is done independent of whether or not the regular radios of the aircraft are operable, or whether or not the aircraft is out of range of air traffic control or other ground transceivers.

[0013] The present invention supplies the above capabilities and more. Although features may be disclosed in a particular embodiment, embodiments of the invention may implement one or more or combinations of the features disclosed herein.

BRIEF DESCRIPTION OF THE DRAWING FIGURES

Figure 1 is a schematic illustration depicting an exemplary embodiment of the system, according to the invention, showing a device for imaging and associated components for managing information.

Figure 2 A is a perspective view of the cockpit, showing two pilots seated therein, as viewed from the rear of the cockpit looking forward.

Figure 2B is a perspective view of the cockpit depicted in Fig. 1, showing a pilot seated therein.

DETAILED DESCRIPTION OF THE INVENTION

[0014] The fundamental invention is disclosed in Figure 1, and further described herein, along with descriptions of options which may be amended to the elements of the invention without limitation or departing from the scope. For the purposes of this disclosure, the invention is called the real-time flight recorder system (RFRS).

[0015] First, referring to Figure 1, an exemplary embodiment of a system according to the invention is illustrated. The RFRS, in the embodiment depicted, is shown including a camera 102 which typically has a wide angle lens 101 A, with a microphone and speaker 101B. The microphone speaker 101B and camera 102 may be contained within a single housing as indicated in Figure 1, or, according to alternate embodiments, may be separately mounted. Preferably, the camera, speaker and microphone are mounted in the cockpit 101 in a location which does not interfere with the operation of the aircraft, but which nevertheless provides a view of the cockpit 101 and pilots, as shown for example, without limitation, in Figures 2A and 2B, where cockpits 201 and 101 are depicted, respectively. For example, as depicted in Figures 2A and 2B, the camera ascertains an image of the cockpit view where a number of the aircraft’s instruments, which are operative during flight, are imaged and recorded. The image may be transmitted to the ground portion 120 during flight (or after the flight), to provide information about the aircraft’s condition during its flight. The information may comprise real time information. According to some alternative embodiments, the information obtained from components (e.g., from cameras and sensors) may be processed by the aircraft portion 100 of the RFRS, and processed data obtained from the image or sensor information may be transmitted from the aircraft portion 100 to the ground portion 120. The component 115 may be configured with software or other artificial intelligence to identify or detect conditions of the aircraft (based on the camera and/or other sensors) and provide alerts when the condition meets or exceeds a threshold. In addition, the single camera 102 indicated in

Figure 1 may in fact be two or more cameras in order to provide complete coverage of a cockpit, without departing from the scope of the invention disclosed herein. In addition, as well known to those practiced in the art, the cameras and microphone and speaker may be of the miniature type, so that their location may be made to be hidden or innocuous or disguised to appear part of other instruments or switches, and therefore not readily recognizable by a hijacker or other unauthorized entrant to the cockpit. The camera 102 and microphone are set to record images and audio in the cockpit 101, primarily during emergencies, but may be set to record at any time, including for the complete duration of a flight, or for training purposes. The recording devices are connected to a system 100 which resides in the aircraft, but other than a power connection (where the system is configured or installed to receive power from the aircraft), is otherwise in no way connected to any other aircraft systems, and is therefore easy to install in aircraft which are already in operation and/or without affecting the aircraft’s airworthiness certificate.

[0016] In addition to the camera 102 and microphone and speaker 101B, the aircraft portion 100 of the RFRS includes means to automatically record and/or transmit images, sound bites or full video when certain sounds or images are noticed by the system. Listening circuitry may be provided. For example, the listening circuitry may process inputs from the camera 102, microphone, or other detecting component (such as, for example, a sensor, which may be included as part of the camera 102, or which may be separately provided an associated with the system components). The optional recording function 102 and recording trigger function 104 and optional continuous full video and audio 105 are compressed, formatted and encrypted 106 in preparation for optional transmission to the ground portion of the RFRS 120 via link 108 to a satellite radio transceiver 109 with suitable antenna 110. According to some embodiments, the ground portion 120 may be configured to include a satellite transceiver 114 for transmitting and receiving communications through a satellite (such as, for example, the satellite link represented by the satellite 112 and links 111,113 depicted in Figure 1). The aircraft portion of the system 100 may also include sensors separate from those of the aircraft, including optionally a barometric pressure sensor and accelerometer sensor and an attitude sensor (not shown). The aircraft portion 100 of the RFRS is shown containing a supervisory computer or control circuitry means 107. According to some embodiments of the invention, all of the functional elements of the aircraft portion 100 of the RFRS may be contained within a single computer or programmable circuit means, as opposed to being separable functions, without departing from the scope. According to other embodiments, the functional elements may be provided in one or more separate arrangements, for example, where the camera 102 and/or other sensors that collect or provide information, are provided separately from the operation circuitry or processing components, and transmission components.

[0017] According to some embodiments, the system components may be provided together in a housing. Embodiments may be configured where one or more components, such as, for example, a camera lens, sensor, or microphone, may be located separate from the housing and disguised among the cockpit instruments, switches, panels and/or other interior components. The component or components which are not situated within the housing preferably are connected to the housing or one or more other components within the housing. This may be done through a suitable connection, which may be wired or wireless (where the transmission would not interfere with aircraft instrumentation).

[0018] In operation, various sensors may operate individually or in concert to register conditions to the supervisory means which cause the images and audio or video and audio to be recorded and optionally immediately transmitted, or saved for later transmission. The trigger points or combinations can be set prior to installation by a program, prior to a single flight by a program, or on a real-time basis by a program, and that program can be optionally updated locally or via the radio link from another location. For example, a recording can be started if the RFRS detects a certain indicator light on the instrument panel, or detects the audio of a warning from the instruments, or detects unusual sounds. The audio can also be set to trigger a recording if it detects a certain word or sequence said by the pilot, such as “mayday” or “help” or a sequence of numbers known only to the pilots but not to unauthorized persons, and in this way the recording and transmission system may be triggered by the pilot during emergencies with minimal additional workload.

[0019] In addition, the RFRS portion 100 on the aircraft may be set to continuously record at all times when in flight, or when power to the aircraft is applied, and sufficient memory can be put in place inside the RFRS aircraft portion 100 to store an entire flight’s worth of audio and video. For example, recording of low motion rank video for 12 hours with low loss compression may take as little as 8 gigabytes of memory, which is easily provided for with modern flash or other nonvolatile memory in the aircraft RFRS 100. This video may be optionally linked to the ground portion of the RFRS 120 via the satellite link, represented by a satellite 112 and links 111,113, at anytime or dumped in reverse time order (that is, the most recent first) upon a signaling of a possible emergency. In this manner transmission may be controlled to prioritize the events leading up to the emergency condition being firstly transmitted to the ground portion of the RFRS 120.

[0020] Optionally, the accelerometer and attitude and cabin barometric pressure sensors may be combined to indicate when the aircraft is taking off and climbing out, or landing, which is when emergencies and incidents occur most often, and the system programmed to only record, or only record and transmit, or only transmit during those periods.

[0021] Many combinations of sensor readings, thresholds, including sounds and images, may be combined in a program on the supervisory means of the RFRS in the aircraft 100, which can be conceived by those practiced in the art of aircraft emergencies and systems, without departing from the scope.

[0022] When audio and video from the RFRS aircraft portion 100 is received by the ground portion 120 via the satellite link (see e.g., satellite 112 and links 111,113), it can be stored and optionally analyzed immediately by either an automated artificial intelligence system contained within a component 115 (which, for example, may be a computer connected to receive information from the aircraft portion 100 of the RFRS via the satellite transceiver 114), or by a human trained to assist pilots in emergencies (not shown). If an aircraft is in a detected or declared emergency situation, the artificial intelligence function or the human may employ the satellite link and the speaker in the aircraft in order to provide advisement to the pilots, when appropriate.

[0023] In addition, the RFRS may provide for distribution of video and audio from an aircraft to authorities, or flight operations departments of airlines, or any other particular entity or machine which has a need to receive such information, or any combination thereof.

[0024] For example, had this RFRS system been in place on Air France 447 over the middle of the Atlantic, the aircraft portion sensors could have detected an attitude and acceleration of the aircraft which were unusual and commenced immediately sending video and audio of the cockpit conditions to Air France flight operations in Paris via the satellite link. The flight operations personnel in Paris, being more objective because they were not present in the cockpit, could have easily seen the multiple instrument panel indicators suggesting that the aircraft was in a stall condition, and could have observed that the pilots were taking the wrong approach to bring the aircraft out of stall, and could have communicated over the satellite link directly into the aircraft flight deck without requiring relay through multiple ground controllers and unknown ground-to-aircraft communications conditions, and possibly saved the aircraft. Even if the attempts to save the aircraft and passengers was ultimately unsuccessful, at the very least the flight operations personnel would have immediate knowledge of the reasons the aircraft went down, instead of having to wait nearly two years for the flight recorders to be found.

[0025] Embodiments of the invention also include aircraft that are configured with the camera 102 and/or components of the system for detecting activity within the aircraft (individuals within the cockpit cabin or other space, smoke, and the like), and of the aircraft (movements of the aircraft, which for example may include altitude, speed, and other detectable movements). The system preferably is deployed in the aircraft separate from other instrumentation of the aircraft itself. The camera 102, microphone, speaker, or other component of the system (including other sensors), may be provided separate from other components of the aircraft, such as the aircraft instrumentation that measure what the system sensors also may measure. The system may be implemented in the aircraft by connecting the system power to a power source of the aircraft, such as, for example, a generator, battery, power terminal or other source. The system may receive power directly from the aircraft power source, or system may be configured with its own power source, such as a rechargeable battery, or a combination of these, where the aircraft power source charges the system battery. However, according to preferred embodiments, the system ascertains information (e.g., such as activity within the aircraft cabin or cockpit, operating conditions, such as, altitude, speed, and others), which is sensed by the system components, and not the aircraft instrumentation. Although the aircraft is configured with instrumentation and sensors that may provide indications on the instrument panel in the cockpit, the RFRS system components are separate from the aircraft instrumentation, and the RFRS system ascertains information using these separately provisioned system components to communicate the information to a location remote from the operating aircraft. The aircraft preferably is configured, as illustrated in Figure 1, to carry the system transceiver 109 and an antenna 110 which are configured with the communications mechanism of the system and sensors (such as the camera 102), to provide the information to a ground portion 120 of the system through a satellite communications link (see e.g., 112 and 111,113). The aircraft may be configured so that the system components that are employed to sense activity are located in different positions within panels of aircrafts, in order to make it more difficult for a wrongdoer to potentially disable the component. For example, a camera lens may be placed and disguised within a particular instrument panel, light, knob, or switch in one aircraft, and in a different location in another aircraft. According to some preferred embodiments, the system components are installed on the aircraft when it is being constructed. According to some other embodiments, the system is installed in an already constructed aircraft.

[0026] The system, method and devices also provide a service that a subscriber may receive. Embodiments of the invention are designed to provide a subscriber system for receiving the ascertained information, where the information for one or more aircraft may be managed among one or more subscribers. For example, the service may be implemented with the system, and information may be communicated to reside in a subscriber account. For example, a portion of the system may comprise one or more computing components, data repositories, or other storage means, which receives the information communicated from the aircraft portion of the system. For example, images from the camera 102 taken periodically (or streaming, or both periodic and streaming when activity or other triggering condition is present or detected) may be transmitted to the remotely situated ground portion of the system. The ground portion may be configured to capture the data from transmissions received from the aircraft portion, and process that information. According to some embodiments, the system is configured as a subscriber system, and subscribers may subscribe to receive access to the information provided by the system, even as the aircraft is in flight. The subscriber side of the system preferably resides in a ground portion of the system which subscribers (who are authorized) may access. The information may be subscribed to, and, the subscriber system may also include an alerting system that is configured to provide notifications to a subscriber. For example, according to one implementation, subscriber notification parameters may be preset (by a subscriber preference or otherwise) to provide an alert to a subscriber upon the subscriber account receiving information from the system of an aircraft that indicates a triggering event has occurred or been detected. A subscriber may subscribe to one or more subscription components, such as aircrafts, carriers, regions, or other distinction, so that the subscriber receives information relevant to particular aircrafts of interest to that subscriber. The subscriber may be provided with approvals and authorizations prior to the subscriber being permitted to access the information. Subscriber communications also may be encrypted. Other subscription requirements may be an account, a contact, a payment system for the subscriber to purchase or maintain a subscription. Among those who may be potential subscribers include for example an aviation authority or agency of a government, law enforcement, military, or other regulatory body or group. The subscriber system may be configured on a computing component, which may comprise a computer with a processor, and a storage component maintained on or in association with the computing component, so that information may be stored and retrieved from the storage component. The storage component may be a hard drive, flash memory, a drive of the computer, a remotely linked drive or other suitable data storage element. The subscriber system computer, for example, may be configured with software containing instructions to process the information from the aircraft portion of the system. The subscriber system computer, for example, may be configured to store and distribute the aircraft information obtained from the aircraft system portion, such as, for example, recordings, to appropriate flight operations people, trainers, authorities, and the like. These organizations or individuals may be subscribers. Subscribers also may be provided with different access levels, so that some subscribers may have access to different information than other subscribers (based on company affiliation, subscriber employment, rank or position level within an organization, and the like). The subscription may be set up as a service, provided and invoiced monthly, annually, or other remuneration method.

[0027] These and other advantages may be obtained through the use of the inventive system, methods and devices disclosed herein. While the invention has been described with reference to specific embodiments, the description is illustrative and is not to be construed as limiting the scope of the invention. For example, the percepting components may comprise or more cameras, and multiple sensors, and may be provided in different locations of the aircraft. The configuration may provide redundancy of sensors or alternate locations for viewpoint locations, and the like. Various modifications and changes may occur to those skilled in the art without departing from the spirit and scope of the invention described herein and as set forth in the appended claims.