Real-time data acquisition and recording system viewer

PATENT TITLE No. 389010

<img file="MX389010B_D0001.tif" />

IMPI

MEXICAN INSTITUTE OF INDUSTRIAL PROPERTY

<td>Headlines):</td><td>WI-TRONIX, LLC</td>

<td>Home:</td><td>E. Boughton Road 631, Suite 240, Bolingbrook, Illinois, 60440, USA</td>

Name: DATA ACQUISITION AND RECORDING SYSTEM VIEWER

<td></td><td>REAL TIME.</td>

<td>Classification:</td><td>CIP: G07C5/08; G06F17/00; H04B1/38; H04N5/77; H04N5/92; H04N5/232; H04N5/247; H04N9/804 CPC: G07C5/0858; G07C5/0866; H04N5/77; H04N5/247; H04N5/9201; H04N5/23238; G07C5/008; H04N9/8042</td>

<td>Inventor(s):</td><td>LAWRENCE B. JORDAN; SAVANKUMAR V. PATEL; BRYAN WEAVER REQUEST</td>

Number: International Filing Date:

MX/a/2018/014070 May 16, 2017

PRIORITY*

Country: Date:

May 16, 2016

May 16, 2016

Number:

62/337,227 62/337,225

Validity: Twenty years

Expiration Date: May 16, 2037

Issue Date: January 3, 2022

The reference patent is granted based on articles 1^either, 2^either fraction V, 6^either section III, and 59 of the Industrial Property Law.

In accordance with article 23 of the Industrial Property Law, this patent is valid for twenty years, non-renewable, counted from the date of filing of the international application and will be subject to the payment of the fee to keep the rights in force. .

Whoever signs this title does so based on the provisions of articles 5^either section I, 9, 10 and 119 of the Federal Law for the Protection of Industrial Property; items 1^either, 3^either section V item a), sub item iii), 4^either and 12th sections I and III of the Regulations of the Mexican Institute of Industrial Property; items 1^either, 3^either, 4^either, 5^either section V subsection a), subsection i¡¡), 16 sections I and III and 30 of the Organic Statute of the Mexican Institute of Industrial Property; 1^either, 3^either and 5^either section I and antepenultimate paragraph of the Agreement Delegating Powers of the Mexican Institute of Industrial Property.

This electronic document has been signed through the use of the advanced electronic signature by the competent public servant, protected by a digital certificate in force at the date of its preparation, and is valid in accordance with the provisions of articles 7 and 9 section I of the Advanced Electronic Signature Law and article 12 of its Regulations. Its integrity and authorship can be verified at www.gob.mx/imDi. Likewise, it was issued in accordance with the provisions of articles 1, section III; 2nd fraction VI; 37, 38 and 39 of the Agreement establishing guidelines on Electronic Services of the Mexican Institute of Industrial Property.

DIVISIONAL DEPUTY DIRECTOR OF SUBSTANTIVE EXAMINATION OF PATENT AREAS MECHANICAL, ELECTRICAL AND INDUSTRIAL DESIGNS AND UTILITY MODELS

PEDRO DAVID FRAGOSO LOPEZ

<img file="MX389010B_D0002.tif" />

Original string:

PEDRO DAVID FRAGOSO LOPEZ|00001000000506606281 |ADMINISTRATIVE SERVICE

TAX|1052||MX/2022/3983|MX/a/2018/014070|PCT patent title|1223|GAGV|Page(s) 2|lsWAybel2t2BPvhjmpuLFJ2+rQI =

Digital stamp:

UO1eguNclravNFKPxDCnsX1kC8aoPECe+mSDIzJlvOelwrtZcBYI29is9DTGO4MK5vD1wuefiaaAlubsTmTUoG4lsg xj1qqmqMnS8rHxSIOduFD5L59tHcFlwplmrGajGcLFjUtWfYKKIblNpqMsYs+uACZzcKeKEg5dOIJJU6R7r13d3¡Jt HvbJV4FdZGgLGM1kcYzl4L+le6Ekwbij3vQY5RkKpar4zlOQfEU7CK9Rh8dODq8zelT3ylNOglwoalDVNEDq1bmDqE

T8DAdJEAjkcDyHLN2GGSXymEBKU/ULbKgBleQNgb/D5XQO+dcsyQ9hXMYUC2KuF4ZUdZ26JQ“ * Additional information on next page.

MX/2022/3983

Arenal No. 550, Santa María Tepepan Town, Xochimilco, CP 16020. CDMX.

Tel: (55) 5624 0400 buzon@impi.gob.mx www.gob.mx/impi Creativity for Wellbeing

<img file="MX389010B_D0003.tif" />

Continuation Priorities

Country: Date: Number:

<img file="MX389010B_D0004.tif" />

IMPI

MEXICAN INSTITUTE OF INDUSTRIAL PROPERTY

<td>US</td><td>May 16, 2016</td><td> 62/337,228</td>

<td>US</td><td>May 15, 2017</td><td> 15/595,650</td>

<td>US</td><td>May 15, 2017</td><td> 15/595,689</td>

Arenal No. 550, Santa María Tepepan Town, Xochimilco, CP 16020. CDMX.

Tel: (55) 5624 0400 buzon@impi.gob.mx www.gob.mx/impi Creativity for Wellbeing

<img file="MX389010B_D0005.tif" />

IMPI

MEXICAN INSTITUTE

OF THE PROPERTY

INDUSTRIAL

SYSTEM VIEWER. REAL TIME DATA ACQUISITION AND RECORDING

DESCRIPTION OF THE INVENTION

This description relates to a system and method for viewing video, images, and data from a real-time data acquisition and recording system used in high-value mobile assets.

High-value mobile assets such as locomotives, aircraft, mass transportation systems, mining equipment, transportable medical equipment, cargo, maritime vessels, and military vessels typically employ black box and/or on-board data acquisition and recording systems. event recording systems. These data acquisition and recording systems, such as event data recorders or flight data recorders, record a variety of system parameters used for incident investigation, crew performance evaluation, fuel efficiency, maintenance planning and predictive diagnostics. A typical data acquisition and recording system comprises digital and analog inputs, as well as pressure switches and pressure transducers, which record data from various integrated sensor devices. Recorded data may include parameters such as speed, distance traveled, location, fuel level, engine revolutions per minute (RPM),

<img file="MX389010B_D0006.tif" />

IMPI

MEXICAN INSTITUTE

OF THE PROPERTY

INDUSTRIAL fluid levels, operator controls, pressures, current and forecast weather and environmental conditions. In addition to basic event and operational data, audio and video data/event recording capabilities are also implemented in many of these same mobile assets. Data is typically pulled from data loggers after an asset-related incident has occurred and an investigation is required, once the data logger has been recovered. Certain situations may arise where the data logger cannot be recovered or the data is not available. In these situations, the data, such as event and operation data, video data, and audio data, acquired by the data acquisition and recording system is needed immediately, regardless of whether physical access to the acquisition and recording system data recording or data is available.

This description generally refers to real-time data acquisition and recording systems used in high-value mobile assets. The teachings herein may provide access to real-time or near-real-time data, such as event and operational data, video data, and audio data, recorded by a real-time data acquisition and recording system on a computer. high-value mobile asset. An implementation of a method for processing, storing, and transmitting data from at least one mobile asset described herein includes receiving, using

<img file="MX389010B_D0007.tif" />

ΙΜΡΙ

MEXICAN INSTITUTE

OF THE PROPERTY

INDUSTRIAL a multimedia management system on board the mobile asset, data based on at least one data signal from at least one of: at least one 360-degree camera; at least one fixed camera; and at least one microphone; receive, using a data logger on board the mobile asset, the data; encoding, using a data logger data encoder, a record comprising a bit stream based on the data; and storing, using an on-board data manager in the data logger, at least one of the data and the record at a first configurable predetermined rate in at least one local memory component of the data logger.

An implementation of a method for displaying data from at least one mobile asset described herein includes receiving, using a web server, a request comprising specific media data from at least one mobile asset and a specific view mode; receiving, using the web server, the specified multimedia data from the at least one mobile asset of a remote memory component; and displaying, using a display device, the specified media data of at least one mobile asset in the specified view.

An implementation of a real-time data acquisition and recording system described herein includes at least one of at least one 360-degree camera, at least one still camera, and at least one microphone; a multimedia management system on board the mobile asset configured to receive data based on

<img file="MX389010B_D0008.tif" />

IMPI

MEXICAN INSTITUTE OF INDUSTRIAL PROPERTY at least one data signal from at least one of at least one 360-degree camera, at least one fixed camera and at least one microphone; an on-board data logger of the mobile asset comprising at least one local memory component, an on-board data manager and a data encoder, the data logger configured to receive data from the media management system; the data encoder configured to encode a record comprising a stream of bits based on the data; and the on-board data manager configured to store at least one of the data and the log at a first configurable predetermined rate in at least one local memory component.

Variations on these and other aspects of the description will be described in more detail below.

BRIEF DESCRIPTION OF THE DRAWINGS

The description herein refers to the accompanying drawings, in which like reference numerals refer to like parts throughout the several views, and in which:

FIGURE 1 illustrates a field implementation of an exemplary real-time data acquisition and recording system in accordance with implementations of this disclosure;

FIGURE 2 is a flow diagram of a process for recording video data, audio data, and/or information from a mobile asset in accordance with implementations of this disclosure;

FIGURE 3 is a flow diagram of a process for displaying recorded video data, audio data and/or information

<img file="MX389010B_D0009.tif" />

IMPI

MEXICAN INSTITUTE OF INDUSTRIAL PROPERTY of the mobile asset in accordance with the implementations of this description;

FIGURE 4 is a diagram illustrating an exemplary fisheye view of a 360-degree camera of the real-time data acquisition and recording system in accordance with implementations of this disclosure;

FIGURE 5 is a diagram illustrating an exemplary 360-degree camera panoramic view of the real-time data acquisition and recording system in accordance with implementations of this disclosure;

FIGURE 6 is a diagram illustrating an exemplary 360-degree quad camera view of the real-time data acquisition and recording system in accordance with implementations of this disclosure.

FIGURE 7 is a diagram illustrating an exemplary dewarped view of the 360-degree camera of the real-time data acquisition and recording system in accordance with implementations of this disclosure.

A real-time data acquisition and recording system described herein provides real-time or near-real-time access to a wide range of data, such as event and operational data, video data, and audio data, from a high asset value for remotely located users such as asset owners, operators and researchers. The data acquisition and recording system records the data, through

<img file="MX389010B_D0010.tif" />

IMPI

MEXICAN INSTITUTE OF INDUSTRIAL PROPERTY from a data logger, related to the asset and transmits it to a remote data repository and remotely located users before, during and after an incident has occurred. Data is transmitted to the remote data warehouse in real time or near real time, making the information available at least until the time of an incident or emergency situation, virtually eliminating the need to locate and download the black box, to investigate an incident involving the asset and eliminate the need to interact with the data logger on the asset to request a download of specific data, locate and transfer files, and use a custom application to view the data. The system of the present disclosure retains the typical recording capabilities and adds the ability to transmit data to a remote data repository and remote end user before, during, and after an incident. In the vast majority of situations, the information recorded in the data logger is redundant and not necessary since the data has already been acquired and stored in the remote data warehouse.

Prior to the system of the present disclosure, data was retrieved from the black box or event recorder after an incident occurred and investigation was required. Data files containing time segments recorded by the black box had to be downloaded and retrieved from the black box and then viewed by a user with software from the black box.

<img file="MX389010B_D0011.tif" />

IMPI

MEXICAN INSTITUTE OF INDUSTRIAL PROPERTY Owner. The user would have to gain physical or remote access to the asset, select the data they want to download from the black box, download the file containing the desired information to a computing device, and locate the appropriate file with the desired data using a custom application that operates on the computing device. The system of the present disclosure has eliminated the need for the user to perform these steps, it only requires the user to use a common web browser to navigate to the desired data. The remotely located user can access a common web browser to navigate to the desired data related to a selected asset to view and analyze operational efficiency and asset safety in real time or near real time.

The remotely located user, such as an asset owner, operator, and/or investigator, can access a common web browser to navigate to desired live and/or historical data related to a selected asset to view and analyze the operational efficiency and security of assets in real time or near real time. The ability to view operations in real-time or near real-time allows for quick behavior assessment and adjustment. During an incident, for example, real-time information and/or data can make it easier to monitor the situation and provide valuable information to first responders. During normal operation, for example, real-time information and/or data may be

<img file="MX389010B_D0012.tif" />

ΙΜΡΙ

MEXICAN INSTITUTE

OF THE PROPERTY

INDUSTRIAL use to audit crew performance and to help provide situational awareness across the network.

The system of the present disclosure uses 360 degree cameras in, on, or near a mobile asset as part of a data acquisition and recording system. Prior to the system of the present disclosure, black box and/or event recorders did not include 360 ​​degree cameras in, on, or near the mobile asset. The system of the present description adds the ability to use and record video using 360-degree cameras as part of the data acquisition and recording system, providing 360-degree views at, on or near the mobile asset to a remote data repository and a remote user and investigators before, during and after an incident involving the mobile asset has occurred. The ability to view operations and/or 360-degree video in real time, or near real time, allows for quick assessment and adjustment of crew behavior. Owners, operators, and investigators can view and analyze operational efficiency, the safety of people, vehicles, and infrastructure, and can investigate or inspect an incident. The ability to view 360-degree video from the mobile asset allows quick assessment and adjustment of crew behavior. During an incident, for example, 360-degree video can make it easier to monitor the situation and provide valuable information to first responders and investigators. During normal operation, for example, you can

<img file="MX389010B_D0013.tif" />

IMPI

MEXICAN INSTITUTE OF INDUSTRIAL PROPERTY use a 360-degree video to audit the performance of the crew and to help understand the situation throughout the network. 360-degree cameras and fixed cameras provide a complete picture to provide video surveillance for law enforcement and/or railway police, critical infrastructure inspection, railroad crossing monitoring, work progress tracking view, audits of the crew both inside the cabin and in the yard, and real-time remote surveillance.

Previous systems required users to download video files containing time segments in order to view them using a proprietary software application or other external video playback applications. The data acquisition and recording system of the present description provides 360-degree video and image information and audio information that can be displayed to a remote user through the use of a virtual reality device and/or a standard web client, which eliminates the need to Download and use external applications to watch the videos. Additionally, remotely located users can view 360-degree videos in various modes using a virtual reality headset or through a standard web client, such as a web browser, eliminating the need to download and use external applications. to watch the video. Previous video systems required the user to download video files that contained time segments of data that could only be viewed with application software.

<img file="MX389010B_D0014.tif" />

IMPI

MEXICAN INSTITUTE OF INDUSTRIAL PROPERTY of the owner or other external video playback applications that the user had to buy separately.

The data may include, but is not limited to, video and image information from cameras located in various locations on, on or near the assets and audio information from microphones located in various locations on, on or near the asset. A 360-degree camera is a camera that provides a 360-degree spherical field of view and/or a 360-degree hemispherical field of view. Using 360-degree cameras at, on, or near an asset provides the ability to use and record video with 360-degree cameras as part of DARS, making the 360-degree view at, at, or near the asset be available to a remote data repository, remotely located users, and investigators before, during, and after an incident.

FIGURE 1 illustrates a field implementation of a first embodiment of an exemplary real-time data acquisition and recording system (DARS) 100 in which aspects of the description may be implemented. DARS 100 is a system that delivers real-time information, video information, and audio information from a data logger 108 on a mobile asset 130 to remotely located end users through a data center 132 . Data logger 108 is installed in vehicle or mobile asset 130 and communicates with any number of various information sources through any combination of wired and/or wireless data links, such as a gateway.

<img file="MX389010B_D0015.tif" />

IMPI

MEXICAN INSTITUTE OF INDUSTRIAL PROPERTY link/wireless router (not shown). The data logger 108 comprises a buffered memory module 110, an on-board data manager 112, and a data encoder 114. In a second embodiment, data logger 108 may also include a non-hardened removable storage device (not shown). An example of a hardened memory module 110 may be, for example, a fail-safe event log memory module that complies with the Code of Federal Regulations and Federal Railroad Administration regulations, a memory unit with of survival that complies with the Code of Federal Regulations and regulations of the Federal Aviation Administration, an enhanced memory module in accordance with any applicable Code of Federal Regulations, or any other suitable enhanced memory device, as known in the art. Wired and/or wireless data links may include any or a combination of discrete signal inputs, standard or proprietary Ethernet, serial connections, and wireless connections.

Data logger 108 collects video data, audio data, and other data and/or information from a wide variety of sources, which may vary depending on the configuration of the asset, via on-board data links. In this implementation, data logger 108 receives data from a video management system 104 that continuously records video data and audio data from 360-degree cameras 102 and fixed cameras 106 that are placed in,

<img file="MX389010B_D0016.tif" />

IMPI

MEXICAN INSTITUTE OF INDUSTRIAL PROPERTY at or near the asset 130 and the video management system 104 stores the video and audio data in the hardened memory module 110, and may also store the video and audio data in the storage device non-reinforced extractable of the second modality. Different versions of the video data are created using different bit rates or spatial resolutions, and these versions are separated into segments of varying length, such as thumbnails, low-resolution five-minute segments, and high-resolution five-minute segments.

Data encoder 114 encodes at least a minimum set of data that is typically defined by a regulatory agency. Data encoder 114 receives video and audio data from video management system 104 and compresses or encrypts the data and time synchronizes the data to facilitate efficient transmission and replication in real time to a remote data repository 120 . Data encoder 114 transmits the encrypted data information to on-board data manager 112 which then sends the encrypted audio and video data to remote data repository 120 via remote data manager 118 located in data center 130 in response to a request at the request of a remotely located user 134 or in response to certain operating conditions that are observed on board the asset 130. Onboard data manager 112 and remote data manager 118 work in unison to manage the data replication process. The administrator

<img file="MX389010B_D0017.tif" />

IMPI

MEXICAN INSTITUTE OF INDUSTRIAL PROPERTY

118 The remote data center at data center 132 can manage the replication of data from a plurality of assets. Video and audio data stored in remote data repository 120 is available to a web server 122 for remote user 5 to access.

Onboard data manager 112 also sends data to a routing depot (not shown). The on-board data manager 112 monitors the video and audio data stored in the hardened memory module 110, and the second mode optional non-hardened removable storage device, by the video management system 104 and determines if is in near real-time mode or real-time mode. In near real-time mode, the on-board data manager 112 stores the encoded data, including video data, audio data 15 and any other data or information, received from the data encoder 114 and any event information in the hardened memory module 110, and the optional non-hardened removable storage device of the second embodiment, and in the targeting bin. After five 20 minutes of encrypted data have been accumulated in the ranging repository, the onboard data manager 112 stores the five minutes of encrypted data in the remote data repository 120 via the remote data manager 118 at the center. 132 via a wireless data link 116 . In real-time mode, the on-board data manager 112 stores the data

<img file="MX389010B_D0018.tif" />

ΙΜΡΙ

MEXICAN INSTITUTE OF INDUSTRIAL PROPERTY encrypted data, including video data, audio data and any other data or information, received from the data encoder 114 and any event information to the remote data repository 120 via the remote data manager 118 in data center 132 via wireless data link 116 . On-board data manager 112 and remote data manager 118 may communicate over a variety of wireless communication links. The wireless data link 116 may be, for example, a wireless local area network (WLAN), a wireless metropolitan area network (WMAN), a wireless wide area network (WWAN), a private wireless system, a cell phone or any other means of transferring data from data logger 108 to, in this example, remote data manager 118 . The process of sending and retrieving video data and audio data remotely from asset 130 requires a wireless data connection between asset 130 and data center 132. When a wireless data connection is not available, the data is stored and queued in the hardened memory module 110, and the second mode optional non-hardened removable storage device, until wireless connectivity is restored. Video, audio, and any additional data recovery processes resume as soon as wireless connectivity is restored.

In parallel with data recording, the data logger 108 continuously and autonomously replicates data in the repository.

<img file="MX389010B_D0019.tif" />

IMPI

MEXICAN INSTITUTE OF INDUSTRIAL PROPERTY

120 remote data. The replication process has two modes, a real-time mode and a near real-time mode. In real time mode, data is replicated to remote data repository 120 every second. In near real-time mode, data is replicated to remote data repository 120 every five minutes. The rate used for the near real time mode is configurable and the rate used for the real time mode can be adjusted to support high resolution data by replicating the data to the remote data repository 120 every 0.10 seconds. Near real-time mode is used during normal operation, under most conditions, to improve the efficiency of the data replication process.

The real-time mode may be initiated based on events occurring in the onboard asset 130 or by a request initiated from the data center 132. A typical data center 132 initiates a request for real-time mode when the remotely located user 134 has real-time requested information from a web client 126 . A typical reason for real-time mode to originate on board the asset 130 is the detection of an event or incident such as an operator initiating an emergency stop request, emergency braking activity, rapid acceleration or deceleration in any axis , or loss of input power to data logger 108. When transitioning from near real-time mode to real-time mode, all data that has not yet been replicated to the repository

<img file="MX389010B_D0020.tif" />

IMPI

MEXICAN INSTITUTE OF INDUSTRIAL PROPERTY

120 Remote data are replicated and stored in the remote data repository 120 and then live replication is started. The transition between near real-time mode and real-time mode typically takes less than five seconds. After a predetermined time has elapsed since the event or incident, a predetermined amount of downtime, or when the user 134 no longer wishes to obtain real-time information from the asset 130, the data logger 108 returns to near-time mode. real. The default amount of time required to initiate the transition is configurable and is typically set to ten minutes.

When the data logger 108 is in real-time mode, the on-board data manager 112 continually attempts to flush its queue to the remote data manager 118, storing the data in the hardened memory module 110 and the removable storage device does not. optional hardening of the second mode and sending the data to the remote data manager 118 simultaneously.

Upon receiving video data, audio data, and any other data or information to be replicated from data logger 108, remote data manager 118 stores the data in remote data repository 120 at data center 130. Remote data repository 120 may be, for example, cloud-based data storage or any other suitable remote data storage. When data is received,

<img file="MX389010B_D0021.tif" />

IMPI

MEXICAN INSTITUTE OF INDUSTRIAL PROPERTY starts a process that causes a data decoder (not shown) to decode the newly replicated data from the remote data repository 120 and send the decoded data to a remote event detector (not shown). Remote data manager 118 stores vehicle event information in remote data repository 120 . When the remote event listener receives the decoded data, it processes the decoded data to determine if an event of interest is found in the decoded data. The decoded information is then used by the remote event detector to detect events, incidents, or other predefined situations in the data that occur with asset 130. Upon detecting an event of interest from previously stored decoded data in remote data repository 120, the remote event detector stores the event information and supporting data in remote data repository 120 .

Video data, audio data, and any other data or information is made available to user 134 in response to a request at the request of user 134 and/or sent by onboard data manager 112 to remote data repository 120 in response. to certain operating conditions observed on board the 130 asset. Video data, audio data, and any other information or information storage in remote data repository 120 is available on web server 122 for user 134 access. Remotely located user 134 can access the video data, audio data and any other

<img file="MX389010B_D0022.tif" />

IMPI

MEXICAN INSTITUTE

OF THE PROPERTY

INDUSTRIAL data or information related to the specific asset 130, or a plurality of assets, stored in the remote data repository 120 using the standard web client 126, such as a web browser or a virtual reality device 128 which, in this implementation , you can display thumbnail images of the selected cameras. The web client 126 communicates the user's request 134 for video, audio, and/or other information to the web server 122 over a network 124 using common web standards protocols and techniques. Network 124 may be, for example, the Internet. Network 124 may also be a local area network (LAN), metropolitan area network (MAN), wide area network (WAN), virtual private network (VPN), cellular telephone network, or any other medium. data transfer from web server 122 to, in this example, web client 126 . The web server 122 requests the desired data from the remote data repository 120 . The web server 122 then sends the requested data to the web client 126 which provides real time playback and display of standard video and 360 degree video. Web client 126 renders the video data, audio data, and any other data or information to user 134 who may interact with the 360-degree video data for viewing and analysis. The user 134 can also download the video data, audio data, and any other data or information using the web client 126 and can then use the virtual reality device 128 to interact with the 360-degree video data for viewing and analysis. .

<img file="MX389010B_D0023.tif" />

IMPI

MEXICAN INSTITUTE OF INDUSTRIAL PROPERTY

The web client 126 can be enhanced with a software application that provides 360-degree video playback in a variety of different modes. User 134 can choose how the software application presents the video playback, such as fisheye view as shown in FIGURE 4, panoramic view as shown in FIGURE 5, dual panoramic view (not shown), quad view as shown in FIGURE 6, and dewarped view as shown in FIGURE 7.

FIGURE 2 is a flow chart showing a process 200 for recording video data, audio data, and/or asset information 130 in accordance with one implementation of this disclosure. Video management system 104 receives data signals from various input components 202, such as 360-degree cameras 102 and fixed cameras 106 that are located on or near asset 130. Video management system 104 stores video data, audio data, and/or information in hardened memory module 110, and optional second embodiment non-hardened removable storage device 204 using any combination of standard formats. such as still images, thumbnails, still image sequences, or compressed video formats. The data encoder 114 creates a register that includes a structured series of bits used to configure and record the information 206 of the data signal. In near real-time mode, system 104

<img file="MX389010B_D0024.tif" />

IMPI

MEXICAN INSTITUTE OF INDUSTRIAL PROPERTY video management stores video data in the hardened memory module 110, and the optional non-hardened removable storage device of the second mode, while sending only limited video data, such as thumbnails or segments off-dash, very low-resolution video to remote data repository 120 208.

In another implementation, the encrypted record is then sent to onboard data manager 112 which sequentially combines a series of records in chronological order into record blocks that include up to five minutes of data. An interim log block includes less than five minutes of data, while a full log block includes a full five minutes of data. Each register block includes all the data necessary to completely decode the included signals, including a data integrity check. At a minimum, a record block must begin with a start record and end with an end record.

In order to ensure that all encrypted signal data is stored in the hardened memory module 110 and optional non-hardened removable storage device of the second mode, if the data logger 108 loses power, the on-board data the manager 112 stores the interim register blocks in the hardened memory module 110, and the optional non-hardened removable storage device of the second embodiment, at a predetermined speed, where the

<img file="MX389010B_D0025.tif" />

IMPI

MEXICAN INSTITUTE OF INDUSTRIAL PROPERTY Default speed is configurable and/or variable. Interim log blocks are saved at least once per second, but can also be saved as often as a tenth of a second. The rate at which interim log blocks are saved depends on the sample rates of each signal. Each interim record block includes the full set of records since the last full record block. The data logger 108 can alternate between two temporary storage locations in the buffer memory module 110 when recording each interim record block to prevent corruption or loss of more than one second of data when the data logger 108 loses power while recording. stores data in the enhanced memory module 110. Each time a new log block is saved to a temporary hardened memory location, it overwrites the previously stored temporary log block in that location.

Every five minutes, in this implementation, when the data logger 108 is in near real-time mode, the on-board data manager 112 stores a complete record block that includes the last five minutes of signal data encoded in a segment of logging to the hardened memory module 110, and optional non-hardened removable storage device of the second mode, and sends a copy of the entire logging block, comprising five minutes of video data, audio data and/or information to the administrator 118 of

<img file="MX389010B_D0026.tif" />

IMPI

MEXICAN INSTITUTE OF INDUSTRIAL PROPERTY remote data to be stored in remote data repository 120 for a predetermined retention period, such as two years. The hardened memory module 110, and optional non-hardened removable storage device of the second embodiment, stores a log segment of the most recent log blocks for a mandatory storage duration, which in this implementation is the federally mandated duration that The data loggers 108 must store the operational or video data in the memory module 110 backed up with an additional 24 hour buffer, and then overwritten.

FIGURE 3 is a flow chart showing a process 300 for viewing data and/or information from asset 130 through a web browser or virtual reality device. When an event occurs or when the authorized user 134 located at a remote location requests a segment of video data stored in the buffer memory module 110 via the web client 126, the on-board data manager 112, depending on the event, it will start sending the video data from the outside in real time, with the best resolution available, given the bandwidth of the wireless data link 116 . Remotely located user 134 initiates a request for specific video and/or audio data in a specific view mode 302 via web client 126 which communicates the request to web server 122 via network 124. Server 122 web requests the specific video and/or audio data from the remote data repository 120 and sends the requested data

<img file="MX389010B_D0027.tif" />

ΙΜΡΙ

MEXICAN INSTITUTE

OF THE PROPERTY

INDUSTRIAL video and/or audio to the client 126 304 web through the 124 network.

The web client 126 displays the video and/or audio data in the display mode specified by the user 134 306. The user 134 may download the specific video and/or audio data for viewing on the virtual reality device 128 . In another implementation, in real-time mode, thumbnails are sent first at one second intervals, then short segments of lower resolution video, then short segments of higher resolution video.

For simplicity of explanation, process 200 and process 300 are represented and described as a series of steps. However, the steps according to this description may occur in several orders and/or simultaneously. Furthermore, steps according to this description may occur with other steps not shown and described herein. Furthermore, not all of the illustrated steps may be required to implement a method in accordance with the disclosed subject matter.

While the present description has been described in conjunction with certain embodiments, it is to be understood that the description is not to be limited to the described embodiments, but rather is intended to cover various modifications and equivalent arrangements included within the scope of the appended claims, to which scope should be given the broadest interpretation to encompass all modifications and equivalent constructions permitted by law.