Geographic image storage, transmission and display system

COMPLETE SPECIFICATION

STANDARD PATENT

<td> APPLICANT:</td><td> NORTHROP GRUMMAN CORPORATION</td>

<td> Invention Title:</td><td> GEOGRAPHIC IMAGE STORAGE, TRANSMISSION AND DISPLAY SYSTEM</td>

The following statement is a full description of this invention, including the best method of performing it known to me:

2006201051 08 Mar 2006 _ 1 .

GEOGRAPHIC IMAGE STORAGE, TRANSMISSION AND DISPLAY SYSTEM

FIELD OF THE INVENTION [0001] The present invention relates to an image storage, transmission and display system, and more particularly relates to a system in which a particular geographic area is identified by a user, and photographic images of that area are downloaded to the user via an input/output device.

BACKGROUND INFORMATION [0002] Images of various geographic regions are available from sources such as aircraft and satellite-mounted cameras. However, such images typically embody a vast amount of data that is difficult to transmit and display to users in a timely fashion.

SUMMARY OF THE INVENTION [0003] The present invention provides a system in which a geographic area of interest can be selected, and images of that area can be downloaded and displayed by a user in an efficient and timely manner. The area of interest may be selected using an input device by various means such as a map, longitude/latitude coordinates, a global positioning system (GPS), and the like. Once the particular geographic area has been selected, photographic images corresponding to the selected area are downloaded to the user. The images may comprise visible photographs, IR photographs or other types of photographic images that may be stored in a suitable data storage system such as a server. In addition to stored image data, the photographic image data may also be generated real time and transmitted to the user using the present system. The images may be captured by any suitable digital or analog device such as an aircraft-mounted camera or a satellite-mounted camera.

[0004] In one embodiment, a relatively low resolution photographic image of a large geographic area is initially displayed to the user, and the user can then select a limited region of the image for closer viewing. The selected limited region of the image may then be downloaded to the user in a higher resolution format, thereby reducing the amount of high resolution data that must be transmitted to the user. Various display devices may be used, such as a personal data assistant (PDA), laptop computer, desktop computer, vehicle-mounted computer, tablet, cell phone and the like. In addition to displaying the desired image, the

759228v3

-22006201051 08 Mar 2006 display device may also be used as the input device to identify the geographic area of interest, e.g., by initially displaying a map of the region from which the specific area of interest is identified. In one embodiment, the photographic image may be overlaid over the map display.

[0005] An aspect of the present invention is to provide a system for displaying geographic images comprising an input device for selecting a geographic area of interest, and a display device for displaying photographic images corresponding to at least a portion of the selected geographic area, wherein the photographic images have different resolutions.

[0006] Another aspect of the present invention is to provide a system for storing, transmitting and displaying geographic images comprising a photographic image data storage device, an input device communicating with the photographic image data storage device for selecting a geographic area of interest, and a display device communicating with the photographic image data storage device for displaying photographic images corresponding to at least a portion of the selected geographic area, wherein the photographic images have different resolutions.

[0007] A further aspect of the present invention is to provide a method of displaying geographic images comprising selecting a geographic area of interest, and displaying photographic images corresponding to at least a portion of the selected geographic area, wherein the photographic images have different resolutions.

[0008] These and other aspects of the present invention will be more apparent from the following description.

BRIEF DESCRIPTION OF THE DRAWINGS [0009] Fig. 1 is a schematic diagram of a geographic image storage, transmission and display system in accordance with an embodiment of the present invention.

[0010] Fig. 2 is a flow diagram illustrating a method of identifying a geographic area of interest, downloading images corresponding to that area to a user, and displaying the images in accordance with a method of the present invention.

[0011] Fig. 3 is a partially schematic illustration of a map including overlaid frames corresponding to stored images taken from the region. The image data from part or all of the selected area may be downloaded and displayed in accordance with an embodiment of the present invention.

759228v3

-32006201051 08 Mar 2006 [0012] Fig. 4 illustrates a relatively large geographic area within which a smaller area is selected from high-resolution image downloading and display in accordance with an embodiment of the present invention.

[0013] Figs. 5-7 illustrate a relatively low resolution image of a large region that may be manipulated to select a smaller region within the area that may be efficiently downloaded and displayed at a significantly higher resolution in accordance with an embodiment of the present invention.

[0014] Fig. 8 is a schematic diagram illustrating details of a system in accordance with an embodiment of the present invention.

[0015] Fig. 9 is an image collection, catalog and logging flow chart illustrating operation of a system in accordance with an embodiment of the present invention.

DETAILED DESCRIPTION [0016] Fig. 1 schematically illustrates a geographic image storage, transmission and display system in accordance with an embodiment of the present invention. Photographic images are captured by an image sensor such as a digital or analog camera. The photographic images may be images taken in the visible region of the electromagnetic spectrum, other wavelengths such as infrared and the like. The image sensor may be mounted on any suitable platform which provides a desirable view of the geographic areas, such as aircraft and satellites.

[0017] As shown in Fig. 1, after the photographic images are captured by the image sensor(s), they may be transmitted to a server for storage and/or transmission. The server may be of any suitable configuration such as Microsoft Windows or Linux based portable computer (PC) systems. The image sensor communicates with the server by any suitable means such as standard commercial video interface (RS-170), video capture card, parallel data interface, network interface or the like. In one embodiment, the server may store multiple images of the same or similar geographic region taken at different times. In many instances, the most recent images are desired. However, in some cases it may be desirable to view earlier images of the particular geographic region.

[0018] In the embodiment shown in Fig. 1, the server communicates with both a high bandwidth user input/output device and a low bandwidth user input/output device. Examples of high bandwidth user input/output devices include personal data assistants, laptop

759228v3

-42006201051 08 Mar 2006 computers, desktop computers, vehicle-mounted computers, tablets, cell phones and the like. Examples of low bandwidth user input/output devices include personal data assistants, laptop computers, desktop computers, vehicle-mounted computers, tablets, cell phones and the like.

[0019] The server communicates with the high bandwidth user input/output device by any suitable means such as a standard wired network interface, or a wireless network interface. The server communicates with the low bandwidth user input/output device by any suitable means such as a standard synchronous or asynchronous serial interface connected to a UHF radio or dial-up phone connection. Although Fig. 1 illustrates an embodiment in which both high bandwidth and low bandwidth user input/output devices are used, it is to be understood that the present system may alternatively use only high bandwidth or low bandwidth user input/output devices.

[0020] Fig. 2 illustrates a method of identifying a geographic area of interest, downloading images corresponding to that area to a user, and displaying the images in accordance with an embodiment of the present invention. A geographic area of interest may be identified by various means such as a map, longitude/latitude coordinates, a global positioning system, photographs and the like. In one embodiment, a geographic map or photograph may be displayed on any suitable device such as a personal data assistant (PDA), laptop computer, desktop computer, vehicle-mounted computer, tablet, cell phone or the like. Alternatively, the user may identify longitude and latitude coordinates corresponding to a particular geographic area of interest. In another embodiment, a GPS system may be used to display a map of a geographic area in which a GPS transceiver is located. When identifying a geographic area of interest, it may be useful to adjust the scale of the region, e.g., by zooming in or zooming out of a map of the region. Furthermore, once a general geographic area of interest has been identified, it may be useful to adjust the viewed area, e.g., by moving East, West, North, South or combinations thereof on a display map of the area.

[0021] As shown in Fig. 2, once the geographic area of interest has been identified, images corresponding to that area may be downloaded. For example, photographic image data corresponding to the identified geographic area may be transmitted from a server via communication means to a high bandwidth and/or low bandwidth user output device, such as those described in connection with Fig. 1. In many cases, it is desirable to download the most recent images available for the geographic area of interest. However, it is also possible to select and download earlier images of the geographic area if desired. By downloading

759228v3

-52006201051 08 Mar 2006 geographic images obtained at different times, it is possible to display time-sequenced images such as movement of objects on the ground over a period of time.

[0022] The photographic images of the identified geographic area of interest are then displayed by means of any suitable display device such as a PDA, laptop computer, desktop computer, vehicle-mounted computer, tablet, cell phone or the like. In one embodiment, where a map is initially used to identify the geographic area of interest, the photographic images corresponding to the identified geographic area may be displayed together on the same display screen. For example, the photographic image may be displayed side by side with the map of the area, or the photographic image may be overlayed on the map of the area.

[0023] Fig. 3 illustrates a map of a geographic region 10 including overlaid frames 11 corresponding to individual photographic images taken of the region which are then combined together. If a selected geographic area 10 corresponds to more than one of the photographic images 11, those multiple images 11 may be combined together before or at the time the specific geographic area is selected for photographic viewing. Multiple images may be geograpically “stitched” together to create a single mosaic image of the geographic area in the requested area.

[0024] Fig. 4 illustrates a photographic image of a relatively large geographic area 10 in which a smaller geographic area 12 is selected for detailed viewing in accordance with an embodiment of the present invention. In accordance with this embodiment, it is possible to download a low-resolution photographic image of the large region 10, followed by downloading a higher-resolution photographic image of the particular small region 12 of interest. In this manner, significantly less data must be transmitted because only a limited portion of the geographic area is downloaded in a high-resolution format. The resolution of the high-resolution image may be at least 10 times greater than the resolution of the lowresolution image, typically at least 100 or 1,000 times greater. For example, a low-resolution image may have a resolution down to 4 or 5 meters per pixel, while a high-resolution image may have a resolution of 1 or 2 meters per pixel, or less.

[0025] The high-resolution and low-resolution photographic images may originate from the same image sensor, or different image sensors. For example, a relatively low resolution digital camera may generate the low-resolution photographic image, while a relatively high-Tesolution digital camera may generate the high-resolution photographic image corresponding to the relatively small geographic area 12 selected. Alternatively, a single

759228v3

-62006201051 08 Mar 2006 digital camera may be used to collect high resolution images, and the resolution of the large geographic area 10 initially transmitted to the user may be electronically reduced in resolution in order to decrease the amount of data image that must be initially transmitted to the user. In this embodiment, after the user selects the smaller geographic area 12 for detailed viewing, the higher-resolution image data may then be transmitted to the user.

[0026] Figs. 5-7 illustrate a photographic image of a large region that may be manipulated to select smaller regions within the area which may be efficiently downloaded and displayed at a significantly higher resolution in accordance with an embodiment of the present invention. In Fig. 5, the user requests photographic images via longitude/latitude coordinates or a map. A server then transmits a relatively low resolution photographic image of an area of interest. This low-resolution image may be downloaded and viewed from the server within a reasonable amount of time, e.g., less than 20 seconds. In Fig. 5, the full image is 9.8 km x 9.8 km at 0.6 m resolution. The full image corresponding to the entire photographic image shown in Fig. 5 includes approximately 270 million image pixels.

[0027] Based upon the PDA display shown in Fig. 5, the user may select a smaller geographic area for higher resolution photographic image display. Fig. 6 illustrates one such higher resolution photographic image display, which may be transmitted to the PDA within a reasonable time, e.g., less than 20 seconds.

[0028] If an even higher resolution image of a smaller geographic area is desired, the user may select, download and display such a photographic image, as illustrated in Fig. 7.

[0029] Table 1 summarizes the PDA display resolution (m/pixel), display coverage, transmission time, percentage of total pixels and cumulative percentage of pixels for various resolution layers, including those illustrated in Figs. 5-7.

Table 1

<td colspan="7"> PDA Display (320 x 240 pixels)</td>

<td> Display m/pixel</td><td> Display Coverage</td><td> Display Coverage (Km2)</td><td> Xmit Time 38.4 Kbps</td><td> % of Total Pixels</td><td> Cumulative % of Pixels</td><td> Layer</td>

<td> 9.6</td><td> 3.0 Km x 2.3 Km</td><td> 7.1 Km2</td><td> 16 sec</td><td> 0.03</td><td> 0.03</td><td> Res 4</td>

<td> 4.8</td><td> l.SKmxl.lKm</td><td> 1.7 Km2</td><td> 16 sec</td><td> 0.03</td><td> 0.06</td><td> Res 3</td>

<td> 2.4</td><td> 0.8 Km x 0.6 Km</td><td> 0.4 Km2</td><td> 16 sec</td><td> 0.03</td><td> 0.09</td><td> Res 2</td>

<td> 1.2</td><td> 0.4 Km x 0.3 Km</td><td> 0.1 Km2</td><td> 16 sec</td><td> 0.03</td><td> 0.12</td><td> Res I</td>

<td> 0.6</td><td> 0.2 Km x 0.1 Km</td><td> &lt;0.1Km2</td><td> 16 sec</td><td> 0.03</td><td> 0.15</td><td> ResO</td>

759228v3

2006201051 08 Mar 2006

-7[0030] As smaller and smaller geographic area photographic images are selected and displayed as shown in Figs. 5-7 and described in Table 1, extremely high resolution imagery is available to the user in a relatively short amount of time. For example, it may take less than one or two minutes to go from the low resolution, large geographic area shown in Fig. 5 to the high resolution, small geographic area shown in Fig. 7. In doing so, only about 0.15 percent of the original 270 million pixels are transmitted to the user.

[0031] Fig. 8 schematically illustrates an image collection, storage and dissemination system in accordance with an embodiment of the present invention. The system includes parallel and serial interfaces to one or more sensors, Input/Output Processor (IOP), storage interface and storage devices (e.g., disk drives or solid state memory), an Information Process Server (IPS), a Narrow Band Link configuration (serial interface, radio and antenna), and a Wide Bank Link configuration (network interface, network switch, network access point, RF convertor hardware and antenna).

[0032] As shown in Fig. 8, the present system collects imagery over the parallel interface through the IOP, and catalogs the data received and stores the data on the storage device(s). Narrow Band or Wide Band user requests received are handled by the IPS processor, which acquires the raw data logged on the storage device(s), creates the requested image product, and returns the product back over the corresponding data link.

[0033] As shown in Fig. 8, the system contains a Narrow Band Link which consists of a military grade tactical radio and compatible antenna configuration. The IPS processor communicates to the tactial radio data rates between 16K-bits and 78K-bits per second.

[0034] Fig. 8 also illustrates the Wide Band Link which consists of a connection from the IPS to the link via an ethemet interface (ΙΟΟΜ-bit per second) to a standard network switch, which is then routed by a network Access Point. Output from the Access Point is then up-converted to radio frequency (RF) through RF hardware for output through the antenna. Conversly, RF data is received through the antenna, down-converted by the RF hardware, pushed to the Access Point and enters the ALA server via the network interface hardware.

[0035] Fig. 9 is a flow chart summarizing the multiple image collection and storage, cataloging, and data logging functions in accordance with the present invention.

[0036] Whereas particular embodiments of this invention have been described above for purposes of illustration, it will be evident to those skilled in the art that numerous

759228v3

2006201051 08 Mar 2006 variations of the details of the present invention may be made without departing from the invention as defined in the appended claims.

[0037] Throughout this specification and the claims which follow, unless the context requires otherwise, the word comprise, and variations such as comprises and comprising, will be understood to imply the inclusion of a stated integer or step or group of integers or steps but not the exclusion of any other integer or step or group of integers or steps.

[0038] The reference to any prior art in this specification is not, and should not be taken as, an acknowledgement or any form of suggestion that the prior art forms part of the common general knowledge in Australia.

759228v3

-92006201051 12 Nov 2008