Systems and methods for processing images with edge detection and snap-to feature.

EDGE DETECTION AND ADJUSTMENT FEATURE

Field of Invention

The inventive concepts described and claimed herein relate generally to digital image processing and, more particularly, but not by way of limitation, to the discovery and selection of points of interest on an edge of interest in a digital image.

Background of the Invention

In the remote sensing / aerial imaging industry, imaging is used to capture views of a geographic area to have the ability to measure objects and structures within images as well as to have the ability to determine geographic locations. points within the image. Such image shaping is described, for example, in US Patent Application Publication No. 2008/0231700. Photogrammetry is the science of obtaining measurements from and between objects displayed within photographs, especially aerial photographs. A person, or user, can interact with the image to select the Points of Interest that can be used to carry out other functions such as: determining the distance between the two points; determine the area delineated by a set of

Ref: 252860

IMPI

MEXICAN INSTITUTE OF INDUSTRIAL FROFIEDAD

<img file="MX339747B_D0001.tif" />

points; determine the volume in a VHimpnsinTial way n other functions. Usually, the largest error induced in the calculation of the function is the human error introduced by the User who selects the Points of Interest. For example, if a measurement from a building to the edge of a trim is desirable and the user selects a point that is close to the edge of the building, but not the actual edge of the building, then the measurement will differ from the actual measurement by distance. in which the selected User is away from the building. In the projected image, several pixels could significantly alter the accuracy of the measurement. Additionally, when multiple Users perform the same measurement, their results may differ significantly.

In view of the above, there is a need for a system and process to allow a User to select Points of Interest based on the edge elements that exist in a geo-referenced image, where Points of Interest are determined without error. caused by human determination of the position of the edge of an element in an image.

Brief Description of the Invention

One method of creating image products includes the following stages. The image data and the position data corresponding to the image data are

IMPI

INSTITUTO MEXICANO OE LA RROFIEUAO INDUSTRIAL

<img file="MX339747B_D0002.tif" />

captured and processed to create georeferenced images. Edge detection procedures are performed on georeferenced images to identify edges and produce georeferenced, edge detected images.

In one embodiment, a computerized system includes a computer system for storing a database of the captured oblique images with the corresponding geolocation data and the corresponding detected edge data. The computer system has a computer-executable logical device which, when executed by a processor, causes the computer system to receive a selection of a geographic point from a User, search the database to find images that contain the selected point, and make the images containing the selected point available to the User.

In another embodiment, a method of providing images to a User includes the following steps. A database stores the captured oblique images that have corresponding geo-location data and corresponding detected edge data. A selection of a geographic point is received from a user and the database is then searched to find images that contain the selected geographic point. Images containing point

IMPI

MtXICANO INSTITUTE OF INDUSTRIAL PROPERTY

<img file="MX339747B_D0003.tif" />

selected geographic area becomes pntnnres. that are available to the user.

In yet another embodiment, a sequence of instructions is stored on at least one non-transient, computer-decipherable medium to run on a computer system capable of displaying and navigating the images. The sequence of instructions includes instructions to cause the computer system to display a geo-referenced representation of the pixels of an edge detected image, wherein the pixel representation includes one or more detected edges in the detected image in the edge, geo-referenced; instructions to cause the computer system to allow the User to select one of the one or more detected edges by moving a cursor over a region of interest (which can be magnified) where the cursor is caused to snap to the selected detected edge when the cursor is within a predetermined distance from the selected detected edge; instructions to cause the computer system to allow the User to accept the selected detected edge as an edge of interest; and instructions to cause the computer system to allow the User to determine and store one or more points of interest along the border of interest.

In yet another embodiment, a system for preparing

IMPI

INSTITUTO MEXICANO M LA «ONEOAD INRUSTtIAL

<img file="MX339747B_D0004.tif" />

and utilizing geo-referenced images includes one or more image and data files accessible by a computer system capable of displaying and navigating digital images, the image and data file including a plurality of image files, the edge information detected corresponding to the plurality of image files, and the position data corresponding to the plurality of image files; and the software for analyzing and displaying the images stored on a non-transitory computer-decodable medium and executable by the computer system. The image analysis and display software causes the computer system to allow a user to download and display, from the image and data archive, a pixel representation of an image that has a plurality of edges detected within the image, and to select a detected edge within the pixel representation by moving a cursor over the pixel representation, wherein the cursor is caused to snap to a selected detected edge when the cursor is within a predetermined distance from the selected detected edge. Image display and analysis software also causes the computer system to allow the user to accept the selected detected edge as an edge of interest; and to allow the user to determine and store one or more points

IMPI

INSTITUTO MEXICANO DE LA PROntOAU INDUSTRIAL

<img file="MX339747B_D0005.tif" />

of interest along the edge of interest.

Thus, using (1) the technology known in the art; (2) the aforementioned general description of the inventive concept (s) currently described and claimed; and (3) the figures and the detailed description of the inventive concepts that follow, the advantages and novelties (of the) inventive concept (s) described and claimed (s) are readily apparent to a person with experience. ordinary in art.

Brief Description of Figures

Like reference numerals in the figures represent and refer to the same item or function. The implementations of the description can be better understood when considering the following detailed description thereof. Such description refers to the accompanying image illustrations, schematic figures, graphs, sketches and annexes. In the figures:

FIG. 1 is a flow chart showing an exemplary process for selecting a Point of Interest on an edge element in a digital photographic image.

Figure 2 is a block diagram of an exemplary computer system in accordance with the present disclosure.

Figure 3 illustrates an exemplary selection of a region of interest on an image.

IMPIéB

INSTITUTO MEXICANO 06 THE PROPERTY

INDUSTRIAL ^ Ñ5l4®

Figure 4 illustrates an exemplary magnification of the region of interest on an image such that the magnified region of interest is a subset of the entire image and surrounding non-magnified areas are still visible to the User.

Figure 5 illustrates an exemplary image showing numerous detected edges.

Figure 6 is a flow chart depicting a User selection of an edge of interest in one embodiment of the invention.

Figure 7 is a flow chart depicting a User selection of a point of interest in one embodiment of the invention.

Figure 8 illustrates an image showing a selected edge of interest and the selected points of interest.

Detailed description of the invention

Before explaining at least one embodiment of the concepts of the invention described here in detail, it will be understood that the concept (s) of the invention is (are) not limited in its application to the construction details and the arrangement of the components or stages or methodologies described in the following description or illustrated in the figures. The concept (s) of the invention described herein are capable of other modalities or

IMPI

INSTITUTO MEXICANO DE LA FXOMEDAU industrial

<img file="MX339747B_D0006.tif" />

to be practiced or carried out in various ways. Also, it is to be understood that the phraseology and terminology used herein is for the purpose of description and should not be construed as limiting.

In the following detailed description of the modes of the description, numerous specific details are described to provide a more complete understanding of the description. However, it will be apparent to a person with ordinary experience in the art that the concepts within the description can be practiced without these specific details. In other cases, well-known features have not been described in detail to avoid unnecessarily complicating the description.

The concept (s) of the invention described here are directed to methods and systems for creating image products, where the image data is captured, in company with the position data corresponding to the image data. Image and position data are processed to create a plurality of geo-referenced images. In an embodiment described here, edge detection procedures are performed on the plurality of geo-referenced images to identify edges and produce geo-referenced edge-detected images that are stored in a database.

IMPÍS

INSTITUTO MEXICANO ΠΕ LA WOftfcDAD industrial data. A computer system that stores such a database of captured oblique images that has corresponding geo-location data and corresponding detected edge data, has computer-executable logic that when executed by a processor causes the computer system to receive a selection of a geographic point from a user, search the database to find the images that contain the selected point, and make the images containing the selected point available to the user. Alternatively, the databases can store the captured oblique images without the edge detected data, and the edges within such images can be detected in real time when the user is viewing the image (s).

In one embodiment, a sequence of instructions is stored on at least one non-transient computer-decipherable medium to be operated on a computer system capable of displaying and navigating the images. With reference now to the figures, and in particular to figure 1, in general, a User downloads a geo-referenced image as in step 10. The sequence of instructions optionally includes instructions for locating and selecting a region of interest (ROI) required by the user from a geo-referenced image downloaded as in step 12. It is

IMPI

INSTITUTO MEXICANO O »LA rnoPIBOAP industrial

<img file="MX339747B_D0007.tif" />

can provide additional instructions for amplifying the user-required region of interest (ROI) to provide an amplified region of interest (MROI) and for performing edge detection procedures on the region of interest (ROI) to provide a detected edge DE as in step 14. However, the detected edge data indicative of the detected edge DE may be provided in the downloaded image. The detected edge DE can be either a linear edge or a non-linear edge. In step 16, the instructions cause the cursor to snap to a detected edge (DE) when the user moves the cursor over the MROI and the cursor near the DE. Additional instructions allow the user to determine the points of interest (POI) on a selected DE as in step 18. It is already known that the accuracy of selecting a POI of an item in a digital photographic image can be improved by amplifying the ROI so that the user can select a point as close as humanly possible to the actual edge of the item. However, computerized edge detection is more accurate for determining an element's edge and provides consistency between multiple users and image interactions. It has been discovered that by combining ROI amplification with edge detection, ED, and a utility of

ΙΜΡΙ

INSTITUTO MEXICANO Di LA INDUSTRIAL PROPERTY

<img file="MX339747B_D0008.tif" />

By adjusting the cursor to snap to the edge ^^ t- ^ nted, a person can obtain synergistic improvements in the accuracy, precision, and consistency of measurements of, and between, objects displayed within the digital photographic image as in stage 20.

Referring now to Figure 2, the User preferably interacts with a first computer system 21 to view and download one or more geo-referenced images to a second computer system 22 (although the geo-referenced images may be stored locally). The first and second computer systems 21 and 22 may be a stand-alone computer, one or more computing device (s), and / or a distributed computing system. The standalone computer can be a personal computer such as a conventional desktop personal computer, a laptop or tablet, or a mobile computer terminal. The second computer system 22 preferably includes one or more processors 23, communication connections 24, input devices 26a and 26b, output devices 28a and 28b including a display device 28a, and at least one media decipherable by computer 30a and 30b. The first computer system 21 can be constructed in a similar way as the second computer system 22. Although the georeferenced image is often provided

IMPI Mexican institute Dt LA TtOtTEDAD INDUSTXIAL

<img file="MX339747B_D0009.tif" />

by downloading from an Internet site, 1^to · Georeferenced can be provided by having access to any source of data files that have image files and position data that correspond to the images. Image capture and geo-location systems may include image capture devices, such as, for example, cameras, digital cameras, digital sensors, conventional load-coupled devices, or other suitable image capture devices. capable of capturing images photographically or electronically, and positioning the information provided, for example, by a global positioning system and / or an inertial navigation unit. These systems and devices are well known to those skilled in the art.

The image analysis and display software may be stored on a non-transient computer-decipherable medium and / or executed by the first and / or second computer systems 21 and 22. The software that provides instructions for the display of a pixel representation Images are commercially available and well known to those skilled in the art. Image display and analysis software may include methods for utilizing oblique images such as those described in US Patent No. 7,787,859, the content of which is incorporated herein for reference.

IMPI

MEXICAN INSTITUTE OF INDUSTRIAL HRORIMad

<img file="MX339747B_D0010.tif" />

Computer-decryptable media 30a and / or 30b includes, for example, non-volatile, read-only memory, random access memory, hard disk memory, removable memory cards, and / or other devices and / or media. memory storage, adequate. Input devices 26a and 26b, such as, for example, a mouse, keyboard, joystick, or other such input device, capable of inputting data and of the interaction of a User with the display software and image analysis are executed by the first and / or second computer systems 21 and 22. In one embodiment, the first computer system 21 runs the image display and analysis software, and the second computer system 22 runs a communications software package, such as a web browser, to communicate with the first system. of computers 21 and display the geo-referenced images. An output device such as a display device 28a may include, for example, a liquid crystal display or the cathode ray tube, and displays the information to the User of the second computer system 22. Additional output devices 28b may include a second display device, printer, speaker, etc. The communications connections 24 connect the second computer system 22 to a network 32, such as, for example, a network

IMPI

INSTITUTO MÍXICANO DS IA RROWSBAtl INDUSTRIAL

<img file="MX339747B_D0011.tif" />

local area network, a wide area network, Tnt-A-rnsi- and / r&gt; the Global Broadband Network to establish communication with the first computer system 21.

The second computer system 22 typically uses a computer monitor as a display device 28a. These display devices often cannot display a complete image with the necessary detail. This may be due to the resolution of the information in the image and the resolution and size of the display surface. When a complete image cannot be displayed on the monitor in its entirety, the image that can be displayed, which is replaced by the complete image, is often a global image, that is, the complete image with the resolution removed to allow the complete image It fits on the display surface of the display device. Referring now to Figure 3 and Figure 4, to obtain the detail necessary to select a point of interest (POI), algorithms and software have been developed to let the user box in a sub-region or region of interest ROI 5 0 of a photograph or full image 52 and enlarge or enlarge this sub-region as a magnified region of interest MROI 54, thus allowing the User to select the points and lines of an enlarged detail image with greater ease and accuracy.

IMPI tNSTmrro Mexican OE LA FROFIEPAD INDUSTRIAL

<img file="MX339747B_D0012.tif" />

The detailed image or the magnified region of interest MROI 54 shows the details of the region of interest ROI 50, but when it shows only the global contexts of the details, the details are lost. Accordingly, in one embodiment, instructions and algorithms known to those skilled in the art are used to amplify a region of interest (ROI) 50 required by the user from image 52 and displaying the entire image or a portion thereof. , accompanied by a subsection of the display screen showing a linear amplification of the ROI 50 required by the user. This allows for the magnification of a particular ROI in an image while preserving the visibility of the larger image as shown in Figure 4.

In some applications, a user-selected non-linear amplification of the ROI 50 can be provided so that the connection between the detailed image and the immediately surrounding portion of the overall image is not obscured. Such methods are also known to those skilled in the art. Although the non-linear magnification creates a distortion-like lens relative to the original image, it still provides increased detail for the ROI 50 while maintaining connection to the immediately surrounding portion of the overall image.

IMPI

MEXICAN INSTITUTE OF THE TR OKI RUAD INDUSTRIAL

<img file="MX339747B_D0013.tif" />

The User typically interprets the image and decides which characteristics are to be measured. By positioning, for example, a mouse cursor, the approximately location of the features can be pinpointed with respect to an algorithm. Semi-automatic feature extraction is already known to those skilled in the art and is used to measure height points as well as to measure specific object corners. A User places the cursor at the same position in image 52 and the cursor snaps to the corner of the desired surface or object. However, the image around this selected point will usually contain gradients of gray values ​​caused by the edges of the objects. Therefore, the adjustment characteristic may be limited by the size and quality of the pixels.

The extraction of lines from digital images has also been investigated for many years. Semi-automatic algorithms have been developed, for example, by mining roads. Most of the algorithms of this class are based on so-called snakes. The extraction of roof-like objects from houses can be improved by algorithms that extract the homogeneous gray value. The algorithms used to find the boundaries of a homogeneous area are usually based on an algorithm of

IMPI

INSTITUTO MEXICANO OE LA MONEDAD

INDUSTRIAL

<img file="MX339747B_D0014.tif" />

growth of the region. A common interactive method is to allow the user to select an appropriate object model, and roughly align the object model to the image. An adjustment algorithm can then be used to find the best match between the edges of the object model and the location of high gradients in the image.⁽¹⁾

In one embodiment of the present disclosure, the sequence of instructions stored on at least one computer-decipherable medium 30a, 30b and / or the computer-decipherable medium 55 of the first computer system 21 for operation on the first and second computer systems 21 and 22 capable of displaying and navigating digital images, includes instructions for performing edge detection procedures on the user required region of interest and / or the full pixel representation of the image. Referring now to Figure 5, detected edges 56 may include linear edges 58 as well as curved edges 60. Edge detection is an image processing tool that identifies discontinuities in a digital image such as points at which the brightness or color of the image changes abruptly. The techniques, procedures, and algorithms for edge detection are commercially available and known from

IMPI Mexican institute DC LA r * Onn&gt; AD industrial

<img file="MX339747B_D0015.tif" />

those skilled in the art.

Referring now to FIG. 6, the sequence of instructions stored on the computer-decipherable media 30a, 30b and / or 55 forms a process 98 as described below. Process 98 includes instructions to allow the User to interact with the amplified region of interest MROI 54 as shown in step 100 to determine points of interest by moving the cursor over the MROI 54. Interaction with the User can be carried out with a mouse, touch panel, electronic pen or other means. Then, the first and second computer systems 21 and 22 detect a cursor position as indicated in step 102. Once the first and second computer systems 21 and 22 detect the cursor position, the instructions cause one or more processors 23 determine the distance between the cursor and a detected edge DE 56 in a step 104. If the distance is less than a predetermined distance, for example within 3 pixel lengths, the cursor is caused to snap with respect to this detected edge DE 56 in one step.

106. Fitting algorithms are already known to those skilled in the art.

In one embodiment, I instructions are provided in a step 108 to keep the cursor on the detected edge of 56 until the User indicates acceptance or

<img file="MX339747B_D0016.tif" />

IMPI iNSTTMTO MífcCAN »

DE LA RKORISDAD hdvstmal rejection of the detected edge DE 56 as an edge of interest

EOI 110 at step 112. During rejection, the cursor is allowed to move freely at step 114 and process 98 branches out at step 100 until the position of the cursor is again detected within a predetermined distance from it. another detected edge DE 56, at which point the cursor will again adjust to the detected edge DE 56. During the acceptance of the detected edge DE 56 as an edge of interest EOI 110, process 98 includes instructions in a step 116 to indicate, mark and store the edge of interest EOI 32 accepted on image 52 as shown, for example, in Figure 8. Acceptance can be indicated by any means of communication with the second computer system 22, such as, for example, by clicking the mouse or selecting Accept from an open dialog block. The rejection can be indicated in a similar way or by the lack of acceptance during a specified time.

Referring now to Figure 7, the sequence of instructions stored on computer-decipherable media 30a, 30b and / or 55 forms a process 118 as described below for the procedure once the user has accepted an edge of interest EOI 110 Process 118 includes instructions to allow the user to move the cursor along the edge of interest EOI 110 as

IMPÍ

INSTITUTO MEXICANO DI LA FTKTTieíIAD

INDUSTRIAL

<img file="MX339747B_D0017.tif" />

in a step 121 to select a POI point of interest 120. The first and / or second computer systems 21 and 22 detect the position of the cursor, and determine whether the cursor is at a distance greater or less than a predetermined distance from the edge of interest EOI 110 as in step 122. If the cursor has moved a greater amount than the predetermined distance from the edge of interest EOI 110, process 118 branches back to step 100 in process 98. If the cursor has moved less than the predetermined distance from the EOI edge of interest 110, the User can select a POI point of interest 120 along the detected edge DE that forms the EOI edge of interest 110. In a step 124, the User can select a POI point of interest 120 and accept or reject the POI, for example, by double clicking with a mouse or by any means of communication with the second computer system 22 as described above to accept or reject EOI edge of interest 110. If POI point of interest 120 is rejected, process 118 branches back to step 122. During acceptance of the POI 120 point of interest, instructions are provided in a step 126 to indicate, mark and store the accepted POI 120 point of interest on the image as shown, for example, in FIG. 8.

The user is then allowed to move the cursor in

IMPI

INSTITUTO MEXICANO IRE LA PROPERTY industrial

<img file="MX339747B_D0018.tif" />

a step 128 for selecting additional i poi points 120. Process 118 asks the user in a step 13 0 as to whether all of the POI 120 points of interest have been selected. The User may indicate by any means of communication with the second computer system 22, such as, for example, by selecting Do It All or Continue from an open dialog block. If additional POI 120 points of interest are desired, process 118 branches back to step 100 of process 98 and additional POI 120 points of interest can be identified using the procedure described above until the User has selected all points of interest. POI 120 interest needed to perform a desired function. If indeed, the user has selected all the necessary POI points of interest 120, the user can select a function to be performed or exit outside the amplified region of the MROI amplified region of interest 54 in a step 132.

Frequently, the User takes measurements of, and between, the objects shown in image 52 by selecting one of the various available measurement modes provided within the image display and analysis software. The User selects the desired measurement mode having access, for example, to a series of menus or advance toolbars or through commands from the

ΙΜΡΙ

MEXICAN INSTITUTE OF INDUSTRIAL PROPERTY

<img file="MX339747B_D0019.tif" />

keyboard. The measurement modes provided by the image analysis and display software may include, for example, a distance mode that makes it possible to measure the distance between two or more selected points, an area mode that makes it possible to measure the area encompassed by several selected and interconnected points, a height mode that makes it possible to measure the height between two or more selected points, and an elevation mode that makes it possible to measure the change in elevation of a selected point relative to one or more of the other selected points.

After selecting the desired measurement mode, the user selects a starting POI 120 and ending POI 120 'ending point on image 52, and the image display and analysis software calculates and automatically displays the desired amount. The accuracy, precision and consistency of these measurements is largely dependent on the procedure used to select POI 120 points of interest. By combining edge detection procedures with a snap function in an amplified region of interest, edges of interest can be identified and individual points of interest along these edges can be selected in a much more exact way. and consistent than what is currently available.

IMPI

INSTITUTO MSXICANO Dt LA PWOPTEDAD industrial

<img file="MX339747B_D0020.tif" />

In an embodiment described here, edge detection procedures are performed on the plurality of geo-referenced images to identify edges and produce edge-detected, geo-referenced images that are stored in a database on one or more computer decipherable media 30a, 30b and 55. Whether it is the first and / or second computer systems 21 and 22 that store such a database of the captured oblique images having corresponding geo-location data and corresponding detected edge data, it has a computer executable logic that when it is executed by a processor causes computer systems 21 and 22 to receive a selection of a geographic point from a user, Search the database to find images that contain the selected point, and make the images available to the user that contain the selected point for the user. Alternatively, the database can store the captured oblique images without detected edge data, and the edges within such images can be detected in real time when the user is viewing the image (s) and / or when the image (s) are being viewed. MROI 54 is showing details of a region of interest 50.

As will be appreciated by those of ordinary skill in the art, changes can be made in the construction and operation of the various components,

IMPI

INSTITUTO MEXICANO Of. IA ΜΟΛΕΟΛΟ industrial or in the stages or the

<img file="MX339747B_D0021.tif" />

elements and assemblies described herein sequence of steps of the methods depart from the spirit and scope of the inventive (s) described herein.

From the above description, it is clear that the inventive concept (s) described here are (are) well adapted to carry out the objects and to achieve the advantages mentioned here as well as those inherent in the inventive concept (s) described here. Although the presently preferred embodiments of the inventive concept (s) described herein have been described for the purposes of this description, it will be understood that numerous changes can be made which will be readily suggested by themselves to those. skilled in the art and that they are made within the spirit (of) inventive concept (s) described and claimed herein.

References (1) George Vosselman, Cartographic Feature Extraction, Delft University of Technology, Delft, The Netherlands.

It is noted that in relation to this date, the best method known to the applicant for putting the aforementioned invention into practice is the one that is clear from the present description of the invention.

described here without (of the) concept (s)

<img file="MX339747B_D0022.tif" />

<img file="MX339747B_D0023.tif" />

IMPI iwrrruTo mwocawo of the INDUSTRIAL ηβηεοΑο