Untitled record

Full description

Sister Ar'a's background

The present disclosure relates generally to the inspection and maintenance of a structure, and specifically to an inspection method using a suspended unmanned aerial vehicle (UAV) with a retractable crawler to inspect and maintain the structure 5. Furthermore, the present disclosure relates generally to the inspection and maintenance of curved ferromagnetic structures, such as pipes, and specifically to methods for automating an unmanned aerial vehicle (UAV) attached to such structures.

Inspection and maintenance of exposed metal assets, such as pipes, storage tanks, etc., can be difficult or impractical for humans in some environments.

In these cases, the use of automated unmanned aerial vehicle (UAV) vehicles can provide a viable alternative. However, it is usually preferable for this inspection and maintenance to be performed in direct contact with the asset, as opposed to hovering at a distance from the asset. However, it can be difficult for an unmanned aerial vehicle (UAV) 15 to land, attach, or maneuver in relation to the asset. Moreover, it may be

Pipes (and other structures with a curved surface) are difficult to inspect or maintain using an unmanned aerial vehicle (UAV), primarily because these assets have curved surfaces that are difficult to land, hang on, or maneuver near.

<p dir="rtl">20 Moreover, inspecting hard-to-access steel assets and gas facilities is a tedious task. For example, it is the periodic inspection of high assets found in refineries</p>

13019

-3-

Gas stations, offshore platforms and other facilities are of utmost importance to prevent malfunctions, leaks and shutdowns. These assets include high-altitude pipes and structures that are difficult to access during inspections. One way to inspect these assets is to erect scaffolds for inspectors to access the asset and perform manual inspection, for example using a sensor

<p dir="rtl">5 Ultrasonic testing (UT) to measure thickness. This scaffolding installation is not expensive for frequent inspections, but it does involve safety concerns mainly in terms of fall and trip hazards.</p>

Chinese Patent No. 106647790 relates to a flight system for an unmanned aircraft with four rotors adapted to a complex environment and a method for flying this aircraft. The drone's flying system includes:

<p dir="rtl">10 Unmanned aerial vehicle with four rotors on a propeller, motor, motor arm, flight control unit, electronic speed regulator, upper drag and damping device, laser detector, depth camera, battery, on-board computer, operating status indicator, positioning system unit GLOBAL POSITIONING SYSTEM, frame and ultrasonic transducer, which arranges the placement of the propeller on the motor; Motor mode setting, flight control unit, status indicator</p>

<p dir="rtl">15 operation, speed regulator, and GLOBAL POSITIONING SYSTEM unit on the aircraft's engine boom; The upper traction and damping device and the actuator arm are connected to the frame; The frame is arranged on the bottom of the unmanned aerial vehicle and is used for stable landing in take-off and landing operations; The laser radar and depth camera are connected to a device at the top of the traction and damping system. Arranges placement</p>

<p dir="rtl">20 The battery is on top of the unmanned aerial vehicle; The on-board computer is arranged in the middle of the plane; The ultrasound transducer is arranged underneath the computer and facing the floor. The unmanned aerial vehicle provided by this invention has a high sensing capacity and is capable of flying automatically and without being tied to a ground station.</p>

Chinese Patent No. 106931963 discloses a method and system for determining locations with a drone.

<p dir="rtl">25 Unmanned aerial vehicle capable of sharing ocean data, and a data sharing platform</p>

13019

-4-

Ocean and drone unmanned aerial vehicle. The positioning method includes the following steps: The environment data sharing platform (1) collects and shares the ocean data; the unmanned aerial vehicle (2) performs visual positioning based on the ocean data. The positioning system includes

<p dir="rtl">5 On an environment data sharing platform (1) capable of collecting and sharing ocean data, and an unmanned aerial vehicle (2) the environment data sharing platform (1) includes a storage device and a port Communication port (4), the storage device (3) saves a database of ocean data, and uses the communication port</p>

<p dir="rtl">10 (4) In accessing the database and in radio communication with the unmanned drone</p>

<p dir="rtl">2) Aerial vehicle. The unmanned aerial vehicle (2) includes a communication device (5) to receive the surrounding data, a processor (6) to process the surrounding data, and a camera and/or laser radar (7) used for visual location identification. 2) The unmanned aerial vehicle carries out visual identification of the location via a camera and/or laser radar.</p>

<p dir="rtl">15 7(laser radar) based on ocean data.</p>

It is for the aforementioned problems and other problems in the technical field that the present disclosure is directed, so as to provide a technical solution for an effective unmanned aerial vehicle (UAV) suspended vehicle with an editable crawler for inspection or maintenance of the structure. It is also for the indicated problems and other problems in the technical field that are addressed

<p dir="rtl">20 The present disclosure provides a technical solution for automated methods for attaching an unmanned aerial vehicle (UAV) to pipes and other assets.</p>

General description of the invention

According to one embodiment, an unmanned aerial vehicle (UAV) is provided to self-suspend on a curved surface from a starting location away from the curved surface. UAV included

<p dir="rtl">25 On: a three-dimensional (3D) depth camera configured to capture and output point clouds</p>

13019

-5-

3D views from an unmanned aerial vehicle (UAV) including a curved surface; A 2D light detection and ranging system (LIDAR detection and ranging) is configured to capture and output 2D slices of scenes; And a control circuit.

The control circuit is configured to: control the depth camera and the light detection and ranging system

<p dir="rtl">5 2D LIDAR (light detection and ranging) to capture 3D point clouds and 2D slices, respectively, of scenes; inserting 3D point clouds captured from the depth camera and 2D slices captured from the LIDAR system; Automated detection and localization of curved surfaces using captured 3D point clouds and captured 2D slices; Automatic guidance of an unmanned aerial vehicle (UAV) from the starting position to another position</p>

<p dir="rtl">10 Landing on a curved surface based on automatic detection and localization of the curved surface.</p>

In one embodiment, the controller is further configured to: use captured 3D point clouds to perform automated detection and placement from a starting position while simultaneously automatically guiding an unmanned aerial vehicle (UAV) to a landing position until an unmanned aerial vehicle approaches Pilot an unmanned aerial vehicle (UAV) from a curved surface; 15 Switching from using 3D point clouds to using 2D captured slices to perform detection.

Automated positioning as soon as an unmanned aerial vehicle (UAV) approaches the curved surface.

In one embodiment, the control circuit is further configured to: automatically guide an unmanned aerial vehicle (UAV) into an alignment position near a curved surface before

<p dir="rtl">20 landing, the alignment position being where the UAV is aligned relative to the landing position; And using the captured two-dimensional slices to automatically guide an unmanned aerial vehicle (UAV) to move directly from the alignment position to the landing position.</p>

In one embodiment, the control circuit is further configured for automatic detection and localization of the curved surface by combining 3D point clouds and 2D splines. The merger includes the following:

13019

-6-

Using one of the 3D point clouds to perform a first pass for automated detection and localization of the curved surface; capturing a corresponding slice from the 2D slices using the exposed curved surface positioned from the first pass; And using a corresponding 2D slice to perform a first pass for automated detection and localization of the curved surface.

<p dir="rtl">5 In one embodiment, the unmanned aerial vehicle (UAV) further includes an inertial measurement unit (IMU) configured to estimate the position of the depth camera, wherein the control circuit is further configured to use the estimated position of the depth camera between the acquisition of a first cloud of 3D point clouds and capturing a second cloud of 3D point clouds to predict the location of the curved surface in the second cloud of 3D point clouds 10 of the exposed curved surface located in the first cloud of 3D point clouds.</p>

In one embodiment, the control circuit is further configured to: use captured 3D point clouds for automated detection and localization of a curved surface; And controlling a 2D light detection and ranging system (LIDAR) to capture 2D slices 15 normal to the exposed and positioned surface.

In one embodiment, the curved surface is cylindrical, and the control circuit is further configured to control a 2D light detection and ranging system (LIDAR) to capture 2D slices normal to the central axis of the cylinder.

In one embodiment, the control circuit is further configured to use a random sample matching method

<p dir="rtl">20 Random sample consensus (RANSAC) for automatic detection and localization of curved surfaces.</p>

In one embodiment, the control circuit is configured to: use captured 3D point clouds, captured 2D slices, or both captured 3D point clouds and captured 2D slices to detect one or more obstacles on the flight path of an air vehicle Unmanned aerial vehicle (UAV) from the starting position to the landing position; And replay

13019

-7-

Automatic guidance of an unmanned aerial vehicle (UAV) to avoid an obstacle or obstacles on the flight path.

In one embodiment, the curved surface is ferromagnetic and the unmanned aerial vehicle (UAV) further includes magnetic legs configured for magnetic coupling

<p dir="rtl">5 to the ferromagnetic curved surface during landing and remains magnetically attached to the ferromagnetic curved surface after landing.</p>

According to another embodiment, a method is provided for automatically suspending an unmanned aerial vehicle (UAV) onto a curved surface from a starting location away from the curved surface. The method includes: pick-up and ejection, using a three-dimensional depth camera

<p dir="rtl">10 A dimensional (3D) depth camera attached to an unmanned aerial vehicle (UAV), for 3D point clouds of UAV scenes including a curved surface; and capture and output, using a 2D light detection and ranging (LIDAR) system attached to an unmanned aerial vehicle (UAV), for 2D slices of scenes; And control the depth camera</p>

<p dir="rtl">15 A LIDAR system for capturing 3D point clouds and 2D slices, respectively, of scenes; Inserting 3D point clouds captured from a depth camera and 2D slices captured from a 2D light detection and ranging system (LIDAR) Automated detection and localization of curved surfaces using captured 3D point clouds and captured 2D slices; Automated guidance of an unmanned aerial vehicle</p>

<p dir="rtl">20 UAV (unmanned aerial vehicle) from the start position to the landing position on the curved surface based on automatic detection and localization of the curved surface.</p>

In one embodiment, the method further comprises: using captured 3D point clouds to perform automated detection and placement from a starting position while simultaneously automatically guiding an unmanned aerial vehicle (UAV) to a landing position until the UAV approaches the surface 25 curved; Switching from using 3D point clouds to using 2D slices

13019

-8-

Dimensions captured to perform automated detection and localization as soon as the UAV approaches a curved surface.

In one embodiment, the method further includes: automatically guiding the unmanned aerial vehicle (UAV) to an alignment position near the curved surface prior to landing,

<p dir="rtl">5 The alignment position is where the UAV is aligned relative to the landing position; And using the captured two-dimensional slices to automatically guide an unmanned aerial vehicle (UAV) to move directly from the alignment position to the landing position.</p>

In one embodiment, the method further includes automated detection and localization of the curved surface by combining 3D point clouds and 2D splines. The combination includes the following: use

<p dir="rtl">10 One of the 3D point clouds to perform a first pass for automatic detection and localization of the curved surface; capturing a corresponding slice from the 2D slices using the exposed curved surface positioned from the first pass; And using a corresponding 2D slice to perform a first pass for automated detection and localization of the curved surface.</p>

In one embodiment, an unmanned aerial vehicle (UAV) is further included

<p dir="rtl">15 On an inertial measurement unit (IMU) attached to the UAV, the method further includes:</p>

Estimating the position of the depth camera using an inertial measurement unit (IMU). And using the estimated position of the depth camera between capturing a first 3D point cloud and capturing a second 3D point cloud to make predictions.

<p dir="rtl">20 The location of the curved surface in the second cloud of 3D point clouds from the exposed curved surface located in the first cloud of 3D point clouds.</p>

In one embodiment, the method further includes: using captured 3D point clouds for automated detection and localization of the curved surface; And control the light detection and ranging system

13019

-9-

2D LIDAR (light detection and ranging) to capture 2D slices normal to the exposed and positioned surface.

In one embodiment, the curved surface is cylindrical, and the method further includes controlling a 2D light detection and ranging system (LIDAR) 5 to capture 2D slices normal to the central axis of the cylinder.

In one embodiment, the method further includes using a random sample consensus (RANSAC) method for automated detection and localization of the curved surface.

In one embodiment, the method further includes: using captured 3D point clouds, captured 2D slices, or both of the captured 3D point clouds

<p dir="rtl">10 Two-dimensional slices captured to detect one or more obstacles on the flight path of a vehicle</p>

Unmanned aerial vehicle (UAV) from the starting position to the landing position; Automatic redirection of an unmanned aerial vehicle (UAV) to avoid an obstacle or obstacles on the flight path.

In one embodiment, the curved surface is ferromagnetic and includes an unmanned aerial vehicle

<p dir="rtl">15 The UAV (unmanned aerial vehicle) also has magnetic legs, and the method further includes attaching the magnetic legs to the ferromagnetic curved surface during landing and remaining magnetically attached to the ferromagnetic curved surface after landing.</p>

Any combinations of the various embodiments and applications disclosed in the present invention may be used. These and other aspects and features will be understood based on the detailed description of the models

<p dir="rtl">20 specified together with the figures and attached protection elements.</p>

Brief explanation of the drawings

Figures 1a and 1b are illustrations of hanging a representative unmanned aerial vehicle (UAV) on a structure (e.g. a tube), where the UAV is

13019

-10-

Having an editable crawler for previewing or maintaining the structure, according to an embodiment. A crawler attached to an unmanned aerial vehicle (UAV) is shown in Figure 1a and not attached to an unmanned aerial vehicle (UAV) (for example, while crawling on a structure) in Figure 1b.

<p dir="rtl">5 Figure 2 is a rectangular diagram of an automated guidance and landing method for landing an unmanned aerial vehicle (UAV) on a curved surface (e.g., a pipe), according to one embodiment.</p>

Figures 3a, 3b, and 3c are oblique projections of the annotation steps for an unmanned aerial vehicle (UAV) to be annotated onto a target structure (e.g., a tube), including approaching the target, aligning with the target, and annotating the target, 10 respectively. According to one embodiment.

Figure 4 is a workflow diagram of an illustrative sequential path planning method for automated deployment and landing of an unmanned aerial vehicle (UAV) on a curved surface (e.g., a pipe), according to one embodiment.

Figure 5 is a diagonal diagram of an illustrative method for detecting, positioning, and aligning targets

<p dir="rtl">15 Using sensor fusion, according to one embodiment.</p>

Figure 6 is a workflow diagram of a method for detecting circuits (e.g., a tube) using 2D light detection and ranging (LIDAR) sensor data collection, according to an embodiment.

Figure 7 is a workflow diagram of an illustrative adjacency detection method, according to one

<p dir="rtl">20 Models.</p>

Figure 8 is a workflow diagram of an illustrative pipelining method based on the Kalman filter framework, according to one embodiment.

13019

-11-

Figure 9 is a workflow diagram of an illustrative piping placement method using image tracking and a Kalman filter, according to one embodiment.

Figure 10 is a workflow diagram of an illustrative method for automatically suspending an unmanned aerial vehicle (UAV) on a curved surface from a starting position away from the curved surface 5, according to one embodiment.

Note that the figures are illustrative and are not intended to quantify scope, and that identical or similar items may have identical or similar figure reference numbers.

Detailed description:

<p dir="rtl">10 In various illustrative embodiments, an unmanned aerial vehicle (UAV) suspended with a releaseable crawler is provided for inspecting or maintaining a structure, such as a difficult-to-access pipe or storage tank. An unmanned aerial vehicle (UAV) is a hybrid UAV with advanced capabilities for conducting contact inspections on ferromagnetic surfaces such as pipes and carbon steel structures.</p>

<p dir="rtl">15 An unmanned aerial vehicle (UAV) can fly into a pipe</p>

It is required to inspect it, descend on it robotically (commonly referred to as hanging), and deploy a releaseable crawler to crawl around the pipe to perform, for example, elaborate maintenance operations.

As noted above, inspection and maintenance of exposed metal assets, such as pipes, storage tanks, etc., can sometimes be difficult or

Not practical for people. For example, a major challenge in the oil and gas industry is the regular inspection of elevated assets located at refineries, gas terminals, offshore platforms, terminals and other facilities. These assets include high-rise pipes and structures that are difficult to access during inspection or maintenance operations. Sometimes, it is

13019

-12-

The only way for people to inspect or maintain these assets is to erect scaffolds so that the inspector or engineer can access the asset and, for example, perform a manual inspection using an ultrasonic testing (UT) sensor to measure the thickness. The erection of scaffolds referred to is not expensive for frequent inspections, but it does present safety concerns5 mainly in terms of fall and trip hazards.

Thus, in illustrative embodiments, an unmanned aerial vehicle with a releasable crawler provides a solution to the above technical problems by having two vehicles in a parent/child configuration. Each vehicle is designed or developed to implement capabilities that best suit it. The vehicles include an unmanned aerial vehicle (UAV).

<p dir="rtl">10 An aerial vehicle is capable of levitating and landing on a tube, a smaller magnetic crawler that is carried by and released from an unmanned aerial vehicle (UAV) after landing or suspension. The crawler can move around pipes (for example, using magnetic wheels) and carry out, for example, scanning operations, such as thickness measurements using an ULTRASONIC TESTING sensor, or</p>

<p dir="rtl">15 Other inspection or maintenance operations. This provides a more feasible approach than having a crawler (unmanned aerial vehicle) entirely around the pipe, which requires larger and heavier engines, and is at risk of colliding with pipes and nearby assets, particularly those with restricted clearances.</p>

In various other illustrative embodiments, automation methods are provided for suspending a 20 unmanned aerial vehicle (UAV) on pipes. Indicated automation methods can provide

There are also systematic methods for suspending an unmanned aerial vehicle (UAV) that are automated and safer. The automation method also provides for the suspension (landing) of an unmanned aerial vehicle (UAV) on hard-to-reach steel structures, including pipes, for example, to carry out inspections. In some of these models, this is done

<p dir="rtl">25 Sending an unmanned aerial vehicle (UAV) to attach, for example</p>

13019

-13-

For example, on an elevated target (such as a pipe), and release a crawling robot that performs a preview operation. In some of these embodiments, the UAV preview task is primarily monitored and controlled by an operator who sends the UAV airborne near the target. It should be noted that while in Some of these models are an unmanned aerial vehicle (UAV) with a releaseable crawler,

<p dir="rtl">5 In other models the UAV does not include an editable crawler.</p>

As noted above, pipes (and other structures with a curved surface) can be difficult to inspect or maintain using an unmanned aerial vehicle (UAV), primarily due to the fact that these assets have curved surfaces that are difficult to land and suspend. Moreover, it can be difficult to access

<p dir="rtl">10 These assets during inspections. Furthermore, erecting scaffolds may not be practical or feasible to access some parts of the asset, at least for human operators. Furthermore, when operating an unmanned aerial vehicle (UAV), human operators can lack the tools to properly perform a hold maneuver on a distant asset. For example, although an operator can have a first-person perspective of the target</p>

<p dir="rtl">15 With an on-board camera, human-level reception accuracy can be inadequate for reliable feedback, and system integrity can be compromised if done manually. Furthermore, robotically landing an unmanned aerial vehicle (UAV) onto a viewing target such as a pipe is a technically challenging task.</p>

Thus, in some embodiments, a complex system comprising one or more environmental sensors is used

<p dir="rtl">20 Multiple devices, intelligent guidance and landing method, and effective mechanical suspension mechanism to achieve successful and effective landing maneuvers on elevated assets, such as pipes. For example, to increase the level of safety as well as the accuracy of maneuvering of an unmanned aerial vehicle (UAV), an automated route is provided using on-board sensors, such as a depth camera and a light detection and ranging (LIDAR) scanner. In some embodiments, a laptop computer is used</p>

<p dir="rtl">25 On board the vehicle to process sensor measurements for accurate detection and placement</p>

13019

-14-

To target the unmanned aerial vehicle (UAV), plan a hold path, and finally control the unmanned aerial vehicle to perform the hold step.

Figures 1a and 1b are illustrations of a representative 5 100 unmanned aerial vehicle (UAV) suspended on a 50 structure (e.g. a tube), where

crawler 100 unmanned aerial vehicle

<p dir="rtl">130 Editable to preview or maintain the structure 50 structure, according to an embodiment. A crawler 130 that is attached to an unmanned aerial vehicle (UAV) 100 is shown in Figure 1a and is not attached to an unmanned aerial vehicle (UAV) 10 100 (e.g., while crawling on structure 50) in appearance</p>

<p dir="rtl">1b. For ease of description, structure 50 is assumed to be larger (e.g. substantially larger) than an unmanned aerial vehicle (UAV) 100. For example, structure 50 is larger in all dimensions than an unmanned aerial vehicle (UAV). 100 unmanned aerial vehicle, or structure 50 comprising a larger landing space</p>

<p dir="rtl">15 It has space for a 100 UAV. Furthermore, for ease of description it is assumed that structure 50 (or any other structure described in the present application) is a pipe, such as a pipe with a diameter of 20.32 cm or more.</p>

Figures 1a and 1b show the parent-child configuration in practice. Figure 1a shows an unmanned aerial vehicle (UAV) after landing on the 50 pipe with 20 crawlers remaining attached to it. Figure 1b shows crawler 130 after editing it with

UAV to perform the inspection. The crawling capability provided by the releasable crawler 130 gives an unmanned aerial vehicle (UAV) important features 100 for inspection and maintenance operations, such as easy access (for example, the landing does not have to be in the same spot where it occurs inspection or maintenance).25 Creeping also provides circumferential and longitudinal surveys. For example, in the oil and gas industry,

13019

-15-

It is necessary to perform complete scans on the 50 pipe to find the minimum steel thickness in a specific area of the 50 pipe. Referenced scans usually include circumferential and longitudinal scans, for which creep is suitable. Creeping also provides efficiency during multiple sampling operations (for example, creeping between multiple sampling positions 5 on the same pipe is more efficient than flying).

In Figures 1a and 2b, the unmanned aerial vehicle 100 uses four articulated magnets 120 (such as permanent magnets or interchangeable permanent magnets). To accommodate landing of the unmanned aerial vehicle (UAV) 100 on The tube is 50 pipe, each of the pieces is separated

<p dir="rtl">10 120 magnetism (or specifically its magnetic field) in a direction perpendicular to the tube</p>

50 pipe When the 100 unmanned aerial vehicle (UAV) lands or gets stuck on the pipe.50 pipe

In some embodiments, the magnetic fields of the four articulated 120 magnets are switchable on and off (for example, to allow

<p dir="rtl">15 (e.g. light detection and ranging, or LIDAR) A laser scanner 110 is included to measure, for example, the relative position of the pipe in relation to an unmanned aerial vehicle (UAV) 100 Unmanned aerial vehicle during an automated landing maneuver as a form of real-time feed. In some embodiments, the mini crawler 130 is connected by a wire</p>

<p dir="rtl">20 (e.g., for power and connectivity) and includes an ULTRASONIC TESTING sensor, four magnetic wheels 140, and two wheel drive motors 140 in symmetrical pairs (e.g., front and rear). The wire is also used for other electronic components and batteries. For carrying out inspection or maintenance, it should be located in the main body of an unmanned aerial vehicle (UAV) 100. This reduces the size and weight of</p>

25 The complexity of the crawl is 130 crawler.

13019

-16-

In some embodiments, the crawler 130 includes a different number of wheels 140 (e.g., two, three, or more than four wheels) of different types (e.g., all-in-one wheels and separate wheels, but not limited to). In contrast to the unmanned ground vehicle (UGV), it must handle...

5 The 130 magnetic crawler with different curvatures and directions (as...

(shown) to inspect or maintain the pipe. As such, in some embodiments, the magnetic crawler 130 includes special transportation systems for navigating the bends of pipes (or similar bends of other curved structures or vessels).

In some embodiments, the connection is between the crawler 130 and an unmanned aerial vehicle

<p dir="rtl">10 100 (UAV) unmanned aerial vehicle wired. For example, using a small roller</p>

From a thin wire, the 130 crawler can be connected to an unmanned aerial vehicle (UAV).

<p dir="rtl">100 Unmanned aerial vehicle for power and connectivity. This could, for example, eliminate the need to embed a battery and other electronic components inside the crawler 130, making it smaller and saving overall weight by using 15 some components already found in an unmanned aerial vehicle (UAV). )</p>

.100 vehicle

In some other embodiments, the connection is between the crawler 130 and an unmanned aerial vehicle

<p dir="rtl">100 (UAV) unmanned aerial vehicle wirelessly. In this case, the creep includes</p>

<p dir="rtl">130 crawler on the battery and its electronic components, to provide a more distinctive vehicle.</p>

<p dir="rtl">20 This can be useful, for example, when you capture an unmanned aerial vehicle (UAV).</p>

100 Unmanned aerial vehicle 130 crawler from the ground and spread it on the pipe

<p dir="rtl">50 pipe, at which point the UAV can fly off to do some more inspections and then return to pick up the 130 crawler. This can also be useful for multiple crawlers 130 (e.g., a swarm of crawlers 130) to preview</p>

25 Multiple assets, where an unmanned aerial vehicle (UAV) operates

13019

-17-

<p dir="rtl">100 It picks them up one by one or in groups from the ground towards their destination and retrieves them when the process is complete. In various embodiments, radio communication can be between the crawler 130 and an unmanned aerial vehicle (UAV) 100 or operator control station, or both the UAV 100 and the operator control station.</p>

<p dir="rtl">5 In one embodiment, the unmanned aerial vehicle (UAV) 100 includes a body structurally designed to enable the 100 UAV to fly (e.g., motors, control and steering devices, etc.). The 100 UAV can further comprise three One or more legs attached to the body and adapted to suspend an unmanned aerial vehicle (UAV) 100 flying on a curved ferromagnetic surface 50.</p>

<p dir="rtl">10 Each leg includes an upper (or main) portion attached to the body, and a lower portion comprising a permanent magnet 120. The lower portion is configured to magnetically attach the leg to the ferromagnetic surface 50 during landing and to maintain the attachment of the leg magnet to the ferromagnetic surface during suspension. Passively hinges the upper and lower parts of the leg, and negatively articulates (i.e., centers) the lower part relative to the upper part in response to the bottom being close to the surface</p>

<p dir="rtl">15 Ferromagnetic 50 during landing. The UAV 100 also includes a releaseable crawler 130 with magnetic wheels 140. The magnetic wheels 140 allow the crawler 130 to detach from an unmanned aerial vehicle (UAV) 100 while suspended, and to maneuver the crawler 130 on the ferromagnetic surface 50 while the crawler is magnetically attached. 130 crawler with ferromagnetic surface 50 after disassembly.</p>

<p dir="rtl">20 In different embodiments, different landing mechanisms may be used for the 100 unmanned aerial vehicle (UAV). These mechanisms include different types of adhesion mechanisms such as magnetic or non-magnetic mechanisms. Examples of magnetic landing mechanisms include magnets that can be deactivated or bypassed by mechanical means during takeoff from the tube 50. The magnetic components referred to include magnetic components</p>

<p dir="rtl">25 Permanent, permanent magnetic parts with armature actuated to assist in separation during take-off, and parts</p>

13019

-18-

Permanent electromagnets and electromagnetic parts. It should be noted, however, that constant power consumption can be a disadvantage of electromagnetic parts. Non-magnetic adhesion mechanisms can be used with non-ferromagnetic surfaces such as stainless steel, composite pipes, and concrete. These mechanisms include tiny prongs and materials

<p dir="rtl">5 Gecko-inspired adhesive (e.g., synthetic spicules), suction cups, clutches, and clutches.</p>

In different embodiments, different payloads or crawler designs are used. For the purpose of simplicity, these payloads or designs fall into two basic categories; They are inspection and maintenance. Inspection payloads and designs include a range of different types of sensors being used

<p dir="rtl">10 Commonly used in the oil and gas field to inspect pipes and structures. For example, in some embodiments, an ULTRASONIC TESTING sensor is used to measure thickness. For ease of description, an ULTRASONIC TESTING thickness sensor is used at times in the present application to represent an illustrative device and use for inspection and maintenance. However, other models are not limited to this device or</p>

<p dir="rtl">15 Usage. For example, other inspection sensors or probes may be used instead of or in addition to an ULTRASONIC TESTING sensor depending on the process, including (but not limited to) eddy current sensors and alternating field measurement sensors. current field measurement (ACFM) sensors.</p>

In other embodiments, the crawler 130 is configured with one or more other tools for use

<p dir="rtl">20 Maintenance. For example, the 130 crawler can be used to perform light maintenance such as cleaning, surface preparation and paint repairs. In still other embodiments, the crawler 130 is configured with one or more cameras and is used for visual inspection. For example, in some embodiments, a camera is used for simple visual inspections, for example where only videos or photographs of areas of interest are required, but for these</p>

13019

-19-

Areas unable to be directly inspected by an unmanned aerial vehicle (UAV).

.100 vehicle

In some embodiments, the crawler 130 is configured to leave markers (such as paint or QR codes) behind areas of interest (such as locations where sensor readings are outside normal levels,

<p dir="rtl">5 (or where faults are detected, or where conditions are normal but the locations have to be marked anyway). These locations would be, for example, where critical thickness levels are detected. In some of these models, after the crawler 130 is reinstalled and flown An unmanned aerial vehicle (UAV) 100 scans these markers and creates a 3D reconstruction of the environment showing the exact location of these markers in some of these models.</p>

<p dir="rtl">10 The 100 UAV uses a vehicle-mounted RGB-D camera to detect tags and calculate their locations in relation to the 100 unmanned aerial vehicle (UAV). Using the UAV's GLOBAL POSITIONING SYSTEM, the absolute positions of the markers can be calculated or determined. It should be noted that while the 100 UAV is scanning the markers, the 130 crawler, for example, can remain on the pipe</p>

<p dir="rtl">15 50 pipe or re-installed with an unmanned aerial vehicle (UAV).</p>

100.

In some embodiments, the crawler 130 uses radio positioning to pinpoint trouble locations (or other sensor reading) on the asset, for example using virtual markers. In other words, fault locations can be located even without physical markers, although this is accurately

<p dir="rtl">20 Say. This is because the crawler is located in relation to an unmanned aerial vehicle</p>

100 An unmanned aerial vehicle (UAV) can be computed (or determined) using radio sensors. For example, in some of these embodiments, a 100 unmanned aerial vehicle (UAV) carries an ultra-wideband (UWB) sensor array. Wide band The radio signals are received by another UWB transmitter mounted on the ski

<p dir="rtl">25 130 crawler. The crawler's relative position can then be measured regardless of whether</p>

13019

-20-

Unmanned aerial vehicle 100 (UAV) in flight or attached to a pipe 50. In some embodiments, when an operator finds a malfunction while crawling, the crawler's position is tagged relative to an unmanned aerial vehicle (UAV) 100 and captured. Using a GLOBAL GPS sensor

<p dir="rtl">5 POSITIONING SYSTEM For the UAV, the absolute positions of these faults can be determined. In some embodiments, when a GPS GLOBAL POSITIONING SYSTEM is not available, the UAV position is estimated based on the flight path and INERTIAL MEASUREMENT UNIT data from its take-off base, where GPS is available. GLOBAL POSITIONING)</p>

<p dir="rtl">10 SYSTEM. This can also be done using sensor fusion methods such as Optical Odometry (VIO), which combines camera images and INERTIAL MEASUREMENT UNIT (IMU) data to compute an estimate of a position relative to a reference point (e.g., take-off base).</p>

In some embodiments, previously computerized (or determined) survey locations are flown from an unmanned aerial vehicle

<p dir="rtl">15 100 (UAV) unmanned aerial vehicle to an operator computer, or ground station. after that,</p>

Inspection locations are displayed visually, for example on a pre-built 3D model of the facility being inspected, or a 3D model that can be created from sensors mounted on an unmanned aerial vehicle (UAV), for example a depth or detection camera. About light and 2D LIDAR (light detection and ranging)

<p dir="rtl">20 Three-dimensional. Furthermore, in these embodiments, the visually displayed locations are appended to the corresponding measured thickness (or other sensed values or information).</p>

Figure 2 is a rectangular diagram of an automated guidance and landing method 200 for landing an air vehicle without

100) on a curved surface (e.g. UAV) unmanned aerial vehicle (UAV) pilot

A pipe, for example pipe 50, according to one embodiment. This method can be applied

<p dir="rtl">25 and other methods described in the present application using a combination of sensors and other devices</p>

13019

-21-

Including computing circuits or other logic circuits configured (for example, programmed) to carry out their assigned tasks. The indicated devices are located on (or near) an unmanned aerial vehicle (UAV). In some illustrative embodiments, the control logic is implemented as computer code adapted to be executed on a computer circuit (such as a microprocessor) to carry out the steps

<p dir="rtl">5 Controls that are part of the method. For ease of description, the processing logic mentioned will be referred to as (e.g., application specific integrated circuit (ASIC specific integrated circuit), FPGA (FIELD PROGRAMMABLE GATE ARRAY), processor, custom circuit, etc.) With the control circuit during the current application to facilitate the description as well, the aforementioned control circuit will be</p>

<p dir="rtl">10 To program with code to implement control logic (or customize the circuit to perform its specified purpose).</p>

Referring to Figure 2, the automated guidance and landing method 200 includes the step of extracting or receiving raw sensor data 210 from various environmental sensors including scanning sensors such as a two-dimensional light detection and ranging system (LIDAR) and cam 3D or infrared (IR) depth imaging

<p dir="rtl">15 The code control circuit is configured to process sensor data 210 to produce actionable information by which the remainder of method 200 can be performed, such as issuing control commands to an unmanned aerial vehicle (UAV) to perform a precision maneuver.</p>

In more detail, method 200 includes a pipe detection and placement step 220. In one embodiment, the control circuit is configured with a code to process data from environmental sensors

<p dir="rtl">20 To identify and detect a pipe and to find its location relative to an unmanned aerial vehicle (UAV). Based on the above, the control circuit is configured by a code to find the location of the 230 UAV with respect to the pipe. The method 200 further includes a path planning step 240. In one embodiment, once the unmanned aerial vehicle (UAV) 230 is located, the control circuit is further configured to plan a path</p>

<p dir="rtl">25 A complete landing 250 from the current location to the desired landing location (for example, on the part</p>

13019

-22-

The method 200 further includes a control and trajectory step 260. In one embodiment, the control circuit is further configured to ensure that an unmanned aerial vehicle (UAV) follows the planned trajectory 250 during a landing maneuver and UAV correction and control It has 270 for any disturbances or deviations from the landing path 250 in actual time 5. Further details of implementing these steps are provided below.

Figures 3a, 3b, and 3c are oblique projections of annotation steps for an aerial vehicle without

Structure 300 is targeted to be attached to an unmanned aerial vehicle (UAV) pilot

(e.g., tube 250), including approaching the target, aligning with the target, and holding onto the target 250, respectively, according to one embodiment. Figure 3a represents a target approach phase, when

<p dir="rtl">10 An unmanned aerial vehicle (UAV) 300 is on its way to approaching the target 250 and determining a suitable landing site. Figure 3b represents the alignment phase with the target, when an unmanned aerial vehicle (UAV) 300 precisely adjusts the landing position (for example, from above the target 250) to make the final landing (or maneuver) as accurate as possible. Figure 3c represents Hold phase, when an unmanned aerial vehicle (UAV) lands</p>

<p dir="rtl">15 300 unmanned aerial vehicle (and becomes centered) on the target 250.</p>

An unmanned aerial vehicle (UAV) includes both a depth camera (e.g., a 3D depth sensor) and a 2D light detection and ranging system (LIDAR). For ease of description, a depth camera acquires (or scans) a rectangular (2D) field of view (field of 20 FOV) using pixels configured by the depth sensor. Each pixel's depth sensor estimates the distance from

The camera is directed to the corresponding area in the pixel's corresponding FOV. This provides an additional dimension (3D) in the data output from the depth camera. To further facilitate description, 2D light detection and ranging (2D LIDAR) or linear FOV (1D) scanning occurs using pixels Equipped with a timing sensor.

<p dir="rtl">25 Based on the timing differences between the laser light from the light detection and binary ranging</p>

13019

-23-

2D LIDAR (light detection and ranging) and light back to the pixels, 2D LIDAR estimates the distance (using the speed of light) between the 2D LIDAR and each pixel in the linear FOV. This provides an additional (2D) dimension in the data output from the 2D LIDAR Light detection and ranging (2D LIDAR).

<p dir="rtl">5 Dimensions, which represents a 2D slice along the scanning direction of 2D LIDAR (light detection and ranging). This is in contrast to a 2D light detection and ranging system (3D LIDAR), which obtains (or scans) a rectangular (2D) FOV using pixels configured with a timing sensor. This system produces 3D output by Add a distance to each pixel</p>

<p dir="rtl">10 Views in the rectangular array.</p>

Using these two sensors (a 3D depth camera and a 2D light detection and ranging system (2D LIDAR), there are different illustrative models of detection, positioning and alignment disclosed in the present application. In a potentially ideal setting, 2D LIDAR (light) detection and ranging could be used.

<p dir="rtl">15 detection and ranging in three dimensions to provide adequate accuracy for a continuous reference feed to estimate the position of the target relative to an unmanned aerial vehicle (UAV) at all times, and there are practical considerations for this. For example, with current 3D light detection and ranging (LIDAR) technology, only a relatively large UAV (e.g. larger than a meter in width) can provide a payload.</p>

<p dir="rtl">20 Enough to hold the indicated sensor. The indicated large unmanned aerial vehicle (UAV) may be impractical for landing on a relatively small pipe (for example, a pipe with a diameter of approximately 20.32 cm).</p>

This is in contrast to 3D depth cameras and 2D light detection and ranging (LIDAR) systems, which are lighter

13019

-24-

Significantly, even considering the combined weight of a 3D depth camera and a 2D LIDAR (light detection and ranging) system.

Furthermore, with current 3D LIDAR (2D light detection and ranging) technology, the cost of a light detection system can be

<p dir="rtl">5 2D light detection and ranging (3D LIDAR) (not to mention a large unmanned aerial vehicle (UAV)) to transport a 3D light detection and ranging system (3D LIDAR) more expensive than Both a 3D depth camera and a 2D light detection and ranging system (LIDAR).</p>

<p dir="rtl">10 Therefore, as a practical matter, a combination of a 3D depth camera and a 2D LIDAR (2D light detection and ranging) system is considered to provide the UAV with the sensor data it needs to perform the detection, positioning, alignment, and feedback operations of the present application. This provides a relatively low-cost, low-payload system compared to a 2D LIDAR light detection and ranging system.</p>

<p dir="rtl">15 and three-dimensional ranging. In other words, in an attempt to expand some or most of the limitations on the size and payload of an unmanned aerial vehicle (UAV), a combination of a 3D depth sensor, 2D light detection and ranging (2D LIDAR) is provided. High tech. The combined weight of both sensors can be much lighter than a high-tech 3D LIDAR system. Furthermore</p>

<p dir="rtl">20 Therefore, the cost of both sensors, compared to a 3D LIDAR system, is reduced by as much. While the proposed sensor combination may not provide the same high accuracy as a 2D light detection and ranging system (3D LIDAR), when using the illustrative methods considered herein, adequate overall performance similar to a positioning reference feed is provided For accurate comment.</p>

13019

-25-

Referring to the illustration in Figures 3a, 3b, and 3c, in this configuration, each sensor (a 3D depth camera and a 2D light detection and ranging system (2D LIDAR) of an unmanned aerial vehicle vehicle 300) is used. UAV) unmanned aerial vehicle for detection and localization at a specified distance range

<p dir="rtl">5 Target pipe 250. For example, a depth camera is used to detect and locate the target 250 from a relatively far distance. Once the 300 UAV approaches the target 250 (e.g., at less than a certain or previously determined distance, or close to the target 250), a 2D light detection and ranging system (LIDAR) is used to detect About the target 250 and position it and align it completely until it is complete</p>

<p dir="rtl">10 comment. Thus there are two zones: a remote zone (for example, when the 300 unmanned aerial vehicle (UAV) is relatively far from the pipe) with only the 300 UAV using the 3D depth camera for detection and localization, and a close zone (at 300 For example, when an unmanned aerial vehicle (UAV) 300 is relatively close to a pipe (250 UAV) a detection system 300 is used with it.</p>

<p dir="rtl">15 2D light detection and ranging (2D LIDAR) only for detection, positioning and alignment.</p>

In this setup, the 3D depth camera provides a 3D point cloud of the sensed surroundings, which helps segment arbitrary target shapes into a 3D pattern, such as a pipe (e.g., pipe 250) that appears as a semi-cylindrical shape in the cloud 20 3D Raster This also helps to estimate the 3D size of the target 250 and select a suitable landing point on the target 250. However, depth cameras become less reliable at distances closer to the 250 target and, therefore, can produce less reliable feedback.

In contrast, a 2D light detection and ranging (LIDAR) system provides more accurate proximity measurements (close to the target 250) compared to a depth camera, where the 250 light detection and ranging pipe appears bilateral

13019

-26-

Dimensions 2D LIDAR (light detection and ranging) As a 2D slice, such as a semicircle, semi-ellipse, straight or near-straight line, depending on the angles determined by the 2D LIDAR (light detection and ranging) scanner Dimensions relative to the 250 pipe. The accuracy feature is indicated for light detection

<p dir="rtl">5 2D LIDAR (light detection and ranging) is particularly noticeable at distances very close to the target 250, where the depth camera becomes less reliable. However, at long distances from the target 250, it may be difficult for a 2D light detection and ranging (LIDAR) system to efficiently detect the desired target 250 using measurements</p>

<p dir="rtl">10 Only 2D (e.g., a linear distance matrix). This is partly because there may be other surrounding objects with similar representations of the target 250 in the 2D representations, and the 2D slice does not include enough of the target 250 to distinguish it from surrounding objects.</p>

Thus, by combining the two sensors, the ambiguity of object detection and positioning can be avoided by using a depth camera at long distances from the target 250, and the accuracy of the alignment can be improved

And feedback by using a 2D light detection and ranging (LIDAR) system at distances close to the target 250. Furthermore, processing power is limited by consuming a single sensor output in each of the two independent steps.

<p dir="rtl">20 Figure 4 is a workflow diagram of a sequential path planning method 400 illustrating the automated deployment and landing of an unmanned aerial vehicle (UAV) on a curved surface (such as a pipe), according to one embodiment. The path planning path 400 according to Figure 4 uses a stepwise path method , in a manner similar to the method shown in the approach to target, alignment with target, and annotation to target illustrations presented in Figures 3a, 3b, and 3c, respectively, herein</p>

<p dir="rtl">25 In the method, the control circuit is programmed to compute the UAV path at three different relays, using ON</p>

13019

-27-

Example one or more of the target detection and localization methods discussed above. The three phases are target approach, target alignment, and target hold, as shown in Figures 3a, 3b, and 3c, respectively. In other words, the complete suspension path consists of these three parts, each of which is configured to be computerized independently of each other by the control circuit.

<p dir="rtl">5 Referring to the path planning method 400 according to Figure 4, processing begins with the approach step 410 to the target. This is the first case in the automatic hold task, where the target is first detected and localized from a relatively far distance. In this case, an unmanned aerial vehicle (UAV) is programmed to hover at a predetermined location of the target. The control circuit is also programmed to receive an actuator's identification of a raw input image of a target for detection and localization algorithms.</p>

<p dir="rtl">10 For example, a user or operator can do this based on a user interface by selecting a region of interest (ROI) containing the target from a live image feed provided by a camera mounted on an unmanned aerial vehicle (UAV). The control circuit is then further programmed to use the raw input target image and the input of the vehicle's on-board depth camera to provide continuous 3D detection and positioning of the target.</p>

<p dir="rtl">15 Relative to UAV. Based on the above, the control circuit is further programmed to use the calculated position of the target (from detection and positioning) to compute a safe path (e.g., a straight line) for an unmanned aerial vehicle (UAV) to follow to move closer to the target and prepare for alignment.</p>

Method 400 further includes the step of determining 420 whether it has become an unmanned aerial vehicle

<p dir="rtl">20 Unmanned aerial vehicle (UAV) close enough to the target (e.g., after moving along a safe path and approaching the target). In one embodiment, the control circuit is further programmed to determine that this condition (approaching The target is completed once the unmanned aerial vehicle (UAV), for example, reaches a specified location or distance from the target,</p>

13019

-28-

The two-dimensional LIDAR (light detection and ranging) system has a greater possibility of viewing the target. At this point, the 2D LIDAR system can perform more accurate positioning (than a 3D depth camera) in preparation for pinning down the target, and the target approach phase ends (when the depth camera is used to detect

<p dir="rtl">5 And objectification.</p>

Method 400 further includes the step of aligning the unmanned aerial vehicle (UAV) vehicle 430 with the target detected using a two-dimensional light detection and ranging system (LIDAR). In this case, the control circuit is also programmed to use only 2D 10D light detection and ranging (LIDAR) measurements to detect the target

It is positioned to improve the orientation of an unmanned aerial vehicle (UAV) relative to the target for accurate feedback. This can include moving the UAV to the appropriate location where the target is pinned, for example directly above the pipe and within a pre-determined proximity range of the pipe. For this purpose, method 400 further includes a step

<p dir="rtl">15 440 Determine whether an unmanned aerial vehicle (UAV) is aligned with the target. In this case, the control circuit is also programmed to use 2D light detection and ranging (2D LIDAR) measurements to determine if the UAV is close enough and in the correct direction relative to the target in order to actually annotate the target. If not, the control circuit is further programmed to continue using</p>

<p dir="rtl">20 2D light detection and ranging measurements (2D LIDAR) to further align an unmanned aerial vehicle (UAV) with the target.</p>

Once aligned with the target, the method 400 further includes a hold step 450 (e.g., final landing maneuver) on the target using light detection and ranging measurements

<p dir="rtl">25 2D LIDAR (light detection and ranging). After the UAV aligns</p>

13019

-29-

With the target, the controller is further programmed to compute a safe path toward the hold point (landing location on the target). In one embodiment, the safe path is characterized by a hold point on the target and a velocity profile toward that point. The controller is further programmed to use light detection measurements Two-dimensional LIDAR (light detection and ranging).

<p dir="rtl">5 For continuous reference feed when controlling an unmanned aerial vehicle (UAV) along a computerized path to land the UAV safely on the target.</p>

Figure 5 is a diagonal diagram of an illustrative method for detecting, positioning, and aligning targets using sensor fusion, according to one embodiment. For ease of description, the 250 pipe and 300 unmanned aerial vehicle (UAV) of Figures 3a, 3b, 10, and 3c are used in the description of the illustrative method according to Figure 5, since the most significant differences are in the programming of the control circuit and the simultaneous use of both depth cameras. 3D and 2D light detection and ranging (LIDAR) system in an unmanned aerial vehicle 300 (UAV).

In more detail, and referring to Figure 5, in this configuration, both depth cameras are used

<p dir="rtl">15 and a simultaneous 2D light detection and ranging system (i.e., in-time) during the entire suspension mission. In this embodiment, a coarse-to-fine proximity tracking sensor fusion method is used. A depth camera is used to efficiently detect and locate the target 250, although its distance accuracy is generally lower than that of a detection system.</p>

<p dir="rtl">20 Light detection and ranging (2D LIDAR) is similar. This is because the additional coverage provided by a 3D depth camera (2D FOV) and the ability to recognize a 3D shape (such as a 250 tube) using this image is superior at greater distances, since the 2D FOV Dimensions: 2D LIDAR (light detection and ranging) (1D FOV)</p>

13019

-30-

It can be more challenging to locate the target 250 in the presence of other structures at these distances.

As shown in Figure 5, the coarse-to-fine approach involves using a localization result from the depth camera (e.g., coarse localization) to suppress the search space for detection measurements.

<p dir="rtl">5 2D light detection and ranging (2D LIDAR) with an unmanned aerial vehicle (UAV) 300 traveling towards the hold point on the target 250. The restricted search area represents the portion of the depth camera FOV most likely to isolate the target 250 (or A portion of interest of the target 250, such as the position of the suspension. (Light detection and ranging (2D LIDAR) measurements are used.</p>

<p dir="rtl">10 2D ranging within the restricted space 250 for target detection and localization (e.g., fine localization), which restricts both the size and direction (e.g., 1D scanning) for performing 2D light detection and ranging (LIDAR) measurements 2D ranging.</p>

In another sensor fusion embodiment, the control circuit for an unmanned aerial vehicle (UAV) is programmed

<p dir="rtl">15 300 unmanned aerial vehicle to use a Kalman filter-based AR sensor fusion scheme</p>

To make more efficient placement estimates by combining independent estimates from each sensor (depth camera, 2D light detection and ranging (2D LIDAR) while accounting for their measurement uncertainties.

In sensor fusion models, measurements from both sensors are combined and used during the suspension task

<p dir="rtl">20 all the time. This differs from the model shown in Figures 3a, 3b, and 3c, where the measurements of each sensor are used in an independent step. Comparing the two basic approaches, by integrating the sensor, positioning accuracy is increased at all times and the path of the unmanned aerial vehicle (UAV) towards the hold point is smoother. However, with sensor fusion, computing time is significantly increased because measurements from both sensors are processed simultaneously</p>

<p dir="rtl">25 The entire suspension process.</p>

13019

-31-

In sensor fusion models, the control unit is programmed to compute a continuous hold path (as opposed to a gradual path according to the sequential path planning method according to Figure 4). In this case, the control circuit is programmed to compute an initial continuous path once the target has been initially detected It was located using a depth camera and a two-dimensional light detection and ranging system

<p dir="rtl">5 2D LIDAR (light detection and ranging). The control circuit is further programmed to characterize this continuous path using the location of the unmanned aerial vehicle (UAV) upon first detection of the target, the hold point on the target, and the velocity profile along the path.</p>

Although multi-sensor configurations provide 10% more efficient object detection, placement, and alignment by using their distinct features, there are situations where each

The sensor is also alone in the self-suspension process. This means that in certain special cases, either a 2D light detection and ranging system (LIDAR) or a 3D depth camera is sufficient to carry out the detection, positioning and self-alignment of the suspension. For example, a dual light detection and ranging system can be used

<p dir="rtl">15 LIDAR (light detection and ranging) is only 2D for efficient target detection, positioning and alignment in a noiseless (or no) environment. Similarly, a 3D depth camera can only be used to provide sufficient detection, positioning and alignment for large targets.</p>

The following describes illustrative examples of detection and localization methods, as they are considered the most important aspects

<p dir="rtl">20 Difficulty in automatically suspending an unmanned aerial vehicle (UAV) on a curved structure. For ease of description, these methods are described from the perspective of programming the control circuit to process sensor data for correct target detection and localization. Moreover, much of this discussion concerns the practical application of detection and localization algorithms to the task of automatically suspending an unmanned aerial vehicle (UAV) onto a structure.</p>

13019

-32-

Curved structure (like a tube). It is also indicated that the steps of these algorithms are implemented by programming the control circuit to execute the corresponding steps.

As previously mentioned, target detection and placement are the two most difficult tasks in the self-suspension maneuver. This is because the accuracy of maneuver control is highly dependent on the accuracy of the positioning reference feed. In illustrative embodiments, different algorithms are discussed (as programmed on the control circuit) to perform this task using a depth camera, a 2D light detection and ranging system (2D LIDAR), or both. ). The depth camera and 2D LIDAR (light detection and ranging) are two different types of sensors for sensing the pipe and detecting its location. Each uses a different methodology to measure distances, and each produces significantly different data from the other. As such, according to the discussion in the present application, localization methodologies or algorithms are specialized to a particular type of one of these sensors.

The depth camera provides a standard volumetric representation (or 3D point cloud), which includes distance measurements of sensed objects in the surrounding 3D environment as observed by 15 rectangular (2D) arrays of sensor pixels. In contrast, a 2D light detection and ranging (LIDAR) system provides a 2D point slice, which includes distance measurements to objects sensed along a line observed by a linear (one-dimensional) array of sensor pixels. The point cloud from a depth camera can be used to perform segmentation and localization of an arbitrary 3D object.

<p dir="rtl">20 In contrast, only 2D light detection and ranging (2D LIDAR) measurements can be used to detect 2D slices of these shapes. As such, detecting and localizing arbitrary objects using only 2D light detection and ranging data (LIDAR) is much more difficult, and is particularly sensitive to noise</p>

<p dir="rtl">25 surroundings (such as similar objects or other sources of spurious signals, which are easy to filter out).</p>

13019

-33-

Using 3D depth data. Target placement accuracy using a depth camera is adequate near the target, for example within a distance of 0.3 to 10 meters (m). However, 2D light detection and ranging (LIDAR) provides And ranging) 2D much greater accuracy at closer distances, especially at much closer distances, on

<p dir="rtl">5 For example, less than 0.05 m.</p>

For ease of description, the models described use algorithms designed to detect pipes or other cylindrical shaped targets, but similar concepts can be generalized in other models to detect other curved surface targets, such as spherical or partially spherical shapes. Eyesight because the data output from a 3D depth camera and a light detection and ranging system

<p dir="rtl">10 2D LIDAR (light detection and ranging) can vary greatly, and different target detection and localization algorithms are provided for each type of sensor. A more detailed description of the algorithms referred to is provided below by reference to Figures 6 to 9. However, a more simplified description is provided below.</p>

Different models are presented for performing detection and positioning using a 3D depth camera. These 15 models include object segmentation from a point cloud, using self-directed measurement fusion, and using

Image-based object tracking for point cloud reduction. However, the above-mentioned are merely examples, and other examples will become clear in light of the present disclosure.

For detection and localization using object segmentation from a point cloud, the main steps include receiving a point cloud from a depth camera, and defining a bounding box in the cloud

<p dir="rtl">20 Raster, perform the segmentation of the object in the bounding box, estimate the location of the center of the target, and repeat some or all of these steps using the results of the previous steps as well as the new input point clouds. In more detail, the control circuit is programmed to receive a point cloud from the depth camera (i.e., process the output data from the depth camera into a 3D point cloud). The control circuit is further programmed to select an initial bounding box containing candidate points representing the target ( tube in this</p>

<p dir="rtl">25 state) in the input point cloud. For example, it could be the initial bounding box</p>

13019

-34-

The referenced is the entire point cloud, a previously defined portion to start from, or an initial estimate of the target area based on a tool (such as GPS (Global Positioning System)).

(IMU) INERTIAL or POSITIONING SYSTEM

MEASUREMENT UNIT, etc.) or a user-defined bounding box, etc

<p dir="rtl">5 for example, and not as a limitation.</p>

The control circuit is further programmed to segment objects in the given bounding box, to find all points that best fit the mathematical model of a tube (for example, a cylinder). Based on the above, the control circuit is further programmed to identify a segmented object that is most likely It represents the intended target pipe (i.e., best matches the position and shape).

<p dir="rtl">10 At this point, the control circuit is further programmed to estimate the location of the center of the target pipe relative to the depth camera. In one embodiment, the mean center of the points identified as the target pipe is used to estimate the center of the pipe. In the next iteration, When a new input point puller is available, a new bounding box is defined around the previously calculated mean center, with additional segmentation and an updated calculation of the mean center assuming that the target has not</p>

<p dir="rtl">15 It moves significantly from the previous location (for example, assuming little movement) relative to the camera's FOV.</p>

In terms of detection and localization using self-directed measurement fusion, this is similar to the point cloud object segmentation discussed above, with only the relaxation of the simple motion assumption. Alternatively, the control circuit is further programmed to receive another input, eg from

<p dir="rtl">20 An inertial measurement unit (IMU) that is rigidly attached to the depth camera (or measures inertia or estimates the position of the depth camera), to predict the location of the target (relative to the FOV of the depth camera) in the next iteration based on the estimated location A new bounding box is created around the estimated position based on the IMU (INERTIAL).</p>

MEASUREMENT UNIT

13019

-35-

An INERTIAL MEASUREMENT UNIT (IMU) can be used to predict the rapid motion or rotation of an unmanned aerial vehicle (UAV), allowing the small motion assumption to be relaxed and a more accurate bounding box prediction for the next iteration.

<p dir="rtl">5 In connection with detection and localization using image-based object tracking for point cloud upsampling, the controller is programmed to use image-based object tracking to restrict the input point cloud to a cloud that corresponds only to the area of the tracked object in the image frame. In one embodiment, to account for failures in the image tracker, an INERTIAL MEASUREMENT UNIT (IMU) is used to predict the location of the object in the image frame</p>

<p dir="rtl">10 Use this to restart the image tracker once the object is lost.</p>

Different models are presented for performing detection and localization using a 2D light detection and ranging system (LIDAR). One such embodiment involves performing circular detection using the random sample consensus (RANSAC) clustering algorithm. In this case, the control circuit is programmed

<p dir="rtl">15 To use a modified RANSAC algorithm to take advantage of the rapid collection of 2D light detection and ranging (LIDAR) measurements. Another model uses linear combination rejection. In this case, the control circuit is also programmed (from the modified RANDOM SAMPLE CONSENSUS model) to delete points that represent line (or quasi-linear) segments in the measurements.</p>

<p dir="rtl">20 2D light detection and ranging (LIDAR) 2D light detection and ranging. Another model uses temporal pipe position and diameter tracking. In this case, the control circuit is programmed to use the correct target detection position and radius initially and track the target in subsequent time frames rather than performing a full detection for each frame.</p>

Another model implements elliptical detection using the Random Sample Consensus (RANSAC) method.

<p dir="rtl">25 RANDOM SAMPLE CONSENSUS AND HOUGH CONVERSION. While describing detection models</p>

13019

-36-

The previous illustrations align a circuit (for example, a one-dimensional part of a 2D light detection and ranging (LIDAR) chip) with 2D data Output, In this methodology, the control circuit is programmed to align 5 ellipsoids with the input points because, according to the direction of light detection and identification

2D LIDAR (light detection and ranging) relative to the target pipe, the resulting points can be best represented by an ellipse than a circle. Furthermore, the ellipsoid can be used to compute the diffraction between LIDAR and the pipe, which can

<p dir="rtl">10 It is useful for aligning an unmanned aerial vehicle (UAV) with the pipe. This model uses the RANSAC (RANDOM) SAMPLE CONSENSUS method with the Hough transformation, and is described in more detail below. Further embodiments will also be demonstrated for 2D light detection and ranging (LIDAR) and localization with the current detection.</p>

<p dir="rtl">15 In more detail, a set of details using a 2D light detection and ranging system (2D LIDAR) are discussed in the present application. 2D light detection and ranging systems (2D LIDAR) are lightweight laser scanners that scan a rotating laser beam at high rotating scanning speeds and measure the escape time of the laser reflected from the environment back to</p>

<p dir="rtl">20 Scanner. This allows 2D LIDAR (light detection and ranging) to provide a detailed 2D scan of all objects and obstacles around the scanner as long as they are in danger of being seen and within a specified range of 2D LIDAR (light detection and ranging). ranging) in two dimensions. Real-time scans are possible due to LIDAR systems' fast scan rate as well as bandwidth</p>

13019

-37-

The small amount required to transfer a single scan due to the point cloud being a 2D point cloud rather than a 3D point cloud.

Various algorithms for positioning pipes using a 2D LIDAR (2D light detection and ranging) system are described further.

<p dir="rtl">5 Detail. Each of the different approaches has strengths and weaknesses, based on factors such as the environment in which they are deployed. Methods include circular detection using clustering and a modified RANDOM SAMPLE CONSENSUS methodology, rejection of linear combinations, time tracking of pipe positions and diameters, and elliptical detection using RANDOM SAMPLE CONSENSUS.</p>

<p dir="rtl">10 CONSENSUS AND HOUGH CONVERSION.</p>

Figure 6 is a workflow diagram of a method of detecting circuits 600 (e.g., a tube) using 2D light detection and ranging (LIDAR) sensor data collection, according to one embodiment. The method 600 detects and places a pipe based on high-resolution 2D LIDAR data using a consensus method methodology

<p dir="rtl">15 RANSAC (RANDOM SAMPLE CONSENSUS) is an intensively modified sample that takes advantage of clustering. The control circuit is programmed to perform fast clustering by angularly sorting the point cloud.</p>

Referring to Figure 6, processing begins with a step of grouping 610 consecutive points closer than 5 centimeters (cm) apart and discarding any group with points fewer than some specified numerical point (assuming, n).

<p dir="rtl">20 In this step, the control circuit is programmed to group nearby points, considering only groups up to a certain threshold size. For example, in one embodiment, the control circuit is programmed to calculate the distance between every two successive points in a line scan, and to group points less than 5 cm apart. The control circuit is also programmed to discard any set with less than a certain threshold number n of points and keep the other sets as satisfied.</p>

13019

-38-

Method 600 further includes running the modified RANDOM SAMPLE CONSENSUS method, which includes the remaining steps 620 through 680, for an arbitrarily large number of iterations (e.g., depending on other factors such as available processing power, size of the point cloud , desired accuracy, etc.) 5 More specifically, method 600 includes a step of aligning 620 circuits across the three random points in

Random set. The goal is to quickly find a circle (for example, a tube spline) that describes the other points in the set. In this case, the control circuit is programmed to randomly select a satisfying set and, based on this set, pick three points in this set and fit a circuit through them.

<p dir="rtl">10 The method 600 further includes the step of calculating 630 the algebraic error for all other points in the given set of the circle using the equation of the circle. This provides a benchmark for how well the group fits into the department. In this case, the control circuit is further programmed to calculate or determine the algebraic error of all other points in the indicated set when introduced into the circuit equation. Method 600 further includes the step of counting 640 the number of points locked in the set that are close to the circle.</p>

<p dir="rtl">15 In this case, the control circuit is further programmed to find a number of trapped points with an error less than a predetermined threshold value (e.g., close to the circle). The method 600 further includes the step of determining 650 whether the number of trapped points is greater than n of size This gives confirmation that the group is actually a circuit (e.g., a pipe). In this case, the control circuit is further programmed to compare the number of recent points to the marginal number n, and if it is less, then the group.</p>

<p dir="rtl">20 (or selected points within the group) do not fit well into the circle (pipe), so the process is repeated starting with step 620.</p>

Otherwise, the number of caged points exceeds a threshold n, and the method 600 further includes the step of aligning 660 a new best circuit across all caged points and calculating the total residual error. In this case, the control circuit is further programmed to find the best-fitting circuit across all 25 locked points and find the error relative to the best-fitting circuit. Method 600 further includes

13019

-39-

670 The step of determining whether the new circuit is better (e.g., lower error) than the previous best circuit. The goal is to find the best circuit (e.g., pipe) that matches 2D light detection and ranging (LIDAR) data. 2D detection and ranging available. In this case, the control circuit is further programmed 5 to compare the total residual error of the most recent circuit/assembly alignment, although not less than that of

With the best circuit/combination fit found in previous iterations, no new best circuit is found, so the process is repeated starting with step 620.

Otherwise, a new best circuit is found, so the method 600 further includes the step of saving 680 the new circuit as the best circuit. In this case, the control circuit is further programmed 10 to save the new circuit as the best circuit in an electronic storage medium for comparison with candidate circuits.

Later, you can repeat the process starting with step 620 in search of a better circuit. The indicated iterations stop when some termination criteria is met (for example, the number of iterations, the amount of computing time, the remaining error is below a previously specified value, etc.). This method works best when complex Unmanned aerial vehicle (UAV) 15 vehicle close enough to the target pipe so that the pipe dominates

Observed groups in the survey line. As the relative size of the pipe shrinks or the amount of other structures increases (particularly other curved structures), the reliability of this method of pipe detection and placement declines dramatically.

In another technique for 2D LIDAR (20) light detection and ranging, linear combination rejection is performed. In this case, the goal is to avoid the risk of false discoveries in the modified RANDOM SAMPLE CONSENSUS algorithm above by identifying groups that are mere line segments and then discarding them from the search. Once they are deleted, the RANDOM SAMPLE method can be started.

<p dir="rtl">25 MODIFIED CONSENSUS. There are many ways to do this. For example, in one</p>

13019

-40-

In one embodiment, the control circuit is programmed to fit at least a least squares rectifier across each plot (or group) and then consider its total residual errors. In another embodiment, the control circuit is programmed to perform a principal component analysis (PCA) on each group She would like to study her second Eigen values. Smaller Eigensecond values indicate a plane set (e.g., 5, a line segment) while larger Eigensecond values can be (e.g.,

Nonlinear groups are good candidates for the RANDOM SAMPLE CONSENSUS method. In another embodiment, the control circuit is further programmed to find the average scalar change in each group and use it as an indication of the normality of the group. The control circuit is further programmed to eliminate groups that exceed Its straightness is beyond a pre-determined limit value of 10.

In another example of 2D LIDAR light detection and ranging, a time tracking of the position and diameter of the pipe is performed. For example, this approach can be used in conjunction with clustering and the modified RANDOM SAMPLE CONSENSUS15 model discussed above. The idea in this case is to use the information discovered about the pipe

pipe in a single frame (for example, 2D light detection and ranging).

2D LIDAR (light detection and ranging) to assist in searching in the next frame. In one such embodiment, the control circuit is programmed such that, if a pipe is found in one frame, the search space is restricted in the next frame which is restricted to the proximity of the pipe

<p dir="rtl">20 In the first frame. This assumes slight unmanned aerial vehicle movements from one frame to another, which is a valid assumption in the given application. However, care must be taken to avoid false detections (false positives), which cause the search window to shift away from the actual pipe location. This can lead to dissociation and failure to bounce back to the true location.</p>

13019

-41-

As such, in another embodiment, the control circuit is further programmed to use the revealed tube diameter in the tire as the likely truth, and then only accepts circuits in future tires that have diameters within a certain tolerance therefor. This greatly improves the performance of the algorithms and identifies false positives.

<p dir="rtl">5 In another method for 2D LIDAR (light detection and ranging), elliptical detection is performed using a RANDOM SAMPLE CONSENSUS and Hough transform while a circuit is aligned with the detector output. 2D LIDAR light detection and ranging methodology for detecting pipes In 2D LIDAR 10 scanning, it assumes that the scanning slice is perpendicular to the longitudinal axis of the target pipe. In general, however, this is true once the longitudinal axis of the tube is positioned, and the 2D LIDAR light detection and ranging scan is adjusted accordingly. Otherwise, the 2D LIDAR slice will fit into an ellipse instead. In other words, if it is an unmanned aerial vehicle (UAV</p>

<p dir="rtl">15 An unmanned aerial vehicle is pointing directly at the pipe and perpendicular to it, the pipe will appear like a circle segment in a 2D light detection and ranging (LIDAR) scan. However, if an unmanned aerial vehicle (UAV) turns to the right or left by a slight angle θ, the pipe no longer appears as a perfect circular segment but as a</p>

<p dir="rtl">20 From an ellipse. The eccentricity of the ellipsoid is directly related to the eccentricity/azimuth angle θ of the UAV with respect to the pipe.</p>

More consistently, the minimum axial length of the ellipsoid is always equal to the diameter of the pipe D as long as the unmanned aerial vehicle (UAV) is level and sloped. It is not possible to compensate for slope angles by measuring the angles through a steering unit

<p dir="rtl">25 Autonomous IMU (INERTIAL MEASUREMENT UNIT) UAV vehicle and data point management</p>

13019

-42-

Light detection and ranging (2D LIDAR) according to these angles. The relationship between the tilt angle θ of the UAV and the length of the major axis L is described as follows: (θ = arccos(D/L). Thus, in one embodiment, the control circuit is programmed to fit an ellipse to light detection and ranging data (2D LIDAR). and)

<p dir="rtl">5 ranging using pipe points in 2D LIDAR (light detection and ranging) data, and then extracting the corresponding maximum and minimum axis lengths to determine the pipe diameter and tilt angles of an unmanned aerial vehicle (UAV). In another embodiment, the control circuit is further programmed to use the Hough transform to render 2D light detection and ranging (LIDAR) data</p>

<p dir="rtl">10 2D detection and ranging are less sensitive to noise when using the RANDOM SAMPLE CONSENSUS method to find the best curve. However, care must be taken to prevent the large dimensionality of the search space from dominating the search time and causing undesirable performance.</p>

A range of localization techniques using a 3D depth camera (e.g. IR) are discussed in

<p dir="rtl">15 Current specification. A depth camera is a good candidate for detecting and locating a straight pipe in a point cloud generated by a depth camera. A point cloud is a standard, volumetric representation of objects in 3D space. Object detection and tracking are studied as part of computer visual research for 2D image processing. Although photographs provide a rich source of features of the captured scene, it is difficult to localize and estimate the 3D pose of objects</p>

<p dir="rtl">20 Using a monocular CamArt. The main challenge comes from depth estimation. Stereo cameras can provide relative convenience at the expense of more computing operations and limited ranges. 3D light detection and ranging (LIDAR) systems provide a standardized, volumetric representation of the sensed environment, making object placement easier and more accurate. However, light detectors and ranging devices are not binary</p>

<p dir="rtl">25 3D LIDAR (light detection and ranging) is feasible to use for</p>

13019

-43-

For certain applications such as aerial applications where a small UAV is used, due to the relatively heavy weight of 2D light detection and ranging (3D LIDAR).

Therefore, the Depth CamArt provides a low-cost alternative solution between the stereo CamArt and detection systems

<p dir="rtl">5 2D light detection and ranging (3D LIDAR). Depth CamArt, commonly known as CamArt RGB-D, combines a regular RGB camera and an IR sensor to provide RGB images as well as estimated depth to each pixel. The depth image can be converted into a point cloud, which provides a standard volumetric representation of the sensed environment. It can provide triple light detection and ranging (2D LIDAR).</p>

<p dir="rtl">10 Dimensions are generally more accurate point clouds compared to Depth CamArt. However, using some filtering techniques, which are discussed in more detail below, point clouds generated by depth cameras can be suitable for automated annotation as described in the automated application. Currently, there are several low-cost depth cameras that are small enough to be mounted on a small unmanned aerial vehicle (UAV), for example</p>

<p dir="rtl">15 Example, CamArt Intel RealSense D435.</p>

Various methods are provided for detecting and locating pipes in a point puller and estimating their position. In this case, the pose is a six-degree-of-freedom (-6 DOF) pose, namely a 3D position of the mid-center of the pipe and 3D rotation angles of its axis. These methods include: point cloud filtering, pipe segmentation, and 20 pipe tracking .

In illustrative point cloud filtering models for 3D depth camera output, the raw input point cloud is generally dense and contains noisy measurements due to sensor imprecision and environmental conditions. Thus, it may be useful to apply some filtering to the input point cloud to obtain more fine-grained object segmentation results. Two filters included

<p dir="rtl">25 From these filters, cut and remove the samples. These filters can be applied, for example,</p>

13019

-44-

Before performing cylindrical fractionation. These filters can be applied individually or collectively (where their effects are compounded). They can also be applied before other methods, such as drum segmentation (e.g., pipe).

In an illustrative clipping model, only objects close to the sensor are of interest and are

<p dir="rtl">5 Other points are less likely to be useful. For example, an input point cloud can be about 10 meters away from the sensor, but a depth of 5 meters is sufficient to image objects of interest. Accordingly, the control circuit is programmed to clip the indicated data points more than 5 meters away. This helps reduce noise and the number of outliers, which improves the accuracy of object segmentation. In addition to the above, this helps reduce fragmentation time by as much</p>

<p dir="rtl">10 Large, and the points processed are also reduced. Similarly, in an illustrative downsampling model, the control circuit is programmed to downsample the input point cloud. This type of filtering also limits the size of the input point cloud, which speeds up further processing, and can be useful in computing environments (e.g. real-time), such as automated annotation.</p>

<p dir="rtl">15 In illustrative pipe segmentation models, the control circuit is also programmed to perform object segmentation after the input point cloud is filtered. There are many ways to perform object segmentation in a point cloud, including, for example, region formation, histogram with a minimum, normal variation, Euclidean distance-based clustering, and model alignment using random sample consensus (RANSAC). . To briefly present the discussion,</p>

<p dir="rtl">20 The present disclosure will focus primarily on model alignment using RANSAC. RANSAC is an alternative method for estimating variables of a mathematical model from a set of observed data that contains unwanted outliers. Therefore, the RANDOM SAMPLE CONSENSUS method can also be described as an outlier detection method. This method is useful in practice because the actual field data contains measurements</p>

<p dir="rtl">25 Other structures surrounding the target object as well as other measurement noise are considered outliers.</p>

13019

-45-

In general, a tube can be represented by a cylindrical model whose parameters can be defined by three model variables, namely the radius of the cylinder, a point (three coordinates) on the central axis of the cylinder, and the direction of three-dimensional rotation (three angles) of the central axis relative to an origin. The mechanism is discussed The main function of the RANSAC algorithm is as follows: A model of the body is defined in this case, which is a cylindrical model with the aforementioned variables (radius, central axis point, axis direction).

Central). A distance function is defined to measure how far a point is from a fitted model. In one embodiment, the Euclidean distance functions as a function of distance.

At each time step 5, an input data frame is provided, which is the input point cloud Pt at time t. In some embodiments, the control circuit is programmed for repeated operation of a method

<p dir="rtl">10 RANDOM SAMPLE CONSENSUS (RANSAC) method up to a pre-determined maximum number of replications. In each iteration, the control circuit is programmed to estimate the parameters of the cylindrical model by fitting the model, and the accuracy of the model is calculated using a specified distance function. The control circuit is also programmed to continue operation of the RANDOM SAMPLE CONSENSUS method and output the best model fit.</p>

<p dir="rtl">15 (model variables) across all iterations with maximum accuracy. These steps are applied for each cloud</p>

Input point Pt.

In some embodiments, the open project point cloud library (PCL) is used to provide various point cloud filtering functions including applying a cylindrical segmentation using the RANDOM SAMPLE method.

<p dir="rtl">20 CONSENSUS. In one such embodiment, PCL is used to perform point cloud filtering and cylindrical segmentation. While point cloud filtering and pipe segmentation using the RANDOM SAMPLE CONSENSUS method work well most of the time to detect and localize the target pipe, they can fail, for example when the input data is noisy enough that it appears patchy.</p>

<p dir="rtl">25 Accompanied by noise, it is false that the segment is better, or when there are two cylinders in the data</p>

13019

-46-

The input,method identifies the faulty cylinder. Accordingly, in some embodiments, to limit detected false pipes and reduce detection ambiguity in the presence of other pipes, mechanisms are proposed to track pipes.

Figure 7 is a workflow diagram of an illustrative adjacent pipe detection method 700, according to one

<p dir="rtl">5 Models. Processing begins with the step of performing 710 detection of cylinders using the RANDOM SAMPLE CONSENSUS method, then the step of selecting 720 valid cylinders, and updating the valid mean center to be the center of the valid cylinder mean. In this case, the control circuit is programmed to perform cylinder detection on the input 3D depth camera data (point cloud) using the random sample matching (RANSAC) method.</p>

<p dir="rtl">10 RANDOM SAMPLE CONSENSUS Once a valid use (e.g., a potential pipe or cylinder) is available, the cylinder identified as the initial candidate for detection in the next frame is selected and the mean center of the valid cylinder is computed. The method 700 further includes a shearing step 730 of the cloud The next input is about the previous valid middle position. In this case, the control circuit is also programmed to assume a small movement of the depth sensor between</p>

<p dir="rtl">15 successive input clouds, and clips the next input cloud around the previously computed valid mean center.</p>

Method 700 further includes the step of applying 740 a RANDOM SAMPLE CONSENSUS based segmentation 740 to the clipped cloud and computing the mean center of the resulting cylinder (or cylindrical cloud of data points). At this point 20, method 700 includes the step of determining 750 if The new intermediate center was next door

to the previous mean center, and if not, the previous valid mean center is maintained, and processing is resumed at step 730. In one embodiment, the control circuit is programmed to determine whether the distance between the previous mean center and the new mean center is less than or equal to a predetermined adjacent distance d, if not, the list is considered invalid, and the position is not updated

<p dir="rtl">25 The valid average is in the new average position. Otherwise, it is the previous and new average position</p>

13019

-47-

adjacent (e.g. within the adjacency distance d), the method 700 further includes a step of updating the current valid center 760 to be the new center. Processing is resumed with a shearing step 730 around the indicated new center.

It should be noted that the success of the aforementioned tracking mechanism relies heavily on the small motion assumption 5 (e.g., the average centers of cylinders identified in successive frames remain close together). Accordingly, in some embodiments, to further improve detection efficiency In the presence of rapid motion, a sensor fusion mechanism is used. In some of these models, the sensor fusion mechanism uses a Kalman filter.

Figure 8 is a workflow diagram of an illustrative piping placement method 800 based on a framework

<p dir="rtl">10 The Kalman filter worked, according to one embodiment. This method helps improve the detection and localization of pipes in light of previous methods. In this case, an inertial measurement unit (IMU) is used that is rigidly attached to the depth camera to provide a more accurate prediction of the detected target even if it is fast moving. Using this prediction, a segmentation based on the RANSAC (RANDOM) method is applied.</p>

<p dir="rtl">15 SAMPLE CONSENSUS on the predicted location of the pipe point cloud, which is a subset of the input point cloud.</p>

Referring to Figure 8, processing begins with the step of finding 810 the initial location of the pipe. In one embodiment, the control circuit is programmed to receive or determine the location and direction of the depth camera (or unmanned aerial vehicle) and, based on that and the location of the pipe

<p dir="rtl">20 Initial (using the RANDOM SAMPLE CONSENSUS) method, the initial location of the tube is determined. Once the initial location of the tube is known, the method further includes the step of obtaining 820 INERTIAL MEASUREMENT UNIT (IMU) measurements and combining them with The last estimate of the pipe's location to predict the next location of the pipe in the FOV of the depth camera. In this case, it is the control circuit</p>

<p dir="rtl">25 Programmed to integrate Inertial Measurement Unit (IMU) measurements</p>

13019

-48-

MEASUREMENT UNIT and estimate the location of the previous pipe under the Kalman filter. In the Kalman filter framework, there are two steps, which are a prediction step followed by an updating step. The INERTIAL MEASUREMENT UNIT (IMU) measurements are used in the prediction step to predict the position of the INERTIAL MEASUREMENT UNIT 5 (or the position of the depth camera attached to it) in a space. The prediction can be made with acceptable accuracy within a short period of time Before it starts to diverge, the prediction step 820 is then completed.

Method 800 further includes the step 830 of obtaining depth measurements around the predicted location, estimating the pipe location from the indicated depth measurements, and then updating the pipe location estimate. In this case, the control circuit is further programmed to update the pipe location estimate 10 using a subset of the input point cloud for the next frame around the location that

predicted (from step 820) to find the pipe and position it in the next frame. The estimated pipe location in the indicated subpoint cloud is then used to update step 830 to correct the predicted location and to avoid divergence. This completes update step 830 The framework continues to repeat the prediction step 820 and the update step 830 to better track the pipe 15 using sensor fusion to interpret the movement of the depth camera during the flight of the unmanned aerial vehicle.

.(UAV) unmanned aerial vehicle

Figure 9 is a workflow diagram of an illustrative piping placement method 900 using image tracking and a Kalman filter, according to one embodiment. In method 900, adding an additional source to track objects helps improve tracking efficiency. Image-based object tracking is used to track the pipe

<p dir="rtl">20 pipe observed by an RGB camera, which is embedded in a depth camera (e.g., RGB-D). The corresponding point cloud is extracted from the original input point cloud. The size of the extracted point cloud is potentially much smaller than the original point cloud, which is It results in less computation associated with object segmentation. Image-based object tracking can fail during fast movement. To mitigate this problem, location-based prediction is used</p>

13019

-49-

INERTIAL MEASUREMENT UNIT (Inertial Measurement Unit (IMU)) as well as for automatic reboot of the image-based object tracker.

In more detail, processing begins with the initiation step 910, where the controller is programmed to receive or identify an initial region of interest (ROI) in the input image,

<p dir="rtl">5 and extracting the corresponding point cloud, computing the initial pipe location estimate (e.g., using the RANDOM SAMPLE CONSENSUS method), and initiating the image tracker using the initial ROI. Method 900 further includes a Kalman filter prediction step 920, where a circuit is The control is also programmed to obtain INERTIAL MEASUREMENT UNIT measurements and forecasting.</p>

<p dir="rtl">10 Pipe location based on the last pipe location estimate. Method 900 further includes an image tracking step 930, where the control circuit is further programmed to feed the next image frame (from the RGB camera) to the object tracker, and the image tracker is triggered to track the object and supply the output ROI. At this point, method 900 further includes a step Determine 940 whether the object tracking was successful, and if not, the method 900 further includes a step of re-initiating the tracker 950</p>

<p dir="rtl">15 Images using the last prediction and resuming the starting step 920.</p>

However, if the tracking is successful, the method 900 further includes the step of extracting 960 the corresponding point cloud using the ROI output from the object tracker. The method 900 further includes a pipe finding step 970, where the controller is further programmed to segment objects on the extracted point cloud to find the pipe. The 20 image tracking step 930 is then re-performed with the newly found pipe. In parallel with the tracking of the indicated objects,

Method 900 also includes a Kalman filter update step 980, where the control circuit is further programmed to update the pipe location estimate from the depth measurements, at which point the Kalman prediction step 920 is repeated with the new IMU (INERTIAL) MEASUREMENT UNIT data.

13019

-50-

In some embodiments, the depth camera and 2D LIDAR sensors are also used to detect and avoid obstacles, for example when automatically controlling the flight path of an unmanned aerial vehicle (UAV) from a take-off position to a landing position on the target surface. 5 In one such embodiment, when the operator manually moves the UAV toward the target, pipes or other structures may be exposed. In this case, the sensor can...

It detects these obstacles and the control circuit can be programmed to avoid a collision by stopping the unmanned aerial vehicle (UAV) and alerting the operator to take a different course.

In another embodiment, when the UAV automatically performs a landing maneuver on a pipe, it can detect nearby obstacles, and the control circuit is programmed to avoid them by planning a path around them.

<p dir="rtl">10 Figure 10 is a workflow diagram of an illustrative method 1000 for automated suspension of an unmanned aerial vehicle</p>

100 Or 300) on a curved surface UAV such as (UAV) unmanned aerial vehicle pilot

(such as pipe 50 or 250) from a starting position away from the curved surface, according to one embodiment.

Parts of the method 1000 or the entire method may be performed using the components and methods shown in Figures 1A through 9. Portions of this method and others described in

<p dir="rtl">15 Current demand for or using a previously programmed logic device, circuit, or processor, such as a programmable logic circuit (PLC), computer, software, or other circuit (for example, an application-specific integrated integrated circuit) ASIC specific integrated circuit, a field-programmable gate array (FPGA) (FIELD PROGRAMMABLE GATE ARRAY) configured with code or logic to perform its assigned tasks.</p>

<p dir="rtl">20 The device, circuit, or processor can, for example, be a dedicated device or</p>

Shared (such as a laptop, single board computer (SBC), workstation, tablet, smartphone, part of a server, or a dedicated physical circuit, such as a field-programmable gate array (FPGA) FIELD PROGRAMMABLE GATE ARRAY or application specific integrated circuit (ASIC).

<p dir="rtl">25 integrated circuit, etc.), a computer server, or part of a server or computer system.</p>

13019

-51-

The device, circuit, or processor may include a non-transitional computer readable medium (CRM), such as read-only memory (ROM memory, flash disks, or a disk drive) that stores instructions when Execution by one or more processors results in the execution of parts of the method 1000 (or other disclosed method). 5 It should be noted that in other embodiments, the order of operations can be changed, and some operations can be omitted. Parts of the method or the entire method 1000 may also be performed using logic, circuits, or processors located on an unmanned aerial vehicle (UAV) configured to implement the method 1000.

In the illustrative method 1000, processing begins with a capture step 1010 and outputs triangular point clouds

<p dir="rtl">10 Dimensions of views from an unmanned aerial vehicle (UAV). Images include curved surface. Capture uses a three-dimensional (3D) depth camera such as a depth camera (for example, an IR depth camera or an RGB-D camera) attached to an unmanned aerial vehicle (UAV). The method 1000 can further include a step of capturing 1020 and outputting two-dimensional slices of scenes. This capture uses a LIDAR system</p>

<p dir="rtl">15 2D attachment to an unmanned aerial vehicle (UAV). The method 1000 further includes a control step 1030 of the depth camera and LIDAR system to capture 3D point clouds and 2D slices, respectively, of the viewer. This can be done using a control circuit (for example, a programmable processor or logic circuit) attached to an unmanned aerial vehicle (UAV) and programmed to perform control tasks.</p>

<p dir="rtl">20 The method 1000 further includes the step of inputting 1040 3D point clouds captured from the depth camera and 2D slices captured from the LIDAR system. In this case, the control circuit can be programmed to input sensor data. The method 1000 further includes a self-detection step 1050 and localization of the curved surface using the captured 3D point clouds and the captured 2D slices. In this case, the control circuit can be programmed to use the random sample matching method</p>

<p dir="rtl">25 RANSAC (RANDOM SAMPLE CONSENSUS) to perform detection and localization. Includes</p>

13019

-52-

The method 1000 further includes an automated guidance step 1060 of an unmanned aerial vehicle (UAV) from a take-off position to a landing position on a curved surface based on automated detection and positioning of the curved surface. In this case, the control circuit can be controlled to automatically control the flight path of an unmanned aerial vehicle (UAV) from the take-off position to the landing position using 3D point clouds.

Walsh uses two-dimensional scents to guide her.

In one embodiment, the curved surface is ferromagnetic and the unmanned aerial vehicle (UAV) further includes magnetic legs. Method 100 further includes magnetically attaching the magnetic stems to the ferromagnetic curved surface 10 during landing and remaining magnetically attached to the ferromagnetic curved surface after landing.

The methods described herein may be performed in part or in whole using software or firmware in machine-readable form on a physical (e.g., non-transitory) storage medium. For example, the software or firmware may be in the form of a computer program that includes Computer software code configured to perform some or all of the steps of any of the 15 methods described in the present application when the program is run on a computer or suitable device

(For example, a PROGRAMMABLE GATE ARRAY (FPGA)), where the computer program can be embedded in a computer-readable medium. Physical storage media includes computer storage devices with computer-readable media such as disks or USB sticks. Flash memory, etc., and does not include published signals. Published signals can be located on physical storage media, however

Published signals themselves are not examples of physical storage media. The program may be suitable for execution on a parallel or serial processor so that the steps of the method can be executed in any suitable order, or simultaneously.

13019

-53-

It should also be noted that numbers that are identical or similar in figures represent elements that are identical or similar across multiple figures, and not all components or steps described and illustrated by reference to figures are necessary for all embodiments or preparations.

Terms used in this disclosure are intended to describe specific models only and are not intended to describe specific models

<p dir="rtl">5 Including it being restricted to Sister Arraa. According to the usage in the description, singular definite and indefinite nouns are general, unless the context indicates otherwise. It is also noted that the terms “including”, “including”, and/or “including” when used in this specification designate the presence of attributes, integers, steps, operations, and/or Identified elements and/or components, but does not prevent the presence or addition of an attribute, integer, step, process, and/or element,</p>

<p dir="rtl">10 And/or one or more components, and/or combinations of the above.</p>

Trend terms are used in the present order only for purposes of convenience and reference, and are not to be construed as restrictive. However, it is known that these terms can be used in reference to a beholder. Accordingly, no limitations are implied or inferred. Furthermore, the purpose of using ordinal numbers (for example, first, second, third) is to differentiate, not to...

<p dir="rtl">15 Statistics. For example, the use of the term “third” does not imply the presence of a corresponding “first” or “second” element. Furthermore, the phrases and terms used in this application are descriptive and not limited to. The use of the expressions "including", "comprising", "containing", "containing" or "involving" and their derivatives are intended to include the elements listed below and their counterparts as well as additional elements.</p>

<p dir="rtl">20 The subject matter described above is provided for clarification and is not limited to. Various modifications and variations may be made to the subject matter described in the present application without following the uses and illustrative embodiments contained in the present application and without departing from the nature and scope of the invention covered by the present disclosure, which are determined by the following set of claims, structures, functions or steps that are equivalent to the following: In protection elements.</p>

13019

-54-