System and method for supporting movable object application development
Abstract
Systems and methods can support application development in a moving object environment. The mobile object manager can establish a connection with the movable object and receive one or more data packets from this movable object. The movable object manager can then provide the information in the one or more data packets to the application on the user terminal.

Term
8.6 yearsto projected expiry
Projected expiry 20 April 2035, counted from filing; an application has no term until it is granted.
- Priority and filed
- Published
- Today
- Projected expiry
60 claims: 11 independent, 49 dependent
- 1可動物体マネージャを介して、可動物体との接続を確立するステップと、 前記可動物体マネージャを介して、1つ又は複数のデータパケットを前記可動物体から受信するステップと、 前記可動物体マネージャを介して、前記1つ又は複数のデータパケット内の情報をユーザ端末上のアプリケーションに提供するステップとを含む可動物体の環境でのアプリケーション開発を支援する方法。
- 2前記可動物体が、無人機、無人車両、ハンドヘルド装置、又はロボットである、請求項1に記載の方法。
- 3前記可動物体マネージャを介して、1つ又は複数のコマンドを前記アプリケーションから受信するステップと、 前記可動物体マネージャを介して、前記1つ又は複数のコマンドを前記可動物体に送信するステップとを更に含む、請求項1に記載の方法。
- 4通信装置を使用して、前記1つ又は複数のデータパケットを前記可動物体から受信するステップを更に含む、請求項1に記載の方法。
- 5前記通信装置が、複数のリンクを介して前記可動物体と通信するように動作し、それぞれの前記リンクを使用して、互いに異なるタイプのデータパケットを伝送する、請求項4に記載の方法。
- 6前記通信装置が、リンクを介して前記可動物体と通信するように動作し、このリンクを使用して、混合タイプのデータパケットを伝送する、請求項4に記載の方法。
- 7通信マネージャを使用して、前記1つ又は複数のデータパケットに関連付けられた通信プロトコルを処理するステップを更に含む、請求項1に記載の方法。
- 8データマネージャを使用して、前記可動物体から受信した情報を、ユーザ端末上の前記アプリケーションに分配するステップを更に含む、請求項1に記載の方法。
- 9前記可動物体マネージャが、インターフェースを介して1つ又は複数のアプリケーションによってアクセスできる、請求項1に記載の方法。
- 10前記インターフェースと1つ又は複数のインターフェース構成要素とを関連付けるステップを更に含み、それぞれの前記インターフェース構成要素が、可動物体内のモジュールを表す、請求項9に記載の方法。
- 11それぞれの前記インターフェース構成要素とリスナを関連付けるステップを更に含み、これにより、前記インターフェース構成要素が、前記表されたモジュールに関連する情報を受信できるようになる、請求項10に記載の方法。
- 12前記可動物体が無人機であり、前記インターフェースが、カメラ構成要素、電池構成要素、ジンバル構成要素、通信構成要素、及び飛行制御装置構成要素を備える、請求項9に記載の方法。
- 13前記インターフェースが地上局構成要素を備え、これが前記飛行制御装置構成要素に関連付けられ、前記地上局構成要素が、1つ又は複数の飛行制御操作を実行するように動作する、請求項12に記載の方法。
- 14認証サーバを使用して前記アプリケーションを認証した後、前記アプリケーションが前記可動物体と通信できるようにするステップを更に含む、請求項1に記載の方法。
- 151つ又は複数のマイクロプロセッサと、 前記1つ又は複数のマイクロプロセッサ上で実行される可動物体マネージャとを備え、 前記可動物体マネージャが、 可動物体との接続を確立し、 1つ又は複数のデータパケットを前記可動物体から受信し、 前記1つ又は複数のデータパケット内の情報をユーザ端末上のアプリケーションに提供するように動作する可動物体の環境でのアプリケーション開発を支援するシステム。
- 16前記可動物体が、無人機、無人車両、ハンドヘルド装置、又はロボットである、請求項15に記載のシステム。
- 17前記可動物体マネージャが、1つ又は複数のコマンドを前記アプリケーションから受信し、 前記1つ又は複数のコマンドを前記可動物体に送信するように動作する、請求項15に記載のシステム。
- 18前記1つ又は複数のデータパケットを前記可動物体から受信するように動作する通信装置を更に備える、請求項15に記載のシステム。
- 19前記通信装置が、複数のリンクを介して前記可動物体と通信するように動作し、それぞれの前記リンクを使用して、互いに異なるタイプのデータパケットを伝送する、請求項18に記載のシステム。
- 20前記通信装置が、リンクを介して前記可動物体と通信するように動作し、このリンクを使用して、混合タイプのデータパケットを伝送する、請求項18に記載のシステム。
- 21前記可動物体マネージャが、通信マネージャを使用して、前記1つ又は複数のデータパケットに関連付けられた通信プロトコルを処理するように動作する、請求項15に記載のシステム。
- 22前記可動物体マネージャが、データマネージャを使用して、前記可動物体から受信した情報をユーザ端末上のアプリケーションに分配するように動作する、請求項15に記載のシステム。
- 23前記可動物体マネージャが、インターフェースを介して1つ又は複数のアプリケーションによってアクセスできる、請求項15に記載のシステム。
- 24前記インターフェースが、1つ又は複数のインターフェース構成要素に関連付けられ、それぞれの前記インターフェース構成要素が、可動物体内のモジュールを表す、請求項23に記載のシステム。
- 25それぞれの前記インターフェース構成要素が、リスナに関連付けられ、これにより、前記インターフェース構成要素が、前記表されたモジュールに関連する情報を受信できるようになる、請求項24に記載のシステム。
- 26前記可動物体が無人機であり、前記インターフェースが、カメラ構成要素、電池構成要素、ジンバル構成要素、通信構成要素、及び飛行制御装置構成要素を備える、請求項23に記載のシステム。
- 27前記インターフェースが地上局構成要素を備え、これが前記飛行制御装置構成要素に関連付けられており、前記地上局構成要素が、1つ又は複数の飛行制御操作を実行するように動作する、請求項26に記載のシステム。
- 28前記アプリケーションを認証した後、前記アプリケーションが前記可動物体と通信できるように動作する認証サーバを更に備える、請求項15に記載のデータ処理方法。
- 29命令を内部に記憶した持続的なコンピュータ読取り可能媒体であって、プロセッサによって実行されると、 可動物体マネージャを介して、可動物体との接続を確立するステップと、 前記可動物体マネージャを介して、1つ又は複数のデータパケットを前記可動物体から受信するステップと、 前記可動物体マネージャを介して、前記1つ又は複数のデータパケット内の情報をユーザ端末上のアプリケーションに提供するステップとを含むステップを実行する、持続的なコンピュータ読取り可能媒体。
- 301つ又は複数のマイクロプロセッサと、 前記1つ又は複数のマイクロプロセッサ上で実行される可動物体マネージャとを備え、 前記可動物体マネージャは、 ユーザ端末上のアプリケーションとの接続を確立し、 前記可動物体内の1つ又は複数の関数モジュールから情報を受信し、 前記1つ又は複数のデータパケットを前記アプリケーションに送信し、 前記1つ又は複数のデータパケットが前記受信した情報を含むように動作する可動物体の環境でのアプリケーション開発を支援するシステム。
- 31可動物体の環境でのアプリケーション開発を支援するための方法であって、 認証サーバを介してアプリケーションからアクティベーション要求を受信するステップであって、前記アクティベーション要求が、可動物体にアクセスするための権限を求める要求を含むステップと、 1つ又は複数のポリシーを前記アクティベーション要求に適用するステップと、 前記アプリケーションに資格がある場合、前記可動物体にアクセスするための権限を前記アプリケーションに付与するステップとを含む方法。
- 32前記可動物体が、無人機、無人車両、ハンドヘルド装置、又はロボットである、請求項31に記載の方法。
- 33前記アプリケーションに関連付けられたアプリケーション識別子に基づいて、アプリケーションキーを前記アプリケーションに割り当てるステップを更に含む、請求項31に記載の方法。
- 34前記アプリケーションキーが、前記アプリケーション識別子と一義的に結びついており、前記アプリケーションキーが、1組の権限に関連付けられている、請求項33に記載の方法。
- 35異なるアプリケーション識別子を有する別のアプリケーションに、異なるアプリケーションキーを割り当てるステップを更に含み、前記異なるアプリケーションキーが、異なる組の権限に関連付けられている、請求項34に記載の方法。
- 36前記アクティベーション要求が、前記アプリケーションキーを含む、請求項33に記載の方法。
- 37前記認証サーバを介して、最大インストレーション数を維持するステップを更に含み、前記最大インストレーション数が、事前構成又は動的構成されるようになっている、請求項31に記載の方法。
- 38アプリケーションがアクティブにされると、アクティブにされたアプリケーションカウントを更新するステップを更に含む、請求項37に記載の方法。
- 39前記アクティブにされたアプリケーションカウントが前記最大インストレーション数以上である場合、前記アクティベーション要求を却下するステップを更に含む、請求項38に記載の方法。
- 40前記最大インストレーション数を増やして、アップグレード要求を処理するステップ、又は 前記最大インストレーション数をゼロに設定して、不法若しくは不適切なアプリケーションを停止するステップを更に含む、請求項37に記載の方法。
- 41前記可動物体にアクセスするためのインターフェースを前記アプリケーションが使用できるようにするステップを更に含む、請求項31に記載の方法。
- 42前記可動物体が無人機であり、前記インターフェースが、カメラ構成要素、電池構成要素、ジンバル構成要素、通信構成要素、及び飛行制御装置構成要素を備える、請求項41に記載の方法。
- 43前記インターフェースが地上局構成要素を備え、これが前記飛行制御装置構成要素に関連付けられ、前記地上局構成要素が、1つ又は複数の飛行制御操作を実行するように動作する、請求項42に記載の方法。
- 44前記地上局構成要素が、特定の組の権限を有するアプリケーションによってのみアクセス可能になるように構成される、請求項43に記載の方法。
- 45可動物体の環境でのアプリケーション開発を支援するためのシステムであって、 1つ又は複数のマイクロプロセッサと、 前記1つ又は複数のマイクロプロセッサ上で実行される認証サーバとを備え、 前記認証サーバは、 アプリケーションからアクティベーション要求を受信し、前記アクティベーション要求が、可動物体にアクセスするための権限を求める要求を含み、 1つ又は複数のポリシーを前記アクティベーション要求に適用し、 前記アプリケーションに資格がある場合、前記可動物体にアクセスするための前記権限を前記アプリケーションに付与するように動作するシステム。
- 46前記可動物体が、無人機、無人車両、ハンドヘルド装置、又はロボットである、請求項45に記載のシステム。
- 47前記認証サーバが、前記アプリケーションに関連付けられたアプリケーション識別子に基づいて、アプリケーションキーを前記アプリケーションに割り当てるように動作する、請求項45に記載のシステム。
- 48前記アプリケーションキーが、前記アプリケーション識別子と一義的に結びついており、前記アプリケーションキーが、1組の権限に関連付けられている、請求項47に記載のシステム。
- 49前記認証サーバが、異なるアプリケーション識別子を有する別のアプリケーションに、異なるアプリケーションキーを割り当てるように動作し、前記異なるアプリケーションキーが、異なる組の権限に関連付けられている、請求項48に記載のシステム。
- 50前記アクティベーション要求が、前記アプリケーションキーを含む、請求項47に記載のシステム。
- 51前記認証サーバが、最大インストレーション数を維持するように動作し、前記最大インストレーション数が、事前構成又は動的構成されるようになっている、請求項47に記載のシステム。
- 52アプリケーションがアクティブにされると、前記認証サーバが、アクティブにされたアプリケーションカウントを更新するように動作する、請求項51に記載のシステム。
- 53前記アクティブにされたアプリケーションカウントが前記最大インストレーション数以上である場合、前記認証サーバが、前記アクティベーション要求を却下するように動作する、請求項47に記載のシステム。
- 54前記認証サーバが、 前記最大インストレーション数を増やして、アップグレード要求を処理し、又は 前記最大インストレーション数をゼロに設定して、不法若しくは不適切なアプリケーションを停止するように動作する、請求項47に記載のシステム。
- 55前記認証サーバは、前記可動物体にアクセスするためのインターフェースを前記アプリケーションが使用できるようにする、請求項45に記載のシステム。
- 56前記可動物体が無人機であり、前記インターフェースが、カメラ構成要素、電池構成要素、ジンバル構成要素、通信構成要素、及び飛行制御装置構成要素を備える、請求項55に記載のシステム。
- 57前記インターフェースが地上局構成要素を備え、これが前記飛行制御装置構成要素に関連付けられており、前記地上局構成要素が、1つ又は複数の飛行制御操作を実行するように動作する、請求項55に記載のシステム。
- 58前記地上局構成要素が、特定の組の権限を有するアプリケーションによってのみアクセス可能になるように構成される、請求項55に記載のシステム。
- 59命令を内部に記憶した持続的なコンピュータ読取り可能媒体であって、プロセッサによって実行されると、 認証サーバを介してアプリケーションからアクティベーション要求を受信するステップであって、前記アクティベーション要求が、可動物体にアクセスするための権限を求める要求を含むステップと、 1つ又は複数のポリシーを前記アクティベーション要求に適用するステップと、 前記アプリケーションに資格がある場合、前記可動物体にアクセスするための権限を前記アプリケーションに付与するステップとを含むステップを実行する、持続的なコンピュータ読取り可能媒体。
- 60可動物体の環境でのアプリケーション開発を支援するシステムであって、 1つ又は複数のマイクロプロセッサと、 前記1つ又は複数のマイクロプロセッサ上で実行されるウェブポータルとを備え、 前記ウェブポータルが、 アプリケーションからアプリケーション開発情報を受信し、 前記受信したアプリケーション開発情報を認証サーバに提供するように動作し、 前記認証サーバは、 アプリケーションからアクティベーション要求を受信し、前記アクティベーション要求が可動物体にアクセスするための権限を求める要求を含み、 1つ又は複数のポリシーを前記アクティベーション要求に適用し、 前記アプリケーションに資格がある場合、前記可動物体にアクセスするための前記権限を前記アプリケーションに付与するように動作するシステム。
Independent claims60
97 paragraphs, as filed
0001The era of unmanned aerial vehicles has arrived. Drones may be used in a number of different fields, including archaeological research, sporting events, disaster relief, and environmental protection. Uncrewed vehicles, as well as other moving objects such as unmanned vehicles, handheld devices, and robots, can provide new directions and unique perspectives for the development of traditional industries.
0002This is a general area that embodiments of the present invention seek to address.
<p num="0003"> This specification describes systems and methods that can support application development in a moving object environment. The mobile object manager can establish a connection with the movable object and receive one or more data packets from this movable object. The movable object manager can then provide the information in the one or more data packets to the application on the user terminal.</p><p num="0004"> Also described herein are systems and methods that can assist in application development in a moving object environment. The authentication server can receive an activation request from the application, which includes a request for permission to access the moving object. The authentication server can then apply one or more policies to the activation request and, if the application is entitled, authorize the application to access moving objects.</p>
0005<figref num="1">It is an exemplary diagram of an application in a moving object environment according to various embodiments of the present invention.</figref><figref num="2">It is an exemplary diagram of using a plurality of data connections to support communication between an application and a moving object according to various embodiments of the present invention.</figref><figref num="3">It is an exemplary illustration of using a high speed connection to support communication between an application and a moving object according to various embodiments of the present invention.</figref><figref num="4">It is an exemplary diagram of supporting the development of a software application in the environment of a movable object by various embodiments of the present invention.</figref><figref num="5">It is an exemplary figure of a movable object manager in a movable object environment according to various embodiments of the present invention.</figref><figref num="6">It is an exemplary diagram of an abstraction of a communication protocol in a moving object environment according to various embodiments of the present invention.</figref><figref num="7">It is an exemplary figure of the packet format in the application environment of a moving object according to various embodiments of the present invention.</figref><figref num="8">A flowchart of using a movable object manager to support a movable object application according to various embodiments of the present invention is shown.</figref><figref num="9">It is an exemplary diagram of supporting a movable object interface in a software development environment according to various embodiments of the present invention.</figref><figref num="10">It is an exemplary diagram of an unmanned aerial vehicle interface according to various embodiments of the present invention.</figref><figref num="11">It is an exemplary diagram of the components for an unmanned aerial vehicle in a software development kit (SDK) according to various embodiments of the present invention.</figref><figref num="12">A flowchart of supporting a movable object interface in a software development environment according to various embodiments of the present invention is shown.</figref><figref num="13">It is an exemplary diagram of using an authentication server to support a security model in a moving object environment according to various embodiments of the present invention.</figref><figref num="14">It is an exemplary diagram of using an authentication server to support a plurality of applications in a moving object environment according to various embodiments of the present invention.</figref><figref num="15">It is an exemplary diagram of how various embodiments of the present invention support a security model in a moving object environment.</figref><figref num="16">A flowchart showing how to support a security model in a moving object environment according to various embodiments of the present invention is shown.</figref>
0006The present invention is shown as an example in each of the drawings in the accompanying drawings, and is not shown for the sake of limitation. In each drawing, the same reference numbers refer to similar elements. It should be noted that references to "some" or "one" or "several" embodiments in the present disclosure do not necessarily refer to the same embodiment, and such references mean at least one. ..
0007In the following description of the present invention, an unmanned aerial vehicle is used as an example of a movable object. Not only that, it will be apparent to those skilled in the art that other types of moving objects can also be used.
0008Illustrative moving object environment FIG. 1 is an exemplary view of an application in a moving object environment according to various embodiments of the present invention. As shown in FIG. 1, the application 112 in the movable object environment 100 can communicate with the movable object 101 via the physical link 110. The movable object 101 can be an unmanned vehicle, an unmanned vehicle, a handheld device, and / or a robot.
0009According to various embodiments of the present invention, the movable object 101 can include various function modules 111. For example, an unmanned aircraft may include a camera module, a battery module, a gimbal module, a communication module, a flight control device module, and the like.
0010As shown in FIG. 1, the application 112 can be implemented on the user terminal 102. For example, the user terminal 102 can be a portable personal computing device, a smartphone, a remote control device, and / or a personal computer.
0011Further, the user terminal 102 can include a communication device (not shown), which processes communication between the application 112 on the user terminal 102 and various modules 111 on the movable object 101. Take a role. For example, a drone can have uplinks and downlinks. Uplinks can be used to transmit control signals, and downlinks can be used to transmit media or video streams.
0012According to various embodiments of the invention, the physical link 110 can be (part of) a network, such as WiFi, Bluetooth®, 3G / 4G, other radio frequency technologies, etc. , Based on various wireless technologies. In addition, the physical link 110 can be based on other computer network technologies such as internet technology.
0013FIG. 2 is an exemplary diagram of using a plurality of data connections to support communication between an application and a moving object according to various embodiments of the present invention. As shown in FIG. 2, using a communication device such as the Range Extender 203 in the movable object environment 200, between the application 221 implemented on the user terminal 202 and the various function modules 210 on the movable object 201. Can handle communications. The movable object 201 can be at least one of an uncrewed vehicle, an unmanned vehicle, a handheld device, and a robot.
0014According to various embodiments of the present invention, the user terminal 202 can be connected to the communication device or range extender 203 via the wireless connection 204. Alternatively, the communication device can be connected to the user terminal 202 via a cable. Further, the user terminal 202 can communicate with the movable object 201 via the remote control (RC) connection 207.
0015Further, the communication device 203 can communicate with the movable object 201 via a plurality of connections 205 to 206, and these connections are used to transmit various types of data packets.
0016As shown in FIG. 2, the movable object 201 includes a media / video server 211 and a data server 212 in addition to the various function modules 210. The media / video connection 205 may be based on the UCP protocol, which can be used to transmit media / video information from the media / video server 211 to the communication device 203 (eg, range extender). The data connection 206 may be based on the TCP protocol, which can be used to transmit data such as flight status information and user commands between the data server 212 and the communication device 203.
0017Further, the data connection 206 can transmit data from the application 221 to the moving object 201 and from the data server 212 to the application 221 (ie, both uplink and downlink).
0018FIG. 3 is an exemplary diagram of using a high speed connection to support communication between an application and a moving object according to various embodiments of the present invention. As shown in FIG. 3, a communication device such as the high-speed communication module 303 in the movable object environment 300 is used between the application 321 implemented in the user terminal 302 and the various function modules 310 on the movable object 301. Can handle communications. The movable object 301 can be at least one of an uncrewed vehicle, an unmanned vehicle, a handheld device, and a robot.
0019According to various embodiments of the present invention, the high-speed communication module 303 operates to communicate with the movable object 301 via the high-speed connection 305, and the high-speed communication can be used for command information, media / video streams, and the like. , Mixed type data packets can be transmitted. Also, the movable object 301, such as an unmanned aerial vehicle, can support both uplink and downlink using the high speed connection 305. Uplinks can be used to transmit control signals, and downlinks can be used to transmit media or video streams and various flight status information.
0020According to various embodiments of the present invention, the user terminal 202 can be connected to the high speed communication module 303 via the cable 304. Alternatively, the high-speed communication module 303 can be connected to the user terminal 302 via a wireless connection. Further, the user terminal 302 can communicate with the movable object 301 via the remote control (RC) connection 307.
0021As shown in FIG. 3, the movable object 301 can include the high-speed communication module 311. The connection 305 between the high speed communication module 303 and the high speed communication module 311 can be based on a high speed communication protocol, for example, various protocols based on Orthogonal Frequency Division Multiplexing (OFDM) technology.
0022The high-speed communication module 311 on the movable object 301 can collect information in parallel in real time from various function modules 310 of the movable object 301. The high-speed communication module 311 can then convert the received data into serial format, which can be transmitted over the connection 305. When the data packet reaches the high-speed communication module 303, the high-speed communication module 303 can convert this data from the serial format to the parallel format.
0023On the other hand, when the high-speed communication module 303 receives various commands from the application 321, the high-speed communication module 303 can transmit these commands to the high-speed communication module 311 in a serial format. When the data packet reaches the high-speed communication module 311 the high-speed communication module 311 can convert the data from serial format to parallel format and distribute this data in parallel to various modules 310 on the movable object 301.
0024Movable object manager FIG. 4 is an exemplary diagram of supporting the development of a software application in a moving object environment according to various embodiments of the present invention. As shown in FIG. 4, the application 403 in the moving object environment 400 can use the moving object manager 402 to access and control the moving object 401, which can be used for various function modules. Use firmware 411 to control. The movable object 401 can be an unmanned vehicle, an unmanned vehicle, a portable computing device, a handheld device, or a robot.
0025According to various embodiments of the present invention, the Movable Object Manager 402 can be part of a software development kit (SDK) that can be used to develop software applications in a Movable Object Environment 400. Assist.
0026As shown in FIG. 4, the movable object manager 402 can establish a connection with the movable object 401 and manage the communication between the application 403 and the movable object 401.
0027For example, the movable object manager 402 can receive one or more data packets from the movable object 401. The movable object manager 402 can then provide the information in the one or more data packets to the application 403. The movable object manager 402 can also receive one or more commands from the application and send the one or more commands to the movable object 401.
0028According to various embodiments of the present invention, the movable object manager 402 can be accessed by application 403 via interface 412.
0029Further, the movable object manager 402 may be configured to be located at various locations in the movable object environment 400. For example, the movable object manager 402 may exist in the user terminal in which the application 403 is implemented. Alternatively, the movable object manager 402 may be present in the remote server, the communication device, or the movable object 401.
0030In addition, the authentication server 404 can be used to provide a security model to support application development in a moving object environment 400.
0031FIG. 5 is an exemplary diagram of a moving object manager in a moving object environment according to various embodiments of the present invention. As shown in FIG. 5, the movable object manager 501 can be used to access and control the movable object 510, which can include various function modules 511-513.
0032For example, the movable object manager 501 can be implemented on the user terminal 505 along with applications such as APP 511 to 513. Alternatively, the Movable Object Manager 510 can be implemented on a separate server or communication device from which the application can gain access to the Movable Object 510. In addition, the movable object manager 502 can be mounted directly on the movable object 510.
0033The movable object manager 501 can include a communication manager 502, a data manager 503, and an interface 504. Communication manager 502 can be used to process one or more data packets associated with a communication protocol. Data Manager 503 can be used to manage the data exchange between the application and the moving object 510. Further, the movable object manager 501 can be provided with an interface 504, which can be accessed by applications 511 to 513 in the movable object environment 500.
0034FIG. 6 is an exemplary illustration of an abstraction of a communication protocol in a moving object environment according to various embodiments of the present invention. As shown in FIG. 6, the communication protocol 600 can include a data link layer 603, a network layer 602, and an application layer 601.
0035The data link layer 603 can be responsible for handling data framing, data checking, and data retransmission. The network layer 602 can play a role of supporting the routing and relaying of data packets. The application layer 601 can play a role in processing various application logics such as controlling the behavior of various function modules in a movable object.
0036According to various embodiments of the invention, the communication protocol 600 is between various modules within a moving object, such as a flight imaging system that can include a camera, a flight remote control device, a gimbal, a digital media processor, and a circuit board. Communication can be supported.
0037In addition, the communication protocol 600 can be used with a variety of physical link technologies such as universal asynchronous receiver / transmitter (UART) technology, controller area network (CAN) technology, and integrated circuit-to-integrated circuit (I2C) technology.
0038FIG. 7 is an exemplary diagram of a packet format of a moving object in an application environment according to various embodiments of the present invention. As shown in FIG. 7, packet 700 can include header 701, application header 702, data 710, and tail 703.
0039Header 701 and trailing 703 can contain control information required to deliver user data over the network. For example, the control information can include source and destination network addresses, error detection codes, and ordering information.
0040The application header 702 can include various sender and receiver information. For example, the transmitting side and the receiving side can exist between various modules in a movable object and between applications on a user terminal.
0041FIG. 8 shows a flow chart of using a movable object manager to support a movable object application according to various embodiments of the present invention. As shown in FIG. 8, in step 801 the movable object manager can establish a connection with the movable object. Then, in step 802, the moving object manager can receive one or more data packets from the moving object. Further, in step 803, the movable object manager can provide the information in the one or more data packets to the application on the user terminal.
0042Movable object interface FIG. 9 is an exemplary diagram of supporting a movable object interface in a software development environment according to various embodiments of the present invention. As shown in FIG. 9, the movable object interface 903 can be used to access the movable object 901 in the software development environment 900, such as the software development kit (SDK) environment.
0043Further, the movable object 901 can include various function modules A to C (911 to 913), and the movable object interface 903 can include various interface components A to C (931 to 933). Each of the interface components A to C (931 to 933) in the movable object interface 903 can represent modules A to C (911 to 913) in the movable object 901.
0044According to various embodiments of the present invention, the movable object interface 903 can provide one or more callback functions to support a distributed computing model between the application and the movable object 901.
0045The application can use a callback function to check if the moving object 901 has received a command. In addition, the application can use the callback function to receive the execution result. Therefore, the application and the movable object 901 can interact even if they are spatially and logically separated.
0046As shown in FIG. 9, interface components A to C (931 to 933) may be associated with listeners A to C (941 to 943). Listeners A to C (941 to 943) can notify interface components A to C (931 to 933) to receive information from the relevant module using the corresponding callback function.
0047Further, the data manager 902 can prepare the data 920 for the movable object interface 903 to separate and package the related functions of the movable object 901. You can also use the data manager 903 to manage the data exchange between the application and the moving object 901. Therefore, application developers do not have to go deep into the complex data exchange process.
0048For example, the DJI SDK can provide a set of callback functions for delivering instance messages and receiving execution results from drones. The DJI SDK can configure the life cycle of DJI callback functions to ensure stable and complete information exchange. For example, the DJI SDK can establish a connection between an unmanned aerial vehicle and an application on a smartphone (eg, using an Android or iOS system). Following the smartphone life cycle, DJI callback functions, such as functions that receive information from unmanned aerial vehicles, utilize patterns within the smartphone system to update statements according to various stages in the smartphone system life cycle. Can be done.
0049FIG. 10 is an exemplary diagram of an unmanned aerial vehicle interface according to various embodiments of the present invention. As shown in FIG. 10, the unmanned aerial vehicle interface 1003 can represent the unmanned aerial vehicle 1001. Therefore, an application in the drone environment 1000, for example APP1004-1006, can access and control the drone 1001.
0050For example, the unmanned aircraft 1001 can be equipped with various modules such as a camera 1011, a battery 1012, a gimbal 1013, a flight control device 1014, and a range extender 1015.
0051Correspondingly, the movable object interface 1003 can include a camera component 1021, a battery component 1022, a gimbal component 1023, a flight control device component 1024, and a range extender component 1025.
0052In addition, the movable object interface 1003 can include a ground station component 1026, which component is associated with flight controller component 1024. Ground station components operate to perform one or more flight control operations, which may require a high level of authority.
0053FIG. 11 is an exemplary diagram of components for an unmanned aerial vehicle in a software development kit (SDK) according to various embodiments of the present invention. As shown in FIG. 11, the drone class 1101 in the SDK 1100 is a set of other components 1102 to 1107 for unmanned aerial vehicles (ie, drones). The drone class 1101 has access to the other components 1102 to 1107, can exchange information with the other components 1102 to 1107, and controls the other components 1102 to 1107.
0054According to various embodiments of the invention, the application may have access to only one instance of drone class 1101. Alternatively, multiple instances of drone class 1101 may exist within the application.
0055The DJI SDK allows applications to connect to an instance of drone class 1101 and upload control commands to the drone. For example, the proper place for the DJI SDK to establish a connection to a drone is the OnCreate () method in the MainActivity class. The DJI SDK can also disconnect the drone in the OnDestory () method of the MainActivity class. After connecting to the drone, the developer can gain access to other classes (eg, camera class 1102 and gimbal class 1104). Drone class 1101 can then be used to call specific functions, such as camera and gimbal functions, to control the behavior of the drone.
0056According to various embodiments of the invention, the application can use battery class 1103 to control the power supply of the drone. The application can also use battery class 1103 to plan and test the schedule of various flight missions.
0057Since batteries are the most regulated element in an unmanned aerial vehicle, applications may consider battery conditions not only for the safety of the unmanned aerial vehicle, but also to ensure that the unmanned aerial vehicle completes its designated mission. For example, battery class 1103 can be configured so that the drone can complete its mission and return immediately if the battery level is low.
0058The DJI SDK allows applications to get the current status and information of a battery by calling the get () function in the DJI drone battery class. The application can also use the set () function to control the frequency of feedback.
0059According to various embodiments of the present invention, the application can use camera class 1102 to define various operations on the camera in a moving object such as an unmanned aerial vehicle. For example, in the DJI SDK, the DJI camera class includes functions for receiving media data in the SD card, acquiring and setting photographic parameters, and taking pictures and recording videos.
0060The application can use camera class 1102 to modify photo and recording settings. For example, developers can use the setCameraPhotoSize () method to adjust the size of the photos they take. Applications can also use media classes to maintain photos and recordings.
0061According to various embodiments of the invention, the application can use gimbal class 1104 to control the field of view of the drone. For example, the DJI gimbal class can be used to configure the actual field of view, for example to set the first field of view for the drone itself. You can also use the DJI gimbal class to automatically stabilize the gimbal and focus in one direction. The application can also use the DJI gimbal class to change the angle of view to detect various objects.
0062According to various embodiments of the invention, the application can use flight control device class 1105 to provide various flight control information and situations for an unmanned aerial vehicle.
0063The DJI main controller class allows applications to monitor flight status, for example using instant messaging. For example, a callback function in the DJI main controller class can return an instant message every 1000 milliseconds (1000 ms).
0064In addition, the DJI main controller class allows application users to inspect instant messages received from the drone interface. For example, pilots can analyze data for each flight to further improve their flight skills.
0065According to various embodiments of the invention, an application can use ground station class 1107 to perform a series of operations to control an unmanned aerial vehicle.
0066For example, the DJI SDK uses the DJI ground station class, which may require an application to have an SDK level 2 key. The DJI ground station class provides manual control of one-key-fly, on-key-go-home drones by app (ie joystick mode), cruising and / or midpoint setting, and various other task scheduling features. It can be realized.
0067According to various embodiments of the invention, an application can use communication components such as Range Extender Class 1106 to establish a network connection between the application and the drone.
0068FIG. 12 shows a flowchart of supporting a movable object interface in a software development environment according to various embodiments of the present invention. As shown in FIG. 12, in step 1201, the system can associate one or more interface components with a movable object interface that represents a movable object. Then, in step 1202, the data manager can receive data from the moving object, which corresponds to one or more modules in the moving object. Further, in step 1203, the data manager can provide the data for the one or more modules in the movable object to the one or more interface components associated with the movable object interface.
0069Using an authentication server to support the security model FIG. 13 is an exemplary diagram of using an authentication server to support a security model in a moving object environment according to various embodiments of the present invention. As shown in FIG. 13, the authentication server 1301 can be used in the application development environment 1300 to support the security model.
0070The application development environment can create (or associate with) the application identifier 1307 for the application 1303 under development. For example, the application identifier 1307 can be a package name on an Android system or a bundle name on an iOS system.
0071As shown in FIG. 13, the developer can submit the application identifier 1307 to the web portal 1309, which can send such information to the authentication server 1301. Therefore, the authentication server 1301 can assign the application key 1305 to the application 1303 under development based on the application identifier 1307 associated with the application 1303 under development.
0072Application 1304 can then be implemented on user terminal 1302 using application key 1305. According to various embodiments of the present invention, the application key 1305 is uniquely associated with the application identifier 1307, and each application key is associated with a set of privileges.
0073As shown in FIG. 13, the authentication server 1301 in the application development environment 1300 can receive the activation request 1310 from the application 1304. For example, activation request 1310 may include application key 1305.
0074According to various embodiments of the invention, this activation request can be a request for authorization 1308 to access a moving object. For example, the movable object can be an unmanned vehicle, an unmanned vehicle, a handheld device, or a robot.
0075As shown in FIG. 13, the authentication server 1301 can apply one or more policies 1306 to activation request 1310 to determine whether application 1304 should be granted permission 1308 to access moving objects. These policies can be defined using different criteria, such as maximum number of installations and / or different roles of the application.
0076According to various embodiments of the present invention, the authentication server 1301 can make a decision based on the application key 1305, which application key is included in activation request 1310. For example, different activation keys may be associated with different sets of permissions.
0077As shown in FIG. 13, if the application 1304 is entitled to the privilege 1308, the authentication server 1301 can grant the application 1304 the privilege 1308 to access the moving object.
0078FIG. 14 is an exemplary diagram of using an authentication server to support a plurality of applications in a moving object environment according to various embodiments of the present invention. As shown in FIG. 14, the authentication server 1420 in the movable object environment 1400 is used to process activation requests from different applications A to B (1401 to 1402) based on one or more policies 1421. You can access and control the movable object 1410.
0079For example, application A (1401) is assigned application key A (1403) and can be associated with a set of privileges, such as privilege A (1407). On the other hand, application B (1401) is assigned application key B (1404), which can be associated with different sets of privileges, such as privilege B (1408).
0080According to various embodiments of the invention, software development kits (SDKs) can include security mechanisms, which are low-level features available to all registered developers, and only available to certified developers. Includes possible high level features. To obtain a high level of authority, the application may need to go through an approval process, which requires real name registration and a statement of purpose of use.
0081FIG. 15 is an exemplary diagram of how various embodiments of the present invention support a security model in a moving object environment. As shown in Figure 15, the application development environment 1500, for example the authentication server 1505 in the software development kit (SDK), can receive an activation request from application 1504 to access and control the moving object 1501. it can.
0082The authentication server can maintain a maximum number of installations, 1511, which can be dynamically preconfigured or configured. The authentication server 1505 can then update the activated application count 1512 when application 1504 is activated.
0083According to various embodiments of the present invention, the authentication server 1505 may reject the activation request if the activated application count 1512 is greater than or equal to the maximum number of installations 1511.
0084In addition, the authentication server 1505 can increase the maximum number of installations 1511 to handle upgrade requests or set the maximum number of installations to zero to stop illegal or inappropriate applications.
0085As shown in FIG. 15, the authentication server 1505 allows an application to access the movable object 1501 using interface 1510. For example, the movable object 1501 can be an unmanned aircraft, the interface 1510 of which comprises a camera component, a battery component, a gimbal component, a communication component, and a flight control device component.
0086Further, the interface 1510 may include a ground station component, which is associated with a flight control component, which operates to perform one or more flight control operations. , Configured to be accessible only by a particular set of privileged applications.
0087In the early stages of application development, the SDK may approve only a small number of applications to secure application development. After the developer finishes development, the SDK can inspect requirements documents, technical documents, installation documents, and associated source code during the approval process. After completing the approval process, the SDK can upgrade the application by increasing the maximum number of installations 1511.
0088According to various embodiments of the present invention, an activation process based on a maximum of 1511 installations can be used to prevent malicious or inappropriate applications.
0089For example, the system can set the maximum number of installations 1511 to zero to prevent malicious applications from becoming active. In addition, the system can disable applications, such as preventing malicious applications from accessing the SDK.
0090FIG. 16 shows a flowchart of how various embodiments of the present invention support a security model in a moving object environment. As shown in FIG. 16, in step 1601, the authentication server can receive an activation request from the application, which includes a request for permission to access the moving object. Further, in step 1602, the authentication server can apply one or more policies to the activation request. Then, in step 1603, if the application is entitled, the authentication server can grant the application permission to access the moving object.
0091Many features of the invention can be performed in or with the help of hardware, software, firmware, or a combination thereof. Therefore, the features of the present invention may be implemented using a processing system (eg, including one or more processors). Exemplary processors include, but are not limited to, one or more general purpose microprocessors (eg, single or multiprocessors), application-specific integrated circuits, application-specific instruction set processors, graphics processors, physics. Processing devices, digital signal processing devices, coprocessors, network processing devices, audio processing devices, encryption processing devices, etc. may be included.
0092A feature of the present invention is a storage medium (s), or storage medium, which internally stores instructions that can be used to program a processing system to perform any of the features presented herein. It can be performed in or with the help of computer program products that are computer readable media (s). Storage media include, but are not limited to, floppy disks, optical disks, DVDs, CD-ROMs, microdrives, and any type of disk including magneto-optical disks, ROM, RAM, EPROM, EEPROM, DRAM, VRAM, flash. It may include memory devices, magnetic or optical cards, nanosystems (including molecular memory ICs), or any type of medium or device suitable for storing instructions and / or data.
0093Stored on any one of machine-readable (s) media, features of the invention can be incorporated into software and / or firmware to control the hardware of the processing system. Also, the results of the present invention can be used to allow the processing system to interact with other mechanisms. Such software or firmware may include, but are not limited to, application code, device drivers, operating systems, and execution environments / containers.
0094Features of the invention may also be implemented in hardware using hardware components such as application specific integrated circuits (ASICs) and field programmable gate array (FPGA) devices. An implementation of a hardware state machine for performing the functions described herein will be apparent to those skilled in the art.
0095Further, conveniently, the invention is programmed according to one or more conventional general purpose or dedicated digital computers, computing devices, machines, or one or more processors, memories, and / or the teachings of the present invention. It may be carried out using a microprocessor containing a computer-readable storage medium. Appropriate software coding can be processed by a skilled programmer on the basis of the teachings of the present disclosure, as will be apparent to those skilled in the art of software.
0096Although various embodiments of the present invention have been described so far, it should be understood that they are presented as examples and are not limiting. It will be apparent to those skilled in the art that various modifications of embodiments and details may be made to embodiments without departing from the spirit and scope of the invention.
0097The present invention has been described so far with the help of function building blocks that show the performance of specified functions and their relationships. For convenience of explanation, the boundaries of these function building blocks have often been arbitrarily defined herein. Alternative boundaries can be defined as long as the specified functions and their relationships are properly executed. Therefore, any such alternative boundaries are within the scope and gist of the present invention.
0098The above description of the present invention has been made for the purpose of exemplification and explanation. This is not exhaustive or does not limit the invention to the exact forms disclosed. The breadth and scope of the invention should not be limited by any of the exemplary embodiments described above. Many modified and modified forms will be apparent to those skilled in the art. These modified and modified forms include any related combination of disclosed features. Each embodiment has been selected and described in order to best illustrate the principles of the present invention and its practical applications, thereby providing a variety of embodiments and suitable for a particular intended use. Along with the modifications, those skilled in the art will be able to understand the present invention. The scope of the present invention is defined by the scope of the appended claims and their equivalents.
17 sheets
Sheet 1 Sheet 2 Sheet 3 Sheet 4 Sheet 5 Sheet 6 Sheet 7 Sheet 8 Sheet 9 Sheet 10 Sheet 11 Sheet 12 Sheet 13 Sheet 14 Sheet 15 Sheet 16 Sheet 17
Every citation, both ways
| Document | Relation | Office | Category | Cited during | Relevant claims |
|---|---|---|---|---|---|
| JP2014505934A | Cites | Japan | XY | Search report | 1-6,8,13-19,21,26-28,7,9-12,20,22-25 |
| JP2014505934A | Cites | Japan | XY | Search report | 1-6,8,13-19,21,26-28,7,9-12,20,22-25 |
| WO2015014116A1 | Cites | World Intellectual Property Organization (WIPO) | A | Search report | – |
| WO2015014116A1 | Cites | World Intellectual Property Organization (WIPO) | A | Search report | – |
| WO2015014116A1 | Cites | World Intellectual Property Organization (WIPO) | A | Search report | – |
| 松倉 隆一: "「家電をWebで制御するサービスプラットフォーム」", 情報処理学会研究報告 エンタテインメントコンピューティング(EC), vol. 第2014−EC−31巻,第48号, JPN6017023424, March 2014 (2014-03-01), pages 1 - 8, ISSN: 0003586153 | Non-patent | – | – | Search report | – |
11 members in 4 offices
Priority claims1
| Document | Office | Kind | Date |
|---|---|---|---|
| 2015076992 | China | W |
Members11
| Document | Office | Kind | |
|---|---|---|---|
| WO2016168972A1 | World Intellectual Property Organization (WIPO) | A1 | |
| US2017064068A1 | United States of America | A1 | |
| JP2017520810AThis record | Japan | A | |
| CN107431613A | China | A | |
| JP6250166B2 | Japan | B2 | |
| US10116785B2 | United States of America | B2 | |
| US2019045044A1 | United States of America | A1 | |
| CN107431613B | China | B | |
| CN113163405A | China | A | |
| US11184474B2 | United States of America | B2 | |
| US2022053079A1 | United States of America | A1 |
16 legal events, as the office reported them to INPADOC
Over the term
Point at a mark for the eventEvents
| Event | Code | |
|---|---|---|
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Certificate of patent or registration of utility modelJAPANESE INTERMEDIATE CODE: R150R150 | R150 | |
| First payment of annual fees (during grant procedure)JAPANESE INTERMEDIATE CODE: A61A61 | A61 | |
| Written decision to grant a patent or to grant a registration (utility model)JAPANESE INTERMEDIATE CODE: A01A01 | A01 | |
| Decision of grant or rejection writtenTRDD | TRDD | |
| Request for written amendment filedJAPANESE INTERMEDIATE CODE: A523A521 | A521 | |
| Notification of reasons for refusalJAPANESE INTERMEDIATE CODE: A131A131 | A131 | |
| Report on accelerated examinationJAPANESE INTERMEDIATE CODE: A971005A975 | A975 | |
| Request for written amendment filedJAPANESE INTERMEDIATE CODE: A523A521 | A521 | |
| Written request for application examinationJAPANESE INTERMEDIATE CODE: A621A621 | A621 | |
| Explanation of circumstances concerning accelerated examinationJAPANESE INTERMEDIATE CODE: A871A871 | A871 |
Numbers
- Publication
- 2017520810
- Application
- 2016534951
Titles2
- Japanese
- 可動物体のアプリケーション開発を支援する方法、システム及びコンピュータ読取り可能媒体
- English
- Methods, systems and computer readable media to assist in the development of moving object applications
Classification
- CPC, 7
- H04L9/088
- H04M1/72415
- H04W12/06
- H04W12/08
- H04W4/50
- H04W4/40
- H04W12/35
- IPC, 4
- G06F9 44
- G06F9 06
- G06F9 445
- H04M1 72415
Designated states5
- Regional, 4
- Zimbabwe
- Turkmenistan
- Türkiye
- Togo
- National, 1
- United States of America