Mobile human interface robot
Abstract
Problem to be solved.To provide a useful mobile human interface robot.
Solution.A mobile human interface robot 100 includes a base 120 that determines a perpendicular center axis line and forward driving direction F, and a holonomic drive system borne by the base 120. The drive system includes first, second, and third passive driving wheels. The driving wheels are equidistantly arranged in the form of a triangle about the perpendicular center axis line, and have driving directions perpendicular to a radial direction axis line with respect to the perpendicular center axis line. A touch sensor system responds to a human touch. A controller issues a drive command to the holonomic drive system on the basis of a touch signal received from the touch sensor system.

Term
7.7 yearsto projected expiry
Projected expiry 26 May 2034, counted from filing; an application has no term until it is granted.
- Priority
- Filed
- Published
- Today
- Projected expiry
38 claims: 12 independent, 26 dependent
- 1移動式ヒューマンインターフェースロボット(100)であって、 垂直中心軸線(Z)及び前方駆動方向(F)を定める基部(120)を有し、 前記基部(120)によって支持されたホロノミック駆動システム(200)を有し、前記駆動システム(200)は、第1、第2及び第3の被動駆動輪(210a,210b,210c)を有し、各駆動輪は、前記垂直中心軸線(Z)回りに三角形の形に互いに間隔を置いて配置されると共に前記垂直中心軸線(Z)に対して半径方向軸線(Slip)に垂直な駆動方向(Drive)を有し、 前記ホロノミック駆動システム(200)と通信状態にあるコントローラ(500)を有し、 前記基部(120)の上方に支持された胴部(140)を有し、 前記コントローラ(500)と通信状態にあるタッチセンサシステム(480)を有し、前記タッチセンサシステム(480)は、人の接触に応動し、 前記コントローラ(500)は、前記タッチセンサシステム(480)から受け取ったタッチ信号に基づいて駆動指令を前記ホロノミック駆動システム(200)に出す、移動式ヒューマンインターフェースロボット(100)。
- 2前記タッチセンサシステム(480)は、少なくとも一部が前記胴部(140)に設けられている、請求項1記載の移動式ヒューマンインターフェースロボット(100)。
- 3前記基部(120)に取り付けられると共に前記胴部(140)を支持した作動可能な伸縮式脚部(130)を更に有し、前記コントローラ(500)は、前記タッチセンサシステム(480)から受け取った前記タッチ信号に応答して前記脚部の高さ(H L )を変更して前記胴部(140)の高さ(H T )を変更し、好ましくは、前記コントローラ(500)は、前記タッチセンサシステム(480)から受け取った前記タッチ信号に応答して前記脚部(130)に命令して前記脚部が前記胴部(140)のユーザ高さ変更に対するアクティブな支援を提供するようにする、請求項1又は2記載の移動式ヒューマンインターフェースロボット(100)。
- 4前記駆動指令は、前記ロボット(100)を前記ロボット(100)と接触状態にあるユーザからの支援により推進する電力減少支援駆動指令を含む、請求項1〜3のうちいずれか一に記載の移動式ヒューマンインターフェースロボット(100)。
- 5前記胴部(140)に取り付けられた頸部(150)及び前記頸部(150)によって支持された頭部(160)を更に有し、前記頸部(150)は、前記頭部(160)を前記胴部(140)に対してパンしたり傾動させたりしたりするよう構成され、好ましくは、前記コントローラ(500)は、頭部タッチを指示する前記タッチセンサシステム(480)から受け取った前記タッチ信号に応答して前記頭部(160)のユーザ関節運動を可能にすると共に頭部タッチの完了後、前記頭部(160)の姿勢を維持する、請求項1〜4のうちいずれか一に記載の移動式ヒューマンインターフェースロボット(100)。
- 6前記コントローラ(500)は、前記タッチセンサシステム(480)から受け取った前記タッチ信号に応答して前記ロボット(100)に対する接触場所に向くよう前記頭部(160)を関節運動させる、請求項5記載の移動式ヒューマンインターフェースロボット(100)。
- 7前記頸部(150)及び前記頭部(160)のうちの少なくとも一方に設けられていて、前記頭部(160)のユーザ操作を検出する操作センサ(158a,158b,165)を更に有する、請求項5記載の移動式ヒューマンインターフェースロボット(100)。
- 8前記タッチセンサシステム(480)は、容量センサ(147t,147b,147f,147r,147l,165)、接触センサ(147t,147b,147r,147l,165)、カメラ(320)、三次元イメージセンサ(450)及びスイッチ(147t,147b,147f,147r,147l,165)のうちの少なくとも1つを有し、好ましくは、タッチセンサシステムは、前記胴部(140)の頂面、底面、右側面、左側面、前面及び後面(144,146,147,148,149)に取り付けられた接触センサ(147t,147b,147f,147r,147l)を有する、請求項1〜7のうちいずれか一に移動式ヒューマンインターフェースロボット(100)。
- 9前記タッチセンサシステム(480)は、前記ロボット(100)の支持面から3フィート(91.5cm)〜5フィート(152.5cm)のところに位置する少なくとも1つの作動可能なタッチ応答式入力(147t,147b,147f,147r,147l,165)を有する、請求項1〜8のうちいずれか一に記載の移動式ヒューマンインターフェースロボット(100)。
- 10前記コントローラ(500)は、受け取ったタッチとは逆の方向における前記ロボット(100)の少なくとも一部分の運動抵抗を増大させると共に/或いは前記タッチセンサシステム(480)から受け取った前記タッチ信号に応答して駆動を停止させるよう前記駆動システム(200)に命令を出す、請求項1〜9のうちいずれか一に記載の移動式ヒューマンインターフェースロボット(100)。
- 11移動式ヒューマンインターフェースロボット(100)であって、 前方駆動方向(F)を有する基部(120)と、 前記基部(120)によって支持された駆動システム(200)と、 前記基部(120)によって支持されていて、毎秒100億を超える命令(million instruction(s) per second :MIPS)を処理することができるロボットコンピュータ(500)と、 前記基部(120)の上方に取り外し可能に支持されると共に前記ロボットコンピュータ(500)とワイヤレス通信状態にある少なくとも1つのタブレット型コンピュータ(310,310a,310b)と、 前記基部(120)の上方に支持されていて、前記少なくとも1つのタブレット型コンピュータ(310,310a,310b)とは別個に少なくとも1つの自由度の範囲内で動くことができるカメラ(320,320a,320b,450,450a,450b)とを有する、移動式ヒューマンインターフェースロボット(100)。
- 12前記駆動システムは、第1、第2及び第3の被動駆動輪(210a,210b,210c)を有し、各駆動輪は、前記垂直中心軸線(Z)回りに三角形の形に互いに間隔を置いて配置されると共に前記垂直中心軸線(Z)に対して半径方向軸線(Slip)に垂直な駆動方向(Drive)を有し、好ましくは、各駆動輪(210a,210b,210c)は、前記駆動輪(210a,210b,210c)の周囲に沿って設けられた第1及び第2の列(232,234)をなすローラ(230)を有し、各ローラ(230)は、前記駆動輪(D R )の転動方向に垂直な転動方向(D r )を有し、より好ましくは、前記ローラ(230)は各々、弧状転動面(235)を備え、前記ローラ(230)は、一緒になって、前記駆動輪(210a,210b,210c)の少なくとも実質的に円形の転動面を構成する、請求項11記載の移動式ヒューマンインターフェースロボット(100)。
- 13前記カメラ(320,320a,320b,450,450a,450b)は、前記ロボット(100)に隣接して位置する空間ボリュームから点群を得るよう位置決めされたボリューム形点群イメージング装置(450,450a,450b)を有し、好ましくは、前記ボリューム形点群イメージング装置(450,450a,450b)は、地面よりも約1フィート(30.5cm)以上、上方の高さ位置のところに位置決めされると共に前記移動式ロボット(100)の移動方向において床平面を含む空間ボリュームから点群を得ることができるよう差し向けられている、請求項11又は12記載の移動式ヒューマンインターフェースロボット(100)。
- 14前記カメラ(320,320a,320b,450,450a,450b)は、前記基部(120)の前縁の近くの領域を視認するよう動くことができる、請求項13記載の移動式ヒューマンインターフェースロボット(100)。
- 15前記タブレット型コンピュータ(310,310a,310b)は、少なくとも150平方インチ(967.7cm 2 )の表示領域を有すると共に/或いは前記ロボット(100)に取り付けられた状態で少なくとも1つの自由度で動くことができるタッチスクリーン(312)を有する、請求項11〜14のうちいずれか一に記載の移動式ヒューマンインターフェースロボット(100)。
- 16前記ロボットコンピュータ(500)と電気通信状態にあるディスプレイ(310b)を更に有し、前記タブレット型コンピュータ(310,310a)は、前記ディスプレイ(310b)に取り外し可能に装着でき、前記ディスプレイ(310b)は、前記タブレット型コンピュータ(310,310a)が前記ディスプレイ(310b)に装着されると不作動状態になり、前記タブレット型コンピュータ(310,310a)が前記ディスプレイ(310b)から取り外されると作動状態になる、請求項15記載の移動式ヒューマンインターフェースロボット(100)。
- 17移動式ヒューマンインターフェースロボット(100)を作動させる方法であって、前記方法は、 ユーザが前記ロボット(100)にタッチしたことに応答して前記ロボット(100)のタッチセンサシステム(480)からタッチ信号を受け取るステップと、 しきい期間を超えるタッチ持続時間の間、前記ロボットを前記ロボット(100)への接触場所に基づく方向に駆動するステップと、 前記しきい期間よりも短いタッチ持続時間の間、前記ロボット(100)のインターフェース(310,320,320a,320b,450,450a,450b)を関節運動させて前記インターフェースが少なくとも実質的に、前記ロボット(100)への前記接触場所に向くようにするステップと、 前記しきい期間よりも短いタッチ持続時間の間、ゼロ速度駆動指令を出すステップとを有する、方法。
- 18駆動指令をホロノミック駆動システム(200)に出して前記ロボット(100)を前記しきい期間、好ましくは0.25秒間よりも長いタッチ持続時間、前記受け取ったタッチの前記ロボット(100)とは逆方向に向く方向に動かすステップを更に有する、請求項17記載の方法。
- 19前記ロボット(100)の頭部(160)を前記ロボット(100)の連結状態の胴部(140)に対してパンすること及び傾動させることのうちの少なくとも一方を行って少なくとも実質的に、前記頭部(160)を前記しきい期間よりも短いタッチ持続時間の間、前記ロボット(110)に対する前記接触場所の方へ向かせるステップを更に有する、請求項17又は18記載の方法。
- 20前記タッチセンサシステム(480)は、容量センサ(147t,147b,147f,147r,147l,165)、接触センサ(147t,147b,147f,147r,147l,165)、カメラ(320)、三次元イメージセンサ(450)及びスイッチ(147t,147b,147f,147r,147l,165)のうちの少なくとも1つを有する、請求項17〜19のうちいずれか一に記載の方法。
- 21移動式ヒューマンインターフェースロボット(100)であって、 駆動モータ(220a,220b,220c)により対応関係をなして駆動される少なくとも1つの駆動輪(210,210a,210b,210c)を有する駆動システム(200)と、 前記駆動システム(200)と通信状態にある存在場所突き止めシステム(1710)と、 前記駆動システム(200)及び前記存在場所突き止めシステム(1710)と通信状態にある電源(105)と、 前記駆動システム(200)の上方に支持されたタッチ応答入力(175)とを有し、 前記タッチ応答入力(175)の作動により、前記駆動システム(200)への配電状態が変更され、それにより前記対応の少なくとも1つの駆動輪(210,210a,210b,210c)への前記駆動モータ(220a,220b,220c)の駆動荷重が少なくとも減少する、移動式ヒューマンインターフェースロボット(100)。
- 22前記タッチ応答入力(175)の作動により、前記駆動システム(200)への配電が終了し、他方、前記存在場所突き止めシステム(1710)への配電を続行させることができると共に/或いは前記対応の駆動モータ(220a,220b,220c)からの少なくとも1つの駆動輪(210,210a,210b,210c)の結合解除が生じる、請求項21記載の移動式ヒューマンインターフェースロボット(100)。
- 23前記駆動システム(200)は、前記タッチ応答入力(175)の作動に応答して、前記ロボット(100)を移動させると共に前記ロボット(100)のユーザ運動を支援することしかできない電力減少駆動指令を出す、請求項21又は22記載の移動式ヒューマンインターフェースロボット(100)。
- 24前記タッチ応答入力(175)は、地面よりも約3フィート(91.5cm)〜約5フィート(152.5cm)上方に位置している、請求項21〜23のうちいずれか一に記載の移動式ヒューマンインターフェースロボット(100)。
- 25前記存在場所突き止めシステム(1710)は、センサシステム(400)と、前記駆動システム(200)、前記センサシステム(400)、前記電源(105)及び前記タッチ応答入力(175)と通信状態にあるコントローラ(500)とを有する、請求項21〜24のうちいずれ一に記載の移動式ヒューマンインターフェースロボット(100)。
- 26前記センサシステム(400)は、慣性測定ユニット(470)、オドメータ、全地球測位システム、レーザスキャナ(440)、ソナー近接センサ(410)及び三次元イメージセンサ(450)のうちの少なくとも1つを有する、請求項25記載の移動式ヒューマンインターフェースロボット(100)。
- 27前記存在場所突き止めシステム(1710)は、三次元イメージセンサ(450)、レーザスキャナ(440)、1つ又は2つ以上のソナー近接センサ(410)、前記少なくとも1つの駆動輪(210,210a,210b,210c)のための駆動輪エンコーダ(138a)及び駆動輪モータフィードバックのうちの少なくとも1つを有する、請求項21〜26のうちいずれか一に記載の移動式ヒューマンインターフェースロボット(100)。
- 28前記駆動システム(200)は、第1、第2及び第3のホロノミック的に駆動される駆動輪(210a,210b,210c)を有し、各駆動輪は、前記ロボット(100)の垂直軸線(Z)回りに三角形の形に間隔を置いて配置されると共に前記垂直軸線(Z)に対して半径方向軸線(Slip)に垂直な駆動方向(Drive)を有する、請求項21〜27のうちいずれか一に記載の移動式ヒューマンインターフェースロボット(100)。
- 29前記駆動システム(200)を支持した基部(120)と、 前記基部(120)から上方に延びる作動可能な伸縮式脚部(130)と、 前記脚部(130)によって支持されていて、地面よりも約3フィート(91.5cm)〜約5フィート(152.5cm)上方に位置する胴部(140)とを更に有し、前記タッチ応答入力(175)は、前記胴部(140)に設けられている、請求項21〜28のうちいずれか一に記載の移動式ヒューマンインターフェースロボット(100)。
- 30前記タッチ応答入力(175)は、接触センサ、容量センサ、作動可能なボタン及びスイッチのうちの少なくとも1つを有する、請求項21〜29のうちいずれか一に記載の移動式ヒューマンインターフェースロボット(100)。
- 31移動式ヒューマンインターフェースロボット(100)であって、 垂直中央軸線(Z)回りに実質的に三角形の形をした対称形状を定めると共に第1、第2及び第3の部分(124a,124b,124c)を有する基部(120)を有し、 前記基部(120)によって支持されたホロノミック駆動システム(200)を有し、前記駆動システム(200)は、第1、第2及び第3の被動駆動輪(210a,210b,210c)を有し、各駆動輪は、前記垂直中心軸線(Z)回りに三角形の形に互いに間隔を置いて配置されると共に対応の前記第1、前記第2及び前記第3の基部部分(124a,124b,124c)により支持され、各駆動輪(210a,210b,210c)は、前記垂直中心軸線(Z)に対して半径方向軸線(Slip)に垂直な駆動方向(Drive)を有し、 前記基部(120)から上方に延びると共に可変高さ(H L )を有する脚部(130)を有し、 前記脚部(130)によって支持された胴部(140)を有し、前記胴部(140)は、前記基部(120)から張り出した底面(144)を有する肩(142)を備え、 前記胴部(140)の前記底面(144)に設けられていて、前記駆動システム(200)の前方駆動方向(F)に沿って下方に向いた胴部イメージングセンサ(450,450a)を有し、前記胴部イメージングセンサ(450,450a)は、前記ロボット(100)の周りのシーンの三次元イメージを捕捉し、 前記胴部(140)によって支持された頸部(150)を有し、 前記頸部(150)によって支持された頭部(160)を有し、前記頸部(150)は、前記頭部(160)を前記垂直中心軸線(Z)に対してパンさせたり傾動させたりし、 前記頭部(160)によって支持されたディスプレイ(310,310a,310b,312)を有する、移動式ヒューマンインターフェースロボット(100)。
- 32前記ディスプレイ(310,310a,310b,312)は、前記頭部(160)に解除可能に取り付けられ、好ましくは、前記ディスプレイ(310,310a,310b,312)は、タッチスクリーン(312)を備えたタブレット型コンピュータ(310,310a,310b)を有する、請求項31記載の移動式ヒューマンインターフェースロボット(100)。
- 33前記ディスプレイ(310,310a,310b,312)は、前記頭部(160)から取り外されると、前記ディスプレイ(310,310a,310b,312)の存在場所を突き止めるロケータ(315)を有する、請求項31又は32記載の移動式ヒューマンインターフェースロボット(100)。
- 34前記頭部(160)に解除可能に取り付けられたタブレット型コンピュータ(310,310b)を更に有し、前記タブレット型コンピュータ(310,310b)は、前記ディスプレイ(310,310a)に解除可能に装着される、請求項31〜33のうちいずれか一に記載の移動式ヒューマンインターフェースロボット(100)。
- 35前記頭部(160)に取り付けられたカメラ(320,320a,32b,450,450a,450b)を更に有する、請求項31〜34のうちいずれか一に記載の移動式ヒューマンインターフェースロボット(100)。
- 36前記頭部(160)に取り付けられると共に前記ロボット(100)の周りのシーンの三次元イメージを捕捉する頭部イメージングセンサ(320,320a,320b,450,450b)を更に有し、前記頭部イメージングセンサ(320,320a,320b,450,450b)は、好ましくは、前記ロボット(100)の周りの空間ボリュームから点群を得ることができるよう位置決めされたボリューム形点群イメージング装置(450,450b)を有する、請求項31〜35のうちいずれか一に記載の移動式ヒューマンインターフェースロボット(100)。
- 37前記頸部(150)に取り付けられると共に前記頭部(160)を前記頸部(150)から遠ざけて支持するアーム(190)を更に有する、請求項31〜36のうちいずれか一に記載の移動式ヒューマンインターフェースロボット(100)。
- 38前記胴部(140)に取り付けられたマニピュレータアーム(180)及び前記マニピュレータアーム(180)の遠位端部に取り付けられたエンドエフェクタ(182)を更に有し、好ましくは、前記マニピュレータアーム(180)は、長さが可変である、請求項31〜37のうちいずれか一に記載の移動式ヒューマンインターフェースロボット(100)。
Independent claims38
145 paragraphs, as filed
The present invention relates to a mobile human interface robot.
This US patent application is a US patent provisional application No. 61 / 346,612 filed on May 20, 2010, a US patent provisional application No. 61 / 356,910 filed on June 21, 2010, US Patent Provisional Application No. 61 / 428,717 filed on December 30, 2010, US Patent Provisional Application No. 61 / 428,734 filed on December 30, 2010, December 30, 2010 35U. Of US Patent Provisional Application Nos. 61 / 428,759 filed in and US Patent Provisional Application Nos. 61 / 429,863 filed on January 5, 2011. S. C. 35U. Of US Patent Application No. 13/032,370 filed on February 22, 2011, pursuant to the provisions of §119 (e). S. C. This is a priority claim application based on §120. These earlier applications are cited by reference and the entire disclosure of these earlier applications is incorporated herein by reference.
A robot is generally an electromechanical device guided by a computer or electronic programming. Mobile robots have the ability to move around in their environment and are not fixed in one physical location. An example of a mobile robot commonly used today is an automatic guided vehicle or an automatic guided vehicle (AGV). AGVs are generally mobile robots that use a vision system or laser for tracing or navigating floor markers or wires. Mobile robots are found in industrial, military or security environments. These mobile robots can also be considered consumer products for entertainment or for performing certain tasks such as vacuum cleaning and home assistance.
0004One aspect of the present invention provides a mobile human interface robot having a vertical central axis and a base that determines the forward drive direction and a holonomic drive system supported by the base. The drive system has first, second and third driven wheels, each of which is spaced around the vertical center axis in a triangular shape and has a radius relative to the vertical center axis. It has a drive direction perpendicular to the direction axis. The robot further comprises a controller in communication with the holonomic drive system, a torso supported above the base and a touch sensor system in communication with the controller. The touch sensor system responds to human contact. The controller issues a drive command to the nonholonomic drive system based on the touch signal received from the touch sensor system.
0005The embodiment of the present invention can include one or more of the following features. In some embodiment, the touch sensor system is at least partially provided on the torso. The robot further has actuable telescopic legs that are attached to the base and support the torso. The controller changes the height of the legs in response to the touch signal received from the touch sensor system to change the height of the torso. The controller commands the legs in response to the touch signal received from the touch sensor system so that the legs provide active assistance for changing the user height of the torso.
0006In some embodiments, the drive command includes a power reduction assist drive command that propels the robot with assistance from a user in contact with the robot. The robot may further have a neck attached to the torso and a head supported by the neck. The neck should be configured to pan or tilt the head with respect to the torso. The controller can enable the user joint movement of the head in response to the touch signal received from the touch sensor system instructing the head touch, and can maintain the posture of the head after the completion of the head touch. In addition, the controller can jointly move the head towards the point of contact with the robot in response to the touch signal received from the touch sensor system. In some embodiments, the robot is provided on at least one of the neck and head and further comprises an operation sensor that detects user manipulation of the head.
0007In some embodiment, the touch sensor system has at least one of a capacitance sensor, a contact sensor, a camera, a three-dimensional image sensor and a switch. The touch sensor system preferably has contact sensors attached to the top surface, bottom surface, right side surface, left side surface, front surface and rear surface of the body. In addition, the touch sensor system may have at least one operable touch-responsive input located 3 feet (91.5 cm) to 5 feet (152.5 cm) from the robot's support surface.
0008The controller, in some embodiments, increases the motion resistance of at least a portion of the robot in the direction opposite to the touch received. In an additional embodiment, the controller commands the drive system to stop driving in response to a touch signal received from the touch sensor system.
0009In another aspect, the mobile human interface robot is supported by a base with a forward drive direction, a drive system supported by the base, and a base, with more than 10 billion instructions (s) per second per second. : MIPS) with a robot computer capable of processing. The robot is detachably supported above the base and at least one tablet computer in wireless communication with the robot computer and at least separately from the at least one tablet computer supported above the base. It also has a camera that can move within one degree of freedom.
0010In some embodiment, the drive system has first, second and third driven drive wheels, each of which is spaced apart from each other in a triangular shape around the vertical center axis. It also has a drive direction perpendicular to the radial axis with respect to the vertical center axis. Each drive wheel has first and second rows of rollers provided along the perimeter of the drive wheel. Each roller has a rolling direction perpendicular to the rolling direction of the drive wheels. Further, each roller is preferably provided with an arcuate rolling surface. The rollers together form at least a substantially circular rolling surface of the drive wheels.
0011In some embodiments, the camera is positioned at a height above about 1 foot (30.5 cm) above the ground and points from a spatial volume that includes a floor plane in the direction of movement of the mobile robot. It is a volume point cloud imaging device that is directed so that a group can be obtained. In an additional embodiment, the camera is a volume point cloud imaging device positioned to obtain a point cloud from a spatial volume located adjacent to the robot. The camera can move to see the area near the front edge of the base.
0012Tablet computers should be at least 150 square inches (967.7 cm)<sup>2</sup>) Has a display area. In addition, the tablet computer can move with at least one degree of freedom while attached to the robot. In some embodiments, the robot has a display that is in telecommunications with the robot computer. The tablet computer can be detachably attached to the display. The display goes into an inactive state when the tablet computer is attached to the display and goes into action when the tablet computer is removed from the display.
0013In yet another aspect, the method of activating the mobile human interface robot comprises the step of receiving a touch signal from the robot's touch sensor system in response to the user touching the robot. For a touch duration that exceeds the threshold period, the method comprises a step of driving the robot in a direction based on the location of contact with the robot. For a touch duration shorter than the threshold period, this method involves the steps of articulating the robot's interface so that the interface is at least substantially oriented towards the point of contact with the robot. For a touch duration shorter than the threshold period, this method has a step of issuing a zero speed drive command.
0014In some embodiment, this method has a step of issuing a drive command to the holonomic drive system to move the robot in a direction opposite to the robot of the received touch, with a touch duration longer than the threshold period. .. The threshold period should be 0.25 seconds. This method performs at least one of panning and tilting the robot's head against the robot's articulated torso, at least substantially substantially shorter than the threshold period. It is good to have a step towards the point of contact with the robot over time.
0015In some embodiments, the touch sensor system has at least one of a capacitance sensor, a contact sensor, a camera, a three-dimensional image sensor and a switch.
0016In yet another aspect of the present invention, a mobile human interface robot has a drive system having at least one drive wheel that is driven in a corresponding relationship by a drive motor, and a location determination system that is in communication with the drive system. It has a power supply that is in communication with the drive system and location location system, and a touch response input supported above the drive system. The activation of the touch response input changes the power distribution state to the drive system, thereby reducing at least the drive load of the drive motor on at least one corresponding drive wheel.
0017In some embodiment, the actuation of the touch response input can terminate the power distribution to the drive system, while continuing the power distribution to the location location system. In response to the activation of the touch response input, the drive system can issue a power reduction drive command that can only move the robot and assist the user movement of the robot in response to the activation of the touch response input. Further, the actuation of the touch response input can result in the disengagement of at least one drive wheel from the corresponding drive motor. The touch response input is located approximately 3 feet (91.5 cm) to approximately 5 feet (152.5 cm) above the ground.
0018In some embodiments, the location location system has a sensor system and a drive system, a sensor system, a power supply and a controller in communication with a touch response input. The sensor system may include at least one of an inertial measurement unit, an odometer, a global positioning system, a laser scanner, a sonar proximity sensor and a three-dimensional image sensor. In additional embodiments, the location determination system is of a three-dimensional image sensor, a laser scanner, one or more sonar proximity sensors, a drive wheel encoder for at least one drive wheel and a drive wheel motor feedback. Have at least one.
0019The drive system has, in some embodiments, first, second and third holononomically driven drive wheels, each of which is spaced in a triangular shape around the robot's vertical axis. And has a drive direction perpendicular to the radial axis with respect to the vertical axis.
0020Mobile human interface robots are supported by a base that supports the drive system, actuable telescopic legs that extend upward from the base, and legs that are approximately 3 feet (91.5 cm) above the ground. It further has a torso located 5 feet (152.5 cm) above and a touch response input is provided on the torso. The touch response input may include at least one of a contact sensor, a capacitance sensor, an actuable button and a switch.
0021Another aspect of the present invention provides a mobile human interface robot, which defines a symmetrical shape in the shape of a substantially triangle around the vertical central axis and forms the first, second and third parts. Has a base to have. The robot has a nonholonomic drive system supported by the base. The drive system has first, second and third driven wheels, each of which is spaced apart from each other in a triangular shape around the vertical center axis and corresponds to the first, second. And supported by a third base portion. Each drive wheel has a drive direction perpendicular to the radial axis with respect to the vertical center axis. The robot further comprises legs extending upward from the base and having variable heights, a torso supported by the legs, and torso imaging sensors provided on the torso. The torso comprises shoulders with a bottom overhanging from the base. The torso imaging sensor is provided on the bottom surface of the torso and points downward along the front drive direction of the drive system. The torso imaging sensor captures a three-dimensional image of the scene around the robot. The robot further has a neck supported by the torso, a head supported by the neck, and a display supported by the head. The neck pans or tilts the head with respect to the vertical center axis.
0022In some embodiment, the display is detachably attached to the head. The display has a locator (eg, a homing device or circuit that outputs a signal for location detection) that locates the display when it is removed from the head. In some embodiments, the display has a tablet computer with a touch screen. Further, it is preferable to provide a tablet computer that is detachably attached to the head and / or other parts of the robot, such as the base, legs and / or torso. In some embodiment, the tablet computer is detachably attached to the display. The robot should have a camera attached to the head.
0023The robot should have a head imaging sensor that is attached to the head and captures a three-dimensional image of the scene around the robot. The head imaging sensor preferably has a volume point cloud imaging device positioned so that the point cloud can be obtained from the spatial volume around the robot.
0024The arm should be attached to the neck and the head should be supported away from the neck. Further, the robot preferably has a manipulator arm attached to the body and an end effector attached to the distal end of the manipulator arm. The manipulator arm should have a variable length.
0025Details of one or more specific examples of the present invention are described in the accompanying drawings and in the description below. Other aspects, other features and other advantages will become apparent from the description, drawings and claims.
<figref num="1">It is a perspective view of an exemplary mobile human interface robot.</figref><figref num="2">It is a schematic diagram of an exemplary mobile human interface robot.</figref><figref num="3">It is a perspective view seen from above of the example mobile human interface robot.</figref><figref num="4A">It is a perspective view seen from the front of the base of an example of a mobile human interface robot.</figref><figref num="4B">It is a perspective view seen from the back of the base shown in FIG. 4A.</figref><figref num="4C">It is a top view of the base shown in FIG. 4A.</figref><figref num="5A">FIG. 6 is a schematic front view of an exemplary base of an exemplary human interface robot.</figref><figref num="5B">FIG. 3 is a schematic plan view of an exemplary base of a mobile human interface robot.</figref><figref num="5C">It is a front view of the holonomic wheel which is an example of a mobile human interface robot.</figref><figref num="5D">It is a side view of the wheel shown in FIG. 5C.</figref><figref num="6A">FIG. 6 is a front-view perspective view of an exemplary torso for a mobile human interface robot.</figref><figref num="6B">FIG. 6 is a front-view perspective view of an exemplary fuselage with a touch detection function for a mobile human interface robot.</figref><figref num="6C">It is a perspective view seen from the bottom of the body part shown in FIG. 6B.</figref><figref num="7">It is a perspective view from the front of the neck of an example of a mobile human interface robot.</figref><figref num="8A">It is a schematic of the circuit configuration of an example of a mobile human interface robot.</figref><figref num="8B">It is a schematic of the circuit configuration of an example of a mobile human interface robot.</figref><figref num="8C">It is a schematic of the circuit configuration of an example of a mobile human interface robot.</figref><figref num="8D">It is a schematic of the circuit configuration of an example of a mobile human interface robot.</figref><figref num="8E">It is a schematic of the circuit configuration of an example of a mobile human interface robot.</figref><figref num="8F">It is a schematic of the circuit configuration of an example of a mobile human interface robot.</figref><figref num="8G">It is a schematic of the circuit configuration of an example of a mobile human interface robot.</figref><figref num="9">FIG. 6 is a perspective view of an exemplary mobile human interface robot with a removable web pad.</figref><figref num="10A">It is a perspective view of a person interacting with an exemplary mobile human interface robot.</figref><figref num="10B">It is a perspective view of a person interacting with an exemplary mobile human interface robot.</figref><figref num="10C">It is a perspective view of a person interacting with an exemplary mobile human interface robot.</figref><figref num="10D">It is a perspective view of a person interacting with an exemplary mobile human interface robot.</figref><figref num="10E">It is a perspective view of a person interacting with an exemplary mobile human interface robot.</figref><figref num="11">It is an exemplary telephone technology schematic that initiates and executes communication with a mobile human interface robot.</figref><figref num="12A">It is a schematic diagram of an exemplary robot system architecture.</figref><figref num="12B">It is a schematic diagram of an exemplary robot system architecture.</figref><figref num="13">It is a schematic diagram of an exemplary mobile human interface robot.</figref><figref num="14">FIG. 6 is a perspective view of an exemplary mobile human interface robot with a large number of sensors directed towards the ground.</figref><figref num="15">It is a schematic of an exemplary control system executed by a controller of a mobile human interface robot.</figref><figref num="16">It is a perspective view of an exemplary mobile human interface robot that receives a human touch command.</figref><figref num="17A">FIG. 5 is a flow chart of an exemplary configuration of operations or steps that perform robot holding so that the user can manually move the mobile human interface robot.</figref><figref num="17B">FIG. 6 is a schematic representation of a retention button system that allows a user to manually move a mobile human interface robot.</figref><figref num="18">FIG. 3 is a perspective view of an exemplary mobile human interface robot to which a manipulator is attached.</figref><figref num="19A">FIG. 5 is a perspective view of an exemplary mobile human interface robot with a manipulator approaching a doorknob.</figref><figref num="19B">FIG. 5 is a perspective view of an exemplary mobile human interface robot in which the manipulator grabs the doorknob and opens the corresponding door.</figref><figref num="19C">FIG. 3 is a perspective view of an exemplary mobile human interface robot with a manipulator releasing a doorknob and moving through a corresponding open door doorway.</figref><figref num="19D">FIG. 5 is a perspective view of an exemplary mobile human interface robot in which the manipulator holds the door open for humans.</figref><figref num="20">It is a perspective view of the example mobile human interface robot in the state where the arm supports the head away from the body of the robot.</figref>
In various figures, the same reference numerals indicate the same elements.
Mobile robots (also called mobile robots) can interact with or interface with people to provide a number of services ranging from home assistance (from home assistance to commercial assistance). In the home assistance example, mobile robots can assist older people in their daily work, such as medication, mobility assistance, and communication assistance (eg, video conferencing, telecommunications, internet access). Etc.), housing or site monitoring (inside and / or outside), human monitoring and / or provision of personal emergency response system (PERS), but not limited to these. In the case of a commercial assistant, the mobile robot can provide video conferencing (for example, in a hospital environment), a sales floor dedicated terminal (POS terminal), an interactive information / marketing terminal, and the like.
Referring to FIGS. 1 and 2, in some specific examples, the mobile robot 100 has a robot body 110 (or chassis) that determines a forward drive direction F. The robot 100 further includes a drive system 200, an interface module 300, and a sensor system 400, each of which is supported by the robot body 110 and is in communication with a controller 500 that coordinates the movements of the robot 100. The power supply 100 (eg, one or more batteries) may be supported in an electrical communication relationship with each of these components as needed, and this power supply may power these components as needed. Send to each of. For example, the controller 500 may include a computer capable of executing more than 1,000 MIPS (1 million instructions per second), and the power supply 1058 powers the computer for more than 3 hours. Provide enough battery to do.
In the illustrated embodiment, the robot body 110 has a base 120, a body 140 supported by at least one leg 130 extending upward from the base 120, and at least one leg 130. The base 120 can at least partially support the drive system 200. The robot body 110 further has a neck 150 supported by a body 140. The neck 150 supports the head 160, which supports at least a portion of the interface module 300. The base 120 has a low center of gravity CG of the base 120 to maintain mechanical stability.<sub>B</sub>And the low overall center of gravity CG of the robot 100<sub>R</sub>Has sufficient weight (eg, by supporting the battery) to maintain.
With reference to FIGS. 3 and 4A-4C, in some specific examples, the base 120 defines a ternary symmetric shape (eg, a triangular shape when viewed in plan view). For example, the base 120 may preferably have a base chassis 122, which is a first, second and third base corresponding to each leg of the three-sided base 120 (see, eg, FIG. 4A). It supports a base body 124 having body portions 124a, 124b, 124c. The base body portions 124a, 124b, 124c are preferably movably supported by the base chassis 122 so that they move independently of the base chassis 122 in response to contact with an object. Due to the trigonal symmetrical shape of the base 120, it is possible to detect a protruding bump (raised portion) over 360 ° around the robot 100. Each base body portion 124a, 124b, 124c preferably has a associated contact sensor (eg, capacitance sensor, reed switch, etc.) that detects the motion of the corresponding base body portions 124a, 124b, 124c with respect to the base chassis 122.
In some embodiment, the drive system 200 allows omnidirectional and / or nonholonomic motion control of the robot 100. As used herein, the term "omnidirectional" means that it is possible to perform substantially any planar direction, i.e., left and right (lateral), back and forth, and rotational movement. These directions are generally referred to herein as x, y and θz, respectively. In addition, the term "holonomic" is used in a manner that is substantially consistent with the literature use of the term, with three planar degrees of freedom in the plane direction, namely two translations and one rotation. It means that you can do it. Therefore, it has the ability to move in a plane direction at a speed composed of virtually any ratio of three plane velocities (front-back, lateral and rotation) and these ratios in a virtually continuous manner. Has the ability to change.
The robot 100 can work in a human environment (eg, an environment typically designed for bipedal occupants) using a wheeled movement system. In some embodiment, the drive system 200 is first, first, equidistant (eg, 120 ° spaced) (ie, ternary symmetrically) about the vertical axis Z. It has second and third drive wheels 210a, 210b, 210c, but other configurations can also be adopted. With reference to FIGS. 5A and 5B, the drive wheels 210a, 210b, 210c are laterally arcuate rolling surfaces (ie, rolling direction D).<sub>R</sub>It is preferable to have a contour shape curved in the lateral direction or the direction perpendicular to the), and this rolling surface can assist the maneuverability of the holonomic drive system 200. The drive wheels 210a, 210b, 210c are coupled to the drive motors 220a, 220b, 220c, respectively, and the drive motors are independent of the other drive motors 220a, 220b, 220c. Can be driven forward (forward) and / or backward (backward). Each drive motor 220a-220c preferably has its own encoder 212 (FIG. 8C), which provides wheel rotation feedback to the controller 500. In some embodiments, each drive wheel 210a, 210b, 210c is attached to or near one of the three vertices of an equilateral triangle and is perpendicular to the bisector of the angle of the end of each triangle. It has a driving direction (front-back direction). By driving the ternary symmetric holonomic base 120 in the front drive direction F, the robot 100 can transition to a non-front drive direction for confinement or autonomous escape from the clutter, and then escape. Can be rotated and / or translated to drive along the front drive direction F.
With reference to FIGS. 5C and 5D, in some embodiment, each drive wheel 210 has an inner row 232 and an outer row 234 of the rollers 230, each row in the rolling direction of the drive wheels 210. D<sub>R</sub>Rolling direction D perpendicular to<sub>r</sub>Have. The rows 232 and 234 of the rollers 230 may be staggered (eg, as a result, one roller 230 belonging to the inner row 232 is between two adjacent rollers 230 belonging to the outer row 234. The roller 230 makes an infinite slip perpendicular to the drive direction of the drive wheels 210. The roller 230 slides in these rolling directions D.<sub>r</sub>It comprises an arc-shaped (eg, convex) outer surface 235 perpendicular to, so that the rollers 230 together define a circular or substantially circular perimeter of the drive wheels 210. The contour shape of the roller 230 affects the overall contour shape of the drive wheels 210. For example, the rollers 230 may include an arc roller outer surface 235 that together constitutes a scalloped rolling surface (eg, as a tread for traction) of the drive wheels 210. However, by configuring the rollers 230 to have a contour shape that constitutes a circular rolling surface as a whole of the drive wheels 210, the robot 100 does not vibrate perpendicular to the wheel tread, but on a flat surface. It can move smoothly. When approaching an object at an angle, using staggered rows 232,234 of rollers 230 (radius r) as treads, climbing an object as high or about as high as the wheel radius R of the drive wheels 210. Can be done.
In the embodiment shown in FIGS. 3 to 5B, the first drive wheel 210 is configured as a leading drive wheel along the front drive direction F, and the remaining two drive wheels 210b, 210c follow it. In this configuration example, in order to drive forward, the controller 500 has the same second and third drive wheels 210b, 210c while the first drive wheel 210a is slipping along the front drive direction F. It is good to issue a drive command to drive in the forward rolling direction at a speed. Further, with this drive wheel configuration, the robot 100 can be stopped in a short time (for example, it causes a rapid negative acceleration with respect to the front drive direction F). This is due to the natural dynamic instability of the three-wheeled design. When the front drive direction F is along the bisector of the angle between the two front drive wheels, the robot 100 rotates around the two "front" wheels when stopped in a short time. A torque is generated to overturn this. On the other hand, by naturally traveling one drive wheel 210a forward, the robot 100 is prevented from falling forward even when it is necessary to be supported or suddenly stopped. However, when accelerating from the stopped state, the controller 500 moves the overall center of gravity CG of the robot 100.<sub>R</sub>The moment of inertia I of the robot 100 can be taken into account.
In some embodiment of the drive system 200, the drive wheels 210a, 210b, 210c have a rolling direction D aligned radially with the vertical axis Z.<sub>R</sub>The vertical axis Z is orthogonal to the X-axis and the Y-axis of the robot 100. The first drive wheel 210a is preferably configured as a leading drive wheel along the front drive direction F, and the remaining two drive wheels 210b and 210c follow it. In this configuration example, in order to advance forward, the control device 500 causes the first drive wheels 210a to advance in the forward rolling direction, and the second and third drive wheels 210b and 210c are the first drive wheels 210a. At the same speed as, however, it is better to issue a drive command to move in the opposite direction.
In another embodiment, the drive system 200 has first and second positions such that the angular bisector of the angle between the two drive wheels 210a, 210b is aligned with the front drive direction F of the robot 100. It is preferable that the drive wheels 210a and 210b of the above are arranged. In this configuration example, in order to drive forward, the controller 500 preferably drives the first and second drive wheels 210a and 210b in the forward rolling direction and at the same speed, and issues a drive command to drive the first and second drive wheels 210a and 210b. , The third drive wheel 210c is driven in the opposite direction or remains idle and is dragged behind the first and second drive wheels 210a, 210b. Since it turns left or right while driving forward, the controller 500 may issue a command to drive the corresponding first or second drive wheels 210a, 210b at a relatively fast / slow speed. Drive systems 200 of other configurations can also be used. The drive wheels 210a, 210b, 210c may have a cylindrical, circular, elliptical or polygonal contour shape.
With reference to FIGS. 1 to 3 again, the base 120 supports at least one leg 130 extending upward from the base 120 in the Z direction. The legs 130 are preferably configured to have a variable height for raising and lowering the body 140 with respect to the base 120. In some embodiment, each leg 130 has first and second leg portions 132, 134 that move relative to each other (eg, nested or telescopic, linear and / or angular). Rather than continuously nesting small diameter extrusions into and out of each other or nesting from a relatively large diameter base extrusion, the second leg portion 134 is the first in the illustrated embodiment. Nesting along this on the leg portion 132 of the base 120, thus allowing other components to be placed along the second leg portion 134 and potentially along with the second leg portion 134. Can move relatively close to. The leg 130 preferably has an actuator assembly 136 (FIG. 8C) that moves the second leg 134 with respect to the first leg 132. The actuator assembly 136 preferably includes a lifting motor 138b and a motor driver 138a in contact with an encoder 138c that provides position feedback to the controller 500.
In general, the nested configuration is the center of gravity CG of the entire leg 130.<sub>L</sub>Has a continuously small diameter extrusion that nests in and out of a relatively large diameter extrusion at the base 120 to keep it as low as possible. Further, it is advisable to provide a strong and / or large diameter component at the bottom to handle the large torque generated at the base 120 when the legs 130 are fully extended. However, this approach raises two issues. First, when a relatively small diameter component is placed at the top of the leg 130, rain, dust or other granules tend to flow or fall along the extrusion, thereby between the extrusions. It enters the space of the above and thus interferes with the fitting of the extrusion part. This creates a very difficult sealing problem when trying to still maintain full mobility / joint motion of the leg 130. Second, it may be desirable to attach a payload or ancillary devices to the robot 100. One common place to attach to ancillary devices is the top of the torso 140. If the second leg portion 134 is nested in and out of the first leg portion, the accessories and components are above the entire second leg portion 134 if it needs to move with the torso 140. Can only be attached to. If not, the component attached to the second leg portion 134 will limit the nested movement of the leg portion 130.
By nesting the second leg portion 134 along this on the first leg portion 130, the second leg portion 134 can move perpendicular to the base 120 with an additional payload attachment location. I will provide a. With this type of configuration, water or air-carried granules do not enter the space between the legs 132, 134 and along the torso 140 outside all leg 132s, 134 (eg, extrusions). Flow. This greatly simplifies the sealing of the joints or joints of the legs 130. Further, the payload / accessory mounting features and / or the second leg portion 134 of the body portion 140 are always exposed and accessible no matter how the leg portion 130 is extended.
With reference to FIGS. 3 and 6A, the leg 130 preferably supports the torso 140, which preferably has a shoulder 142 that covers the base 120 and extends above it. In the illustrated embodiment, the torso 140 has a downward facing bottom surface 44 (eg, closer to the base) forming at least a portion of the shoulder 142 and an opposite upward facing top surface 146 between them. The side surface 148 extends. The torso 140 has various shapes or geometric shapes, such as a circle or an ellipse, with a central portion 141 supported by the legs 130 and a peripheral free portion 143 extending laterally beyond the lateral extent of the legs 130. It should be in the shape of a shape, thus forming an overhang that constitutes the downward facing surface 144. In some embodiments, the torso 140 has a polygonal or other complex shape that constitutes the shoulder, and the shoulder is an overhang that extends beyond the leg 130 while covering the base 120. There is.
The robot 100 preferably has one or more attachment ports 170 (eg, mechanical and / or electrical interconnect points) that accept the payload. The accessory port 170 is arranged so that the received payload does not obstruct or interfere with the sensor of the sensor system 400 (eg, attached to the bottom surface 144 and / or the top surface 146 of the body 140). good. In some embodiment, as shown in FIG. 6A, the fuselage 140 is provided in the rear portion 149 of the fuselage 140, eg, one or two that accepts the payload in the basket 340. It has one or more accessory ports 170 so that sensors attached to the front portion 147 of the fuselage 140 or other portions of the robot body 110 are not disturbed.
The outer surface of the body 140 should be sensitive to contact or touch by the user to receive touch commands from the user. For example, when the user touches (touches) the top surface 146 of the body 140, the robot 100 has the height H of the body.<sub>T</sub>With respect to the floor (for example, the height H of the leg 130 supporting the torso 140)<sub>L</sub>React by lowering (by reducing). Similarly, when the user touches the bottom surface 144 of the torso 140, the robot 100 raises the torso 140 with respect to the floor (for example, the height H of the legs 130 supporting the torso 140).<sub>L</sub>Respond by raising (by increasing). Further, upon receiving the user's touch on the front, rear, right or left portion of the side surface 148 of the body 140, the robot 100 responds to the received touch command in the corresponding directions (eg, rear, front, left and right, respectively). Respond by moving to. The outer surface of the body 140 is in communication with the controller 500 and preferably has a capacitance sensor to detect user contact.
With reference to FIGS. 6B and 6C, in some embodiment, the fuselage 140 includes a top panel 145t, a bottom panel 145b, a front panel 145f, a rear panel 145b, a right panel 145r and a left panel 145l. It has a main body 145. Each panel 145t, 145b, 145f, 145r, 145l can move independently of the other panels. Further, each panel 145t, 145b, 145f, 145r, 145l is in communication with the controller 500 and is associated with detecting motion and / or contact with each panel, and / or contact sensor 147t, It is preferable to have 147b, 147f, 147r, 147l.
With reference to FIGS. 1 to 3 and 7 again, the body 140 supports the neck 150, which pans and engages the head 160 with the body 140. In the illustrated embodiment, the neck 150 has a rotator 152 and a chiller 154. The rotator 152 has an angular motion range θ of about 90 ° to about 360 °.<sub>R</sub>(For example, around the Z axis) can be provided. Other ranges can also be adopted. Further, in some embodiments, the rotator 152 maintains a continuous 360 ° of head 160 relative to the body 140 at an unrestricted rotation speed while maintaining electrical communication between the head 160 and the rest of the robot 100. Has an electrical connector or contact that allows the rotation of the robot. The tilter 154 preferably has the same or similar electrical connectors or contacts that allow rotation of the head 150 with respect to the body 140 while maintaining electrical communication between the head 160 and the rest of the robot 100. .. The rotator 152 preferably has a rotator motor 151 coupled to or engaged with a ring 153 (eg, a toothed ring rack). The tilter 154 has its head at an angle θ with respect to the body 140, separately from the rotator 152.<sub>T</sub>It can be moved (eg, around the Y axis). In some embodiments, the tilter 154 has a tilter motor 155, the tilter motor 155 having the head 160 at an angle θ of ± 90 ° with respect to the Z axis.<sub>T</sub>Move between. Other ranges, such as ± 45 °, can also be adopted. The robot 100 is preferably configured such that the legs 130, torso 140, neck 150 and head 160 remain within the perimeter of the base 120 in order to maintain stable mobility of the robot 100. In an exemplary schematic circuit diagram shown in FIG. 10F, the neck 150 has a pan-tilt assembly 151, which is a motor driver 156a, 156b corresponding to a rotator 152 and a tilter 154. And included with the encoders 158a, 158b.
The head 160 should be sensitive to contact or touch operations by the user to receive touch commands from the user. For example, when the user pulls the head 160 forward, the head 160 tilts forward, providing passive resistance, and then holds its position. Further, when the user pushes or pulls the head 160 vertically downward, the torso 140 can be lowered to lower the head 160 (due to the reduction in the length of the legs 130). The head 160 and / or neck 150 preferably has a strain gauge and / or contact sensor 165 (FIG. 7) that detects the user's contact or operation.
8A to 8G are schematic diagrams of an example of the circuit configuration of the robot 100. 8A-8C provide an exemplary schematic of the circuit configuration for the base 120, which is a proximity sensor such as a sonar proximity sensor 410 and a cliff proximity sensor 420, contact sensor 430. , Laser scanner 440, sonar scanner 460 and drive system 200 may be accommodated. The base 120 may further accommodate the controller 500, the power supply 105 and the leg actuator assembly 136. The body 140 preferably accommodates a microcontroller 145, a microphone 330, a speaker 340, a scanning 3-D image sensor 410a and a body touch sensor system 480, and the body touch sensor system 480 allows the controller 500 to be a user. Can receive and respond to a touch or touch (eg, move the body 140 with respect to the base 120, pan and / or tilt the neck 150, and / or command in response. By sending it to the drive system 200). The neck 150 preferably accommodates the pan / tilt assembly 151, which has a pan motor 152 having a corresponding motor driver 156a and an encoder 138a and a corresponding motor driver 156b and an encoder 138b. It is preferable to include a tilt motor 154. The head 160 preferably accommodates one or more web pads 130 and a camera 312.
With reference to FIGS. 1-3 and 9, in some embodiment, the head 160 supports one or more parts of the interface module 300. The head 160 preferably has a dock 302 that leasably accepts one or more computing tablets 310, also called a web pad or tablet PC, each of which has a touch screen 312. good. The web pad 310 can be directed forward, backward or upward. In some embodiment, the web pad 310 has a touch screen, optional I / O (eg, buttons and / or connectors, such as micro USB, etc.), a processor and memory in communication with the processor. An example web pad 310 is Apple Incorporated (Apple). Inc.) Apple iPad. In some embodiments, the web pad 310 functions as or assists the controller 500 and controls the robot 100. In some embodiments, the dock 302 is mounted firmly attached to the first computing tablet 310a (eg, a wired interface for data transmission at relatively high bandwidth, eg gigabit speed) and the dock. It has a second computing tablet 310b that is detachably connected. The second web pad 310b can be attached to the first web pad 310a as shown in FIG. 9 or the second web pad 310b is opposite to the head 160 with respect to the first web pad 310a. It can be attached to the side facing side or the other side. In an additional embodiment, the head 160 supports a single web pad 310, which may be fixedly attached or detachably attached to it. The touch screen 312 can detect, monitor, and / or reproduce where the user touches the touch screen to receive user input and provide a touch-interactive graphical user interface. In some embodiments, the web pad 310 has a touch screen cola (caller) that allows the user to find it when it is removed from the robot 100.
In some embodiment, the robot 100 has a number of web pad docks 302 attached to one or more parts of the robot body 110. In the embodiment shown in FIG. 27, the robot 100 optionally has a web pad dock 302 attached to the legs 130 and / or the torso 140. As a result, the user can, for example, dock the web pad 310 with the robot 100 at various heights to accommodate users of different heights and capture images using the camera of the web pad 310 at different vantage points. Alternatively, a large number of web pads 310 can be attached to the robot 100.
The interface module 300 preferably has a camera 320 attached to the head 160 (see, eg, FIG. 2), which is used to capture images from a high vantage point of the head 160. Can be (for example, for video conferencing). In the embodiment shown in FIG. 3, the camera 320 is attached to the neck 150. In some embodiments, the camera 320 is only activated when the web pads 310, 310a are removed or undocked from the head 160. When the web pads 310, 310a are attached or docked to the head 160 in the dock 302 (and optionally cover the camera 320), the robot 100 uses the camera of the web pad 310a to capture the image. be able to. In such cases, the camera 320 is preferably placed behind the docked web pad 310, which is in operation when the web pad 310 is removed from or undocked from the head 160. When the 310 is attached to or docked with the head 160, it becomes inactive. The web pad 310 preferably has a locator 315 that positions the web pad 310 when it is removed from the head 160. The locator 315 may be a light emitter (eg, visible light, infrared light, etc.), a signal emitter (eg, radio emitter) and / or a light or signal detector.
The robot 100 can be provided via the interface module 300 (eg, using a web pad 310, a camera 320, a microphone 330 and / or a speaker 340) for video conferencing (eg, at 24 fps). Video conferencing should be multi-party. Robot 100 can provide eye contact between both parties in a video conference by maneuvering the head 160 so that it faces the user. Further, the robot 100 preferably has a gaze angle of less than 5 ° (eg, an angle away from the axis perpendicular to the front face of the head 160). At least one 3-D image sensor 450 and / or camera 320 attached to the robot 100 can capture a full-scale image including body language. Controller 500 can synchronize audio and video (eg, with a difference of less than 50 milliseconds). In the embodiment shown in FIGS. 10A-10E, the robot 100 adjusts the height of the web pad 310 and / camera 320 attached to the head 160 (by raising and lowering the torso 140). A video conference can be provided to people standing or sitting by / or by panning and / or tilting the head 160. The camera 320 can move within at least one degree of freedom separately from the web pad 310. In some embodiments, the camera 320 has an objective lens located at a distance of more than 3 feet from the ground but less than 10 percent of the height of the web pad from the apex of the display area of the web pad 310. .. Further, the robot 100 can zoom the camera 320 to obtain an enlarged picture or image around the robot 100. The head 160 preferably has one or more speakers 340 so that audio is emitted from the head 160 near the web pad 310 displaying the video conference.
In some embodiments, the robot 100 can accept user input into the web pad 310 (eg, via a touch screen) as shown in FIG. 10E. In some embodiment, the web pad 310 is a display or monitor, and in other embodiment, the web pad 310 is a tablet computer. The web pad 310 preferably has an easy and intuitive control device that provides high interactivity, such as a touch screen. The web pad 310 can move with at least one degree of freedom in 150 square inches (967.7 cm).<sup>2</sup>) It is preferable to have a monitor display 312 (for example, a touch screen) having a display area equal to or larger than that.
Robot 100 can provide EMR integration in some embodiments by providing video conferencing between doctors and patients and / or other doctors or nurses. The robot 100 preferably includes a pass-through consultation command. For example, the robot 100 may have a stethoscope configured to convey the listening to a video conferencing user (eg, a doctor). In another embodiment, the robot is a connector 1710 that allows direct connection to Class II medical devices such as electronic stesoscopes, otoscopes and ultrasound to send medical data to remote users (doctors). Has.
In the embodiment shown in FIG. 10B, the user is for remote control of the robot 100, video conferencing (eg, using the camera and microphone of the web pad 310) and / or the use of software applications on the web pad 310. The web pad 310 can be removed from the web pad dock 302 attached to the head 160. The robot 100 is attached to the head 160 to obtain various vantage points for video conferencing, navigation, etc. while the web pad 310 is being removed from the web pad dock 302. It is preferable to have cameras 320a and 320b.
For interactive applications that are executable on the computer 500 and / or in communication with the controller 500, it may be necessary to provide the robot 100 with two or more displays. A number of webpads 310 associated with the robot 100 include "FaceTime", telestration, HD look at this-cam (eg, webpad 310 with a built-in camera). In the case of) Can provide a combination of, can act as a remote operator control unit (OCU) for remote control of the robot 100, and / or can provide a local user interface pad.
In some embodiment, the robot 100 is an intermediary security device 350, also called a bridge, that allows communication between the web pad 310 and the controller 500 (and / or other components of the robot 100) (FIG. 9). Has. For example, the bridge 350 can convert the communication of the web pad 310 from a web pad communication protocol to a robot communication protocol (eg, Ethernet with gigabit capacity). The bridge 350 can authenticate that the web pad 310 is not fraudulent and allows communication conversion between the web pad 310 and the controller 500. In some embodiments, the bridge 350 has an authorization chip that authorizes / verifies communication traffic between the web pad 310 and the robot 100. The bridge 350 can inform the controller 500 when it has checked and authorized the web pad 310 that the bridge is trying to communicate with the robot 100. Further, after authorization, the bridge 350 informs the web pad 310 of the communication authorization. The bridge 350 can be attached to the neck 150 or head (or as shown in FIGS. 2 and 3) or anywhere else on the robot 100.
Session Initiation Protocol (SIP) is an IETF-defined signaling protocol that is widely used to control multimedia communication sessions, such as voice and video calls, via Internet Protocol (IP). This protocol can be used to create, modify, or terminate a two-party (unicast) or multi-party (multicast) session containing one or several media streams. Modifications may include changing addresses or ports, inviting many participants, and adding or removing media streams. Examples of other viable applications include video conferencing, streaming multimedia distribution, instant messaging, presence information, file transmission, and the like. Voice over the Internet Protocol (voice over IP, or VoIP) is part of a system of methodologies, communication protocols and transmission technologies for the delivery of voice communications and multimedia sessions over Internet Protocol (IP) networks, such as the Internet. .. Other terms that are often found and often used synonymously with VoIP are IP telephony technology, Internet telephones, broadband voice (VoBB), broadband telephone technology and broadband telephones.
FIG. 11 provides an exemplary telephone technique that includes dialogue with the bridge 350 to initiate and perform communication via the robot 100. The SIP of telephone A makes a call to the SIP application server. SIP activates the VoIP dial function, which causes the HTTP post request to be sent to the VoIP web server. The HTTP post request should behave like a call back (callback function) phone function. The SIP application server sends a call to phone A to indicate that the call has started. The VoIP server initiates a call to the callback phone number included in the HTTP post request via the PSTN. The callback phone number ends with the SIP DID provider, which is configured to send a callback to the SIP application server. The SIP application server matches the incoming call with the original phone of phone A and replies both calls with an okay answer. Media sessions are phone A and SIP It is established with the DID provider. Phone A can hear the artificial call made by VoIP. Once the VoIP confirms that it has responded to the callback leg, the VoIP initiates a call to the destination, eg, via the bridge 350 to the robot 100). The robot 100 answers the call and the VoIP server bridges the media from the SIP DID provider to the media from the robot 100.
12A and 12B are schematics of an exemplary robot system architecture 1200, wherein the exemplary robot system architecture includes a robot 100 (or a portion thereof, eg, a controller 500 or a drive system 200), a computing device 310 (head 160). It is preferable to have a cloud 1220 (eg, cloud computing) and a portal 1230 that are removable or fixedly attached to it.
The robot 100 can provide various basic robot features, such as mobility (eg, drive system 200), reliable, reliable, and secure. It may include a robot intelligence system, such as a control system running on a manipulator in communication with a controller 500, a power source 105, a detection system 400 and an optional controller 500. The control system can provide directional and speed control, body attitude control, navigation and basic robot applications. The detection system 400 includes vision (eg, via camera 320), depth map imaging (eg, by 3-D imaging sensor 450), collision detection, obstacle detection and obstacle avoidance and / or inertial measurement (eg, via inertial measurement). (Via inertial measurement unit 470) can be provided.
The computing device 310 is a tablet computer, a portable electronic device, for example, a mobile phone or a portable information terminal, or a dumb tablet display (for example, a tablet that acts as a monitor for an atom scale PC in the robot body 110). good. In some embodiments, the tablet computer may have a touch screen that displays the user interface and accepts user input. The computing device 310 can execute one or more robot applications 1210, such robot applications such as security, medication compliance, telepresence, behavioral guidance, social networking, active alerts, home management), etc. It is good to include a software application for (eg, stored in memory and runn on a processor). The computing device 310 can provide various communication functions (eg, wireless connectivity and / or cellular communication, improved application development tools, voice recognition and personal or object recognition functions. In the examples of, an interactive / COMS feature operating system, such as an object provided by Google Inc., an iPad OS provided by Apple Inc., and other smartphones. Use an operating system or government system, such as RSS A2.
The cloud 1220 provides cloud computing / or cloud storage functions. Cloud computing can provide internet-based computing, which allows shared servers to provide resources, software and data to computers and other devices on demand. For example, the cloud 1220 may be a cloud computing service that includes at least one server computing device, which is a hypertext transfer protocol wrapper by an abstraction layer and a server virtual machine embodied therein. It is good to include. The server computing device is preferably configured to parse the HTTP request and send an HTTP response. Cloud computing should be a technology that uses the Internet and a central remote server to maintain data and applications. Cloud computing allows users to access and use applications without installation, and Internet access allows users to access personal files on any computer. Cloud computing enables relatively efficient computing by centralizing storage, memory, processing and bandwidth. The cloud 1220 can provide scaleable on-demand computing output, storage and bandwidth.
The cloud storage device 1222 is preferably a model of a networked computer data storage device in which data is generally stored on a large number of virtual servers hosted by a third party. By enabling communication between the robot 100 and the cloud 1220, the information collected by the robot 100 can be reliably viewed by a qualified user via the web usage information portal.
The portal 1230 is preferably a web-based user portal that collects and / or provides information such as personal information, home status information, and anger robot status information. The information is integrated with the information of the third party, thereby providing additional functions and resources to the user and / or the robot 100. The robot system architecture 1200 can facilitate proactive data acquisition. For example, application 1210 running on computing device 310 may collect data and reports about actions performed by robot 100 and / or humans or the environment visible by robot 100 (using detection system 400). it can. It can be said that this data is a characteristic attribute of the robot 100.
In some embodiments, the portal 1230 is a personal portal website on the World Wide Web. Portal 1230 can provide personalized features and routes to other content. Portal 1230 can provide services from a wide variety of sources using distributed applications, varying numbers and formats of middleware and hardware. In addition, the Business Portal 1230 can share workplace coordination and provide content available on a number of platforms, such as personal computers, personal digital assistants (PDAs) and mobile phones / mobile phones. Information, news and updates are examples of content that can be delivered via portal 1230. The personal porter 1230 can be associated with a particular topic that provides friend information on a social network or provides a link to external content that can help others.
Referring again to FIG. 6A, the interface module 300 is attached to a microphone 330 (eg, or a microphone array) that receives audio input and a robot body 110 and has one or more speakers 340 that output audio. Is good. The microphone 330 and the speaker 340 can each communicate with the controller 500. In some embodiments, the interface module 300 has a basket 360, which is preferably configured to hold brochures, emergency information, household items and other items.
With reference to FIGS. 1-4C, 13 and 14, the sensor system 400 can make intelligent decisions about the actions the robot 100 takes in the robot's environment in order to achieve reliable and robust autonomous motion. It is good to have several different types of sensors that can be used in relation to each other to produce sufficient perception of the robotic environment. The sensor system 400 preferably has one or more types of sensors supported by the robot body 110, such as obstacle detection collision avoidance (ODOA) sensors, communication sensors, navigation sensors, and the like. Can be mentioned. For example, these sensors include proximity sensors, contact sensors, three-dimensional (3-D) imaging / depth map sensors, cameras (eg, visible and / or infrared cameras), sonars, radars, lidars (Light Detection And Ranging). (Light detection ranging, which may require an optical remote detection method that measures the characteristics of scattered light to detect the range and / or other information of a distant target), LADAR (Laser Detection). and Ranging), etc., but not limited to these. In some embodiment, the sensor system 400 is a ranged sonar sensor 410 (eg, nine around the base 120), a proximity cliff detector 420, a contact sensor 430, a laser scanner 440, one or two. It is preferable to have the above 3-D imaging / depth sensor 450 and imaging sonar 460.
There are several challenges in placing the sensor on a robot platform. First, the sensors need to be arranged so that they have the maximum coverage of the area of interest around the robot 100. Second, the sensors may need to be arranged so that the robot 100 itself produces the absolute minimum degree of occlusion to the sensor, and in essence, the sensors are "these are" by the robot itself. It must not be placed so that it is "blindfolded". Third, the placement and installation of the sensors should not be exposed to the rest of the platform's industrial design. In terms of aesthetics, a robot with a sensor mounted inconspicuously can be considered more "attractive" than a robot without it. In terms of practicality, the sensor should be mounted so that it does not interfere with normal robot operation (does not get caught in obstacles, etc.).
In some embodiment, the sensor system 400 is in communication with the controller 500 and within one or more zones or parts of the robot 100 to detect nearby or invading obstacles. The proximity sensors 410, 420 have a set or array of proximity sensors 410, 420 provided at or near the base body portions 124a, 142b, 142c of the robot body 110 (eg, the proximity sensors 410, 420 have an object. Focused infrared (IR) emitter-sensor element, sonar sensor, ultrasonic sensor and / or imaging sensor (eg, 3-D depth map image sensor) that provides a signal to the controller 500 when within a given range of robot 100. ) Is good.
In the embodiments shown in FIGS. 4A-4C, the robot 100 forms an array configured around the base body 120 (eg, at substantially equal spacing) and with an upward field of view. It has a sonar type proximity sensor 410. First, second and third sonar proximity sensors 410a, 410b, 410c are provided at or near the first (front) base body portion 124a and are at least one of the sonar proximity sensors. Is located near the outermost radial edge 125a of the first base body 124a. Fourth, fifth and sixth sonar proximity sensors 410d, 410e, 410f are provided on or near the second (right) base body portion 124b, at least one of the sonar proximity sensors. It is located near the outermost radial edge 125b of the second base body 124b. Seventh, eighth and ninth sonar proximity sensors 410g, 410h, 410i are provided on or near the third (right) base body portion 124c, at least one of the sonar proximity sensors. It is located near the outermost radial edge 125c of the third base body 124c. This configuration provides at least three detection zones.
In some embodiments, a set of sonar proximity sensors 410 (eg, 410a-410i) provided around the base body 120 is positioned upwards (eg, substantially in the Z direction) and is optional. As a result, the angle is formed so as to move outward from the Z axis, and thus the detection curtain 412 is formed around the robot 100. The sonar proximity sensors 410a-410i do not guide sonar emissions (radiated waves) upwards or at least towards other parts of the robot body 110 (eg, do not detect motion of the robot body 110 with respect to itself). It is better to have a shroud or emission guide 414. The emission guide 414 is preferably of a shell or semi-shell type. In the illustrated embodiment, the base body 120 extends laterally beyond the legs 130, and sonar proximity sensors 410 (eg, 410a-410i) are mounted on the base body 120 (eg, substantially) around the legs 130. (Along the perimeter of the body base 120). Further, the upwardly oriented sonar proximity sensors 410 are spaced apart from each other to form a continuous or substantially continuous sonar detection curtain 412 around the leg 130. The sonar detection curtain 412 can be used to detect an obstacle having a high-altitude lateral protrusion, such as a table top surface, a shelf, or the like.
The upward-viewing sonar proximity sensor 410 mainly provides a function of looking at an object located in a horizontal plane, for example, the upper surface of a table. Due to these aspect ratios, these objects may not be visible to other sensors in other sensor systems, such as the laser scanner 440 or the imaging sensor 450, and may therefore pose problems for the robot 100. .. An upward-viewing sonar proximity sensor 410, provided along the perimeter of the base 120, provides a means of visually recognizing or detecting objects / obstacles of these types. Further, the sonar proximity sensor 410 is arranged around the widest part around the base in a state of being slightly inclined outward so as not to be shielded or obstructed by the body 120 or the head 160 of the robot 100. Well, thus, there is no false belief in detecting parts of the robot 100 itself as a result. In some embodiment, the sonar proximity sensor 410 is provided around the body 140 outside the field of view of the sonar proximity sensor 410, thus freeing the attached payload or ancillary device, eg basket 340. Arranged (upper and outer) to leave an acceptable volume behind. The sonar proximity sensor 410 is preferably retracted into the base body 124 so as not to be visible and to provide an external feature that is caught or hits an obstacle.
The sensor system 400 serves as a backup for one or more sonar proximity sensors 410 (eg, in the direction opposite to the forward drive direction F) to detect obstacles. For example, it is preferable to have a rear proximity sensor 410j). The rear sonar proximity sensor 410j preferably has an emission guide 414 that directs its sonar detection field 412. Further, the rear sonar proximity sensor 410j can be used for distance measurement for determining the distance between the robot 100 and a detected object (for example, as a "backup alert") in the field of view of the rear sonar proximity sensor 410j. In some embodiments, the rear sonar proximity sensor 410j is provided retracted into the base body 120 so as not to provide visual or functional irregularities in the form of a housing.
With reference to FIGS. 3 and 4B, in some embodiment, the robot 100 enables drive wheels 210a, 210b, 210b to detect cliffs before the drive wheels 210a, 210b, 210c hit the cliffs (eg, stairs). , 210c has a cliff proximity sensor 420 located near or around them. For example, the cliff proximity sensor 420 may be arranged at or near each of the outermost radial edges 125a-125c of the base bodies 124a-124c and at locations between them. In some cases, cliff detection was tilted towards each other using infrared (IR) proximity or actual range detection and forming overlapping emission and detection fields and thus detection zones where the floor should be predicted. It is carried out using an infrared emitter 422 and an infrared detector 424. IR proximity detection should have a relatively narrow field of view, may be determined by the surface albedo for reliability, and should have varying range accuracy for each surface. As a result, it is preferable to arrange a large number of separate sensors all around the robot 100 to appropriately detect cliffs from a large number of points on the robot 100. Further, the IR proximity sensor is typically unable to distinguish between a cliff and a safety event immediately after, for example, the robot 100 has climbed the threshold.
The cliff proximity sensor 420 can detect when the robot 100 encounters a falling edge of the floor, for example, when the robot encounters a series of stairs. The controller 500 (which executes the control system) can execute a behavior that causes the robot 100 to take an action such as a change in the moving direction when the edge is detected. In some embodiment, the sensor system 400 has one or more secondary cliff sensors (eg, other sensors configured to perform cliff detection and optionally other forms of detection). The cliff detection proximity sensor 420 is configured to provide early detection of cliffs and to provide data to distinguish between actual cliffs and safety events (eg, climbing a threshold), and these fields of view are robots. It should be positioned downwards and outwards to include at least a portion of the body 110 and a region located away from the robot body 110. In some embodiment, the controller 500 increases the distance through the edge of the supporting work surface (eg, the floor), the edge of the work surface and / or the distance between the robot body 110 and the work surface. Perform a cliff detection routine to identify and detect. In this embodiment, 1) early detection of potential cliffs (which may allow the realization of rapid movement speeds in unknown environments), and 2) the controller 500 tells the cliff event to be true. Increased reliability of autonomous mobility by receiving cliff imaging information from the cliff detection proximity sensor 420 to know if it is unsafe or if it can safely cross the cliff (eg, up the threshold). 3) It is possible to reduce cliff false confidence (eg, due to the use of edge detection for multiple separate IR proximity sensors with a narrow field of view). An additional sensor, placed as a "wheel drop" sensor, is used for the sake of security and to detect situations where the range detection camera is unable to reliably detect certain types of cliffs. it can.
With the detection of thresholds and stairs, the robot 100 can effectively plan either to cross the threshold that can be climbed or to avoid stairs that are too high. This also applies to randomly placed objects on the work surface where the robot 100 may or may not be able to safely cross. In the case of obstacles or thresholds that Robot 100 sees, it is necessary for Robot 100 to be able to make a smooth transition to maximize smoothness and minimize instability due to sudden acceleration. You can go up knowing these heights, which can moderately slow down if done. In some embodiment, sill and staircase detection is based on object height on the work surface along with geometric recognition (eg, sill or stain with electrical cable, eg sock distinction). The threshold can be recognized by edge detection. The controller 500 receives imaging data from the cliff detection proximity sensor 420 (or another imaging sensor attached to the robot 100), executes an edge detection routine, and issues a drive command based on the result of the edge detection routine. be able to. The controller 500 can also identify an object by using pattern recognition. The threshold detection allows the robot 100 to orient its orientation with respect to the threshold to maximize its ability to climb smooth stairs.
Proximity sensors 410, 420 can function alone or, as a modification, in combination with one or more contact sensors 430 (eg, bump switches) for the sake of completeness. For example, one or more contact or bump sensors 430 attached to the robot body 110 can detect whether the robot 100 has physically encountered an obstacle. Such sensors can utilize physical properties within the robot 100, such as capacitance or physical displacement, to ascertain when the robot encounters an obstacle. In some embodiment, each base body portion 124a, 124b, 124c of the base 120 detects movement of the corresponding base body portion 124a, 124b, 124c with respect to the base chassis 122 (see, eg, FIG. 4A). It has a related contact sensor 430 (for example, a capacitive sensor, a reed switch, etc.). For example, the base bodies 124a, 124b, 124c can move radially with respect to the Z axis of the base chassis 122 to enable three-way bump detection.
With reference to FIGS. 1 to 4C, 13 and 14, again, in some embodiment, the sensor system 400 is a laser scanner 440 that is attached to the front portion of the robot body 110 and is in communication with the controller 500. Have. In the illustrated embodiment, the laser scanner 440 is oriented forward (eg, forward) to or above the first base 124a (eg, to obtain maximum imaging coverage along the robot drive direction F). It is attached to the base body 120 (which has a field of view along the drive direction F). Further, placing the laser scanner at or near the anterior tip of the triangular base 120 causes the outside angle of the robot base (eg, 300 °) to be greater than the field of view 442 (eg, about 285 °) of the laser scanner 440. Largely, thus, means that the base 120 is prevented from obstructing or obstructing the detection field of view 442 of the laser scanner 440. Do not block the field of view of the laser scanner 440 to minimize the portion of the laser scanner that protrudes beyond the base body 124 (eg, for aesthetics and to minimize the risk of getting caught in obstacles). It is preferable that the base body 124 is provided in a retracted state as much as possible.
The laser scanner 440 scans the area around the robot 100, and the controller 500 uses the signal received from the laser scanner 440 to create an environment map or object map of the scanned area. Controller 500 can use object maps for navigation, obstacle detection and obstacle avoidance. In addition, the controller 500 can use sensory inputs from other sensors in the sensor system 400 to create and / or navigate object maps.
In some embodiments, the laser scanner 440 is a scanning lidar, which uses a laser that quickly scans an area in one direction as the "main" scanning line and a depth for each pixel generated in this line. A time-of-flight imaging element that uses a phase difference or similar technique can be used to assign to (return to a two-dimensional depth line in the scan plane). To create a three-dimensional map, lidar can perform an "auxiliary" scan in a second direction (eg, by "nodding" the scanner). This mechanical scanning technique, if not captured, for example technologies such as "flash" LIDAR / LADAR and "Swiss Ranger" focal plane imaging element sensors, depth at each pixel. Or complete by technology using a semiconductor stack to allow time-of-flight calculations for a complete 2-D matrix of pixels (by an encoded illuminator or illuminating laser) to provide a series of depths at each pixel. Can be a thing.
The sensor system 400 preferably has one or more three-dimensional (3-D) image sensors 450 in communication with the controller 500. If the 3-D image sensor 450 has a limited field of view, the controller 500 or sensor system 400 will have a 3-D image. The page sensor 450a can be operated in a lateral scanning manner to create a relatively wide field of view, which allows for robust ODOA. Referring to FIGS. 1 to 3 and 14, in some embodiment, the robot 100 is attached to the front portion of the robot body 110 and has a field of view along the front drive direction F (for example, the robot). It has a scanning 3-D image sensor 450a (so that it has the maximum imaging coverage along the driving direction F of the). The scanning 3-D image sensor 450a is mainly for obstacle detection / obstacle avoidance (ODOA). Can be used for. In the illustrated embodiment, for example, as shown in FIG. 3, the shoulder 142 is attached to the torso 140 in a state of being retracted into the torso 140 (for example, flush with or beyond the bottom surface 144). Mounted below or on the bottom surface 144, which prevents contact between the user and the scanning 3-D image sensor 450a. The scanning 3-D image sensor 450a is used for obstacle detection and obstacle avoidance (ODOA) (for example, the robot body 11). It is preferable that the robot 100 is arranged so as to have a downward field of view 452 in front of the robot 100 (in the presence of interference by the base 120 of 0 or other parts), practically downward and facing away from the robot body 110. By arranging the scanning 3-D image sensor 450a on or near the front edge of the body 140, the field of view of the 3-D image sensor 450 (eg, about 285 °) is reduced to the 3-D image sensor 450. In contrast, it can be smaller than the outer surface angle of the body 140 (eg, 300 °), thus blocking or obstructing the detection field of view 452 of the scanning 3-D image sensor 450a. Is blocked. In addition, the scanning 3-D image sensor 450a (and associated actuators) does not block its field of view (eg, for aesthetic purposes and to minimize catching on obstacles) as much as possible on the torso 140. It is better to install it in a retracted state. The distracting scanning motion of the scanning 3-D image sensor 450a is invisible to the user and thus reduces the experience of distracting interactions. Unlike protruding sensors or features, the retracted scanning 3-D image sensor 450a interacts unintentionally with the environment, especially when moving or scanning (people, obstacles). Etc.) will not be likely to occur. This is because there are virtually no moving parts that extend beyond the envelope of the body 140.
In some embodiment, the sensor system 400 has an additional 3-D image sensor 450 provided on the base body 120, legs 130, neck 150 and / or head 160. In the embodiment shown in FIG. 1, the robot 100 has a 3-D image sensor 450 provided on a base body 120, a body 140, and a head 160. In the embodiment shown in FIG. 2, the robot 100 has a 3-D image sensor 450 provided on a base body 120, a body 140, and a head 160. In the embodiment shown in FIG. 13, the robot 100 has 3-D image sensors 450 provided on the legs 130, the torso 140 and the neck 150. Other forms can also be adopted. One 3-D image sensor 450 (eg, attached to the neck 150 above the head 160) can be used for people's recognition, gesture recognition and / or video conferencing, while the other. Another 3-D image sensor 450 (eg, attached to the base 120 and / or the leg 130) can be used for navigation and / or obstacle detection and obstacle avoidance.
A forward-facing 3-D image sensor 450 provided on the neck 150 and / or the head 160 can be used to recognize the faces and / or gestures of individuals around the robot 100. For example, using the signal input from the 3-D image sensor 450 provided on the head 160, the controller 500 creates a three-dimensional map of the user's face that is visually recognized and / or captured, and the created three-dimensional map. Can be recognized by comparing with a known 3-D image of a person's face and confirming a match with one of the known 3-D facial images. Face recognition is preferably used to identify the user as an acceptable user of the robot 100. Further, one or more of the 3-D image sensors 450 determines an individual gesture visually recognized by the robot 100, and optionally, the determined gesture (for example, pointing or waving). Can be used to react on the basis of cues and / or hand signals. For example, the controller 500 can issue a drive command in response to a recognized hand pointing in a particular direction.
The 3-D image sensor 450 may be able to generate data in the following formats: (i) depth map, (ii) intensity image using reflectance and / or (iii) normal intensity image. is there. The 3-D image sensor 450 can obtain such data by image pattern matching, flight time measurements and / or phase lag shifts of light emitted from the source and reflected from the target.
In some embodiment, a combination of algorithms in which inference or control software that can be executed on a processor (eg, the processor of robot controller 500) is executed using various types of data generated by the sensor system 400. Use. The inference software processes the data collected from the sensor system 400 and outputs data for making a navigational decision about where the robot 100 can move, for example, without colliding with an obstacle. By accumulating imaging data over time around the robot, inference software can apply effective methods to selected segments of detected images to improve depth measurements on the 3-D image sensor 450. .. This should include the use of suitable temporary and spatial averaging techniques.
The reliability of performing robotic collision-free movements is based on (i) confidence levels obtained by high-level inference over time and (ii) depth perception sensors that accumulate three main forms of data for analysis. Of course, the three main formats of data are (a) depth image, (b) active illumination image and (c) ambient illumination image. Algorithms for recognizing various types of data should be run for each of the images obtained by the depth perception imaging sensor 450. The collected data can improve the reliability level as compared with a system using only one of various data.
The 3-D image sensor 450 can obtain an image including depth and brightness data from a scene around the robot 100 containing one or more objects (eg, a sensor-visible portion of a room or work area). The controller 500 is preferably configured to determine occupancy data about the object based on the capture of reflected light from the scene. In addition, the controller 500 issues drive commands to the drive system 200, at least in part, based on occupied data to bypass obstacles (ie, objects in the scene) in some embodiments. The 3-D image sensor 450 can repeatedly capture the scene depth image for real-time determination by the controller 500 and advance through the scene without causing the robot 100 to collide with an object in the scene. For example, the speed or frequency at which depth image data is obtained by the 3-D image sensor 450 can be controlled by the shutter speed of the 3-D image sensor 450. In addition, the controller 500 receives an event trigger (eg, from another sensor component of the sensor system 400 (eg, from proximity sensors 410, 420) and informs the controller 500 that there is an object nearby or is at risk. The controller 500 can allow the 3-D image sensor 450 to capture the depth image and increase the frequency of obtaining occupancy information in response to the event trigger.
In some embodiment, the robot has a sonar scanner 460 to obtain acoustic imaging of the area around the robot 100. In the embodiments shown in FIGS. 1 and 3, the sonar scanner 460 is provided in the front portion of the base body 120.
With reference to FIGS. 1, 3B and 14, in some embodiment, the robot 100 is a laser scanner or laser rangefinding 440 for complete detection and a sonar proximity sensor facing backwards for safety. We are using 410j, both of which are directed parallel to the ground G. The robot 100 preferably has first and second 3-D image sensors 450a, 450b (depth cameras) to allow robust detection around the robot 100. The first 3-D image sensor 450a is attached to the body portion 140 in a state of being directed downward at a constant angle with respect to the ground G. By tilting the first 3-D image sensor 450a downward, the robot 100 receives a dense sensor coverage of the area immediately in front of or adjacent to the robot 100, which coverage of the robot 100 in the forward direction. Suitable for short-time travel. The rearward-facing sonar 410j performs object detection when the robot is moving backwards. When backward movement is common to the robot 100, the robot 100 is a third 3-D image sensor 450 oriented back and forth to provide dense sensor coverage of the area immediately behind or adjacent to the robot 100. It is good to have.
The second 3-D image sensor 450b is attached to a head 160 capable of panning and tilting the neck 150. The second 3-D image sensor 450b should be useful for remote drive. This is because the operator can see where the robot 100 is going. The neck 150 allows the operator to tilt and / or pan the second 3-D image sensor 450b to see both near and far objects. Panning the second 3-D image sensor 450b widens the associated horizontal field of view. During high speed travel, the robot 100 tilts the second 3-D image sensor 450b slightly downward to widen the field of view of both the 3-D image sensors 450a and 450b in the overall or combined state, and the robot 100 becomes an obstacle. Can be given enough time to avoid (because high speed generally means that the time to respond to an object is short). At slower speeds, the robot 100 can tilt the second 3-D image sensor 450b upwards or substantially parallel to the ground G to track the person the robot 100 is following. Further, while driving at a relatively low speed, the robot 100 can pan the second 3-D image sensor 450b to widen its field of view around the robot 100. The first 3-D image sensor 450a should remain fixed (eg, should not move relative to the base 120) when the robot is driving to increase the robot's perceptual range. ..
In some embodiment, at least one of the 3-D image sensors 450 is a robot at a height above 1 foot (30.5 cm) or 2 feet (61.0 cm) above the ground. A volume point cloud imaging device (eg, speckle or thyme) attached to 100 and capable of obtaining a point cloud from a spatial volume including a floor plane in the direction of movement of the robot (by the omnidirectional drive system 200). Of flight camera) is good. In the embodiments shown in FIGS. 1 and 3, the first 3-D image sensor 450a is at least 1 or 2 feet above the ground and at a height above the ground (or about 1 or 2 feet above the ground). Forward drive direction F to capture an image of the volume including the floor (eg, volumetric point cloud) while being mounted and driven (eg, for object detection and object avoidance) at (height). It is good to be sent along. The second 3-D image sensor 450b is located on the head 160 (eg, about 3 feet above the ground (eg, about 3 feet (91. 91.)) so that the skeleton recognition and definition point cloud can be obtained from the spatial volume located adjacent to the robot 100. It is shown mounted (5 cm) or 4 feet (122.0 cm) or more (at an upper height position). The controller 500 can execute skeleton / digital recognition software to analyze the captured volume point group data.
Referring again to FIGS. 2 and 4A-4C, the sensor system 400 is the overall center of gravity CG of the robot 100.<sub>R</sub>It is preferable to have an inertial measurement unit (IMU) 470 in communication with the controller 500 to measure and monitor the moment of inertia of the robot 100 with respect to.
The controller 500 can monitor the deviation of the feedback from the IMU 470 with respect to the threshold signal corresponding to the normal unrestricted operation. For example, if the robot begins to lean from an upright position, the robot may be "clothed" or disturbed in a different way, or someone may suddenly add a heavy payload. In these cases, emergency measures to ensure safe operation of Robot 100, including, but not limited to, elusive manipulation, recalibration and / or generation of audio / visual warnings. ) May be required.
Since the robot 100 can work in a human environment, the robot 100 can interact with humans and work in a space designed for humans (without considering the constraints of the robot). Robot 100 can limit its drive speed and acceleration when in a crowded, restrained or highly dynamic environment, for example at a cocktail party or a busy hospital. However, the robot 100 may encounter situations where it is safe to drive relatively quickly, for example in a long, empty corridor, for example when something crosses the robot's motion path. It is possible to slow down suddenly.
When accelerating from a stop, the controller 500 moves the robot's overall center of gravity CG.<sub>R</sub>It is possible to prevent the robot from tipping over in consideration of the moment of inertia of the robot 100 from. Controller 500 can use a model of its attitude including its current moment of inertia. When supporting the payload, the controller 500 has an overall center of gravity CG.<sub>R</sub>It is possible to monitor the motion of the robot moment of inertia by measuring the effect of the load on the robot. For example, the torso 140 and / or the neck 150 may have a strain gauge to measure strain. If this is not possible, the controller 500 may apply a test registration command to the drive wheels 210 and use the IMU470 to measure the actual linear and angular acceleration of the robot to empirically determine safety limits. Can be done.
During a sudden deceleration, the commanded load on the second and third drive wheels 210b, 210c (rear wheels) is reduced, during which the first drive wheels 210a (front wheels) slip in the forward drive direction. Supports robot 100. When the loads of the second and third drive wheels 210b, 210c (rear wheels) are asymmetric, the robot 100 "yaws", which reduces dynamic stability. When the IMU470 (for example, a gyro) is used, this deviation can be detected and commands can be issued to the second and third drive wheels 210b and 210c to reorient the robot 100.
With reference to FIGS. 3-4C and 6A, in some embodiments, the robot 100 has a large number of antennas. In the illustrated embodiment, the robot 100 has a first antenna 490a and a second antenna 490b, both of which are attached to the base 120 (provided that these antennas are optional to the robot 100 and others. (It may be attached to a portion of, for example, a leg 130, a torso 140, a neck 150 and / or a head 160). By using a large number of antennas, sound signal reception and signal transmission become possible. The use of multiple antennas provides the robot 100 with multiple inputs and multiple outputs, or MIMO, which is the use of multiple antennas for transmitters and / receivers to improve communication performance. .. MIMO results in a significant increase in data throughput and link range without additional bandwidth or transmit power. MIMO achieves this with high spectral performance (high number of bits per second per hertz of bandwidth) and link reliability or versatility (reduced fading). In view of these characteristics, MIMO is a state-of-the-art wireless communication standard such as IEEE. It is an important part of 802.11n (WiFi), 4G, 3GPP Long Term Evolution, WiMAX and HSPA +. In addition, the robot 100 can act as a WiFi bridge, hub or hotspot for other nearby electronics. The mobility and use of MIMO in Robot 100 allows the robot to be a relatively reliable WiFi bridge.
MIMO can be subdivided into three main categories: pre-coding, spatial multiplexing or SM and diversified coding diversity coding. Recording is a type of multi-stream beam shaping and is considered to be all spatial processing that occurs at the transmitter. In beam forming (single layer beam forming), the same signal is emitted from each of the transmitter antennas with the appropriate phase (and possibly gain) weighted so that the signal output is maximized at the receiver input. To. The advantage of beam shaping is that it increases the gain of the received signal and reduces the multipath fading effect by making the signals emitted from different antennas stronger than each other. In the absence of scattering, beam formation results in a well-defined directional pattern. If the receiver has a large number of antennas, the transmitting beam formation cannot maximize the signal levels at all of the receiving antennas at the same time, so it is better to use recording with a large number of streams. .. Recording may require knowledge of channel state information (CSI) at the transmitter.
Spatial multiplexing requires a MIMO antenna configuration. Spatial multiplexing divides a high-rate signal into a number of low-rate streams, each stream being transmitted from different transmitting antennas on the same frequency channel. If these signals reach the receiver antenna array with sufficiently different spatial signatures, the receiver can separate these streams into parallel (nearly parallel) channels. Spatial multiplexing is a very powerful technique that increases channel capacity at high signal-to-noise ratio (SNR). The maximum number of spatial streams is limited by the reduction in the number of antennas at the transmitter or receiver. Spatial multiplexing can be used with or without knowledge of transmission channels. Spatial multiplexing can also be used for simultaneous transmission to multiple receivers, called spatial access multiple access. Good separability can be ensured by scheduling receivers with different spatial signatures.
Diversity coding techniques are best used when there is no knowledge of the channel at the transmitter. The diversity method sends a single stream (unlike many streams of spatial multiplexing), but is coded using a technique called spatiotemporal coding. The signal is emitted from each of the transmitting antennas with a fully or nearly orthogonal coding. Diversity coding utilizes independent fading on a large number of antenna links to increase signal diversity. Since there is no knowledge of channels, no beam shaping is done or there is no array gain from diversity coding. Spatial multiplexing can also be combined with programming if the channel is known at the transmitter, or with diversity coding if there is a trade-off in decoding reliability.
Some embodiment has a third antenna 490c and / or a fourth antenna 490d provided on the body 140 and / or head 160, respectively (see, eg, FIG. 3). .. In such a case, the controller 500 can determine an antenna arrangement configuration that achieves a threshold signal level for robust communication (eg, raising and lowering the body 140 and / or rotating and / or tilting the head 160). By letting, for example, by moving the antennas 490a-490d). For example, the controller 500 can raise the third and fourth antennas 490c and 490d by issuing a command to raise the height of the body 140. Further, the controller 500 can issue a command to rotate and / or tilt the head 160 to further orient the fourth antenna 490d with respect to the other antennas 490a-490c.
Referring to FIG. 15, in some embodiment, the controller 500 executes a control system 510 including a control arbitration system 510a and a behavior system 510b that are in communication with each other. The control arbitration system 510a allows applications 520 to be dynamically added to or removed from control system 510, and each application 520 can control the robot 100 without having to know about any other application 520. It will be easier to be able to. In other words, the control arbitration system 510a provides a simple priority control mechanism between the application 520 of the robot 100 and the resource or resource 530. Resource 530 may include a drive system 200, a sensor system 400 and / or any payload or controllable device that is in communication with the controller 500.
The application 520 is better stored in the memory of the robot 100 or has a good communication relationship with the robot 100, thereby operating at the same time (eg, a processor) and at the same time controlling the robot 100. Application 520 can access behavior 600 of behavior system 510b. The independently deployed applications 520 can be dynamically combined at runtime to share the robot resources 530 of the robot 100 (eg, drive system 200, arms, head, etc.). A low-level policy is implemented in application 520 that dynamically shares robot resource 530 at run time. The policy determines which application 520 controls the robot resource 530 required by that application 520 (eg, the priority hierarchy of the applications 520). Application 520 can start and stop dynamically and run completely independently of each other. Also, the control system 510 allows complex behaviors 600 that can be combined with each other to support each other.
The control arbitration system 510a has one or more resource controllers 540, a robot manager 550 and one or more control arbiters 560. These components do not have to be in a common process or computer, nor do they need to be started in any particular order. The components of the resource controller 540 form the interface of the control arbitration system 510a for the application 520. There is an instance of this component for every application 520. Resource controller 540 extracts and summarizes the complexity of authentication, distributed resource control arbiters, command buffering, and so on. The robot manager 550 emphasizes application 520 prioritization by controlling which application 520 has exclusive control of any of the robot resources 530 at any particular time point. Since this is the upper central coordinator, there is only one instance of Robot Manager 550 for each robot part. The robot manager 550 executes a priority policy having a linear priority order of the resource controller 540, and constantly monitors the resource control arbiter 560 that enables hardware control. The control arbiter 560 receives commands from all applications 520, issues a single command based on the application priority, and publishes it for its associated resource 530. The control arbiter 560 also receives state feedback from its associated resource 530 and sends it back to application 520. The robot resource 530 may be a network of functional modules (eg, actuators, drive systems and groups thereof) with one or more hardware controllers. The command of the control arbiter 560 is specific to the resource 530 in performing a particular action.
A dynamics model 570 that can be run on controller 500 is configured to computerize the center of gravity (CG), moment of inertia, and cross product of inertia of various parts of robot 100 to evaluate the current state of the robot. Is good. The dynamics model 570 can also model the shape, weight and / or moment of inertia of these components. In some embodiments, the dynamics model 570 is an inertial moment unit 470 (IMU) or a portion thereof (IMU) that is provided on the robot 100 and is in communication with the controller 500 to calculate various centers of gravity of the robot 100. For example, communicate with an accelerometer and / or a gyro). The dynamics model 570 can be used by the controller 500 along with other programs 520 or behavior 600 to determine the working envelope of the robot 100 and its components.
Each application 520 has an action selection engine 580 and a resource controller 540, one or more behaviors 600 are connected to the action selection engine 580, and one or more action models 590 are the action selection engines. It is connected to 580. The behavior system 510b provides predictive modeling, which allows the behavior 600 to be coordinatedly determined with respect to the robot's behavior by evaluating the possible outcomes of the robot's behavior. In some embodiments, the behavior 600 is a hierarchical state-full that combines sensory feedback from multiple sources with a priori limits and information into evaluation feedback for the robot's acceptable behavior. It is a plug-in component that provides an evaluation function. Since the behavior 600 is pluggable to the application 520 (eg, present inside or outside the application 520), the behavior 600 can be removed or added without modifying the application 520 or any other part of the control system 510. Can be done. Each behavior 600 is a stand-alone policy. In order to make the behavior 600 powerful, it is possible to attach the outputs of a large number of behaviors 600 together to the inputs of different behaviors so that they can have complex combination functions. The behavior 600 is adapted to execute a manageable portion of all jurisdictions of the robot 100.
The action selection engine 580 is a cooperative element of the control system 510, and when all the inputs of the behavior 600 are given, the action selection engine 580 searches for the optimum action and executes a rapid optimized action selection cycle (prediction / correction cycle). The action selection engine 580 has three stages: nomination, action selection search and completion. In the nomination stage, each behavior 600 is notified and the action selection cycle is started, which is given a cycle start time, a current state and a robot actuator space limitation. Based on internal policies or external inputs, each behavior 600 determines whether it wants to participate in this action selection cycle. During this stage, a list of active behavioral prototypes is generated, and these inputs will influence the choice of commands to be executed on the robot 100.
In the action selection search stage, the action selection engine 580 produces an executable result from an available action space, also called an action space. The action selection engine 580 uses the action model 590 to simulate the actions of each command in different time steps depending on the planned target time in the future, resulting in a set of executable commands (within limits) and response results. provide. The action selection engine 580 calculates a favorable result based on the result evaluation of the behavior 600, sends a corresponding command to the control arbitration system 510a, and informs the action model 590 of the command selected as feedback.
At the completion stage, the directives corresponding to the coordinated best scored results are combined with each other as global directives, and such global directives are executably provided to the resource controller 540 on the robot resource 530. The best results are provided as feedback to the active behavior 600 and will be used in future evaluation cycles.
The sensor signal received from the sensor system 400 allows the interaction with one or more behaviors 600 to perform the action. For example, using the control system 510, the controller 500 may move from the corresponding action space (eg, a set of possible actions or movements for that particular component) to each robot component (eg, action for a motor or actuator). Or motion command) to ensure coordinated movement of each robot component in an efficient way to avoid collisions with each robot component itself and with objects around the robot 100 known to the robot 100. Let me do it. Controller 500 is capable of issuing coordinated commands by a robot network, such as an EtherIO network, as described in US Patent Application No. 61 / 305,069 filed February 16, 2010. This U.S. patent application is cited by reference, and the description thereof is a part of the present specification.
The control system 510 may provide the adaptive speed / acceleration of the drive system 200 to maximize the stability of the robot 100 in different forms / positions as the robot 100 is moving around a given area. it can.
In some embodiment, the controller 500 issues a command to the drive system 200, which drives the robot 100 according to the directional set value and the speed set value. One or more behaviors 600 evaluate the expected outcome of executable directives using the signals received from the sensor system 400, and one of these directives is made executable to handle obstacles. It should be chosen (alone or in combination with other commands as an overall robot command). For example, the signal from the proximity sensor 410 allows the control system 510 to change the commanded speed or orientation of the robot 100. For example, the control system 510 can issue a command in principle as a result of a signal from the proximity sensor 410 due to the presence of a nearby wall. In another case, the collision signal from the contact sensor resulting from the encounter with the chair allows the control system 510 to issue a command to change direction. In other cases, the speed setting value of the robot 100 cannot be reduced in response to the contact sensor, and / or the orientation setting value of the robot 100 cannot be changed in response to the proximity sensor 410.
The behavior system 510b includes a mapping behavior 600a that gives rise to the occupancy map 1700 and / or the robot map 1820, a speed behavior 600c configured to adjust the speed setting of the robot 100 (eg, a behavior routine that can be executed on the processor) and It is preferable to have a directional behavior 600d configured to change the directional setting value of the robot 100. The velocity and directional behaviors 600c, 600d can be configured to be executed simultaneously or independently. For example, the velocity behavior 600c may be configured to poll one of the sensors (eg, a set of proximity sensors 410, 420), and the orientation behavior 600d may be configured to poll another sensor (eg, a set of proximity sensors 410, 420). It should be configured to poll the bump sensor).
Referring to FIGS. 15 and 16, the behavior system 510b is configured to respond to a user 1600 touching the body 140 for remote control (eg, guidance of the robot 100). It is preferable to have 600a (eg, a behavior routine that can be executed on the processor). The torso touch remote control behavior 600a becomes effective when the sensor system 400 detects that the torso has been in contact (eg, human contact) for at least a threshold period (eg, 0.25 seconds). Is good. For example, movements and / or movements that are in communication with the controller 500 and are associated with the top panel 145t, bottom panel 145b, front panel 145f, rear panel 145b, right panel 145r and left panel 145l of the body body 145. Alternatively, the contact sensors 147t, 147b, 147f, 147r, 147l can detect motion and / or contact with their respective panels as shown in FIGS. 6B and 6C. Once enabled, the body touch remote control behavior 600a receives the direction of contact force (eg, detected from the elliptical location of the touch and calculated by the computer), and utilizes local X / Y coordinates (holonomic mobility). And issue a speed command to the drive system 200. Obstacle detection and obstacle avoidance behavior should be turned off while the torso touch remote control behavior 600a is enabled. When the detected touch location, force and direction change, the body touch remote control behavior 600a corresponds to the detected contact force direction by changing the speed command. The torso touch remote control behavior 600a may preferably execute a stop routine when the sensor system 400 no longer detects contact with the robot 100 over a threshold period (eg, 2 seconds). The stop routine allows the drive system 200 to stop driving after about 0.5 seconds if the sensor system 400 no longer detects contact with the robot 100 (eg, with the body 140). The body touch remote control behavior 600a is a robot.
The body touch remote control behavior 600a is a support drive command that allows the user to push the robot 100 while receiving drive support from the drive system 200 (eg, although the robot 100 itself cannot be moved). A partial speed command) that supports the movement of the robot 100 by the user can be issued to the drive system 200.
The body touch remote operation behavior 600a can receive sensor signals from the touch sensor system 480 (eg, button, capacitance sensor, contact sensor, etc.), and a part of the touch sensor system is on the body 140 and the robot 100. It is better to install it somewhere else, for example, the head 160). The torso touch remote control behavior 600a places the torso 140 at a height H of 3 to 5 feet above the ground G so that at least a portion of the touch sensor system 480 is located at a height accessible to a typical user.<sub>T</sub>It is better to position it at the place.
In some embodiment, the torso touch remote control behavior 600a recognizes the touch and specific posture of the user arranging the robot 100. For example, when the user 1600 pushes down on the body 140, the sensor system 400 detects a downward force applied to the body 140 and sends a corresponding signal to the controller 500. The torso touch remote control behavior 600a receives an index of downward force applied to the torso 140, whereby the control system 510 issues a command to the length H of the leg 130.<sub>L</sub>Reduced, thereby increasing the height H of the torso 140<sub>T</sub>To reduce. Similarly, when the user 1600 pushes up or pulls up on the torso 140, the torso touch remote control behavior 600a receives an index of upward force applied to the torso 140 from the sensor system 400, thereby causing the control system 510 to receive an index of upward force. Issue a command and the height H of the leg 130<sub>L</sub>Increased, thereby increasing the height H of the torso 140<sub>T</sub>To increase.
When the user 1600 pushes / pulls / or rotates the head 160, the torso touch remote control behavior 600a is generated from the sensor system 400 (eg, a strain gauge / motion / contact sensor provided on the neck 150). An indicator of user behavior can be received (from 165), and the control system 510 can respond by issuing a command to move the head 160 accordingly and then to hold its posture. ..
In some embodiment, the robot 100 provides passive resistance and / or active support to the user's operation of the robot 100. For example, motors 138b, 152, 154 that actuate the legs 130 and neck 150 provide passive resistance and / or active support to the user's operation of the robot 100 to provide operational feedback to the user. It provides and provides assistance in moving relatively heavy components, eg, lifting the torso 140. This allows the user to move various robot components without having to bear the entire weight of the corresponding component.
The behavior system 510b preferably has a tap attention behavior 600b (eg, a behavior routine that can be executed on a processor) configured to direct the attention of the robot 100 to the user. The tap caution behavior 600b causes the sensor system 400 to make contact (eg, contact with a person) on the body 140 (or some other part of the robot 100) for a period shorter than the threshold period (eg, 0.25 seconds). When it is detected that it has been received, it should be valid. Further, the tap caution behavior 600b should be effective only when the body touch remote control behavior 600a is in an inactive state. For example, even if the touch to the body 140 is detected for 0.2 seconds, the body touch remote control behavior 600a is not triggered, but the tap attention behavior 600b is triggered. The tap attention behavior 600b can use a contact location with respect to the torso 140, whereby the head 160 can tilt and / or pan (by actuating the neck 150) to look at the user. When the head 160 reaches the position where the head 160 is looking in the direction of the touch location, the stop reference of the behavior 600b can be reached. In some embodiment, the behavior system 510b is configured to stop the drive system 200 from driving (eg, put the robot 100 in a stopped state) with a tap stop behavior 600c (eg, a behavior that can be performed on a processor). Routine). The tap stop behavior 600c was detected by the sensor system 400 that the body 140 received contact (eg, contact with a person) and issued a zero speed drive command to the drive system 200 to cancel the previous drive command. It should be effective in some cases. When the robot advances and the user wants to stop the robot, the user may tap the torso 140 (or some other part of the robot 100) or the touch sensor. In some embodiments, the tap stop behavior 600c should only be activated when high priority behaviors such as the torso touch remote control behavior 600a and tap attention behavior 600b are not valid. The tap stop behavior 600c can be terminated when the sensor system 400 no longer detects a touch on the body 140 (or somewhere else on the robot 100).
Referring to FIGS. 1-6A, in some embodiment, the robot provides the ability to deactivate the drive system 200 or parts thereof (eg, shut off the motor driver output), and thus the robot 100. You will be able to push freely from place to place. The motor encoder 212 is preferably in an operating state, and therefore the location of the robot 100 remains located (eg, its location can be determined). When the robot 100 is in the stationary state, the drive wheels 210a to 210c can be locked due to the corresponding motors 220a to 220c being in the operating state, thus making it difficult for a person to move the robot 100. By shutting off the robot, the torque applied to the wheel motors 220a to 220c can be released, but the robot 100 to which the electric power is not supplied loses all location determination and communication. The robot 100 preferably has a hold button 175 (also called an emergency stop button). Although the holding button 175 is shown provided on the body 140, the holding button can also be attached to other parts of the robot body 110. The hold button 175 is provided on the body 140 in some embodiments. This is because the torso 140 is at a height accessible to the average user (eg, a height H of about 3-5 feet above the ground G).<sub>T</sub>Because it can be maintained.
FIG. 17A is an exemplary flow chart describing how to use the hold button 175. FIG. 17B is an exemplary schematic of the auxiliary button system 1700. In some embodiment, the hold button 175 stops power distribution to the drive wheel motors 220a-220c and thus directs power to the location location system 1710 (eg, controller 500 (control system 510) and sensor system 400). The drive wheels 210a to 210c are unlocked / disengaged without stopping the electric power of the drive wheels 210a to 210c. For example, when the holding button 175 is activated, the power to the entire robot 100 can be turned off except for the location determination system 1710. When the robot 100 is being moved while pressing the hold button 175, the drive wheel motors 220a-220c do not resist the wheel movement, and the location determination system 1710 keeps track of the robot's position. In some embodiments, the location determination system 1710 includes a 3-D image sensor 450, a laser scanner 440 for laser distance detection, a sonar proximity sensor 410 for sonar distance detection, and drive wheel motors 220a-220c and an encoder. Has feedback from 212.
By removing the high voltage power from the drive wheel motors 220a to 220c, the drive torque applied to the drive wheels 210a to 210c can be eliminated. In some cases it may be necessary to use mechanical hardware such as clutches or other wheel engagement methods. The circuit associated with the hold button 175 may need to prevent current from flowing through the motors 220a-220c due to external torque when the robot 100 is manually moved by the user. Using a corresponding motor disconnecting device, such as a relay, prevents current from flowing into the motors 220a-220c due to external torque when the robot 100 is manually moved by the user. can do. As a variant, a device that allows the motor DC bus to rise would block the flow of motor current. If this is done in a way that energy is only available to the user moving the robot 100, the drive wheel motors 220a-220c are still prevented from moving under robotic commands, which may be the purpose of the hold button 175. ..
In some embodiment, the operation of the hold button 175 does not stop the power to the drive system 200, rather the drive system 200 does not propel the robot 100 and supports the user movement of the robot 100. The support mode for urging the drive motors 220a to 220c is set only in this way.
The drive system 200 is preferably configured such that a lateral resistance of less than 50 N is required to move the robot 100 at the floor level. Further, the base 120 is preferably sized to have a wheel base of less than 2 feet in any direction, and even if the robot 100 is pushed at 100 N at a height of 4 feet (122 cm) above the road surface, the robot 100 can be pushed. I can't take you.
Referring to FIG. 18, in some embodiment, the robot 100 has a manipulator 180 (eg, articulated or non-articulated) with an end effector 182. If the manipulator 180 is attached to a fixed base (ie, a non-moving base), the manipulator 180 must have joints and any one of the joints ends with multiple degrees of freedom. It has at least one, if not, two or more degrees of freedom to allow the movement of the effector 182. However, multiple joints with multiple motors increase the cost and complexity of the manipulator 180.
This problem can be mitigated by attaching the manipulator 180 to a mobile platform. This is because some degrees of freedom are gained by the mobility of the platform itself. However, if the mobility paradigm is based on tracked or parallel wheeling, the manipulator will allow the end effector 182 to move in any direction while maintaining the orientation of the mobility platform. Will still require some degree of articulation. For example, if the manipulator 180 is facing straight forward on a tracked vehicle and it is desirable to move the end effector 182 directly to the side while maintaining the orientation of the tracked vehicle, then the manipulator 180 can be moved to the manipulator 180 without moving the tracked vehicle. Some kind of joint connection method is required. This is because a vehicle with a track cannot move directly to the side (that is, perpendicular to the driving direction of the track).
The nonholonomic drive system 200 (due to the actuation of the legs 130) associated with the variable height of the torso 140 allows the realization of infinite freedom of movement of the articulated articulated manipulator 180 attached to the torso 140. Therefore, the end effector can be moved along any vector in real space while maintaining the orientation of any given robot. In addition, attachment of the manipulator 180 to the head 160, which can be moved with the neck 150, provides additional movement and reach of the manipulator 180. Due to the nonholonomic mobility of the base 120, x, y and θ<sub>z</sub>Degrees of freedom are provided. The vertical actuation of the legs 130 causes the torso 140 to move in the Z direction with respect to the "z" degrees of freedom. Therefore, the robot 100 has x of the end effector 182 without the joint movement of the manipulator 180 itself. , Y and θ motions can be provided.
In addition to reducing the cost and complexity of the manipulator 180, this scheme greatly simplifies the computer processing required to control the end effector 182 in various directions. Complex logic and control algorithms are required to be able to move the end effector 182 in a particular direction by analyzing motion or controlling multiple joints with multiple degrees of freedom. However, by attaching the manipulator to the body 140 of the disclosed robot 100, each degree of freedom (x, y, z, θ) rather than utilizing joint control that affects two or more of the degrees of freedom.<sub>z</sub>) Allows independent control, making the mathematical method behind the analyzed motion algorithm relatively easy. In addition, this results in relatively low computer processor overhead requirements, reducing costs and improving reliability.
Traditionally, the method of opening and / or passing doors or doorways for robots is to keep the door open using a "chock" or to use a robot with multiple degrees of freedom or a wide range of motion manipulators. While manipulating through the doorway (eg, nonholonomic movements (y and θ)<sub>z</sub>Includes a step of keeping the door open continuously with (only) (this requires a separate corrugated motion). The nonholonomic drive system 200 allows the robot 100 to open the door (free hung or auto-close) and pass through the corresponding doorway.
With reference to FIGS. 19A-19C, in some embodiment, the behavior system 410a is a manipulator behavior 600d (eg, which allows the control system 510 to issue a command to open the door 1902 and successfully pass through the corresponding doorway 1900 (eg,). , Routines that can be executed on computing processors). The method of opening the door 1902 is to operate the robot 100 (eg, rotate and / or translate) to orient and position the end effector 182 of the manipulator 180 so that the end effector 182 is the doorknob 1904 of the door 1902. Has a step that allows you to operate. The end effector 182 is preferably configured to open and close to grab and rotate an object (eg, twist around the axis of the manipulator 180). The method comprises the step of grasping the doorknob 1904 with an end effector 182 and twisting the doorknob 1904 (or raising and lowering the body 140 to twist and actuate the lever 1904) to release the doorknob 1904. The method further comprises the step of manipulating the robot 100 to pull or push the door 1902 to open, and then operating the robot holononomically through the corresponding doorway 1900. Robot 100 can grab the doorknob 1904 on the opposite side of the door and then pull or push the door 1902 to close it.
To open and close the relatively heavy door 1902 with the relatively small lightweight robot 100, after releasing the doorknob 1904 (eg, by turning the doorknob or twisting the lever), operate the robot 100 as close as possible to the doorknob 1904. On the other hand, the extension length of the manipulator 180 is reduced to minimize the distance between the doorknob 1904 and the base 120. This method increases the normal force applied to the drive wheels 210a-210c by pushing up the doorknob 1904 (eg, by extending the legs 130, eg by lifting the torso 140), thereby increasing traction. It has more steps.
Once the door 1902 is opened to successfully pass through the self-closing door 1902 (from either direction), the robot 100 is already located near the door 1902, and the base 120 is rotated and / or moved to chock. Works like. In some embodiments, the manipulator 180 has a passive or active pan degree of freedom (DOF) to maintain contact between the end effector 182 and the doorknob 1904. Once through the doorway 1900, this method releases the end effector 182 and retracts the manipulator 180 (eg, x, y, θ of the base 120).<sub>z</sub>(By using DOF), it has a step of smoothly passing through the doorway 1900 while maintaining a continuous contact state between the door 1902 and the robot base 120. No sliding contact motion with respect to the door 1902 is required, thus avoiding scratching of the robot 100 or door 1902 and avoiding friction between them, thereby improving the workability of the robot. In some embodiment, the robot 100 maintains all related components on the base 120 within the vertical volume section defined by the base 120, so that there is only contact between the door 1902 and the base 120. Since the contact with the door 1902 is close to the ground, the traction and stability of the robot 100 can be maximized.
In the embodiment shown in FIG. 19D, the robot 100 has an extendable manipulator 180 attached to the head 160. While operating (eg, holononomically), the robot 100 grabbed and released the doorknob 1904, pushed the corresponding door 1902 to open it, then exited the doorway 1900 to distract the door and opened the door 1902. Allow people to pass through the doorway while left unattended.
Referring to FIG. 20, in some embodiment, the robot 100 has a base 120, a torso 140 supported by at least one leg 130 extending upward from the base 120 and at least one leg 130. It has a robot body 110 provided. The base 120 preferably supports at least some portion of the drive system 200. The robot body 110 further has a neck 150 supported by a body 140. The neck 150 supports an arm 190 that supports the head 160, which is preferably articulated. The head 160 preferably supports at least a portion of the interface module 300. The arm 190 allows the robot 100 to move the head 160 above and away from the neck 150. In the illustrated embodiment, the robot 100 moves the head 160 onto the conveyor belt and looks at the article on the conveyor belt using the camera 320 or the 3-D image sensor 450 on which the robot 100 is attached to the head 160. To be able to. During a video conferencing session with a remote user, the robot 100, and a local user located adjacent to the robot 100, the remote user and / or the local user points the robot 100 at an object of interest to the robot 100. Can be taken to detect and / or be visually recognizable. Further, the control system 510 preferably limits the movement of the head 160 away from the neck 150 so as to maintain the stability of the robot 100. In some embodiments, the control system 510 has an overall center of gravity CG.<sub>R</sub>The head 160 is maintained within the perimeter of the base 120 so as not to move beyond the perimeter of the base 120.
Various embodiment of the systems and techniques described herein can be realized with digital electronic circuits, integrated circuits, specially designed ASICs (application specific integrated circuits), computer hardware, firmware, software and / or combinations thereof. .. These various implementations can include implementations in the state of one or more computer programs that are executable and / or interpretable on a programmable system and are such programmable. The system is combined to receive data and instructions from the storage system, at least one input device and at least one output device, and send the data and instructions to the storage system, at least one input device and at least one output device. Alternatively, it includes at least one programmable processor, which may be of general purpose.
These computer programs (also referred to as programs, software, software applications or code) include machine instructions for programmable processors and can be implemented in high-level procedures and / or object-oriented programming languages and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" are used to provide machine instructions and / or data to a programmable processor that includes a machine-readable medium that receives machine instructions as machine-readable signals. It means any computer program product, device and / or device used (eg, magnetic disk, optical disk, memory, programmable logic device (PLD)). The term "machine readable signal" means any signal used to provide machine instructions and / or data to a programmable processor.
The content of the invention and the embodiment of the functional operations described herein are digital electronic circuits or computer software, firmware or structures disclosed herein and equivalent examples of these structures or one or two of them. It can be embodied in hardware that includes more than one combination. Embodiments of the content of the invention described herein are on one or more computer program products, i.e., a computer-readable medium that is executable by or controls the operation of the data processing apparatus. It can be embodied as one or more modules of computer program instructions encoded in. The computer-readable medium may be a machine-readable storage device, a machine-readable storage board, a memory device, a component of an object that sends a machine-readable propagation signal, or a combination of one or more of these. The term "data processor" includes all devices, devices and machines for processing data, such as programmable processors, computers or multiple processors or computers. The device, in addition to the hardware, is the code that gives rise to the execution environment for the computer program in question, such as the processor firmware, protocol stack, database management system, operating system, or a combination of one or more of these. It is good to include the code that makes up the. Propagation signals are artificially generated signals, such as machine-generated electrical signals, optical signals or electromagnetic signals, that are generated to encode information for transmission to a suitable receiver device.
Computer programs (also called programs, software, software applications, scripts or code) are writable in any form of programming language, including compiled or interpreted languages, which can be written as stand-alone programs or modules, components, subroutines or It is available in any form included as another unit suitable for use in a computer processing environment. Computer programs do not always support files in the file system. A program may be another program or data (eg, a file that stores one or more modules, subprograms, or parts of code) in a single file or multiple coordinated files dedicated to the program in question. , One or more scripts stored in a make-up language sentence) can be stored in a part of a file. Computer programs are available to run on a number of computers that are located on one computer or site or distributed in multiple locations and are connected to each other by a communication network.
The process and logic flow described herein is one or more programmable to execute one or more computer programs to perform a function by working on the input data and producing an output. It can be implemented by the processor. Processes and logic flows can also be implemented by dedicated purpose logic circuits, such as FPGAs (Field Programmable Gate Arrays) or ASICs (Application Specific Integrated Circuits), and devices can also be embodied as these.
Suitable processors for executing computer programs include, for example, both general purpose microprocessors and dedicated purpose processors and any one or more processors of any type of digital computer. Generally, the processor receives instructions and data from read-only storage or read-write storage. An essential element of a computer is a processor that executes instructions and one or more storage devices that store instructions and data. In general, a computer includes or receives data from one or more mass storage devices that store data, such as magnetic disks, magneto-optical disks or optical disks, and / or transmits data to them. Operated to do. However, the computer need not have such a device. In addition, computers can be embedded in other devices, such as mobile phones, personal digital assistants (PDAs), mobile audio players, and Global Positioning System (GPS) receivers, to name a few. Computer-readable media suitable for storing computer program instructions and data include non-volatile memory, media and storage devices of all forms, such as semiconductor storage devices such as EPROM. EPROM and flash memory devices, magnetic disks such as internal hard disks or removable disks, opto-magnetic disks, CDROMs and DVD-ROM disks. The processor and memory are preferably captured by dedicated purpose logic circuits or incorporated into them.
Examples of embodiment of the contents of the present invention described herein include, for example, a back-end component as a data server or a middleware component, such as an application server, or a front-end computer, eg, a user described herein by a user. Any combination of client computers with a graphical user interface or web browser that allows interaction with examples of the content of the invention or any combination of such backend, middleware or frontend components. It can be embodied in the including computing system. The components of such a system can be connected to each other by any form or medium of digital data communication, such as a communication network. Examples of communication networks include local area networks ("LAN") and wide area networks ("WAN"), such as the Internet.
The computing system may include clients and servers. Clients and servers are generally separated from each other and typically interact over a communication network. The client-server relationship arises from computer programs that run on their respective computers and have a client-server relationship with each other. The present specification has many details, which should not be mediated by the scope of the invention or the claims of the invention, but rather are specific to a particular embodiment of the invention. It should be mediated as an explanation of the features. Certain features described herein in relation to separate reification examples can also be embodied in combination in a single reification example. Conversely, the various features described in the context of a single reification example can also be embodied separately or in any suitable subcombination state in a number of reification examples. Further, the features described above as being used in a particular combination, and in some cases initially claimed in that state, may include one or more features from the claimed combination. , In some cases, the combination can be removed from the combination and the claimed combination can be changed to a sub-combination or a variant of the rust combination.
Similarly, operations or steps are described in the drawings in a particular order, which means that such operations or steps are performed in the specific order or sequential order shown or that all illustrated operations or steps are desired. It should not be understood as a prerequisite that it should be carried out to achieve the purpose of. In certain situations, multiple tasking and parallel processing may be advantageous. Moreover, the separation of the various system components in the embodiments described above should not be understood to require such separation in any embodiment, and the program components and systems described above are generally single software products. It should be understood that they can be integrated with each other or packaged into a large number of software products.
Many reification examples have been explained. Nevertheless, it will be appreciated that various modifications can be made without departing from the spirit and scope of the invention. Therefore, there are other embodiment within the scope of the invention described in the claims. For example, the acts described in the claims can be carried out in a different order, and even in this case, the desired result can be achieved.
43 sheets
Sheet 1 Sheet 2 Sheet 3 Sheet 4 Sheet 5 Sheet 6 Sheet 7 Sheet 8 Sheet 9 Sheet 10 Sheet 11 Sheet 12 Sheet 13 Sheet 14 Sheet 15 Sheet 16 Sheet 17 Sheet 18 Sheet 19 Sheet 20 Sheet 21 Sheet 22 Sheet 23 Sheet 24 Sheet 25 Sheet 26 Sheet 27 Sheet 28 Sheet 29 Sheet 30 Sheet 31 Sheet 32 Sheet 33 Sheet 34 Sheet 35 Sheet 36 Sheet 37 Sheet 38 Sheet 39 Sheet 40 Sheet 41 Sheet 42 Sheet 43
Every citation, both ways
| Document | Relation | Office | Cited during |
|---|---|---|---|
| KR20240068018A | Cited by | Republic of Korea | Search report |
| JP2001199356A | Cites | Japan | Search report |
| JP2005157689A | Cites | Japan | Search report |
| JP2005157689A | Cites | Japan | Search report |
| JP2007196300A | Cites | Japan | Search report |
| JP2008241535A | Cites | Japan | Search report |
| JP2008241535A | Cites | Japan | Search report |
| JP2009174898A | Cites | Japan | Search report |
| JP2009174898A | Cites | Japan | Search report |
| JP2009528514A | Cites | Japan | Search report |
| JP2009528514A | Cites | Japan | Search report |
| JP2010015194A | Cites | Japan | Search report |
| JP2010015194A | Cites | Japan | Search report |
| JPH11112965A | Cites | Japan | Search report |
| JPH11112965A | Cites | Japan | Search report |
| JPH11149315A | Cites | Japan | Search report |
| JPH11149315A | Cites | Japan | Search report |
107 members in 9 offices
Priority claims14
| Document | Office | Kind | Date |
|---|---|---|---|
| 61346612 | United States of America | – | |
| 34661210 | United States of America | P | |
| 61356910 | United States of America | – | |
| 35691010 | United States of America | P | |
| 61428717 | United States of America | – | |
| 61428734 | United States of America | – | |
| 61428759 | United States of America | – | |
| 201061428717 | United States of America | P | |
| 201061428734 | United States of America | P | |
| 201061428759 | United States of America | P | |
| 61429863 | United States of America | – | |
| 201161429863 | United States of America | P | |
| 13032370 | United States of America | – | |
| 201113032370 | United States of America | A |
Members107
| Document | Office | Kind | |
|---|---|---|---|
| CA2800372A1 | Canada | A1 | |
| US2011288684A1 | United States of America | A1 | |
| WO2011146254A2 | World Intellectual Property Organization (WIPO) | A2 | |
| WO2011146256A2 | World Intellectual Property Organization (WIPO) | A2 | |
| WO2011146259A2 | World Intellectual Property Organization (WIPO) | A2 | |
| CA2822980A1 | Canada | A1 | |
| CA2824606A1 | Canada | A1 | |
| CA2928262A1 | Canada | A1 | |
| US2012173018A1 | United States of America | A1 | |
| WO2012091801A2 | World Intellectual Property Organization (WIPO) | A2 | |
| WO2012091804A2 | World Intellectual Property Organization (WIPO) | A2 | |
| WO2012091807A2 | World Intellectual Property Organization (WIPO) | A2 | |
| WO2012091814A2 | World Intellectual Property Organization (WIPO) | A2 | |
| US2012182392A1 | United States of America | A1 | |
| US2012185094A1 | United States of America | A1 | |
| US2012185095A1 | United States of America | A1 | |
| US2012185096A1 | United States of America | A1 | |
| WO2011146254A3 | World Intellectual Property Organization (WIPO) | A3 | |
| AU2011256720A1 | Australia | A1 | |
| IL223155A0 | Israel | A0 | |
| GB2493887A | United Kingdom | A | |
| GB2494081A | United Kingdom | A | |
| WO2012091801A3 | World Intellectual Property Organization (WIPO) | A3 | |
| EP2571660A2 | European Patent Office (EPO) | A2 | |
| EP2571661A2 | European Patent Office (EPO) | A2 | |
| AU2011352997A1 | Australia | A1 | |
| WO2012091804A3 | World Intellectual Property Organization (WIPO) | A3 | |
| WO2012091814A3 | World Intellectual Property Organization (WIPO) | A3 | |
| WO2012091807A3 | World Intellectual Property Organization (WIPO) | A3 | |
| AU2011353004A1 | Australia | A1 | |
| WO2011146256A3 | World Intellectual Property Organization (WIPO) | A3 | |
| WO2011146259A3 | World Intellectual Property Organization (WIPO) | A3 | |
| AU2011353004B2 | Australia | B2 | |
| AU2011352997A8 | Australia | A8 | |
| GB201313403D0 | United Kingdom | D0 | |
| GB201313410D0 | United Kingdom | D0 | |
| JP2013537487A | Japan | A | |
| JP2013537651A | Japan | A | |
| EP2647475A2 | European Patent Office (EPO) | A2 | |
| DE112011104644T5 | Germany | T5 | |
| DE112011104645T5 | Germany | T5 | |
| GB2501209A | United Kingdom | A | |
| EP2659320A2 | European Patent Office (EPO) | A2 | |
| EP2659321A2 | European Patent Office (EPO) | A2 | |
| GB2502213A | United Kingdom | A | |
| GB201319513D0 | United Kingdom | D0 | |
| AU2013263851A1 | Australia | A1 | |
| JP2014505934A | Japan | A | |
| JP2014509417A | Japan | A | |
| EP2647475A3 | European Patent Office (EPO) | A3 | |
| GB2509814A | United Kingdom | A | |
| US2014200713A1 | United States of America | A1 | |
| EP2769809A1 | European Patent Office (EPO) | A1 | |
| JP2014176966AThis record | Japan | A | |
| JP2014195868A | Japan | A | |
| JP2014197403A | Japan | A | |
| JP2014197411A | Japan | A | |
| GB201416267D0 | United Kingdom | D0 | |
| JP2014209381A | Japan | A | |
| GB2501209B | United Kingdom | B | |
| GB2509814B | United Kingdom | B | |
| JP5629390B2 | Japan | B2 | |
| US8918209B2 | United States of America | B2 | |
| US8918213B2 | United States of America | B2 | |
| US8930019B2 | United States of America | B2 | |
| US8935005B2 | United States of America | B2 | |
| JP2015016549A | Japan | A | |
| US2015073598A1 | United States of America | A1 | |
| US2015073646A1 | United States of America | A1 | |
| US9014848B2 | United States of America | B2 | |
| GB2519433A | United Kingdom | A | |
| EP2659321B1 | European Patent Office (EPO) | B1 | |
| JP2015092348A | Japan | A | |
| AU2015202200A1 | Australia | A1 | |
| AU2013263851B2 | Australia | B2 | |
| US2015158182A1 | United States of America | A1 | |
| JP5735102B2 | Japan | B2 | |
| AU2011352997B2 | Australia | B2 | |
| GB2519433B | United Kingdom | B | |
| GB201510218D0 | United Kingdom | D0 | |
| AU2011256720B2 | Australia | B2 | |
| AU2015218522A1 | Australia | A1 | |
| JP5803043B2 | Japan | B2 | |
| GB2494081B | United Kingdom | B | |
| GB2527207A | United Kingdom | A | |
| GB2493887B | United Kingdom | B | |
| JP5852706B2 | Japan | B2 | |
| GB2527207B | United Kingdom | B | |
| CA2822980C | Canada | C | |
| JP5946147B2 | Japan | B2 | |
| US9400503B2 | United States of America | B2 | |
| JP5963372B2 | Japan | B2 | |
| IL223155A | Israel | A | |
| JP6028304B2 | Japan | B2 | |
| US9498886B2 | United States of America | B2 | |
| JP6039611B2 | Japan | B2 | |
| AU2015218522B2 | Australia | B2 | |
| JP2017050018A | Japan | A | |
| AU2017201879A1 | Australia | A1 | |
| US9902069B2 | United States of America | B2 |
16 legal events, as the office reported them to INPADOC
Over the term
Point at a mark for the eventEvents
| Event | Code | |
|---|---|---|
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Certificate of patent or registration of utility modelJAPANESE INTERMEDIATE CODE: R150R150 | R150 | |
| First payment of annual fees (during grant procedure)JAPANESE INTERMEDIATE CODE: A61A61 | A61 | |
| Written decision to grant a patent or to grant a registration (utility model)JAPANESE INTERMEDIATE CODE: A01A01 | A01 | |
| Decision of grant or rejection writtenTRDD | TRDD | |
| Request for written amendment filedJAPANESE INTERMEDIATE CODE: A523A521 | A521 | |
| Notification of reasons for refusalJAPANESE INTERMEDIATE CODE: A131A131 | A131 | |
| Request for written amendment filedJAPANESE INTERMEDIATE CODE: A523A521 | A521 | |
| Written request for extension of timeJAPANESE INTERMEDIATE CODE: A601A601 | A601 | |
| Notification of reasons for refusalJAPANESE INTERMEDIATE CODE: A131A131 | A131 |
Numbers
- Publication
- 2014176966
- Application
- 108606
Titles2
- Japanese
- 移動式ヒューマンインターフェースロボット
- English
- Portable human interface robot
Classification
- CPC, 9
- B25J5/007
- G05D1/0227
- G05D1/024
- G05D1/0251
- G05D1/0255
- G05D1/0272
- G05D1/0274
- Y10S901/01
- B25J13/084
- IPC, 1
- B25J5 00