Gesture recognizers with delegates for controlling and modifying gesture recognition
Abstract
Touch-based gestures provide a method of being perceived as appropriate input in other situations (different applications and / or different modes or situations within a first application). A software application includes a plurality of views and application states. Display one or more views, and individual views include individual gesture recognizers with corresponding delegates to individual gesture recognizers. Includes detecting one or more events and using individual gesture recognizers to process individual events for one or more events. The processing of individual events processes individual events in individual gesture recognizers according to the individual gesture definitions that correspond to the individual gesture recognizers. Run the software application according to the information received from the individual gesture recognizer. [Selection diagram] Fig. 6A

Term
9.6 yearsto projected expiry
Projected expiry 16 May 2036, counted from filing; an application has no term until it is granted.
- Priority
- Filed
- Published
- Today
- Projected expiry
61 claims: 20 independent, 41 dependent
- 11つ以上のイベント・センサを有しており、複数のビューとソフトウェア・アプリケーションのアプリケーション状態とを含むソフトウェア・アプリケーションを実行するように構成された電子デバイスにおいて、 前記複数のビューのうちの1つ以上のビューを表示する工程であって、前記表示される1つ以上のビューの個別のビューは1つ以上の個別のジェスチャ・レコグナイザを含み、個別のジェスチャ・レコグナイザは対応するデリゲートを有する、工程と、 1つ以上のイベントを検出する工程と、 前記個別のジェスチャ・レコグナイザを用いて前記1つ以上のイベントの個別のイベントを処理する工程であって、前記個別のジェスチャ・レコグナイザに対応する個別のジェスチャ定義に従って前記個別のジェスチャ・レコグナイザにおいて前記個別のイベントを処理する工程と、前記アプリケーション状態に従って1つ以上の値を判定するために前記対応するデリゲートを実行する工程と、前記個別のジェスチャ・レコグナイザによる前記個別のイベントの処理の結果に従うとともに前記対応するデリゲートにより判定された前記1つ以上の値に従って、前記個別のイベントに対応する情報を前記ソフトウェア・アプリケーションに条件付きで送信する工程とを含む、工程と、 前記個別のイベントに対応する、前記個別のジェスチャ・レコグナイザから受信された情報に従って前記ソフトウェア・アプリケーションを実行する工程とを有することを特徴とする方法。
- 2前記1つ以上のイベント・センサは1つ以上のタッチを検出するように構成されたタッチセンサ面を含み、前記1つ以上のイベントは前記1つ以上のタッチを含み、前記個別のイベントを処理する工程は個別のタッチを処理する工程を含むことを特徴とする請求項1に記載の方法。
- 3前記対応するデリゲートにより判定された前記1つ以上の値に従って前記個別のジェスチャ・レコグナイザにおいて前記個別のタッチを条件付きで受信する工程をさらに有することを特徴とする請求項2に記載の方法。
- 4前記個別のタッチを処理する工程は、前記対応するデリゲートにより判定された前記1つ以上の値が所定のタッチ無視基準に合致する場合に、前記個別のジェスチャ・レコグナイザが前記個別のタッチを無視する工程を含むことを特徴とする請求項2又は3に記載の方法。
- 5前記個別のタッチを処理する工程は、前記対応するデリゲートにより判定された前記1つ以上の値が所定のタッチ無視基準に合致する場合に、前記個別のジェスチャ・レコグナイザが前記個別のタッチを受信することを阻止する工程を含むことを特徴とする請求項2又は3に記載の方法。
- 6前記個別のジェスチャ・レコグナイザにおいて前記個別のタッチを処理する工程は、前記検出されたタッチが前記個別のジェスチャ定義に整合している場合に、前記個別のジェスチャ・レコグナイザにおける対応する状態遷移を、前記対応するデリゲートにより前記状態遷移が可能にされたときに可能にする工程を含むことを特徴とする請求項2又は3に記載の方法。
- 7前記個別のジェスチャ・レコグナイザにおいて前記個別のタッチを処理する工程は、前記検出されたタッチが前記個別のジェスチャ定義に整合している場合に、前記個別のジェスチャ・レコグナイザにおける対応する状態遷移を、前記対応するデリゲートにより前記状態遷移が可能にされたときに条件付きで可能にする工程を含むことを特徴とする請求項2又は3に記載の方法。
- 8前記個別のジェスチャ・レコグナイザにおいて前記個別のタッチを処理する工程は、第2ジェスチャ・レコグナイザに対応するデリゲートにより判定された1つ以上の値に従って前記第2ジェスチャ・レコグナイザにおいて前記個別のタッチを同時に処理する工程を含むことを特徴とする請求項2乃至7の何れか1項に記載の方法。
- 9前記個別のジェスチャ・レコグナイザにおいて前記個別のタッチを処理する工程は、前記個別のジェスチャ・レコグナイザに対応するデリゲートにより判定された1つ以上の値に従って第2ジェスチャ・レコグナイザにおいて前記個別のタッチを同時に処理する工程を含むことを特徴とする請求項2乃至7の何れか1項に記載の方法。
- 10前記表示される1つ以上のビューは複数のジェスチャ・レコグナイザを含み、前記方法は、前記複数のジェスチャ・レコグナイザのうちの少なくとも部分集合に別個のデリゲートを割り当てる工程を含むことを特徴とする請求項1乃至9の何れか1項に記載の方法。
- 11イベントを検出するための1つ以上のイベント・センサと、 1つ以上のプロセッサと、 メモリと、 前記メモリに格納され、前記1つ以上のプロセッサにより実行されるように構成された1つ以上のプログラムとを備える電子デバイスであって、 前記1つ以上のプログラムは、複数のビューとソフトウェア・アプリケーションのアプリケーション状態とを含むソフトウェア・アプリケーションを含み、 前記ソフトウェア・アプリケーションは、 前記複数のビューのうちの1つ以上のビューを表示するための命令であって、前記表示される1つ以上のビューの個別のビューは1つ以上の個別のジェスチャ・レコグナイザを含み、個別のレコグナイザは個別のデリゲートを有する、命令と、 前記個別のジェスチャ・レコグナイザを用いて前記1つ以上のイベントの個別のイベントを処理するための命令であって、前記個別のジェスチャ・レコグナイザに対応する個別のジェスチャ定義に従って前記個別のジェスチャ・レコグナイザにおいて前記個別のイベントを処理するための命令と、前記アプリケーション状態に従って1つ以上の値を判定するために前記個別のデリゲートを実行するための命令と、前記個別のジェスチャ・レコグナイザによる前記個別のイベントの処理の結果に従うとともに前記個別のデリゲートにより判定された前記1つ以上の値に従って、前記個別のイベントに対応する情報を前記ソフトウェア・アプリケーションに条件付きで送信するための命令とを含む、命令と、 前記個別のイベントに対応する、前記個別のジェスチャ・レコグナイザから受信された情報に従って前記ソフトウェア・アプリケーションを実行するための命令とを有することを特徴とする電子デバイス。
- 121つ以上のイベント・センサを有する電子デバイスの1つ以上のプロセッサによる実行のための1つ以上のプログラムを格納するコンピュータ可読記憶媒体であって、 前記1つ以上のプログラムは、複数のビューとソフトウェア・アプリケーションのアプリケーション状態とを含むソフトウェア・アプリケーションを含み、 前記ソフトウェア・アプリケーションは、 前記複数のビューのうちの1つ以上のビューを表示するための命令であって、前記表示される1つ以上のビューの個別のビューは1つ以上の個別のジェスチャ・レコグナイザを含み、個別のジェスチャ・レコグナイザは個別のデリゲートを有する、命令と、 1つ以上のイベントを検出するための命令と、 前記個別のジェスチャ・レコグナイザを用いて前記1つ以上のイベントの個別のイベントを処理するための命令であって、前記個別のジェスチャ・レコグナイザに対応する個別のジェスチャ定義に従って前記個別のジェスチャ・レコグナイザにおいて前記個別のイベントを処理するための命令と、前記アプリケーション状態に従って1つ以上の値を判定するために前記個別のデリゲートを実行するための命令と、前記個別のジェスチャ・レコグナイザによる前記個別のイベントの処理の結果に従うとともに前記個別のデリゲートにより判定された前記1つ以上の値に従って、前記個別のイベントに対応する情報を前記ソフトウェア・アプリケーションに条件付きで送信するための命令とを含む、命令と、 前記個別のイベントに対応する、前記個別のジェスチャ・レコグナイザから受信された情報に従って前記ソフトウェア・アプリケーションを実行するための命令とを有することを特徴とするコンピュータ可読記憶媒体。
- 13タッチセンサ面を有しており、複数のビューとソフトウェア・アプリケーションのアプリケーション状態とを含むソフトウェア・アプリケーションを実行するように構成された電子デバイスにおいて、 前記複数のビューのうちの1つ以上のビューを表示する工程であって、前記表示される1つ以上のビューの個別のビューは1つ以上の個別のジェスチャ・レコグナイザを含み、個別のジェスチャ・レコグナイザは対応するデリゲートを有する、工程と、 前記タッチセンサ面において、前記表示されたビューのうちの1つ以上のビューの範囲に入るタッチ位置をそれぞれが有する1つ以上のタッチを検出する工程と、 前記1つ以上のタッチの個別のタッチを処理する工程であって、 前記アプリケーション状態に従って受信タッチ値を取得するために前記個別のジェスチャ・レコグナイザに対応する前記デリゲートを実行する工程と、 前記受信タッチ値が所定の基準を満たす場合に前記個別のジェスチャ・レコグナイザにおいて前記個別のタッチを処理する工程と、 前記個別のタッチに対応する情報を前記ソフトウェア・アプリケーションに条件付きで送信する工程とを含む、工程と、 前記個別のタッチに対応する、前記個別のジェスチャ・レコグナイザから受信された情報に従って前記ソフトウェア・アプリケーションを実行する工程とを有することを特徴とする方法。
- 14前記複数のビューは複数のジェスチャ・レコグナイザを含み、 前記方法は、前記複数のジェスチャ・レコグナイザの少なくとも部分集合に別個のデリゲートを割り当てる工程を含み、 前記1つ以上のタッチの前記個別のタッチを処理する工程は、 前記複数のジェスチャ・レコグナイザのうちの候補ジェスチャ・レコグナイザの集合を識別する工程と、 割り当てられたデリゲートを有する候補ジェスチャ・レコグナイザ毎に、前記アプリケーション状態に従って受信タッチ値を取得するために前記割り当てられたデリゲートを実行する工程と、 前記取得された受信タッチ値に従って前記候補ジェスチャ・レコグナイザの部分集合を含む1つ以上の受信ジェスチャ・レコグナイザを識別する工程と、 受信ジェスチャ・レコグナイザのうちの1つ以上の受信ジェスチャ・レコグナイザの各ジェスチャ・レコグナイザにおいて前記個別のタッチを処理する工程とを含むことを特徴とする請求項13に記載の方法。
- 15前記1つ以上の受信ジェスチャ・レコグナイザの各ジェスチャ・レコグナイザにおいて前記個別のタッチを処理する工程は、前記個別のジェスチャ・レコグナイザに対応する個別のジェスチャ定義に従って、デリゲートが割り当てられた個別の受信ジェスチャ・レコグナイザにおいて前記個別のタッチを処理する工程と、前記アプリケーション状態に従って1つ以上の値を判定するために前記割り当てられたデリゲートを実行する工程と、前記個別のジェスチャ・レコグナイザによる前記個別のタッチの処理の結果に従うとともに前記割り当てられたデリゲートにより判定された前記1つ以上の値に従って前記個別のタッチに対応する情報を前記ソフトウェア・アプリケーションに条件付きで送信する工程とを含み、 前記方法は、前記タッチのうちの1つ以上に対応する、前記受信ジェスチャ・レコグナイザのうちの1つ以上から受信された情報に従って前記ソフトウェア・アプリケーションを実行する工程を含むことを特徴とする請求項14に記載の方法。
- 16前記個別の受信ジェスチャ・レコグナイザにおいて前記個別のタッチを処理する工程は、前記検出されたタッチが前記個別のジェスチャ定義に整合している場合に、前記個別のジェスチャ・レコグナイザにおける対応する状態遷移を、前記割り当てられたデリゲートにより前記状態遷移が可能にされたときに可能にする工程を含むことを特徴とする請求項15に記載の方法。
- 17前記個別の受信ジェスチャ・レコグナイザにおいて前記個別のタッチを処理する工程は、前記検出されたタッチが前記個別のジェスチャ定義に整合している場合に、前記個別のジェスチャ・レコグナイザにおける対応する状態遷移を、前記割り当てられたデリゲートにより前記状態遷移が可能にされたときに条件付きで可能にする工程を含むことを特徴とする請求項15に記載の方法。
- 18前記個別の受信ジェスチャ・レコグナイザにおいて前記個別のタッチを処理する工程は、第2ジェスチャ・レコグナイザに割り当てられた前記デリゲートにより判定された1つ以上の値に従って前記第2ジェスチャ・レコグナイザにおいて前記個別のタッチを同時に処理する工程を含むことを特徴とする請求項15乃至17の何れか1項に記載の方法。
- 19前記個別の受信ジェスチャ・レコグナイザにおいて前記個別のタッチを処理する工程は、前記個別のジェスチャ・レコグナイザに割り当てられた前記デリゲートにより判定された1つ以上の値に従って第2ジェスチャ・レコグナイザにおいて前記個別のタッチを同時に処理する工程を含むことを特徴とする請求項15乃至17の何れか1項に記載の方法。
- 20タッチセンサ面と、 1つ以上のプロセッサと、 メモリと、 前記メモリに格納され、前記1つ以上のプロセッサにより実行されるように構成された1つ以上のプログラムとを備える電子デバイスであって、 前記1つ以上のプログラムは、複数のビューとソフトウェア・アプリケーションのアプリケーション状態とを含むソフトウェア・アプリケーションを含み、 前記ソフトウェア・アプリケーションは、 前記複数のビューのうちの1つ以上のビューを表示するための命令であって、前記表示される1つ以上のビューの個別のビューは1つ以上の個別のジェスチャ・レコグナイザを含み、個別のジェスチャ・レコグナイザは対応するデリゲートを有する、命令と、 前記タッチセンサ面において、前記表示されたビューのうちの1つ以上のビューの範囲に入るタッチ位置をそれぞれが有する1つ以上のタッチを検出するための命令と、 前記1つ以上のタッチの個別のタッチを処理するための命令であって、 前記アプリケーション状態に従って受信タッチ値を取得するために前記個別のジェスチャ・レコグナイザに対応する前記デリゲートを実行するための命令と、 前記受信タッチ値が所定の基準を満たす場合に前記個別のジェスチャ・レコグナイザにおいて前記個別のタッチを処理するための命令と、 前記個別のタッチに対応する情報を前記ソフトウェア・アプリケーションに条件付きで送信するための命令とを含む、命令と、 前記個別のタッチに対応する、前記個別のジェスチャ・レコグナイザから受信された情報に従って前記ソフトウェア・アプリケーションを実行するための命令とを有することを特徴とする電子デバイス。
- 21タッチセンサ面を有する電子デバイスの1つ以上のプロセッサによる実行のための1つ以上のプログラムを格納するコンピュータ可読記憶媒体であって、 前記1つ以上のプログラムは、複数のビューとソフトウェア・アプリケーションのアプリケーション状態とを含むソフトウェア・アプリケーションを含み、 前記ソフトウェア・アプリケーションは、 前記複数のビューのうちの1つ以上のビューを表示するための命令であって、前記表示される1つ以上のビューの個別のビューは1つ以上の個別のジェスチャ・レコグナイザを含み、個別のジェスチャ・レコグナイザは対応するデリゲートを有する、命令と、 前記タッチセンサ面において、前記表示されたビューのうちの1つ以上のビューの範囲に入るタッチ位置をそれぞれが有する1つ以上のタッチを検出するための命令と、 前記タッチセンサ面における1つ以上のタッチの個別のタッチを処理するための命令であって、 前記アプリケーション状態に従って受信タッチ値を取得するために前記個別のジェスチャ・レコグナイザに対応する前記デリゲートを実行するための命令と、 前記受信タッチ値が所定の基準を満たす場合に前記個別のジェスチャ・レコグナイザにおいて前記個別のタッチを処理するための命令と、 前記個別のタッチに対応する情報を前記ソフトウェア・アプリケーションに条件付きで送信するための命令とを含む、命令と、 前記個別のタッチに対応する、前記個別のジェスチャ・レコグナイザから受信された情報に従って前記ソフトウェア・アプリケーションを実行するための命令とを有することを特徴とするコンピュータ可読記憶媒体。
- 22タッチセンサ面を有しており、ソフトウェア・アプリケーションを実行するように構成された電子デバイスにおいて、 前記ソフトウェア・アプリケーションの1つ以上のビューを表示する工程であって、前記表示される1つ以上のビューは複数のジェスチャ・レコグナイザを含み、前記複数のジェスチャ・レコグナイザは、個別のジェスチャに応答して単一の動作メッセージを送信するように構成された少なくとも1つの離散ジェスチャ・レコグナイザと、認識された個別のジェスチャの認識された連続するサブイベントにおいて動作メッセージを送信するように構成された少なくとも1つの連続ジェスチャ・レコグナイザとを含む、工程と、 1つ以上のタッチを検出する工程と、 前記ジェスチャ・レコグナイザのうちの1つ以上を使用しての前記タッチのそれぞれを処理する工程であって、個別のジェスチャ・レコグナイザに対応する個別のジェスチャ定義に従って前記個別のジェスチャ・レコグナイザにおいて個別のタッチを処理する工程と、前記個別のジェスチャ・レコグナイザにおける前記個別のタッチの処理の結果に従って1つ以上の個別の動作メッセージを前記ソフトウェア・アプリケーションに条件付きで送信する工程とを含む、工程と、 前記タッチの1つ以上に対応する前記ジェスチャ・レコグナイザの1つ以上から受信された1つ以上の動作メッセージに従って前記ソフトウェア・アプリケーションを実行する工程とを有することを特徴とする方法。
- 23各ジェスチャ・レコグナイザはジェスチャ・レコグナイザ状態の集合を有することを特徴とする請求項22に記載の方法。
- 24前記離散ジェスチャ・レコグナイザは、前記離散ジェスチャ・レコグナイザの初期状態に対応するジェスチャ可能状態と、前記個別のジェスチャの認識に対応するジェスチャ認識済状態と、前記離散ジェスチャ・レコグナイザが前記1つ以上のタッチを前記個別のジェスチャとして認識することに失敗したことに対応するジェスチャ失敗状態とを含むジェスチャ・レコグナイザ状態の第1集合を有し、 前記連続ジェスチャ・レコグナイザは、ジェスチャ可能状態と、前記個別のジェスチャの初期認識に対応するジェスチャ開始済状態と、前記個別のタッチの位置の個別の変化に対応するジェスチャ変更済状態と、前記認識された個別のジェスチャの完了に対応するジェスチャ終了済状態と、前記個別のジェスチャの認識の割り込みに対応するジェスチャ・キャンセル済状態と、前記連続ジェスチャ・レコグナイザが前記1つ以上のタッチを前記個別のジェスチャとして認識することに失敗したことに対応するジェスチャ失敗状態とを含むジェスチャ・レコグナイザ状態の第2集合を有することを特徴とする請求項23に記載の方法。
- 25前記ジェスチャ認識済状態及び前記ジェスチャ終了済状態は、同一のジェスチャ・レコグナイザ状態値を有することを特徴とする請求項24に記載の方法。
- 26前記ソフトウェア・アプリケーションはアプリケーション状態を有し、前記1つ以上の個別の動作メッセージを条件付きで送信する工程は、前記ソフトウェア・アプリケーションの前記アプリケーション状態にさらに従って前記1つ以上の個別の動作メッセージを条件付きで送信する工程を含むことを特徴とする請求項22乃至25の何れか1項に記載の方法。
- 27前記方法は、前記個別のジェスチャ・レコグナイザからの追加の情報を要求する工程をさらに有し、前記ソフトウェア・アプリケーションを実行する工程は、前記追加の情報にさらに従って前記ソフトウェア・アプリケーションを実行する工程を含むことを特徴とする請求項22乃至26の何れか1項に記載の方法。
- 28前記追加の情報は、前記個別のジェスチャ・レコグナイザにおいて処理された個別のタッチの個数及び位置を含むことを特徴とする請求項27に記載の方法。
- 29前記少なくとも1つの離散ジェスチャ・レコグナイザは、タップ・ジェスチャ・レコグナイザと、スワイプ・ジェスチャ・レコグナイザとのうちの1つ以上を含み、前記少なくとも1つの連続ジェスチャ・レコグナイザは、長押しジェスチャ・レコグナイザと、ピンチ・ジェスチャ・レコグナイザと、パン・ジェスチャ・レコグナイザと、回転ジェスチャ・レコグナイザと、変換ジェスチャ・レコグナイザとのうちの1つ以上を含むことを特徴とする請求項22乃至28の何れか1項に記載の方法。
- 30前記少なくとも1つの離散ジェスチャ・レコグナイザは、タップ・ジェスチャ・レコグナイザと、スワイプ・ジェスチャ・レコグナイザとを含み、前記少なくとも1つの連続ジェスチャ・レコグナイザは、長押しジェスチャ・レコグナイザと、ピンチ・ジェスチャ・レコグナイザと、パン・ジェスチャ・レコグナイザと、回転ジェスチャ・レコグナイザと、変換ジェスチャ・レコグナイザとを含むことを特徴とする請求項22乃至29の何れか1項に記載の方法。
- 31タッチセンサ面と、 1つ以上のプロセッサと、 メモリと、 前記メモリに格納され、前記1つ以上のプロセッサにより実行されるように構成された1つ以上のプログラムとを備える電子デバイスであって、 前記1つ以上のプログラムは、ソフトウェア・アプリケーションを含み、 前記ソフトウェア・アプリケーションは、 前記ソフトウェア・アプリケーションの1つ以上のビューを表示するための命令であって、前記表示される1つ以上のビューは複数のジェスチャ・レコグナイザを含み、前記複数のジェスチャ・レコグナイザは、個別のジェスチャに応答して単一の動作メッセージを送信するように構成された少なくとも1つの離散ジェスチャ・レコグナイザと、認識された個別のジェスチャの認識された連続するサブイベントにおいて動作メッセージを送信するように構成された少なくとも1つの連続ジェスチャ・レコグナイザとを含む、命令と、 1つ以上のタッチを検出するための命令と、 前記ジェスチャ・レコグナイザのうちの1つ以上を使用しての前記タッチのそれぞれを処理するための命令であって、個別のジェスチャ・レコグナイザに対応する個別のジェスチャ定義に従って前記個別のジェスチャ・レコグナイザにおいて個別のタッチを処理するための命令と、前記個別のジェスチャ・レコグナイザにおける前記個別のタッチの処理の結果に従って1つ以上の個別の動作メッセージを前記ソフトウェア・アプリケーションに条件付きで送信するための命令とを含む、命令と、 前記タッチの1つ以上に対応する前記ジェスチャ・レコグナイザの1つ以上から受信された1つ以上の動作メッセージに従って前記ソフトウェア・アプリケーションを実行するための命令とを有することを特徴とする電子デバイス。
- 32タッチセンサ面を有する電子デバイスの1つ以上のプロセッサによる実行のための1つ以上のプログラムを格納するコンピュータ可読記憶媒体であって、 前記1つ以上のプログラムは、ソフトウェア・アプリケーションを含み、 前記ソフトウェア・アプリケーションは、 前記ソフトウェア・アプリケーションの1つ以上のビューを表示するための命令であって、前記表示される1つ以上のビューは複数のジェスチャ・レコグナイザを含み、前記複数のジェスチャ・レコグナイザは、個別のジェスチャに応答して単一の動作メッセージを送信するように構成された少なくとも1つの離散ジェスチャ・レコグナイザと、認識された個別のジェスチャの認識された連続するサブイベントにおいて動作メッセージを送信するように構成された少なくとも1つの連続ジェスチャ・レコグナイザとを含む、命令と、 1つ以上のタッチを検出するための命令と、 前記ジェスチャ・レコグナイザのうちの1つ以上を使用しての前記タッチのそれぞれを処理するための命令であって、個別のジェスチャ・レコグナイザに対応する個別のジェスチャ定義に従って前記個別のジェスチャ・レコグナイザにおいて個別のタッチを処理するための命令と、前記個別のジェスチャ・レコグナイザにおける前記個別のタッチの処理の結果に従って1つ以上の個別の動作メッセージを前記ソフトウェア・アプリケーションに条件付きで送信するための命令とを含む、命令と、 前記タッチの1つ以上に対応する前記ジェスチャ・レコグナイザの1つ以上から受信された1つ以上の動作メッセージに従って前記ソフトウェア・アプリケーションを実行するための命令とを有することを特徴とするコンピュータ可読記憶媒体。
- 33タッチセンサ面と、 1つ以上のプロセッサと、 メモリと、 1つ以上のプログラムとを備える電子デバイスであって、 前記1つ以上のプログラムは前記メモリに格納され、前記1つ以上のプロセッサにより実行されるように構成され、前記1つ以上のプログラムは請求項1乃至10、13乃至19及び22乃至30の何れか1項に記載の方法を実行するための命令を含むことを特徴とする電子デバイス。
- 341つ以上のプログラムを格納するコンピュータ可読記憶媒体であって、前記1つ以上のプログラムは、タッチセンサ面を有する電子デバイスによって実行される場合に、請求項1乃至10、13乃至19及び22乃至30の何れか1項に記載の方法を実行させる命令を含むことを特徴とするコンピュータ可読記憶媒体。
- 35タッチセンサ面と、 請求項1乃至10、13乃至19及び22乃至30の何れか1項に記載の方法を実行するための手段とを含むことを特徴とする電子デバイス。
- 36タッチセンサ面を有する電子デバイスに用いられる情報処理装置であって、 請求項1乃至10、13乃至19及び22乃至30の何れか1項に記載の方法を実行するための手段とを含むことを特徴とする情報処理装置。
- 371つ以上のタッチを受信するように構成されたタッチセンサ面ユニットと、 前記タッチセンサ面ユニットに結合された処理ユニットとを備える電子デバイスであって、 前記処理ユニットは、 複数のビューとソフトウェア・アプリケーションのアプリケーション状態とを含むソフトウェア・アプリケーションを実行し、 前記複数のビューのうちの1つ以上のビューの表示を可能にし、前記表示される1つ以上のビューの個別のビューは1つ以上の個別のジェスチャ・レコグナイザを含み、個別のジェスチャ・レコグナイザは対応するデリゲートを有しており、 前記表示されたビューのうちの1つ以上のビューの範囲に入るタッチ位置をそれぞれが有し、前記タッチセンサ面において受信された1つ以上のタッチを検出し、 前記1つ以上のタッチの個別のタッチを処理し、前記個別のタッチについて、 前記アプリケーション状態に従って受信タッチ値を取得するために前記個別のジェスチャ・レコグナイザに対応する前記デリゲートを実行し、 前記受信タッチ値が所定の基準を満たす場合に前記個別のジェスチャ・レコグナイザにおいて前記個別のタッチを処理し、 前記個別のタッチに対応する情報を前記ソフトウェア・アプリケーションに条件付きで送信し、 前記個別のタッチに対応する、前記個別のジェスチャ・レコグナイザから受信された情報に従って前記ソフトウェア・アプリケーションを実行するように構成されることを特徴とする電子デバイス。
- 38前記複数のビューは複数のジェスチャ・レコグナイザを含み、 前記処理ユニットは、 前記複数のジェスチャ・レコグナイザの少なくとも部分集合に別個のデリゲートを割り当て、 前記1つ以上のタッチの個別のタッチを処理することを、 前記複数のジェスチャ・レコグナイザのうちの候補ジェスチャ・レコグナイザの集合を識別し、 割り当てられたデリゲートを有する候補ジェスチャ・レコグナイザ毎に、前記アプリケーション状態に従って受信タッチ値を取得するために前記割り当てられたデリゲートを実行し、 前記取得された受信タッチ値に従って前記候補ジェスチャ・レコグナイザの部分集合を含む1つ以上の受信ジェスチャ・レコグナイザを識別し、 受信ジェスチャ・レコグナイザのうちの1つ以上の受信ジェスチャ・レコグナイザの各ジェスチャ・レコグナイザにおいて前記個別のタッチを処理することによって行うように構成されることを特徴とする請求項37に記載の電子デバイス。
- 39前記処理ユニットは、 前記1つ以上の受信ジェスチャ・レコグナイザの各ジェスチャ・レコグナイザにおいて前記個別のタッチを処理することを、 前記個別のジェスチャ・レコグナイザに対応する個別のジェスチャ定義に従って、デリゲートが割り当てられた個別の受信ジェスチャ・レコグナイザにおいて前記個別のタッチを処理し、 前記アプリケーション状態に従って1つ以上の値を判定するために前記割り当てられたデリゲートを実行し、 前記個別のジェスチャ・レコグナイザによる前記個別のタッチの処理の結果に従うとともに前記割り当てられたデリゲートにより判定された前記1つ以上の値に従って前記個別のタッチに対応する情報を前記ソフトウェア・アプリケーションに条件付きで送信することによって行い、 前記タッチのうちの1つ以上に対応する、前記受信ジェスチャ・レコグナイザのうちの1つ以上から受信された情報に従って前記ソフトウェア・アプリケーションを実行するように構成されることを特徴とする請求項38に記載の電子デバイス。
- 40前記処理ユニットは、前記個別の受信ジェスチャ・レコグナイザにおいて前記個別のタッチを処理することを、前記検出されたタッチが前記個別のジェスチャ定義に整合している場合に、前記個別のジェスチャ・レコグナイザにおける対応する状態遷移を、前記割り当てられたデリゲートにより前記状態遷移が可能にされたときに可能にすることによって行われるように構成されることを特徴とする請求項39に記載の電子デバイス。
- 41前記処理ユニットは、前記個別の受信ジェスチャ・レコグナイザにおいて前記個別のタッチを処理することを、第2ジェスチャ・レコグナイザに割り当てられた前記デリゲートにより判定された1つ以上の値に従って前記第2ジェスチャ・レコグナイザにおいて前記個別のタッチを同時に処理することによって行うように構成されることを特徴とする請求項39又は40に記載の電子デバイス。
- 42前記処理ユニットは、前記個別の受信ジェスチャ・レコグナイザにおいて前記個別のタッチを処理することを、前記個別のジェスチャ・レコグナイザに割り当てられた前記デリゲートにより判定された1つ以上の値に従って第2ジェスチャ・レコグナイザにおいて前記個別のタッチを同時に処理することによって行うように構成されることを特徴とする請求項39又は40に記載の電子デバイス。
- 431つ以上のイベントを検知するように構成された1つ以上のイベント検知ユニットと、 前記1つ以上のイベント検知ユニットに結合された処理ユニットとを備える電子デバイスであって、 前記処理ユニットは、 複数のビューとソフトウェア・アプリケーションのアプリケーション状態とを含むソフトウェア・アプリケーションを実行し、 前記複数のビューのうちの1つ以上のビューの表示を可能にし、前記表示される1つ以上のビューの個別のビューは個別のジェスチャ・レコグナイザを含み、個別のジェスチャ・レコグナイザは対応するデリゲートを有しており、 前記1つ以上のイベント検知ユニットにより1つ以上のイベントを検出し、 前記個別のジェスチャ・レコグナイザを用いて前記1つ以上のイベントの個別のイベントを処理することを、 前記個別のジェスチャ・レコグナイザに対応する個別のジェスチャ定義に従って前記個別のジェスチャ・レコグナイザにおいて前記個別のイベントを処理し、前記アプリケーション状態に従って1つ以上の値を判定するために前記対応するデリゲートを実行し、前記個別のジェスチャ・レコグナイザによる前記個別のイベントの処理の結果に従うとともに前記対応するデリゲートにより判定された前記1つ以上の値に従って、前記個別のイベントに対応する情報を前記ソフトウェア・アプリケーションに条件付きで送信することによって行い、 前記個別のイベントに対応する、前記個別のジェスチャ・レコグナイザから受信された情報に従って前記ソフトウェア・アプリケーションを実行するように構成されることを特徴とする電子デバイス。
- 44前記1つ以上のイベント検知ユニットは1つ以上のタッチを検出するように構成されたタッチセンサ・ユニットを含み、前記1つ以上のイベントは前記1つ以上のタッチを含み、前記処理ユニットは個別のタッチを処理するように構成されることを特徴とする請求項43に記載の電子デバイス。
- 45前記処理ユニットは、前記対応するデリゲートにより判定された前記1つ以上の値に従って前記個別のジェスチャ・レコグナイザが前記個別のタッチを条件付きで受信することを可能にするように構成されることを特徴とする請求項44に記載の電子デバイス。
- 46前記処理ユニットは、前記対応するデリゲートにより判定された前記1つ以上の値が所定のタッチ無視基準に合致する場合に、前記個別のジェスチャ・レコグナイザが前記個別のタッチを無視することを可能にするように構成されることを特徴とする請求項44又は45に記載の電子デバイス。
- 47前記処理ユニットは、前記対応するデリゲートにより判定された前記1つ以上の値が所定のタッチ無視基準に合致する場合に、前記個別のジェスチャ・レコグナイザが前記個別のタッチを受信することを阻止することによって、前記個別のタッチを処理するように構成されることを特徴とする請求項44又は45に記載の電子デバイス。
- 48前記処理ユニットは、前記検出されたタッチが前記個別のジェスチャ定義に整合している場合に、前記個別のジェスチャ・レコグナイザにおける対応する状態遷移を、前記対応するデリゲートにより前記状態遷移が可能にされたときに可能にすることによって、前記個別のジェスチャ・レコグナイザにおいて前記個別のタッチを処理することを特徴とする請求項44又は45に記載の電子デバイス。
- 49前記処理ユニットは、前記検出されたタッチが前記個別のジェスチャ定義に整合している場合に、前記個別のジェスチャ・レコグナイザにおける対応する状態遷移を、前記対応するデリゲートにより前記状態遷移が可能にされたときに条件付きで可能にすることによって、前記個別のジェスチャ・レコグナイザにおいて前記個別のタッチを処理するように構成されることを特徴とする請求項44又は45に記載の電子デバイス。
- 50前記処理ユニットは、第2ジェスチャ・レコグナイザに対応するデリゲートにより判定された1つ以上の値に従って前記第2ジェスチャ・レコグナイザにおいて前記個別のタッチを同時に処理することによって、前記個別のジェスチャ・レコグナイザにおいて前記個別のタッチを処理するように構成されることを特徴とする請求項44乃至49の何れか1項に記載の電子デバイス。
- 51前記処理ユニットは、前記個別のジェスチャ・レコグナイザに対応するデリゲートにより判定された1つ以上の値に従って第2ジェスチャ・レコグナイザにおいて前記個別のタッチを同時に処理することによって、前記個別のジェスチャ・レコグナイザにおいて前記個別のタッチを処理するように構成されることを特徴とする請求項44乃至49の何れか1項に記載の電子デバイス。
- 52前記表示される1つ以上のビューは複数のジェスチャ・レコグナイザを含み、前記処理ユニットは、前記複数のジェスチャ・レコグナイザのうちの少なくとも部分集合に別個のデリゲートを割り当てるように構成されることを特徴とする請求項43乃至51の何れか1項に記載の電子デバイス。
- 531つ以上のタッチを受信するように構成されたタッチセンサ面ユニットと、 前記タッチセンサ面ユニットに結合された処理ユニットとを備える電子デバイスであって、 前記処理ユニットは、 ソフトウェア・アプリケーションを実行し、 前記ソフトウェア・アプリケーションの1つ以上のビューの表示を可能にし、前記表示される1つ以上のビューは複数のジェスチャ・レコグナイザを含み、前記複数のジェスチャ・レコグナイザは、個別のジェスチャに応答して単一の動作メッセージを送信するように構成された少なくとも1つの離散ジェスチャ・レコグナイザと、認識された個別のジェスチャの認識された連続するサブイベントにおいて動作メッセージを送信するように構成された少なくとも1つの連続ジェスチャ・レコグナイザとを含み、 前記タッチセンサ面ユニットで受信された1つ以上のタッチを検出し、 前記ジェスチャ・レコグナイザのうちの1つ以上を使用しての前記タッチのそれぞれを処理することを、個別のジェスチャ・レコグナイザに対応する個別のジェスチャ定義に従って前記個別のジェスチャ・レコグナイザにおいて個別のタッチを処理し、前記個別のジェスチャ・レコグナイザにおける前記個別のタッチの処理の結果に従って1つ以上の個別の動作メッセージを前記ソフトウェア・アプリケーションに条件付きで送信することによって行い、 前記タッチの1つ以上に対応する前記ジェスチャ・レコグナイザの1つ以上から受信された1つ以上の動作メッセージに従って前記ソフトウェア・アプリケーションを実行するように構成されることを特徴とする電子デバイス。
- 54各ジェスチャ・レコグナイザはジェスチャ・レコグナイザ状態の集合を有することを特徴とする請求項53に記載の電子デバイス。
- 55前記離散ジェスチャ・レコグナイザは、前記離散ジェスチャ・レコグナイザの初期状態に対応するジェスチャ可能状態と、前記個別のジェスチャの認識に対応するジェスチャ認識済状態と、前記離散ジェスチャ・レコグナイザが前記1つ以上のタッチを前記個別のジェスチャとして認識することに失敗したことに対応するジェスチャ失敗状態とを含むジェスチャ・レコグナイザ状態の第1集合を有し、 前記連続ジェスチャ・レコグナイザは、ジェスチャ可能状態と、前記個別のジェスチャの初期認識に対応するジェスチャ開始済状態と、前記個別のタッチの位置の個別の変化に対応するジェスチャ変更済状態と、前記認識された個別のジェスチャの完了に対応するジェスチャ終了済状態と、前記個別のジェスチャの認識の割り込みに対応するジェスチャ・キャンセル済状態と、前記連続ジェスチャ・レコグナイザが前記1つ以上のタッチを前記個別のジェスチャとして認識することに失敗したことに対応するジェスチャ失敗状態とを含むジェスチャ・レコグナイザ状態の第2集合を有することを特徴とする請求項54に記載の電子デバイス。
- 56前記ジェスチャ認識済状態及び前記ジェスチャ終了済状態は、同一のジェスチャ・レコグナイザ状態値を有することを特徴とする請求項55に記載の電子デバイス。
- 57前記ソフトウェア・アプリケーションはアプリケーション状態を有し、前記処理ユニットは、前記ソフトウェア・アプリケーションの前記アプリケーション状態にさらに従って前記1つ以上の個別の動作メッセージを条件付きで送信するように構成されることを特徴とする請求項53乃至56の何れか1項に記載の電子デバイス。
- 58前記処理ユニットは、 前記個別のジェスチャ・レコグナイザからの追加の情報を要求し、 前記追加の情報にさらに従って前記ソフトウェア・アプリケーションを実行するように構成されることを特徴とする請求項53乃至57の何れか1項に記載の電子デバイス。
- 59前記追加の情報は、前記個別のジェスチャ・レコグナイザにおいて処理された個別のタッチの個数及び位置を含むことを特徴とする請求項58に記載の電子デバイス。
- 60前記少なくとも1つの離散ジェスチャ・レコグナイザは、タップ・ジェスチャ・レコグナイザと、スワイプ・ジェスチャ・レコグナイザとのうちの1つ以上を含み、前記少なくとも1つの連続ジェスチャ・レコグナイザは、長押しジェスチャ・レコグナイザと、ピンチ・ジェスチャ・レコグナイザと、パン・ジェスチャ・レコグナイザと、回転ジェスチャ・レコグナイザと、変換ジェスチャ・レコグナイザとのうちの1つ以上を含むことを特徴とする請求項53乃至59の何れか1項に記載の電子デバイス。
- 61前記少なくとも1つの離散ジェスチャ・レコグナイザは、タップ・ジェスチャ・レコグナイザと、スワイプ・ジェスチャ・レコグナイザとを含み、前記少なくとも1つの連続ジェスチャ・レコグナイザは、長押しジェスチャ・レコグナイザと、ピンチ・ジェスチャ・レコグナイザと、パン・ジェスチャ・レコグナイザと、回転ジェスチャ・レコグナイザと、変換ジェスチャ・レコグナイザとを含むことを特徴とする請求項53乃至60の何れか1項に記載の電子デバイス。
Independent claims61
220 paragraphs, as filed
The present invention relates to user interface processing that generally includes, but is not limited to, devices and methods for recognizing gesture inputs.
Electronic devices generally include user interfaces that can be used to interact with computing devices. User interfaces include display and / or input devices such as keyboards, mice and touch sensor surfaces for interacting with various aspects of the user interface. For some devices that have a touch sensor surface as an input device, the first set of touch-based gestures (eg, tap, double tap, horizontal swipe, vertical swipe, pinch in, pinch out, two-finger swipe) One or more are recognized as appropriate inputs in a particular situation (eg, a particular mode of the first application), and other different sets of touch-based gestures are in other situations (eg, different applications and / or). Recognized as appropriate input in different modes or situations within the first application). As a result, the software and logic required to recognize and respond to touch-based gestures can be complex, adding new applications to computing devices each time an application is updated. May need to be modified each time. These and similar problems can occur in user interfaces that utilize input sources other than touch-based gestures.
<p num="0003"> Therefore, it is desirable to have a comprehensive framework or mechanism that recognizes gestures and events from other input sources, as well as touch-based gestures and events, which is a fact of all application programs on computing devices. Above It is easily adaptable to all situations or modes and requires little or no modification when an application is updated or a new application is added to a computing device.</p>
<p num="0004"> To address the above shortcomings, some embodiments have one or more event sensors to run a software application that includes multiple views and the application state of the software application. The method is performed on an electronic device configured in. The method includes displaying one or more of the plurality of views. The individual views of one or more of the displayed views include one or more gesture recognizers. Each gesture recognizer has a corresponding delegate. The method includes detecting one or more events and processing the individual events of the one or more events using the individual gesture recognizer. The processing of the individual event determines the step of processing the individual event in the individual gesture recognizer according to the individual gesture definition corresponding to the individual gesture recognizer, and one or more values according to the application state. One or more of the steps according to the step of executing the corresponding delegate of the individual gesture recognizer and the processing of the individual event by the individual gesture recognizer and determined by the corresponding delegate. A step of conditionally transmitting information corresponding to the individual event to the software application according to the value is included. Further, the method includes executing the software application according to the information received from the individual gesture recognizer corresponding to the individual event.</p><p num="0005"> According to some embodiments, the electronic device is stored in one or more event sensors, one or more processors, a memory, and the memory for detecting an event, by the one or more processors. Includes one or more programs configured to run. The one or more programs include software applications having multiple views and application states. The software application includes instructions for displaying one or more of the plurality of views. The individual views of one or more of the displayed views include one or more individual gesture recognizers, each of which has a corresponding delegate. The software application further includes instructions for detecting one or more events and instructions for processing the individual events of the detected event using the individual gesture recognizer. The instructions for processing the individual gesture event are the instruction for processing the individual event in the individual gesture recognizer according to the individual gesture definition corresponding to the individual gesture recognizer, and the application state. According to an instruction to execute the corresponding delegate to determine one or more values according to, and the result of processing the individual event by the individual gesture recognizer and determined by the corresponding delegate. It includes instructions for conditionally transmitting information corresponding to the individual event to the software application according to one or more values. Further, the software application includes instructions for executing the software application according to the information received from the individual gesture recognizer corresponding to the individual event.</p><p num="0006"> According to some embodiments, the computer-readable storage medium stores one or more programs for execution by one or more processors of an electronic device having one or more event sensors for detecting events. .. The one or more programs include a software application that includes a plurality of views and application states of the software application. The software application includes instructions for displaying one or more of the plurality of views. The individual views of one or more of the displayed views include one or more individual gesture recognizers, each of which has a corresponding delegate. The software application includes instructions for detecting one or more events and instructions for processing the individual events of the detected event using the individual gesture recognizer. The instructions for processing the individual event are the instruction for processing the individual event in the individual gesture recognizer according to the individual gesture definition corresponding to the individual gesture recognizer, and the instruction for processing the individual event according to the application state. The one or more according to the instruction to execute the corresponding delegate to determine one or more values and the result of processing the individual event by the individual gesture recognizer and determined by the corresponding delegate. Includes instructions for conditionally transmitting information corresponding to the individual event to the software application according to the value of. Further, the software application includes instructions for executing the software application according to the information received from the individual gesture recognizer corresponding to the individual event.</p><p num="0007"> According to some embodiments, the method is performed on an electronic device that has a touch sensor surface and is configured to run a software application that includes multiple views and application states of the software application. The method includes displaying one or more of the plurality of views. The individual views of one or more of the displayed views include individual gesture recognizers. Each gesture recognizer has a corresponding delegate. The method further comprises detecting one or more touches on the touch sensor surface, each having a touch position within the range of one or more of the displayed views. The method further comprises processing the individual touches of the one or more touches. The step of processing the individual touch includes the step of executing the delegate corresponding to the individual gesture recognizer in order to acquire the received touch value according to the application state, and the case where the received touch value satisfies a predetermined criterion. Includes a step of processing the individual touches in the individual gesture recognizer. The step of processing the individual touches also includes the step of conditionally transmitting information corresponding to the individual touches to the software application. Further, the method includes executing the software application according to the information received from the individual gesture recognizer corresponding to the individual touch.</p><p num="0008"> According to some embodiments, the electronic device is a touch sensor surface, one or more processors, a memory, and one stored in the memory and configured to be executed by the one or more processors. Including the above programs. The one or more programs include a software application that includes a plurality of views and application states of the software application. The software application includes instructions for displaying one or more of the plurality of views. The individual views of one or more of the displayed views include one or more individual gesture recognizers. Each gesture recognizer has a corresponding delegate. The software application also includes instructions on the touch sensor surface to detect one or more touches, each having a touch position within the range of one or more of the displayed views. The software application further includes instructions for processing individual touches of the one or more touches. The instructions for processing the individual touches include an instruction for executing the delegate corresponding to the individual gesture recognizer to acquire the received touch value according to the application state, and the received touch value is predetermined. Includes instructions for processing the individual touches in the individual gesture recognizer if the criteria are met. The instructions for processing the individual touches also include instructions for conditionally transmitting information corresponding to the individual touches to the software application. Further, the software application includes an instruction for executing the software application according to the information received from the individual gesture recognizer corresponding to the individual touch.</p><p num="0009"> According to some embodiments, the computer-readable storage medium stores one or more programs for execution by one or more processors of an electronic device having a touch sensor surface. The one or more programs include a software application that includes a plurality of views and application states of the software application. The software application includes instructions for displaying one or more of the plurality of views. The individual views of one or more of the displayed views include one or more individual gesture recognizers. Each gesture recognizer has a corresponding delegate. The software application also includes instructions on the touch sensor surface to detect one or more touches, each having a touch position within the range of one or more of the displayed views. The software application further includes instructions for processing individual touches of the one or more touches. The instructions for processing the individual touches include an instruction for executing the delegate corresponding to the individual gesture recognizer to acquire the received touch value according to the application state, and the received touch value is predetermined. Includes instructions for processing the individual touches in the individual gesture recognizer if the criteria are met. The instructions for processing the individual touches also include instructions for conditionally transmitting information corresponding to the individual touches to the software application. Further, the software application includes an instruction for executing the software application according to the information received from the individual gesture recognizer corresponding to the individual touch.</p><p num="0010"> According to some embodiments, the method is performed on an electronic device that has a touch sensor surface and is configured to run a software application. The method comprises displaying one or more views of the software application. The displayed one or more views include multiple gesture recognizers. The plurality of gesture recognizers include at least one discrete gesture recognizer and at least one continuous gesture recognizer. The discrete gesture recognizer is configured to send a single action message in response to individual gestures, and the continuous gesture recognizer sends action messages in recognized successive sub-events of the recognized individual gestures. Configured to send. The method also includes a step of detecting one or more touches and a step of processing each of the touches using one or more of the gesture recognizers. The steps of processing the individual touches include processing the individual touches in the individual gesture recognizer according to the individual gesture definition corresponding to the individual gesture recognizer, and the individual touches in the individual gesture recognizer. The method further comprises one of the gesture recognizers corresponding to one or more of the touches, comprising the step of conditionally transmitting one or more individual action messages to the software application according to the result of the processing of. Including the step of executing the software application according to one or more operation messages received from the above.</p><p num="0011"> According to some embodiments, the electronic device is a touch sensor surface, one or more processors, a memory, and one stored in the memory and configured to be executed by the one or more processors. Including the above programs. The one or more programs include software applications. The software application includes instructions for displaying one or more views of the software application. The displayed one or more views include multiple gesture recognizers. The plurality of gesture recognizers include at least one discrete gesture recognizer and at least one continuous gesture recognizer. At least one discrete gesture recognizer is configured to send a single action message in response to individual gestures, and at least one continuous gesture recognizer is a recognized contiguous subevent of a recognized individual gesture. Is configured to send an action message at. The software application also includes instructions for detecting one or more touches and instructions for processing each of the touches using one or more of the gesture recognizers. The instructions for processing individual touches are the instruction for processing individual touches in the individual gesture recognizer according to the individual gesture definition corresponding to the individual gesture recognizer, and the instructions in the individual gesture recognizer. Includes instructions for conditionally sending one or more individual action messages to the software application according to the result of processing individual touches. The software application further includes instructions for executing the software application according to one or more action messages received from one or more of the gesture recognizers corresponding to one or more of the touches.</p><p num="0012"> According to some embodiments, the computer-readable storage medium stores one or more programs for execution by one or more processors of an electronic device having a touch sensor surface. The one or more programs include software applications. The software application includes instructions for displaying one or more views of the software application. The displayed one or more views include multiple gesture recognizers. The plurality of gesture recognizers include at least one discrete gesture recognizer and at least one continuous gesture recognizer. At least one discrete gesture recognizer is configured to send a single action message in response to individual gestures, and at least one continuous gesture recognizer is a recognized contiguous subevent of a recognized individual gesture. Is configured to send an action message at. The software application also includes instructions for detecting one or more touches and instructions for processing each of the touches using one or more of the gesture recognizers. The instructions for processing the individual touches are the instruction for processing the individual touches in the individual gesture recognizer according to the individual gesture definition corresponding to the individual gesture recognizer, and the instructions in the individual gesture recognizer. Includes instructions for conditionally sending one or more individual action messages to the software application according to the result of processing the individual touches. The software application further includes instructions for executing the software application according to one or more action messages received from one or more of the gesture recognizers corresponding to one or more of the touches.</p><p num="0013"> According to some embodiments, an electronic device comprises one or more event detection units configured to detect one or more events and a processing unit coupled to the one or more event detection units. Including. The processing unit is configured to execute a software application that includes a plurality of views and application states of the software application, allowing the display of one or more of the plurality of views. The individual views of one or more of the displayed views include one or more individual gesture recognizers, each of which has a corresponding delegate. The processing unit detects one or more events by the one or more event detection units, and processes the individual events of the one or more events by using the individual gesture recognizer. The individual gesture recognizer processes the individual event according to the individual gesture definition corresponding to the gesture recognizer, executes the corresponding delegate to determine one or more values according to the application state, and said. Conditionally transmit information corresponding to the individual event to the software application according to the result of processing the individual event by the individual gesture recognizer and according to the one or more values determined by the corresponding delegate. It is configured to do by doing. The processing unit is configured to execute the software application according to the information received from the individual gesture recognizer corresponding to the individual event.</p><p num="0014"> According to some embodiments, the electronic device includes a touch sensor surface unit configured to receive one or more touches and a processing unit coupled to the touch sensor surface unit. The processing unit is configured to execute a software application that includes a plurality of views and application states of the software application, allowing the display of one or more of the plurality of views. The individual views of one or more of the displayed views include one or more individual gesture recognizers, each of which has a corresponding delegate. Each of the processing units has a touch position within the range of one or more of the displayed views and is configured to detect one or more touches received on the touch sensor surface. To. The processing unit processes the individual touches of the one or more touches by executing the delegate corresponding to the individual gesture recognizer to obtain a received touch value according to the application state and receiving the reception. When the touch value meets a predetermined criterion, the individual gesture recognizer processes the individual touch, and the information corresponding to the individual touch is conditionally transmitted to the software application. Will be done. The processing unit is configured to execute the software application according to the information received from the individual gesture recognizer corresponding to the individual touch.</p><p num="0015"> According to some embodiments, the electronic device includes a touch sensor surface unit configured to receive one or more touches and a processing unit coupled to the touch sensor surface unit. The processing unit is configured to run a software application and allow the display of one or more views of the software application. The displayed one or more views include a plurality of gesture recognizers, the plurality of gesture recognizers including at least one discrete gesture recognizer and at least one continuous gesture recognizer. At least one discrete gesture recognizer is configured to send a single action message in response to individual gestures, and at least one continuous gesture recognizer is a recognized contiguous subevent of the recognized individual gesture. Is configured to send an action message at. The processing unit is configured to detect one or more touches received by the touch sensor surface unit and process each of the touches using one or more of the gesture recognizers. .. The processing unit processes the individual touches in the individual gesture recognizer according to the individual gesture definition corresponding to the individual gesture recognizer, and the individual in the individual gesture recognizer. It is configured to conditionally send one or more individual action messages to the software application according to the result of the touch processing. The processing unit is configured to execute the software application according to one or more action messages received from one or more of the gesture recognizers corresponding to one or more of the touches.</p>
<figref num="1A">、</figref><figref num="1B">It is a block diagram which shows the electronic device which concerns on some embodiments.</figref><figref num="2">It is a figure of the input / output processing stack of the exemplary electronic device which concerns on some embodiments.</figref><figref num="3A">It is a figure which shows the exemplary view hierarchy which concerns on some embodiments.</figref><figref num="3B">It is a block diagram which shows an exemplary component for event processing which concerns on some embodiments.</figref><figref num="3C">It is a block diagram which shows the exemplary class and instance of the gesture recognizer which concerns on some embodiments.</figref><figref num="4A">、</figref><figref num="4B">、</figref><figref num="4C">、</figref><figref num="4D">It is a flowchart which shows the exemplary state transition machine which concerns on some embodiments.</figref><figref num="5A">It is a block diagram which shows the flow of the event information which concerns on some embodiments.</figref><figref num="5B">、</figref><figref num="5C">It is a high-level flowchart which shows the gesture recognition method which concerns on some embodiments.</figref><figref num="6A">、</figref><figref num="6B">It is a flowchart which shows the exemplary method of processing an individual event according to the information acquired from the delegate which concerns on some embodiments.</figref><figref num="7A">、</figref><figref num="7B">FIG. 5 is a flow chart illustrating an exemplary method of processing individual touches according to received touch values obtained from delegates according to some embodiments.</figref><figref num="8A">、</figref><figref num="8B">FIG. 5 is a flow chart illustrating an exemplary method of processing individual touches in a software application including a discrete gesture recognizer and a continuous gesture recognizer according to some embodiments.</figref><figref num="9">It is a functional block diagram of the electronic device which concerns on some embodiments.</figref><figref num="10">It is a functional block diagram of the electronic device which concerns on some embodiments.</figref><figref num="11">It is a functional block diagram of the electronic device which concerns on some embodiments.</figref>
Throughout the drawings, similar reference numerals indicate corresponding parts.
Hereinafter, embodiments will be referred to in detail. An example of the embodiment is shown in the attached drawing. In the following detailed description, many specific details are described in order to provide a complete understanding of the present invention. However, it will be apparent to those skilled in the art that the invention may be practiced without these particular details. In other examples, well-known methods, procedures, components, circuits and networks are not described in detail so as not to unnecessarily obscure aspects of the embodiments.
Although terms such as first and second are used herein to describe the various elements, it will be understood that these elements should not be limited by these terms. These terms are used only to distinguish one element from another. For example, without departing from the scope of the present invention, the first contact may be referred to as the second contact, and similarly, the second contact may be referred to as the first contact. The first contact and the second contact are both contacts, but they are not the same contact.
The terminology used herein to describe the invention is intended to describe a particular embodiment and is not intended to limit the invention. As used in the description of the invention and the appended claims, the singular forms "a", "an" and "the" are intended to include the plural unless otherwise stated in the context. It is understood that the term "and / or" as used herein refers to and includes any and all possible combinations of one or more of the listed related items. right. The terms "equipped" and / or "equipped", as used herein, identify the presence of defined features, numbers, steps, actions, elements and / or components, but one. It will be further understood that the existence or addition of these other features, numbers, steps, actions, elements, components and / or groups thereof is not excluded.
As used herein, the term "if" means "in response" or "sometimes" or "in response to a judgment" or "in response to a detection," depending on the circumstances. Is interpreted to mean. Similarly, the phrase "if determined" or "if [specified state or event] is detected" may be "determined" or "in response to the determination" or, depending on the situation. It is interpreted to mean "when (specified state or event) is detected" or "in response to the detection of (specified state or event)".
As used herein, the term "event" refers to an input detected by one or more sensors on the device. In particular, the term "event" includes touch on the touch sensor surface. An event contains one or more sub-events. Sub-events typically indicate changes to the event (eg, touch touch-down, touch move, and lift-off can be sub-events). Sub-events in a sequence of one or more sub-events are, among other things, key press, key press hold, key press release, button press, button press hold, button press release, joystick move, mouse move, mouse button press, mouse button release, It includes many forms including, but not limited to, penstylas touch, penstylas movement, penstylas release, voice commands, detected eye movements, biometric inputs and detected user physiological changes. Since an event can include a single subevent (eg, a short lateral movement of the device), the term "subevent" as used herein also refers to an event.
As used herein, the terms "event recognizer" and "gesture recognizer" are used interchangeably to indicate a recognizer that can recognize gestures or other events (eg, device movements). ..
As mentioned above, in some devices that have a touch sensor surface as an input device, the first set of touch-based gestures (eg, two or more taps, double taps, horizontal swipes, vertical swipes) is specific. Recognized as a suitable input in a situation (eg, a particular mode of the first application), other different sets of touch-based gestures are in other situations (eg, different applications and / or different modes within the first application). Or the situation) is recognized as an appropriate input. In addition, two or more appropriate inputs (or gestures) can interfere with or conflict with each other (eg, after detecting a single tap, recognize the single tap as a complete single tap gesture or double tap. You need to decide if you want to recognize it as part of the gesture). As a result, the software and logic required to recognize and respond to touch-based gestures can be complex, each time an application is updated or a new application is added to a computing device. May need modification.
When using touch-based gestures to control an application running on a device that has a touch sensor surface, the touch has both temporal and spatial aspects. A temporal aspect, called a phase, indicates when the touch started, whether it was moving or stationary, and when it ended, that is, when the finger was lifted off the screen. A spatial aspect of a touch is a collection of views or user interface windows from which the touch occurred. The view or window in which the touch is detected corresponds to the program level of the view hierarchy. For example, the lowest level view where a touch is detected is called a hit view, and the set of events that are recognized as appropriate input is at least part of the hit view of the first touch that initiates a touch-based gesture. Can be determined based on the target.
1A and 1B are block diagrams showing various embodiments of electronic devices 102 and 104 according to some embodiments. Electronic device 102 or 104 is any electronic device including, but not limited to, desktop computer systems, laptop computer systems, netbook computer systems, mobile phones, smartphones, personal digital assistants or navigation systems. May be good. The electronic device is also a portable electronic device having a touch screen display configured to present the user interface (eg, touch sensor display 156 in FIG. 1B), a touch screen configured to present the user interface. Computers including displays, computers with touch sensor surfaces and displays configured to present a user interface, or consumer electronic devices, mobile phones, video game systems, electronic music players, tablet PCs, electronic book reading systems, Any other form of computing device including, but not limited to, e-books, PDAs, e-organizers, e-mail devices, laptops, netbooks or other computers, kiosk computers, vending machines, smart devices, etc. May be good. Electronic device 102 or 104 includes user interface 113.
In some embodiments, the electronic device 104 includes a touch screen display. In these embodiments, the user interface 113 may include an on-screen keyboard (not shown) used by the user to interact with the electronic devices 102 and 104. Alternatively, the keyboard may be separate and separate from the electronic device 104 (or electronic device 102). For example, the keyboard can be a wired or wireless keyboard coupled to the electronic device 102 or 104.
In some embodiments, the electronic device 102 is a display 126 and one or more input devices 128 (eg, keyboard, mouse, trackball, microphone, physical buttons (groups), touchpad) coupled to the electronic device 102. Etc.) and include. In these embodiments, one or more of the input devices 128 may be separate and optionally separate from the electronic device 102. For example, one or more input devices may include one or more of a keyboard, mouse, trackpad, trackball and electronic pen that may optionally be separated from the electronic device. Optionally, the device 102 or 104 may include one or more accelerometers, a gyroscope, a GPS system, a speaker, an infrared (IR) sensor, a biometric sensor, a camera, and the like. It should be noted that the above description of the various exemplary devices, the input device 128 or the sensor 130, is not important to the operation of the embodiments described herein and is arbitrary described herein as an input device. An input device or sensor device is similarly well described as a sensor, and any input device or sensor device described as a sensor is similarly well described as an input device. In some embodiments, the signals generated by one or more sensors 130 are used as an input source to detect events.
In some embodiments, the electronic device 104 includes a touch sensor display 156 (ie, a display having a touch sensor surface) coupled to the electronic device 104 and one or more input devices 128. In some embodiments, the touch sensor display 156 has the ability to detect two or more separate simultaneous (or partially simultaneous) touches, and in these embodiments, the display 156 is described herein. It may also be referred to as a multi-touch display or multi-touch sensor display in the book.
In some embodiments of the electronic device 102 or 104 described herein, the input device 128 is located at the electronic device 102 or 104. In other embodiments, one or more of the input devices 128 are separate and separate from the electronic device 102 or 104, eg, one or more of the input devices 128 are cables (eg, USB cables). Alternatively, it can be coupled to electronic device 102 or 104 via a wireless connection (eg, a Bluetooth® connection).
When using input device 128 or performing touch-based gestures on the touch sensor display 156 of electronic device 104, the user is a subevent processed by one or more CPU 110s of electronic device 102 or 104. Generate a sequence of. In some embodiments, one or more CPUs 110 of the electronic device 102 or 104 process a sequence of sub-events to recognize the event.
Electronic devices 102 or 104 typically include one or more single-core or multi-core processing units (CPU or CPU groups) 110, as well as one or more networks or other communication interfaces 112, respectively. Including. The electronic device 102 or 104 includes memory 111 and one or more communication buses 115 for interconnecting components, respectively. The communication bus 115 includes a circuit configuration (sometimes referred to as a chipset) that interconnects system components (not shown herein) and controls communication between them. As briefly described above, electronic device 102 or 104 includes a user interface 113 that includes a display (eg, display 126 or touch sensor display 156). Further, the electronic device 102 or 104 typically includes an input device 128 (eg, keyboard, mouse, touch sensor surface, keypad, etc.). In some embodiments, the input device 128 includes an on-screen input device (eg, the touch sensor surface of the display device). Memory 111 may include fast random access memory such as DRAM, SRAM, DDRRAM or other random access solid memory device, one or more magnetic disk storage devices, optical disk storage devices, flash memory devices or other non-volatile solids. It may include a non-volatile memory such as a storage device. Memory 111 may optionally include one or more storage devices located away from CPU (group) 110. The non-volatile memory device (s) within the memory 111 or in its place the memory 111 includes a computer-readable storage medium. In some embodiments, the memory 111 (of the electronic device 102 or 104) or the computer-readable storage medium of the memory 111 stores the following programs, modules and data structures, or subsets thereof. Operating system 118 that contains procedures that handle various basic system services and perform hardware-dependent tasks. Electronic devices 102 or 104 via one or more individual communication interfaces 112 (wired or wireless) and one or more communication networks such as the Internet, other wide area networks, local area networks and metropolitan area networks. Communication module 120 used to connect to other devices respectively. User interface module 122 used to display a user interface, including user interface objects, on display 126 or touch sensor display 156. A control application 124 used to control processing (eg, hit view determination, thread management and / or event monitoring, etc.). In some embodiments, the control application 124 includes a run-time application. In other embodiments, the runtime application includes a control application 124. Event delivery system 130 that can be implemented in various other embodiments within operating system 118 or in application software 132. However, in some embodiments, some aspects of the event delivery system 130 are implemented in operating system 118, and other aspects (eg, at least a subset of event handlers) are implemented in application software 132. Application software 132 that includes one or more software applications (eg, email applications, web browser applications, text messaging applications, etc.). Each software application typically has an application state that indicates the state of the software application and its components (eg, gesture recognizers and delegates) at least at run time. See application internal state 317 (Figure 3B) described below. Application state showing the state of software applications and their components (eg gesture recognizers and delegates) and display showing applications, views or other information that occupy various areas of the touch sensor display 156 or display 126. One or more of the state and the sensor state, including information obtained from the various sensors 130 of the device, the input device 128 and / or the touch sensor display 156, and the position information regarding the position and / or orientation of the device. Includes device / global internal state 134.
Each of the above identified elements may be stored in one or more of the memory devices described above and corresponds to a set of instructions that perform the functions described herein. The set of instructions can be executed by one or more processors (eg, one or more CPU 110s). The different subsets of these modules combine in different embodiments, as the above identified modules or programs (ie, sets of instructions) do not need to be implemented as separate software programs, procedures or modules. Can be or can be reconstructed like any other. In some embodiments, memory 111 may store a subset of the modules and data structures identified above. In addition, memory 111 may store additional modules and data structures not described above.
FIG. 2 is a diagram of an input / output processing stack 200 of an exemplary electronic device or device (eg, device 102 or 104) according to some embodiments of the present invention. The device hardware (eg, electronic circuit configuration) 212 is at the base level of the I / O processing stack 200. Hardware 212 includes various hardware interface components such as the components shown in FIGS. 1A and / or 1B. Hardware 212 may also include one or more of the sensors 130 described above. All other elements of the I / O processing stack 200 (132, 204-210) process the input received from the hardware 212 and are one of the hardware user interfaces (eg, display, speaker, device vibration actuator). It is a software procedure or part of a software procedure that produces various outputs presented via).
The driver or set of drivers 210 communicates with hardware 212. The driver 210 receives the input data from the hardware 212 and processes the received input data. The core operating system (OS) 208 can communicate with the driver (s) 210. The core OS 208 processes the raw input data received from the driver (group) 210. In some embodiments, the driver 210 may be considered part of the core OS 208.
The set 206 of OS application programming interfaces (OS APIs) is a software procedure that communicates with the core OS 208. In some embodiments, API 206 is included in the device's operating system, but at a higher level than core OS 208. API206 is designed to be used by applications running on electronic devices or devices described herein. The user interface (UI) API 204 can use OS API 206. Application software (application) 132 running on the device can use UI API 204 to communicate with the user. The UI API 204 then communicates with lower level elements and finally with various user interface hardware, such as the multi-touch display 156.
Each layer of the I / O processing stack 200 can use layers below it, but this is not always required. For example, in some embodiments, application 132 may communicate with OS API 206. Generally, layers above OS API layer 206 are considered private, so there is no need to directly access core OS 208, driver (group) 210 or hardware 212. Applications 132 and UI API 204 within the tier typically call OS API 206 directly, which in turn accesses the core OS 208, driver (group) 210, and hardware 212 tiers.
Alternatively, one or more hardware elements 212 of the electronic device 102 or 104 and, for example, a driver (group) 210 (shown in FIG. 2), a core OS (operating system) 208 (shown in FIG. 2), Software that runs on a device, such as operating system API software 206 (shown in Figure 2) and application user interface API software 204 (shown in Figure 2), is input device (group) 128 and / or touch sensor display. Currently to detect input events (which can correspond to gesture sub-events) in one or more of 156 and determine when and when input events correspond to events delivered to application 132. Generates or updates various data structures (stored in the memory of device 102 or 104) used by a set of active event recognizers. The method of event recognition, the apparatus, and the embodiment of the computer program will be described in more detail below.
FIG. 3A shows an exemplary view hierarchy 300, which is the search program displayed in the outermost view 302 in this example. The outermost view 302 generally includes the entire user interface with which the user can interact directly, including dependent views such as: Search results panel 304 that groups search results and can be scrolled vertically. Search field 306 that accepts text input. Home column 310 to group applications for quick access.
In this example, each dependent view contains a lower level dependent view. In another example, the number of view levels in hierarchy 300 may vary at different branches of the hierarchy, with one or more dependent views having lower level dependent views and one or more other dependent views. Has no such lower level dependent views. Continuing with reference to the example shown in FIG. 3A, the search results panel 304 includes dependent views 305 (subordinate to panel 304) separated by search results. As used herein, this example shows one search result in a dependent view called map view 305. The search field 306 includes a dependent view, referred to herein as the content clear icon view 307, which allows the user to perform a particular action (eg, single-touch gesture or single) on the content clear icon in view 307. Clears the contents of the search field when performing a tap gesture). Home column 310 includes dependent views 310-1, 310-2, 310-3 and 310-4 corresponding to the contacts application, email application, web browser and iPod® music interface, respectively.
The touch subevent 301-1 is represented in the outermost view 302. Assuming that the location of the touch subevent 301-1 is on both the search results panel 304 and the map view 305, the touch subevents will also be on these views as 301-2 and 301-3, respectively. Be expressed. Actively involved views of touch subevents include view search results panel 304, map view 305, and outermost view 302. With reference to Figures 3B and 3C, additional information regarding sub-event delivery and active involvement views is provided below.
Views (and corresponding program levels) can be nested. In other words, a view can include other views. Thus, a software element (s) associated with a first view (eg, an event recognizer) may include or be linked to one or more software elements associated with a view within the first view. Some views are related to applications, while others are related to higher level OS elements such as graphical user interfaces and window managers.
FIG. 3B is a block diagram showing an exemplary component for event processing according to some embodiments (eg, event processing component 390). In some embodiments, memory 111 (in FIGS. 1A and 1B) includes an event recognizer global method 311 (eg, in operating system 118) and a separate application 132-1.
In some embodiments, the event recognizer global method 311 includes an event monitor 312, a hit view determination module 313, an active event recognizer determination module 314, and an event dispatcher module 315. In some embodiments, the event recognizer global method 311 is located within the event delivery system 130. In some embodiments, the event recognizer global method 311 is implemented in operating system 118. Instead, the event recognizer global method 311 is implemented in application 132-1. In yet another embodiment, the event recognizer global method 311 is implemented as part of a stand-alone module or another module stored in memory 111 (eg, a contact / operation module (not shown)).
The event monitor 312 receives event information from the sensor 130, the touch sensor display 156 and / or the input device 128. Event information includes events (eg, user touches on the touch sensor display 156 that are part of a multi-touch gesture or movement of device 102 or 104) and / or sub-events (eg, touches across the touch sensor display 156). Contains information about (movement). For example, event information about a touch event includes one or more of the touch position and time stamp. Similarly, event information about a swipe event includes two or more of the swipe position, time stamp, direction and speed. The sensor 130, the touch sensor display 156 and the input device 128 transmit event and sub-event information to the event monitor 312 directly or via a peripheral interface, which reads and stores the event information. The sensor 130 includes one or more of a proximity sensor, an accelerometer (group), a gyroscope, a microphone and a video camera. In some embodiments, the sensor 130 also includes an input device 128 and / or a touch sensor display 156.
In some embodiments, the event monitor 312 sends a request to the sensor 130 and / or peripheral interfaces at predetermined intervals. In response, the sensor 130 and / or the peripheral interface sends event information. In other embodiments, the sensor 130 and the peripheral interface transmit event information only if there is a critical event (eg, reception of input for a period that exceeds a predetermined noise threshold and / or a period longer than a predetermined period). ..
The event monitor 312 receives the event information and determines the application view 316.2 of the application 132-1 and the application 132-1 to which the event information is distributed.
In some embodiments, the event recognizer global method 311 also includes a hit view determination module 313 and / or an active event recognizer determination module 314.
The hit view determination module 313 provides a software procedure for determining the position in one or more views where an event or subevent has occurred when the touch sensor display 156 displays two or more views. To do. The view consists of controls and other elements that the user sees on the display.
Another aspect of the user interface associated with application 132-1 is set view 316, which is sometimes referred to herein as an application view or user interface window, in which information is displayed and touched. A base gesture occurs. The application view (for each application) where the touch is detected corresponds to a particular view in the application's view hierarchy. For example, the lowest-level view in which a touch is detected may be called a hit view, and the set of events recognized as appropriate input is the hit view of the first touch that initiates a touch-based gesture. Can be determined on the basis of at least partly.
The hit view determination module 313 receives information related to the event and / or the sub-event. If the application has multiple views organized in a hierarchy, the hit view determination module 313 identifies the hit view as the lowest view in the hierarchy to handle the event or subevent. In most situations, a hit view is the lowest level view at which a start event or subevent (ie, the first event or subevent in a sequence of events and / or subevents that form a gesture) occurs. When a hit view is identified by the hit view determination module, the hit view typically receives all events and / or subevents for the same touch or input source identified as the hit view.
The active event recognizer determination module 314 determines one or more views in the view hierarchy that should receive a particular sequence of events and / or subevents. In some application situations, the active event recognizer determination module 314 determines that only the hit view should receive a particular sequence of events and / or subevents. In other application situations, the active event recognizer determination module 314 determines that all views, including the physical location of an event or subevent, are active involvement views and therefore all active involvement views. Determines that a particular sequence of events and / or subevents should be received. In other applications, even though touch events and / or sub-events are generally restricted to the area associated with one particular view, the higher-level views in the hierarchy still exist as active involvement views. ..
The event dispatcher module 315 distributes event information to an event recognizer (also referred to herein as a "gesture recognizer") (eg, an event recognizer 320-1). In an embodiment including the active event recognizer determination module 314, the event dispatcher module 315 delivers event information to the event recognizer determined by the active event recognizer determination module 314. In some embodiments, the event dispatcher module 315 stores the event information in the event queue, and the event information is read by the individual event recognizer 320 (or the event receiver 331 of the individual event recognizer 320).
In some embodiments, application 132-1 includes an application internal state 317 showing the current application view (s) displayed on the touch sensor display 156 when the application is active or running. .. In some embodiments, device / global internal state 134 is used by the event recognizer global method 311 to determine the currently active application (s), and application internal state 317 delivers event information. Used by event recognizer global method 311 to determine the previous application view 316.
In some embodiments, the application internal state 317 indicates or is ready to display the resume information used when application 132-1 resumes execution and the information being displayed by application 132-1. Of the user interface state information that indicates the information that is in, the state queue that allows the user to return to the previous state or view of application 132-1, and the redo / undo queue of the previous action performed by the user. Contains additional information such as one or more of (eg, 344 in Figure 3C). In some embodiments, the application internal state 317 further includes contextual information / text and metadata 318.
In some embodiments, application 132-1 includes one or more application views 316, where each application view is an instruction to handle touch events that occur within a separate view of the application's user interface. including. At least one application view 316 of application 132-1 contains one or more event recognizers 320 and one or more event handlers 322. Typically, a separate application view 316 includes a plurality of event recognizers 320 and a plurality of event handlers 322. In other embodiments, one or more of the event recognizers 320 are isolated, such as user interface kits (not shown) or higher level objects to which application 132-1 inherits methods and other attributes. It is part of the module. In some embodiments, the individual application view 316 also includes one or more of a data update unit, an object update unit, a GUI update unit, and / or an event data receiver.
The individual event recognizer 320-1 receives the event information from the event dispatcher module 315 and identifies the event from the event information. The event recognizer 320-1 includes an event receiving unit 331 and an event comparing unit 332.
Event information includes information about an event (eg, touch) or a sub-event (eg, touch movement). Depending on the event or sub-event, the event information also includes additional information such as the location of the event or sub-event. If the event or sub-event is related to the movement of the touch, the event information may also include the speed and direction of the sub-event. In some embodiments, the event involves rotation of the device from one direction to another (eg, from portrait to landscape or vice versa), and event information is also referred to as the device's current (also called device orientation). ) Contains the corresponding information about the direction.
The event comparison unit 332 compares the event information with one or more predetermined gesture definitions (also referred to herein as "event definitions") and determines an event or subevent based on the comparison, or an event. Or, determine or update the status of the sub-event. In some embodiments, the event comparison unit 332 includes one or more gesture definitions 333 (also referred to herein as "event definitions" as described above). Gesture definition 333 includes definitions of gestures (eg, a given sequence of events and / or subevents) such as gesture 1 (334-1) and gesture 2 (334-2). In some embodiments, the gesture definition 333 subevents include, for example, touch start, touch end, touch move, touch cancel and multiple touches. In one example, the definition of gesture 1 (334-1) is a double tap on the displayed object. Double tapping is, for example, the first touch (touch start) to the display object for a given phase of the gesture, the first lift-off (touch end) for the next given phase of the gesture, and the subsequent given phase of the gesture. Includes a second touch to the display object (touch start) and a second lift-off (touch end) for the last predetermined phase of the gesture. In another example, the definition of Gesture 2 (334-2) is the dragging of a display object. For example, dragging includes touching (or touching) a display object, moving the touch across the touch sensor display 156, and lifting off the touch (end of touch).
In some embodiments, the event recognizer 320-1 also includes information for event delivery 335. The information for event delivery 335 includes a reference to the corresponding event handler 322. Optionally, the information for event delivery 335 includes behavior-target pairs (groups). In some embodiments, in response to the recognition of a gesture (or part of a gesture), event information (eg, an action message (group)) is one or more objects identified by an action-target pair (group). Will be sent to. In other embodiments, the action-target pair is activated in response to the recognition of the gesture (or part of the gesture).
In some embodiments, the gesture definition 333 includes a gesture definition for an individual user interface object. In some embodiments, the event comparison unit 332 performs a hit test to determine which user interface object is associated with the subevent. For example, in an application view where three user interface objects are displayed on the touch sensor display 156, if a touch is detected on the touch sensor display 156, the event comparison unit 332 will have three, if any. Run a hit test to determine which of the user interface objects is associated with a touch (event). If each display object is associated with a separate event handler 322, the event comparison unit 332 uses the results of the hit test to determine if that event handler 322 should be invoked. For example, the event comparison unit 332 selects the event handler 322 associated with the event and object that initiates the hit test.
In some embodiments, the definition of individual gesture 333 delays the delivery of event information until it is determined whether the sequence of events and / or sub-events corresponds to the event type of the event recognizer. Including operation.
If the individual event recognizer 320-1 determines that the series of events and / or subevents do not match any of the events in gesture definition 333, the individual event recognizer 320-1 goes into an event failure state. The individual event recognizer 320-1 then ignores subsequent events and / or subevents of the touch-based gesture. In this situation, if there are other event recognizers that remain active for the hit view, these event recognizers will track and process the events and / or subevents of the running touch-based gesture. Continue to do.
In some embodiments, the individual event recognizer 320-1 includes an event recognizer state 336. Event recognizer state 336 includes the state of individual event recognizer 320-1. An example of the event recognizer state will be described in more detail below with reference to FIGS. 4A-4D.
In some embodiments, event recognizer state 336 includes recognizer metadata and attribute 337-1. In some embodiments, the recognizer metadata and attribute 337-1 can be configured to indicate A) how the event delivery system should perform event and / or subevent delivery to the actively involved event recognizer. Attributes, flags and / or lists, B) Configurable attributes, flags and / or lists that indicate how the event recognizers interact with each other, C) How the event recognizers receive event information. Configurable attributes, flags and / or lists, D) configurable attributes, flags and / or lists, E) events and / or how the event recognizer may recognize gestures. Includes configurable attributes, flags and / or lists indicating whether subevents are delivered to different levels of the view hierarchy, F) one or more of references to the corresponding event handler 322.
In some embodiments, the event recognizer state 336 includes event / touch metadata 337-2. Event / touch metadata 337-2 contains event / touch information about the individual event / touch detected and corresponds to gesture definition 333. Event / touch information includes one or more of individual event / touch positions, timestamps, velocities, directions, distances, magnifications (or changes in magnification) and angles (or changes in angles).
In some embodiments, the individual application views include one or more delegates 321. The individual delegate 321 is assigned to the individual event recognizer 320. Instead, the individual event recognizer 320 has a corresponding delegate 321 but the delegate 321 does not necessarily have to be assigned to the individual recognizer 320 at run time, and the delegate to the event recognizer is established before the application runs. (For example, the delegate to the event recognizer may be indicated by the delegate attribute of the application view and may be established when the corresponding application view 316 is initialized). In some embodiments, some event recognizers do not have an assigned (or corresponding) delegate. Event recognizers that do not have a corresponding delegate run according to default rules, such as the default rule that governs event-aware exclusivity. In some embodiments, some event recognizers have multiple assigned (or corresponding) delegates. Delegate can also be used to modify the behavior of the corresponding event recognizer and coordinate the behavior of multiple event recognizers. In some embodiments described below, delegates modify multiple aspects of the behavior of individual event recognizers when assigned to individual event recognizers.
In some embodiments, the individual event recognizer 320 provides an event handler 322 associated with the individual event recognizer 320 when one or more specific events and / or subevents of the gesture are recognized. to start. In some embodiments, the individual event recognizer 320 delivers event-related event information to the event handler 322.
Event handler 322, when invoked, creates and / or updates data, creates and updates objects, prepares display information, and displays that display information on display 126 or touch sensor display 156. Perform one or more of the transmissions.
In some embodiments, the individual application view 316-2 comprises view metadata 323. View metadata 323 contains data about the view. Optionally, view metadata includes the following attributes that affect the delivery of events and / or subevents to the event recognizer. Stop attribute 324-1 that prevents event and / or sub-event delivery to event recognizers related to the view as well as its ancestors in the view hierarchy when set for the view. Prevents event and / or sub-event delivery to the event recognizer associated with that view when set for a view, but does not deliver events and / or sub-events to its ancestors in the view hierarchy. Allow skip attribute 324-2. Non-hit skip attribute 324-3 that prevents delivery of events and / or subevents to the view-related event recognizer, unless the view is a hit view when configured for a view. As described above, the hit view determination module 313 identifies the hit view as the lowest view in the hierarchy in which the subevent should be processed. Other view metadata 324-4.
In some embodiments, the first active involvement view in the view hierarchy may be configured to prevent the delivery of individual subevents to the event recognizer associated with that first active involvement view. This behavior can realize the skip attribute 324-2. When the skip attribute is set for an application view, delivery of individual subevents is still performed for event recognizers associated with other actively involved views in the view hierarchy.
Instead, the first active involvement view in the view hierarchy delivers a separate subevent to the event recognizer associated with the first active involvement view, unless the first active involvement view is a hit view. Is configured to prevent. This behavior can realize the skip attribute 324-3 except for conditional hits.
In some embodiments, the second active involvement view in the view hierarchy is a separate event recognizer associated with the second active involvement view and an event recognizer associated with the ancestors of the second active involvement view. It is configured to prevent the delivery of sub-events. This behavior can realize the stop attribute 324-1.
The above description of user touch event handling on the touch sensor display also applies to other forms of user input for operating an electronic device 102 or 104 with an input device, all of which begins on the touch screen. Mouse movements and mouse button presses, with or without single or multiple keyboard presses or holdings, which are not, for example, can be used as inputs corresponding to events and / or subevents that define recognized gestures. Adjusting user movements, taps, drags, crawls, etc. on the touchpad, pen stylus inputs, device movements (eg, rotation), voice commands, detected eye movements, biometric inputs and / or any combination thereof. It is understood that this also applies to.
FIG. 3C is a block diagram showing exemplary classes and instances of a gesture recognizer (eg, event handling component 390) according to some embodiments.
A software application (eg, application 132-1) has one or more event recognizers 340. In some embodiments, the individual event recognizer (eg, 340-2) is an event recognizer class. A separate event recognizer (eg, 340-2) includes an event recognizer-specific code 341 (eg, a set of instructions that define the behavior of the event recognizer) and a state machine 342.
In some embodiments, application state 317 of a software application (eg, application 132-1) includes an instance of an event recognizer. Each instance of the event recognizer is an object that has a state (eg, event recognizer state 336). The "execution" of an individual event recognizer instance is achieved by executing the corresponding event recognizer-specific code (eg, 341) and updating or maintaining state 336 of the event recognizer instance 343. The state 336 of the event recognizer instance 343 includes the state 338 of the state machine 342 of the event recognizer instance.
In some embodiments, application state 317 includes multiple event recognizer instances 343, each corresponding to an event recognizer associated with a view of the application (also referred to as "attached"). In some embodiments, application state 317 includes multiple instances (eg, 343-1 to 343-L) of individual event recognizers (eg, 345-2). In some embodiments, application state 317 includes instances 343 of multiple event recognizers (eg, 340-1 to 340-R).
In some embodiments, a separate instance of the gesture recognizer 343-2 comprises an event recognizer state 336. As mentioned above, the event recognizer state 336 includes the recognizer metadata and the attribute 337-1 and the event / touch metadata 337-2. Event recognizer state 336 also includes a view hierarchy reference (group) 337-3 that indicates the view to which individual instances of the gesture recognizer 343-2 are attached.
In some embodiments, the recognizer metadata and attribute 337-1 include, or include a subset or superset of: Other active involvement views (except for any other event recognizer listed in Exception List 353) when the event recognizer recognizes a gesture when configured for the event recognizer. Exclusiveness flag 339 indicating that the delivery of events and / or subevents to any event recognizer should be stopped. Subsequent events and / or subevents are events in the exclusive state when a particular event recognizer becomes exclusive by receiving an event or subevent, as indicated by the corresponding exclusivity flag 339. Delivered only to the recognizer (as well as any other event recognizer listed in Exception List 353). Exclusivity exception list 353. When included in the event recognizer state 336 for an individual event recognizer, this list 353 shows that the event recognizer continues to receive events and / or subevents even after the individual event recognizer has entered the exclusive state. In some cases, a set of these recognizers is shown. For example, if the event recognizer for a single-tap event is in an exclusive state and the currently involved view contains an event recognizer for a double-tap event, Listing 353 doubles even after the single-tap event is detected. List the double-tap event recognizers so that the tap event is recognized. Therefore, exclusivity exception list 353 allows the event recognizer to recognize different gestures that share a common sequence of events and / or subevents. For example, single-tap gesture recognition does not preclude subsequent recognition of double or triple-tap gestures by other event recognizers. Waiting list 351. If included in the event recognizer state 336 for an individual event recognizer, this list 351 must be in the event failed or event canceled state before the individual event recognizer can recognize the individual event. Shows a set of these recognizers, if any. In essence, the listed event recognizers have a higher priority for recognizing events than event recognizers with waiting list 351. When configured for an event recognizer, the event recognizer (touch start subevent or finger) until it is determined that the sequence of events and / or subevents does not correspond to this event recognizer gesture type. Delayed touch initiated flag 352 that delays sending events and / or subevents (including down subevents and subsequent events) to the individual hit views of the event recognizer. This flag can be used to prevent the hit view from referencing any of the events and / or subevents when the gesture is recognized. If the event recognizer cannot recognize the gesture, the touch-initiated subevent (and subsequent touch-end subevents) may be delivered to the hit view. In one example, by delivering such a subevent to the hit view, the user interface can easily highlight the object without invoking actions related to the object. When configured for an event recognizer, the event recognizer keeps subevents (eg, end-of-touch subevents) until it determines that the sequence of subevents does not correspond to the type of event for this event recognizer. Delayed touch end flag 363 that delays sending to an individual hit view or level of the event recognizer. This can be used to prevent the hit view from acting on the end-of-touch subevent if the gesture is later recognized. Canceled touches may be sent to the hit view or level unless a touch end subevent is sent. If the event is recognized, the application performs the corresponding action and the end-of-touch subevent is delivered to the hit view or level. If configured for an event recognizer and it is determined that the sequence of events and / or sub-events does not correspond to this event recognizer gesture type, touch or cancel the input to the event recognizer. Touch cancel flag 364 to send to individual hit views of the event recognizer. Cancellation of a touch or input sent to the hit view indicates that the previous event and / or subevent (eg, touch-initiated subevent) has been cancelled. By canceling the touch or input, the event recognizer state enters the event canceled state 418 (Fig. 4B).
In some embodiments, one or more event recognizers may be adapted to delay delivering one or more subevents of a sequence of subevents until after the event recognizer recognizes the event. .. This behavior reflects delayed events. For example, consider a single-tap gesture for a view that allows multiple tap gestures. In this case, the tap event becomes a "tap + delay" recognizer. In essence, when the event recognizer implements this behavior, the event recognizer delays event recognition until it is certain that the sequence of subevents actually corresponds to the event definition. This behavior may be appropriate if the recipient view is unable to respond properly to the canceled event. In some embodiments, the event recognizer delays updating the event awareness state to a separate active involvement view until the event recognizer is confident that the sequence of subevents does not correspond to its event definition. Delayed touch started flag 352, delayed touch end flag 363 and touch cancel flag 364 are provided to tailor event recognizer and view state information updates to specific needs, as well as sub-event delivery technology. The flag.
In some embodiments, the recognizer metadata and attribute 337-1 include, or include a subset or superset of: State machine state / phase 338 that indicates the state of the state machine (eg, 342) for an individual event recognizer instance (eg, 343-2). The state / phase 338 of the state machine can have various state values such as "event possible", "event recognized", and "event failure" as described below. Alternatively or additionally, state machine state / phase 338 is "touch phase started" indicating that the touch data structure defines a new touch that is not referenced by the previous touch data structure. It can have various phase values such as. The Touch Phase Moved value indicates that the defined touch has moved from its previous position. A "touch phase stationary" value may indicate that the touch remains in the same position. A "touch phase finished" value can indicate that the touch has finished (eg, the user has lifted his or her finger off the surface of the multi-touch display). A Touch Phase Canceled value indicates that the touch has been canceled by the device . A canceled touch is a touch that is not necessarily terminated by the user, but the device has determined to ignore it. For example, the device determines that the touch was unintentionally generated (ie, the result of putting a portable multi-touch capable device in the pocket), so the touch can be ignored. The state / phase 338 values of the state machine may be integers (referred to herein as "gesture recognizer state values"). Actions in which individual event recognizer instances are identified in response to recognizing an event or touch as a gesture or part of a gesture Actions in which each pair identifies the target to which it sends a message-Target pairs (groups) ) 345. Delegate 346, which is a reference to the corresponding delegate when the delegate is assigned to an individual event recognizer instance. Delegate 346 contains a null value if the delegate is not assigned to a separate event recognizer instance. Valid attributes 347 that indicate whether individual event recognizer instances are enabled. In some embodiments, the individual event recognizer instance does not process the event or touch if the individual event recognizer instance is not valid (eg, invalid).
In some embodiments, exception list 353 can also be used by a non-exclusive event recognizer. Subsequent events and / or sub-events are listed in the event recognizer's exception list 353 that recognizes the event or sub-event, especially if the non-exclusive event recognizer recognizes the event or sub-event. Not delivered to exclusive event recognizers associated with the currently active view except the recognizer.
In some embodiments, the event recognizer will utilize the touch cancel flag 364 along with the delayed touch end flag 363 to prevent unwanted events and / or subevents from being delivered to the hit view. Can be configured in. For example, the definition of a single-tap gesture and the first half of a double-tap gesture are the same. If the single-tap event recognizer recognizes the single-tap normally, unnecessary actions can be performed. If the deferred touch end flag is set, the single-tap event recognizer will not send sub-events to the hit view until the single-tap event is recognized. Also, the single-tap event recognizer waiting list may identify the double-tap event recognizer, which causes the single-tap event recognizer to remain uneventable until the double-tap event recognizer becomes uneventable. Do not recognize single taps. By using the wait list, you avoid performing actions related to single taps when double tap gestures are performed. Instead, only the actions associated with the double tap are performed in response to the recognition of the double tap event.
With particular reference to the form of user touch on the touch sensor surface, as mentioned above, touch and user gestures may include actions that do not have to be instantaneous, for example touch moving a finger relative to the display for a period of time. Alternatively, it may include a holding action. However, the touch data structure defines the state of the touch at a particular time (ie, more generally the state of any input source). Therefore, the values stored in the touch data structure may change during a single touch, thereby communicating the single touch state at different times to the application.
Each touch data structure can contain various entries. In some embodiments, the touch data structure may include data that at least corresponds to a touch-specific entry in event / touch metadata 337-2, such as: .. First Touch to View entry 348, which indicates whether the touch data structure defines the first touch for a particular view (because the view was instantiated). "Information per touch" entry 349 that contains "timestamp" information that indicates the specific time (for example, the time of the touch) to which the touch data structure is associated. Optionally, the "per-touch information" entry 349 includes other information such as the position of the corresponding touch. Optional tap count entry 350 that indicates the number of consecutive taps performed at the position of the first touch. A tap can be defined as a quick finger press and lift to a touch-sensitive panel at a particular position. Multiple consecutive taps can occur if the finger is quickly and continuously pressed and released again at the same position on the panel. The event delivery system 130 can count taps and relay this information to the application via a "tap count" entry 350. Multiple taps at the same location may be considered useful and easy-to-remember commands for touch-enabled interfaces. Therefore, by counting the taps, the event delivery system 130 can reduce some data processing from the application again.
Thus, each touch data structure is not only generated by a separate touch (or other input source) at a particular time (eg, whether the touch is stationary or moving, etc.), but also the touch. Other information related to (location, etc.) can be defined. Therefore, each touch data structure can define a particular touch state at a particular moment. One or more touch data structures that reference the same time can be added to the touch event data structure that can define the state of all touches that a particular view is receiving at the moment (as mentioned above). Some touch data structures may also refer to touches that have been terminated and are no longer received). Multiple touch event data structures may be sent to the software that implements the view over time in order to provide the software with contiguous information that describes the touches that are occurring in the view.
The ability to handle complex touch-based gestures, including optionally multi-touch gestures, can further complicate various software applications. In some examples, such additional complexity may be necessary to achieve a high degree of desired interface functionality. For example, games often require pressing multiple buttons at the same time, or combining accelerometer data with touches on the touch sensor surface, thus handling multiple simultaneous touches that occur in different views. May need the ability to do. However, some simpler applications and / or views do not need to require advanced interface features. For example, a simple soft button (ie, a button displayed on a touch sensor display) may work well with a single touch rather than a multi-touch feature. In these examples, the underlying OS is unnecessary or excessive for the software components associated with the view that are intended to be able to operate with only a single touch (eg, single touch or single tap to a soft button). Touch data (eg, multi-touch data) can be transmitted. Software components may need to process this data, so they must be characterized by all the complexity of a software application that handles multiple touches, even if only a single touch is relevant to the relevant view. sell. Software components that were previously easy to program in a mouse interface environment (ie, various buttons, etc.) can become quite complex in a multi-touch environment, which increases the cost of developing software for the device. Can be done.
To reduce the complexity of recognizing complex touch-based gestures, delegates can be used to control the behavior of the event recognizer according to some embodiments. As described below, the delegate is, for example, whether the corresponding event recognizer (or gesture recognizer) can receive event (eg, touch) information, or the corresponding event recognizer (or gesture recognizer) is a state machine. Can transition from an initial state (eg, eventable state) to another state, and / or without preventing other event recognizers (groups) (or gesture recognizers (groups)) from recognizing the event or The corresponding event recognizer (or gesture recognizer) as the corresponding gesture for the event (eg, touch) without being blocked by another event recognizer (s) (or gesture recognizer (s)) that recognizes the event. It can be determined whether they can be recognized at the same time.
However, the above description of the complexity of user touch evaluation and processing with respect to the touch sensor surface also applies to all forms of user input for operating the electronic device 102 or 104. Not all user input is initiated on the touch screen, for example with a single or multiple keyboard presses or holds that can be used as input corresponding to events and / or subevents that define the recognized event. Or without mouse movement and mouse button press, device rotation or other movement, user movement such as tapping, dragging, scrolling, pen stylus input, voice command, detected eye movement, biometric input, detected It is understood that this also applies to adjusting the user's physiological changes and / or any combination thereof.
4A-4D are flowcharts for exemplary state machines according to some embodiments. Gesture recognizers include discrete gesture recognizers and continuous gesture recognizers. Discrete gesture recognizers are typically useful in recognizing simple gestures that occur within a given time period (eg, tap or swipe gestures), and more essentially one in gesture recognition. It is for recognizing gestures that only the action message or only one set of action messages needs to be delivered to the application. Continuous gesture recognizers are useful in recognizing gestures (eg, pan, pinch, or rotating gestures) that involve touch movement (and therefore require tracking of touch position), and more essentially, movement. It is for recognizing the gestures that the sequence of messages needs to be delivered to the application during the gestures. In some embodiments, the discrete event recognizer state machine 400 and the continuous event recognizer state machine 402 have different states.
FIG. 4A shows a discrete event recognizer state machine 400 containing three states according to some embodiments. By managing the state transitions of the event recognizer state machine 342 based on the received event and / or subevent, the event recognizer effectively represents the event definition. For example, a tap gesture can be effectively defined by a sequence of two sub-events or, in some cases, three sub-events. First, touch should be detected, which is subevent 1. For example, the touch subevent can be that the user's finger is touching the touch sensor surface in a view that includes an event recognizer with an event recognizer state machine 342. Second, the touch does not move substantially in any given direction (eg, any movement of the touch position is less than a predetermined threshold, which is, for example, a distance (eg, 5 mm) or the number of pixels in the display. An optional measured delay with a sufficiently short delay (which can be measured as, for example, 5 pixels) is subevent 2. Finally, the end of touch (eg, releasing the user's finger from the touch sensor surface) is subevent 3. By coding the event recognizer state machine 342 to transition between states based on receiving these subevents, the event recognizer state machine 342 effectively represents the tap gesture event definition. The discrete event recognizer state machine 400 is an exemplary implementation of the event recognizer state machine 342 configured to recognize the tap gesture described above.
Regardless of the type of event, the event recognizer state machine 342 (including the event recognizer state machine realized as a discrete event recognizer state machine 400) starts at the event ready state 410, which is the event recognizer state machine. Indicates the initial state. The event recognizer state machine 342 may proceed to any of the remaining states depending on the event and / or subevent received.
If an event or subevent that is not the first or subevent of the gesture definition is received, starting from the eventable state 410, the discrete event recognizer state machine 400 transitions to the event failure state 430.
Starting from the event ready state 410, if an event or subevent containing a gesture definition for the gesture is received, the discrete event recognizer state machine 400 transitions to the event recognized state 420. However, even if the received event or subevent contains a gesture definition for the gesture, the discrete event recognizer state machine 400 has one or more of the corresponding event recognizer metadata (eg, attributes), as determined by the corresponding delegate. It is possible to transition to the event failure state 430 according to the value of and / or the application state.
In some embodiments, after transitioning to the event-recognized state 420, the corresponding event recognizer checks for a delay flag (eg, delayed touch end flag 363) (441). If the delay flag is on (YES in 441), the corresponding event recognizer delays the delivery of event information until the delay flag is turned off (442).
In some embodiments, the corresponding event recognizer includes a wait list 351 and the corresponding event recognizer waits for the event recognizers listed in the wait list 351 to reach a given state. For example, if the view includes a single-tap gesture recognizer and a double-tap gesture recognizer, the single-tap gesture recognizer may be configured to wait for the double-tap gesture recognizer to fail. In fact, for the single-tap gesture recognizer to transition to the event-recognized state 420, the double-tap gesture recognizer must (or be conditioned on) failing to recognize the event. .. As a result, if there is a tap event, the single-tap gesture recognizer will recognize the tap event unless the tap event is part of a multi-tap gesture.
After a delay and wait (442), the corresponding gesture recognizer, if present, delivers the event to the application (443). In some embodiments, the event is delivered in the form of an action message. In some embodiments, the action message is delivered according to action-target pair (group) 345. In some embodiments, the corresponding gesture recognizer activates the action-target pair (group) 345.
FIG. 4B shows a continuous event recognizer state machine 402 containing six states according to some embodiments.
As mentioned above, the continuous event recognizer state machine 402 starts from the event ready state 410.
If an event or subevent is received that is not part of the gesture-defined event and / or subevent start sequence, starting from event-enabled state 410, the discrete event recognizer state machine 400 transitions to event-failed state 430. ..
A continuous event recognizer state if an event or subevent is received that is part of the start sequence of an event (group) and / or subevent (group) of a given gesture definition, starting from event-enabled state 410. Machine 402 transitions to the event started state 412. Similar to the discrete gesture recognizer 400, the continuous event recognizer state machine 402 has a continuous event recognizer state machine 402, even if the received event or subevent contains part of the gesture-defined event (s) and / or subevent (s) start sequence. , The corresponding event recognizer metadata (eg, attributes), one or more values determined by the corresponding delegate, and / or the event failure state 430 may be transitioned according to the application state.
From event started state 412, the continuous event recognizer state machine 402 is an event if the next event or subevent received is an intermediate event or subevent but not the last event or subevent in a given gesture definition. It transitions to the changed state 414 and stays. The continuous event recognizer state machine 402 may remain in the event modified state 414 as long as the sequence of received events and / or subevents remains part of the gesture definition. The continuous event recognizer state machine 402 is in the event modified state 414 at some point, and if it receives an event or subevent that is not part of the gesture definition, it transitions to the event failure state 430, thereby ( Determines that the current event (if present) is not the type of event corresponding to this event recognizer (ie, the event recognizer corresponding to continuous event recognizer state machine 402). On the other hand, the continuous event recognizer state machine 402 is in the event started state 412 or the event changed state 414, and when it receives the last event or subevent of the gesture definition, it transitions to the event finished state 416, which is Completes normal event recognition.
In some embodiments, each gesture / recognizer state has a gesture / recognizer state value. In some embodiments, the event recognized state 420 (for the discrete event recognizer state machine 400) and the event terminated state 416 (for the continuous event recognizer state machine 402) are gestures made by one type of gesture recognizer. It has the same gesture recognizer state value so that software components configured to respond to recognition can also respond to other types of gesture recognizers.
A continuous event recognizer state when a given interrupt event (for example, predefined in operating system 118 or control application 124, such as an incoming call) occurs during event-initiated state 412 or event-changed state 414. Machine 402 transitions to the event canceled state 418.
Regardless of the type of gesture recognizer, each gesture recognizer (eg, 343) may be reconfigured so that the corresponding event recognizer state machine (eg, 342) returns to the event ready state 410.
4C and 4D show the role of delegates in state transitions according to some embodiments. In Figures 4C and 4D, actions (or decisions made) by one or more delegates are indicated by shaded boxes (eg, 450-456).
FIG. 4C shows the role of the delegate in the state transition of the discrete event recognizer state machine 400 according to some embodiments. In the examples described below, the state machine 400 corresponds to a particular discrete event recognizer with a corresponding delegate.
If an event or subevent is detected, starting from event ready state 410, the delegate corresponding to the event recognizer determines whether the event recognizer should receive the event or subevent (450). If the delegate returns a value that prevents the corresponding event recognizer from receiving the event or subevent, then the corresponding event recognizer does not receive the event or subevent (or ignores the event or subevent). ). As a result, the corresponding event recognizer remains in the event ready state 410. If there is no delegate that prevents the corresponding event recognizer from receiving the event or subevent, the default behavior of the corresponding event recognizer is to receive the event or subevent.
The "should receive" action 450 by the delegate of the set of event recognizers can be used to determine which event recognizer receives which touch on the touch sensor display or touch sensor surface. For example, in a view that allows the user to use two touches to reposition two objects individually and simultaneously or to select two different objects, the delegates for the two event recognizers are one. The event recognizer may be configured to allow only the first touch of the two touches to be received, and the second event recognizer to allow only the second touch of the two touches to be received. .. Therefore, all information about each of the two touches is addressed only to the event recognizer, who is authorized to receive that touch by the corresponding delegate. A fairly complex multi-touch input can be recognized and processed by using multiple event recognizers and corresponding delegates to determine which touch is processed by which event recognizer.
If the event recognizer is allowed to receive an event or subevent, the delegate (or control application 124 or operating system 118) corresponding to the event recognizer will be aware of the event or subevent by the event recognizer. It is blocked by another event recognizer that is already aware of the event (451). The initial level of this blocking can be overridden by the delegate based on the default exclusivity rules. If recognition of an event or subevent is blocked, the corresponding delegate (or operating system 118 or control application 124) is allowed simultaneous recognition of the event by the event recognizer according to one or more values determined by the delegate. Ruka (452) is also decided. For example, if the event recognizer is on the event recognizer's exclusivity exception list 353 that first recognized the event, the delegate allows simultaneous recognition by both event recognizers. In another example, if the event recognizer exclusivity flag 339 is not set, the delegate allows simultaneous recognition by both event recognizers. If simultaneous recognition is not allowed, the event recognizer transitions to event failure state 430.
If the corresponding event recognizer is not prevented from recognizing the event or subevent (NO in 451), or if simultaneous recognition is allowed (YES in 452), the corresponding event recognizer is the event or subevent. Determine if the event matches the corresponding gesture definition (453). If the event or subevent does not match the corresponding gesture definition (NO in 453), the corresponding gesture recognizer transitions to event failure state 430.
If the event or subevent matches the corresponding gesture definition (YES in 453), the corresponding delegate (or operating system 118 or control application 124) is in event-enabled state 410 according to one or more values determined by the delegate. Determine if you can transition from (Should Start 454). If the delegate does not allow the event recognizer to transition from event-enabled state 410 (NO in 454), the corresponding event recognizer goes into event-failed state 430. If the event recognizer is allowed to transition from the event-enabled state 410 (YES in 454), the corresponding event recognizer transitions to the event-recognized state 420.
Does the corresponding event recognizer (or operating system 118 or control application 124) allow other event recognizers to recognize the event or subevent when the corresponding event recognizer transitions to the event recognized state 420? (455) is also determined. In some embodiments, the default is the same for all other event recognizers, unless the delegate (or application) of at least one of the event recognizers sets the attribute to allow simultaneous recognition. It is to prevent the recognition of the event of. If the delegate corresponding to the event recognizer that recognizes the event or subevent determines that the delegate allows other event recognizers to recognize the event or subevent (YES in 455), then the delegate (or) The operating system 118 or control application 124) sets the attributes of other event recognizers so that they can recognize the event or subevent at the same time (456). If the delegate does not allow the other event recognizer to recognize the event or subevent, the other event recognizer is prevented from recognizing the event or subevent.
In some embodiments, before making an individual event recognizer unaware of an event or subevent, the delegate of that event recognizer is to see if it allows simultaneous recognition of the event or subevent. Called (see 452). In these embodiments, simultaneous recognition may be possible by either a first event recognizer delegate or a second event recognizer delegate that recognizes the event. As shown by 452 and 455 in FIG. 4C, in these embodiments, the decision as to whether to allow simultaneous recognition is made only if the event meets the event definition of at least one event recognizer.
The delegate behavior described above is used to customize the interaction of event recognizers when implemented in a delegate to the set of event recognizers used by the application view (or set of views displayed at the same time). sell. The delegate can implement an exception to the default exclusivity rule, otherwise allowing only one event recognizer to recognize gestures based on the same event (s) received. Delegate users who implement exceptions to the default exclusivity rules, thereby allowing simultaneous event recognition by compatible event recognizers, facilitate the implementation of many useful features in software applications. The use of delegates to modify and control the behavior of event recognizers allows for compact representation and implementation of complex relationships such as mutually exclusive sets of mutually compatible gestures.
FIG. 4D shows the role of the delegate in the state transition of the continuous event recognizer state machine 402 according to some embodiments. In the examples described below, the state machine 402 corresponds to a particular continuous event recognizer with a corresponding delegate. The behavior of all delegates shown in Figure 4C and described above with reference to Figure 4C is equally applicable to continuous event recognizers with corresponding delegates, therefore the behavior of delegates shown in Figure 4D is illustrated. It has the same reference code as 4C. The only difference is that the name of one state on the state machine has changed from "event recognized" 420 on state machine 400 to "event started" 412 on state machine 402.
The corresponding event recognizer transitions from the event started state 412 to another state as described above. For brevity, the transition from event modified state 414 to event failed state 416 is not shown.
The table below presents the processing of an exemplary sub-event sequence (eg, single tap) for the event recognizer state described above in tabular form. In this example, the sub-event sequence contains a single tap and the view has two tap gesture recognizers, namely a single tap gesture recognizer and a double tap gesture recognizer. Again in this example, both gesture recognizers are configured to receive and recognize subevent sequences at the same time. Simultaneous recognition may be allowed by the delegate assigned to the single-tap gesture recognizer or the delegate assigned to the second-tap gesture recognizer.<img id="000003" he="37" wi="142" file="JP2016177828A_D0001.tif" img-format="tif" img-content="drawing" />
Before the distribution of sub-event information starts (sequence number 0), both gesture recognizers are in the event ready state 410. Both gesture recognizers remain in the event ready state 410, even after detecting the finger-down subevent (sequence number 1) and measuring the delay (sequence number 2). In response to detecting finger lift-off (sequence number 3), the single-tap gesture recognizer transitions to the event-recognized state 420. After detecting an additional delay, the single-tap gesture recognizer stays in the event-recognized state 420 until reset, and when reset, the single-tap gesture recognizer returns to the event-enabled state 410. On the other hand, if the additional measured delay exceeds a given time period (for example, the period during which the double-tap gesture recognizer expects a second finger-down subevent), the double-tap gesture recognizer will be in event failure state 430. Transition to.
The table below presents in tabular form the processing of an exemplary subevent sequence when the behavior of one gesture recognizer changes. In this example, the sub-event sequence contains a single tap and the view has two tap gesture recognizers, namely a single tap gesture recognizer and a double tap gesture recognizer. Again in this example, the single-tap gesture recognizer is not allowed by the delegate to receive sub-events.<img id="000004" he="37" wi="142" file="JP2016177828A_D0001.tif" img-format="tif" img-content="drawing" />
The single-tap gesture recognizer remains in the event-enabled state 410 because the delegate does not allow it to receive sub-events. When the measured second delay exceeds a predetermined threshold (sequence number 4), the double-tap gesture recognizer transitions to the event failure state 430.
The table below presents in tabular form the processing of an exemplary subevent sequence when the behavior of one gesture recognizer changes. In this example, the sub-event sequence contains a single tap and the view has two tap gesture recognizers, namely a single tap gesture recognizer and a double tap gesture recognizer. Again in this example, neither gesture recognizer is allowed to recognize the subevent sequence at the same time.<img id="000005" he="37" wi="142" file="JP2016177828A_D0001.tif" img-format="tif" img-content="drawing" />
As described above, after the finger lift-off is detected (sequence number 3), the single-tap gesture recognizer transitions from the event-enabled state 410 to the event-recognized state 420. In general, the first gesture recognizer that recognizes the subevent sequence prevents other gesture recognizers that have not yet recognized the subevent sequence from recognizing the subevent sequence. Unless simultaneous recognition is allowed, the blocked gesture recognizer transitions to event failure state 430. In this case, simultaneous recognition is not allowed, so if the single-tap gesture recognizer recognizes a sub-event sequence (at sequence number 3), the double-tap gesture recognizer transitions to event failure state 430 and is reset. It stays in that state until.
The table below is an exemplary subevent when the behavior of one gesture recognizer is modified by the delegate and the behavior of the two gesture recognizers is adjusted according to the behavior performed by the delegate of one or both recognizers. The processing of the sequence is presented in tabular format. In this example, the sub-event sequence contains a single tap and the view has two tap gesture recognizers, namely a single tap gesture recognizer and a double tap gesture recognizer. Again, the single-tap gesture recognizer is not allowed to start (ie, transition from the eventable state 410).<img id="000006" he="37" wi="142" file="JP2016177828A_D0001.tif" img-format="tif" img-content="drawing" />
After detecting the finger lift-off (sequence number 3), the single-tap gesture recognizer attempts to transition from the event-enabled state 410 to the event-recognized state 420. However, the delegate assigned to the single-tap gesture recognizer does not allow the state transition to the event-recognized state 420, and as a result, the single-tap gesture recognizer transitions to the event-failed state 430. When the measured delay exceeds a predetermined threshold (sequence number 4), the double-tap gesture recognizer transitions to the event failure state 430.
The table below shows an exemplary subevent sequence when the behavior of one gesture recognizer is modified and the behavior of two gesture recognizers is adjusted according to the behavior performed by the delegates of one or both recognizers. Present the process in tabular format. In this example, the sub-event sequence contains a single tap and the view has two tap gesture recognizers, namely a single tap gesture recognizer and a double tap gesture recognizer. Again in this example, the single-tap gesture recognizer waits for (or requires) the failure of the double-tap gesture recognizer.<img id="000007" he="37" wi="142" file="JP2016177828A_D0001.tif" img-format="tif" img-content="drawing" />
After detecting the finger lift-off (sequence number 3), the single-tap gesture recognizer attempts to transition from the event-enabled state 410 to the event-recognized state 420. However, due to the "wait" requirement or the failure requirement (that the double-tap gesture recognizer fails), the single-tap gesture recognizer delays the transition to the event-recognized state 420. If the double-tap gesture recognizer fails because the measured second delay exceeds a predetermined threshold (sequence number 4), the single-tap gesture recognizer transitions to the event-recognized state 420. The "wait" and / or failure requirements may be implemented using a delegate or in a gesture recognizer.
The following table presents the processing of an exemplary subevent sequence in tabular form. In this example, the sub-event sequence contains a pan gesture that includes multiple intermediate sub-events, and the view has two gesture recognizers, namely a single tap gesture recognizer and a pan gesture recognizer. Again in this example, both gesture recognizers are allowed to recognize the subevent sequence at the same time.<img id="000008" he="47" wi="139" file="JP2016177828A_D0001.tif" img-format="tif" img-content="drawing" />
Before the distribution of sub-event information starts (sequence number 0), both gesture recognizers are in the event ready state 410. Even after detecting the finger-down subevent (sequence number 1) and measuring the delay (sequence number 2), the single-tap gesture recognizer remains in the eventable state 410, while panning. The gesture recognizer transitions to the event started state 412. In response to finger movement detection (sequence number 3), the single-tap gesture recognizer transitions to event-failed state 430 because the sub-event does not match the gesture definition for single-tap. The single-tap gesture recognizer then remains in event failure state 430 until reset. However, the pan gesture recognizer transitions to the event-modified state 414 in response to the detection of finger movement (sequence number 4), and in some embodiments, an action message (including a new position of finger contact). Group) is sent. After detecting additional finger movements (sequence numbers 4 and 5), the pan gesture recognizer remains in the event-modified state 414, while sending an action message (group) each time it detects a finger movement. Send. When a finger lift-off is detected (sequence number 6), the pan gesture recognizer transitions to the event finished state 416.
Considering the flow of event information and the interaction between event recognizers, FIG. 5A is a block diagram showing the flow of event information according to some embodiments. The event dispatcher module 315 (eg, in operating system 118 or control application 124) receives event information and sends the event information to the application (eg 132-1). In some embodiments, application 132-1 has multiple views of view hierarchy 506 (eg, 508, 510 and 512 corresponding to view 316) and multiple gesture recognizers (516-1 ~) in the views. 516-6) and included. Application 132-1 also includes one or more event handlers 520 that correspond to target values in target-action pairs (eg, 522-1, 522-2, and 522-3). The event dispatcher module 315 receives hit view information from the hit view determination module 313 and is an event recognizer (s) attached to the hit view (eg, 512) or hit view (eg, 512). Send event information to. In some embodiments, only a subset of gesture recognizers attached to the hit view are allowed (or configured to receive) event information (or touch information). These gesture recognizers who are authorized to receive event information are referred to herein as "receive gesture recognizers". In FIG. 5A, gesture recognizers 516-1 and 516-2 are within the set of receiving gesture recognizers 514. As a result, the event dispatcher module 315 sends event information to both gesture recognizers 516-1 and 516.2 in the set of receiving gesture recognizers.
In some embodiments, the gesture recognizer may prevent or prevent the other gesture recognizer from recognizing an event or subevent as the corresponding gesture. In this example, gesture recognizer 1 (516-1) prevents gesture recognizer 2 (516-2) from recognizing an event or subevent as the corresponding gesture. As a result, in this example, only gesture recognizer 1 (516-1) sends the action message (518) to the corresponding target-action pair (eg, target: action 1 (522-1)).
5B and 5C are flowcharts showing gesture recognition methods according to some embodiments. FIG. 5B shows a flowchart in which the gesture recognizer calls the corresponding delegate, and FIG. 5C shows a flowchart in which the software application calls the delegate corresponding to the individual gesture recognizer. In FIGS. 5B and 5C, each column represents the processing performed by each entity or component (eg, software application, gesture recognizer or delegate).
In FIG. 5B, a software application (eg, application 132-1) displays one or more of a plurality of views (eg, 506, 508, 510) (530). Multiple views include multiple gesture recognizers (eg, 516-1 to 516-6). A software application (eg, application 132-1) assigns a separate delegate to at least a subset of multiple gesture recognizers (532). In some embodiments, individual gesture recognizers are assigned to the corresponding delegates (533-1). In some embodiments, individual delegates are assigned to the corresponding gesture recognizers (533-2). Alternatively, the correlation between the delegate and the gesture recognizer may be established before run time. Throughout the following description, each reference to an assigned delegate may also mean a corresponding delegate, and each reference to a gesture recognizer to which a delegate is assigned also means a gesture recognizer corresponding to a particular delegate. You may.
A software application (eg, application 132-1) detects one or more events (534) and uses one or more of gesture recognizers (eg, 320) to process each event (536). ).
Individual events are processed in individual gesture recognizers (of one or more gesture recognizers (eg 320)) (538). To illustrate the behavior of the delegate, assume that the individual gesture recognizer that handles the event has a corresponding delegate. The individual gesture recognizer calls the assigned delegate, and the assigned delegate is executed to determine one or more values according to the application state (540). In response, the individual gesture recognizer conditionally sends the information corresponding to the individual event to the software application according to one or more values determined by the assigned delegate (542).
The software application runs according to the information received from one or more of the gesture recognizers corresponding to one or more of the events (544).
In other words, in these embodiments, the individual gesture recognizer calls an assigned delegate to obtain one or more values that determine the behavior of the gesture recognizer. As mentioned above, the behavior of the gesture recognizer modified by the corresponding delegate includes whether to receive touch / event information, whether to transition from the eventable state, and / or whether to allow simultaneous recognition. .. Delegate behavior also controls which gesture recognizer receives which touch, and which gesture recognizer is "event-aware" (in some cases, accompanied by the coordinated behavior of the delegates of other gesture recognizers). The behavior of two or more gesture recognizers is adjusted by determining whether the transition to the "event started" state is permitted and by enabling or disabling simultaneous recognition.
In Figure 5C, a software application (eg, application 132-1) displays one or more of a plurality of views (eg, 506, 508, 510) (530). Multiple views include multiple gesture recognizers (eg, 516-1 to 516-6). A software application (eg, application 132-1) assigns a separate delegate to at least a subset of multiple gesture recognizers (532). In some embodiments, individual gesture recognizers are assigned to the corresponding delegates (533-1). In some embodiments, individual delegates are assigned to the corresponding gesture recognizers (533-2). Alternatively, the correlation between the delegate and the gesture recognizer may be established before run time.
A software application (eg, application 132-1) detects one or more touches (535) and uses one or more of the gesture recognizers to process each of the one or more touches (546). ). When processing each of one or more touches, the software application identifies a set of candidate gesture recognizers among multiple gesture recognizers (548). In some embodiments, the candidate gesture recognizer is a gesture recognizer attached to a hit view (eg, gesture recognizers 516-1, 516.2 and 516-3 in FIG. 5A).
Delegate assigned to the individual candidate gesture recognizer is executed to get the "received touch value" according to the application state (550). The "received touch value" is used to determine whether an individual candidate gesture recognizer can receive event / touch information (eg, step 450 of "should be received" in FIGS. 4C and 4D).
A collection of received gesture recognizers is identified based on the received touch values obtained from the individual delegates (552). The set of received gesture recognizers includes a subset of candidate gesture recognizers (552). In some embodiments, the set of received gesture recognizers includes all candidate gesture recognizers that do not have individual delegates assigned. If two or more of the candidate gesture recognizers have corresponding delegates, then each such candidate gesture recognizer delegate runs to determine if the candidate gesture recognizer can receive event / touch information. Will be done. The "received touch value" obtained from the delegate corresponding to the candidate gesture recognizer is used to identify the set of received gesture recognizers.
Individual touches are processed in the set of receiving gesture recognizers (554). Corresponds to the individual gesture recognizer if the event or gesture is recognized as a result of processing the individual touches by the individual gesture recognizer (see "Does it meet the definition" 453 in Figures 4C and 4D)? The delegate (if any) is called to determine if recognition of the event or gesture is allowed. This corresponds to the "should start" operation 454 described above with reference to FIGS. 4C and 4D. The delegate returns one or more values that indicate whether state transitions are allowed. The individual gesture recognizer conditionally sends the information corresponding to the individual event to the software application according to one or more values determined by the assigned delegate (542). Software applications run according to information received from one or more of the gesture recognizers that correspond to individual touches (545).
In other words, in these embodiments, the software application (or operating system 118 or control application 124) is individualized to obtain a value indicating a candidate gesture recognizer (if any) to handle the individual touch. Call the delegate corresponding to the candidate gesture recognizer. In addition, other aspects of gesture recognizer behavior can be further modified by the assigned delegate.
6A and 6B are flowcharts illustrating exemplary methods of processing individual events according to information obtained from delegates according to some embodiments.
Method 600 has a software application (eg, 130) with one or more event sensors, including multiple views (eg, application view 316) and application states of the software application (eg, 317). For example, it is performed on an electronic device (eg, device 102 or 104) that is configured to perform 132) (602).
The device displays one or more of the views (604). The individual views of one or more displayed views include one or more gesture recognizers (eg, event recognizer 320-1). In some embodiments, at least a subset of the displayed one or more views includes one or more gesture recognizers, and the remaining views of the displayed one or more views do not include a gesture recognizer.
The individual gesture recognizers of one or more gesture recognizers have corresponding delegates. In some embodiments, not all gesture recognizers have a corresponding delegate (ie, in some embodiments, some gesture recognizers do not have a corresponding delegate). In some embodiments, individual gesture recognizers correspond to two or more delegates, and each delegate determines a different value for the corresponding gesture recognizer for different conditions (eg, first delegate). Determines "should be received" 450, and the second delegate determines "whether recognition is blocked" 451 etc.). In some embodiments, two or more gesture recognizers correspond to the same delegate (eg, utilize the same delegate).
In some embodiments, the device assigns a separate delegate (eg, delegate 321-1) to a separate gesture recognizer (eg, 320-1) (606) (eg, see step 532 in Figure 5B). ). Instead, individual gesture recognizers have corresponding delegates, so delegates do not need to be assigned at run time. It is understood that all references to the event / gesture recognizer's assigned delegates herein are similarly applicable to the corresponding delegates of the event / gesture recognizer, and all references to the corresponding delegates are It is understood that it is equally applicable to the assigned delegate.
In some embodiments, the one or more views displayed include multiple gesture recognizers (608), and the device assigns a separate delegate to at least a subset of the multiple gesture recognizers. In other words, the device may have fewer delegates than the number of gesture recognizers, as some gesture recognizers do not have to have assigned delegates.
The device detects one or more events (610). In some embodiments, the device uses a sensor 130, an input device 128 and / or a touch sensor display 156 to detect one or more events.
The device uses individual gesture recognizers to process individual events for one or more events (612). The processing of individual events is to process individual events in individual gesture recognizers according to the individual gesture definitions corresponding to the individual gesture recognizers (for example, using event comparison unit 332 with event and gesture definition 333). (Compare), execute the corresponding delegate to determine one or more values according to the application state (for example, 540 in Figure 5B), and the result of processing individual events by individual gesture recognizers. Whether the information corresponding to an individual event (eg, whether the gesture recognizer recognizes an event such as a "tap gesture" or "swipe gesture", according to one or more values determined by the corresponding delegate as well as according to Includes the conditional transmission of relevant event information, such as event location and time stamps, and / or other additional information) to the software application.
In some embodiments, the delegate has an instruction to determine event recognizer attributes (such as "should start", "should receive", and "simultaneous recognition") and if executed 1 Returns one or more corresponding values. In some embodiments, the value for the event recognizer attribute can be set by the software application according to the application state. In some embodiments, the value for the attribute is predefined by the developer. In some embodiments, the internal attributes of the individual event / gesture recognizer have default values, which can be overridden by the delegate corresponding to the event / gesture recognizer.
For example, whether one or more values determined by the corresponding delegate of the gesture recognizer, for example, the gesture recognizer can transition from the event ready state 410 to the event recognized state 420 or the event started state 412. Or if the gesture recognizer is allowed to recognize the event (based on one or more values that indicate whether the gesture recognizer can recognize the event at the same time even though the blocking gesture recognizer is present). , The device sends information corresponding to individual events. In some embodiments, the device sends information corresponding to an individual event only if the event matches the corresponding gesture definition or part thereof. In addition, application state or other conditions can prevent individual gesture recognizers from sending information corresponding to individual events.
The device runs a software application (eg 132-1) according to the information received from the individual gesture recognizer corresponding to the individual event (614). For example, a software application (eg 132-1) contains multiple event handlers 322, and one or more of the event handlers 322 are invoked according to the information received from the individual gesture recognizers (eg 132-1). , Behavior-The event handler 322 listed in target pair 345 is invoked).
In some embodiments, one or more event sensors (eg, 130) include a touch sensor surface (eg, 156 or a separate touch sensor surface) configured to detect one or more touches. One or more events include one or more touches, and processing individual events involves processing individual touches (616). In some embodiments, one or more event sensors (eg, 130) include an accelerometer, and one or more events also include rotation or other movement of an electronic device.
In some embodiments, the device conditionally receives individual touches in individual gesture recognizers according to one or more values determined by the assigned delegate (618). For example, only if one or more values determined by the corresponding delegate (eg, "received touch value") allow individual gesture recognizers to receive individual touches (eg, Figure 4C and Figure). "Should I receive?" 450) in 4D, individual gesture recognizers receive individual touches.
In some embodiments, processing individual touches causes the individual gesture recognizer to ignore the individual touches if one or more values determined by the corresponding delegate meet certain touch neglect criteria. Including doing (620). In these embodiments, instead of receiving the individual touches conditionally as described in step 618, the individual gesture recognizer ignores the individual touches.
In some embodiments, processing an individual touch means that the individual gesture recognizer receives the individual touch if one or more values determined by the corresponding delegate meet a given touch neglect criterion. Including preventing doing (622). In these embodiments, the gesture recognizer does not need to conditionally receive or ignore the individual touches, as the individual touches are blocked and therefore do not reach the individual gesture recognizers. In some embodiments, blocking an individual gesture recognizer from receiving an individual touch instructs the event dispatcher module 315 not to send event information to the corresponding gesture recognizer. Including.
In some embodiments, processing individual touches in an individual gesture recognizer is such that the detected touch is consistent with the individual gesture definition (eg, the individual touch is in or part of the gesture definition). If so, enable the corresponding state transitions in the individual gesture recognizers when the corresponding delegates enable the state transitions (eg, Should Start in Figures 4C and 4D 454. ) Is included (624). In some embodiments, state transitions are enabled when the state transitionable value (eg, the "should start" value) is determined by the corresponding delegate to meet the state transition criteria.
In some embodiments, processing individual touches in individual gesture recognizers causes the corresponding state transitions in individual gesture recognizers if the detected touches are consistent with the individual gesture definitions. Includes conditional enabling when state transitions are enabled by the corresponding delegate (626). In other words, state transitions are conditionally enabled, even if the corresponding delegate allows (eg, does not block) the transition. For example, the conditions for state transitions are whether the individual touch / event meets the gesture definition or part of it, whether the individual gesture recognizer is allowed to receive the individual touch / event, and /. Or whether the recognition of individual touch / event is blocked.
In some embodiments, processing individual touches in individual gesture recognizers causes the corresponding state transitions in individual gesture recognizers if the detected touches are consistent with the individual gesture definitions. Includes (conditionally) disabling state transitions when they are prevented / disabled by another gesture recognizer that recognizes the gesture. In particular, gesture recognizers can be paired (or grouped) so that one gesture recognizer can prevent other gesture recognizers (groups) from transitioning to event-recognized state 420 or event-initiated state 412. (For example, if the 1st gesture recognizer is configured to prevent the 2nd gesture recognizer, the 1st gesture recognizer will respond to the 2nd gesture recognizer when it recognizes the event / touch. Prevents the second gesture recognizer from recognizing the event / touch regardless of the value returned by the delegate).
In some embodiments, multiple gesture recognizers are assigned on a priority basis (eg, a sequence of code, a sequence of instantiations, a view hierarchy corresponding to an individual gesture recognizer, or by a developer or software application. Enumerated (based on the priority given). When two or more gesture recognizers recognize individual touches at the same time, the highest priority gesture recognizer prevents all other gesture recognizers from recognizing individual touches.
In some embodiments, processing individual touches in the individual gesture recognizer is a separate touch in the second gesture recognizer according to one or more values determined by the delegate corresponding to the second gesture recognizer. Includes processing at the same time (628). For example, even if another gesture recognizer blocks the recognition of the event, the delegate corresponding to the second gesture recognizer may allow the second gesture recognizer to process the gesture in the second gesture recognizer ( For example, step 452) in FIGS. 4C and 4D.
In some embodiments, processing individual touches in individual gesture recognizers is a separate touch in a second gesture recognizer according to one or more values determined by the delegate corresponding to the individual gesture recognizer. Includes processing at the same time (630). For example, even if another gesture recognizer blocks the recognition of the event, the delegate corresponding to the gesture recognizer may allow the second gesture recognizer to process the gesture in the second gesture recognizer (eg,). Steps 455 and 456 in FIGS. 4C and 4D).
In some embodiments, processing individual touches in individual gesture recognizers is performed in the second gesture recognizer according to the values determined by the delegates corresponding to the first gesture recognizer and the second gesture recognizer, respectively. Includes processing individual touches at the same time.
7A and 7B are flowcharts illustrating exemplary methods of processing individual touches according to received touch values obtained from delegates according to some embodiments.
Method 700 has a touch sensor surface (eg, 156) and is configured to run a software application that includes multiple views (eg, 316) and application states of the software application (eg, 317). Performed on an electronic device (eg, device 104) (702).
The device displays one or more of a plurality of views (eg, 316) (704). The individual views of one or more displayed views include one or more gesture recognizers (eg, 320-1 or 343-2), and the individual gesture recognizers of one or more gesture recognizers correspond. It has a delegate (eg 321-1 or 346).
The device detects one or more touches on the touch sensor surface (eg, 156 or 130) (706). Each one or more touches has a touch position within the displayed one or more views.
The device handles individual touches for one or more touches (708) (eg, using event comparison unit 332 to determine if the individual touches meet gesture definition 333 (453)). Processing individual touches means performing a delegate corresponding to an individual gesture recognizer to get the received touch value according to the application state (eg, 550 in Figure 5C) and the received touch value is a given criterion. (For example, in some embodiments, a predetermined criterion is that the individual gesture recognizer is the receiving gesture recognizer 552) to process the individual touches in the individual gesture recognizer (eg, in some embodiments). , 554) and conditionally send information corresponding to individual touches to software applications (eg, 542) (710).
In some embodiments, the plurality of views comprises a plurality of gesture recognizers (712) (eg, application views 316 and recognizer 320 in FIG. 3B, views 508, 510 and 512 in FIG. 5A, and gesture recognizer 516. ). Separate delegates correspond to at least a subset of multiple gesture recognizers. Optionally, the device assigns a separate delegate (eg, 321) to at least a subset of multiple gesture recognizers (eg, 320). Processing individual touches of one or more touches identifies a set of candidate gesture recognizers among multiple gesture recognizers (eg, 548) and for each candidate gesture recognizer with a corresponding delegate. To execute the corresponding delegate to get the received touch value according to the application state (for example, 550), and one or more received gestures containing a subset of candidate gesture recognizers according to the obtained received touch value. It involves identifying the recognizer (eg, 552) and processing individual touches in each gesture recognizer of one or more of the receiving gesture recognizers (eg, 554).
In some embodiments, identifying a set of candidate gesture recognizers among a plurality of gesture recognizers includes identifying a set of gesture recognizers attached to a hit view. Optionally, identifying a set of candidate gesture recognizers among multiple gesture recognizers involves identifying a set of gesture recognizers that includes gesture definitions that correspond to individual touches. Further, in some embodiments, identifying a set of receive gesture recognizers identifies a subset of candidate gesture recognizers whose corresponding delegates provide individual receive touch values that meet the receive touch criteria. Includes (eg, receive touch values indicate that the corresponding gesture recognizer can receive individual touches).
In some embodiments, processing individual touches in each gesture recognizer of one or more received gesture recognizers has an individual delegate with a corresponding delegate according to the individual gesture definition corresponding to the individual gesture recognizer. Processing individual touches in the receiving gesture recognizer, executing delegates to determine one or more values according to application state, and following the results of processing individual touches by individual gesture recognizers and delegates Includes the conditional transmission of information corresponding to individual touches to software applications according to one or more values determined by (718). The device runs the software application according to the information received from one or more receiving gesture recognizers that correspond to one or more touches.
In some embodiments, processing individual touches in the individual receiving gesture recognizer is such that the detected touches are consistent with the individual gesture definition (eg, the individual touch is the gesture definition or part thereof). To enable the corresponding state transitions in the individual gesture recognizers when the state transitions are enabled by the corresponding delegates (eg, should start in Figures 4C and 4D). 454) includes (720).
In some embodiments, processing individual touches in individual receiving gesture recognizers corresponds to the corresponding state transitions in individual gesture recognizers if the detected touches are consistent with the individual gesture definitions. Includes conditionally enabling state transitions when the corresponding delegate enables state transitions (722). For example, the conditions for state transitions are whether the individual touch / event meets the gesture definition or part of it, and whether the individual gesture recognizer is allowed to receive the individual touch / event. Includes whether individual touch / event recognition is blocked and / or whether system-level instructions (eg, shutdown processing or other processing with higher priority than the application) prevent state transitions.
In some embodiments, processing individual touches in the individual receiving gesture recognizer is separate in the second gesture recognizer according to one or more values determined by the delegate corresponding to the second gesture recognizer. Includes processing touches simultaneously (724). For example, the delegate corresponding to the second gesture recognizer allows the second gesture recognizer to process gestures in the second gesture recognizer (eg, step 452 in FIGS. 4C and 4D).
In some embodiments, processing individual touches in individual receiving gesture recognizers is separate in the second gesture recognizer according to one or more values determined by the delegate corresponding to the individual gesture recognizer. Includes processing touches simultaneously (eg, steps 455 and 456 in FIGS. 4C and 4D) (726).
The device runs the software application according to the information received from the individual gesture recognizers that correspond to the individual touches (eg, 545) (716). For example, a software application (eg 132-1) contains multiple event handlers 322, and one or more of the event handlers 322 are invoked according to the information received from the individual gesture recognizers (eg 132-1). , Behavior-The event handler 322 listed in target pair 345 is invoked).
8A and 8B are flowcharts illustrating exemplary methods of processing individual touches in a software application that includes a discrete gesture recognizer and a continuous gesture recognizer according to some embodiments.
Method 800 is performed on an electronic device (eg, device 104) that has a touch sensor surface and is configured to run a software application (802).
The device displays one or more views (eg, 316) of a software application (eg, 132-1) (804). One or more views displayed include multiple gesture recognizers (eg, 320). The plurality of gesture recognizers includes at least one discrete gesture recognizer (eg, FIGS. 4A and 4C) and at least one continuous gesture recognizer (eg, FIGS. 4B and 4D).
In some embodiments, the discrete gesture recognizer is configured to send a single action message in response to individual gestures, and the continuous gesture recognizer is a recognized contiguous sub of the recognized individual gestures. It is configured to send an action message at the event.
In some embodiments, the discrete gesture recognizer is configured to send a single set of action messages in response to individual gestures. Multiple Targets-A single set of action messages contains multiple action messages when action pairs are assigned to individual discrete gesture recognizers. A single set of action messages contains a single action message when the action pair is assigned to a separate discrete gesture recognizer.
In some embodiments, each gesture recognizer has a set of gesture recognizer states (eg, FIGS. 4A-4D) (822).
In some embodiments, the discrete gesture recognizer has a first set of gesture recognizer states including: (824). Gesture-enabled state 410 corresponding to the initial state of the discrete gesture recognizer. Gesture recognized state 420 corresponding to the recognition of individual gestures. Gesture failure state 430 corresponding to the failure of the discrete gesture recognizer to recognize one or more touches as individual gestures.
In some embodiments, the continuous gesture recognizer has a second set of gesture recognizer states, including: Gesture-enabled state 410 corresponding to the initial state of the continuous gesture recognizer. Gesture started state 412 corresponding to the initial recognition of individual gestures. Gesture changed state 414 corresponding to individual changes in the position of individual touches. Gesture finished state 416 corresponding to the completion of the recognized individual gesture. Gesture canceled status 418 corresponding to the interrupt of recognition of individual gestures. Gesture failure state 430 corresponding to the failure of the continuous gesture recognizer to recognize one or more touches as individual gestures.
In some embodiments, the gesture recognizer state has an assigned value (eg, a gesture recognizer state value). In some embodiments, the gesture-recognized state and the gesture-finished state have the same gesture recognizer state value (826).
In some embodiments, at least one discrete gesture recognizer comprises one or more of a tap gesture recognizer and a swipe gesture recognizer (828), and at least one continuous gesture recognizer is a long press. Includes one or more of the Gesture Recognizer, Pinch Gesture Recognizer, Pan Gesture Recognizer, Rotating Gesture Recognizer and Converted Gesture Recognizer.
In some embodiments, at least one discrete gesture recognizer includes a tap gesture recognizer and a swipe gesture recognizer (830), and at least one continuous gesture recognizer includes a long press gesture recognizer. Includes Pinch Gesture Recognizer, Pan Gesture Recognizer, Rotating Gesture Recognizer, and Transform Gesture Recognizer.
The tap gesture recognizer is configured to recognize tap gestures. The swipe gesture recognizer is configured to recognize a swipe gesture (eg, a flick of a touch on the touch sensor surface). The long press gesture recognizer is configured to recognize long press gestures (eg, press and hold touch). The pinch gesture recognizer is configured to recognize pinch gestures (eg, contact and relative movement of two or more touches). The pan gesture recognizer is configured to recognize pan gestures (eg, a touch and a consistent movement of one or more touches). The rotary gesture recognizer is configured to recognize rotation (eg, contact and rotational movement of two or more touches). The transform gesture recognizer is configured to recognize transform gestures (eg, simultaneous movement of two or more touches representing pan, rotate, and pinch).
In some embodiments, at least one discrete gesture recognizer (eg, one or more of the discrete gesture recognizers described above) and at least one continuous gesture recognizer (eg, one or more of the continuous gesture recognizers described above) , Distributed to the software library so that software developers can use the software library to incorporate them into third-party software. In comparison, the view has a view style (eg, the color, size and shape of the user interface objects and frames). In some embodiments, a given view style is a UI interface API (eg, for example, that allows a software developer to use a software library (or template) to develop a software application with a given view style. It will be distributed as part of 204) in Figure 2.
The device detects one or more touches (808 in Figure 8A). In some embodiments, the device uses a sensor 130, an input device 128 and / or a touch sensor display 156 to detect one or more events.
The device uses one or more gesture recognizers to handle each touch (810). The processing of individual touches is to process individual touches in individual gesture recognizers according to the individual gesture definitions corresponding to individual gesture recognizers (for example, using event comparison unit 332 with event and gesture definition 333). To determine if an event meets gesture definition 333 or part of it) and each of one or more action messages according to the result of processing individual touches in the individual gesture recognizer software application. Includes conditional transmission (eg, sending an action message when an individual touch meets a gesture definition) (812).
In some embodiments, the software application has an application state (814). Conditionally sending one or more individual action messages involves conditionally sending one or more individual action messages according to the application state of the software application. For example, the application state of a software application can delay or prevent sending one or more individual action messages (eg, if system resources are overused, higher priority processing is processed. If you need to).
The device runs the software application according to one or more action messages received from one or more of the gesture recognizers corresponding to one of the touches (816). For example, a software application (eg 132-1) contains multiple event handlers 322, and one or more of the event handlers 322 is launched according to an action message received from one or more of the gesture recognizers. Will be done.
In some embodiments, the device requires additional information from an individual gesture recognizer (818). Running a software application involves running the software application according to additional information. For example, individual gesture recognizers can provide additional information (eg, detailed information such as time stamps for each sub-event, amount of jitter, speed, direction, duration, magnification, angle, etc.).
In some embodiments, the additional information includes the number and location of individual touches processed by the individual gesture recognizer (820).
According to some embodiments, FIG. 9 shows a functional block diagram of an electronic device 900 configured according to the principles of the invention as described above. Functional blocks of a device can be implemented by hardware, software, or a combination of hardware and software to implement the principles of the invention. Those skilled in the art will appreciate that the functional blocks described in FIG. 9 may be combined or separated into sub-blocks to realize the principles of the invention as described above. Accordingly, the description herein may support any possible combination, separation or further definition of the functional blocks described herein.
As shown in FIG. 9, the electronic device 900 includes one or more event detection units 902 configured to detect one or more events and a processing unit 906 coupled to one or more event detection units 902. And include. In some embodiments, the electronic device also includes a display unit 904 coupled to a processing unit 906. In some embodiments, the processing unit 906 includes an execution unit 908, a displayable unit 910, a detection unit 912, an event processing unit 914, a transmission unit 916, a recognizerable unit 918, a blocking unit 920, and an allocation unit 922.
Processing unit 906 executes a software application (eg, in execution unit 908) that includes multiple views and application states of the software application, and displays one or more of the views (eg, in execution unit 908). It is configured to enable (with the viewable unit 910 for the display unit 904). The individual views of one or more of the displayed views include individual gesture recognizers, and the individual gesture recognizers have corresponding delegates. The processing unit 906 processes individual events (for example, in the event processing unit 914) in the individual gesture recognizer according to the individual gesture definition corresponding to the individual gesture recognizer, and determines one or more values according to the application state. Run the corresponding delegate (for example, in execution unit 908) to follow the results of processing the individual event by the individual gesture recognizer and to the individual event according to one or more values determined by the corresponding delegate. One or more events detected by one or more event detection units 902 (eg, in detection unit 912) by conditionally sending the corresponding information to the software application (eg, in transmission unit 916). It is configured to detect and use a separate gesture recognizer to process the individual events of one or more events (eg, in event processing unit 914). Processing unit 906 is configured to execute software applications (eg, in execution unit 908) according to information received from individual gesture recognizers that correspond to individual events.
In some embodiments, one or more event detection units 908 include a touch sensor surface unit configured to detect one or more touches (eg, touch sensor surface unit 1002 in FIG. 10). One or more events include one or more touches, and processing unit 906 is configured to process individual touches (eg, in event processing unit 914).
In some embodiments, the processing unit 906 conditionally receives individual touches according to one or more values determined by the individual gesture recognizer by the corresponding delegate (eg, at the recognizer capable unit 918). ) Configured to enable.
In some embodiments, processing unit 906 allows individual gesture recognizers to ignore individual touches if one or more values determined by the corresponding delegate meet a given touch neglect criterion. Is configured to handle individual touches (eg, with event processing unit 914 and / or recognizerable unit 918).
In some embodiments, processing unit 906 prevents individual gesture recognizers from receiving individual touches if one or more values determined by the corresponding delegate meet a given touch neglect criterion. By doing so, the individual touches are configured to be processed (eg, by the event handling unit 914 and / or the blocking unit 920).
In some embodiments, processing unit 906 allows the corresponding state transitions in the individual gesture recognizers by the delegates to which the state transitions correspond, if the detected touches are consistent with the individual gesture definitions. By enabling when done, the individual gesture recognizers are configured to handle individual touches (eg, with the event processing unit 914 and / or the recognizing unit 918).
In some embodiments, processing unit 906 allows state transitions in individual gesture recognizers with corresponding delegates when the detected touches are consistent with individual gesture definitions. By conditionally enabling when done, individual gesture recognizers are configured to handle individual touches (eg, with event processing units 914 and / or possible units 918).
In some embodiments, the processing unit 906 simultaneously processes individual touches in the second gesture recognizer according to one or more values determined by the delegate corresponding to the second gesture recognizer, thereby making individual gestures. -The recognizer is configured to handle individual touches (eg, in the event processing unit 914).
In some embodiments, the processing unit 906 simultaneously processes individual touches in the second gesture recognizer according to one or more values determined by the delegate corresponding to the individual gesture recognizer, thereby making the individual gestures individual gestures. -The recognizer is configured to handle individual touches (eg, in the event processing unit 914).
In some embodiments, the one or more views displayed include multiple gesture recognizers, and processing unit 906 has separate delegates to at least a subset of the multiple gesture recognizers (eg, in allocation unit 922). ) Configured to allocate.
According to some embodiments, FIG. 10 shows a functional block diagram of an electronic device 1000 configured according to the principles of the invention as described above. Functional blocks of a device can be implemented by hardware, software, or a combination of hardware and software to implement the principles of the invention. Those skilled in the art will appreciate that the functional blocks described in FIG. 10 may be combined or separated into sub-blocks to realize the principles of the invention as described above. Accordingly, the description herein may support any possible combination, separation or further definition of the functional blocks described herein.
As shown in FIG. 10, the electronic device 1000 includes a touch sensor surface unit 1002 configured to receive one or more touches and a processing unit 1006 coupled to the touch sensor surface unit 1002. In some embodiments, the electronic device 1000 also includes a display unit 1004 coupled to a processing unit 1006. In some embodiments, the processing unit 1006 includes an execution unit 1008, a displayable unit 1010, a detection unit 1012, a touch processing unit 1014, a transmission unit 1016, an allocation unit 1018, an identification unit 1020, and a recognizerable unit 1022.
Processing unit 1006 runs a software application (eg, in execution unit 1008) that contains multiple views and the application state of the software application, and displays one or more of the views (eg, in execution unit 1008). It is configured to enable (in the displayable unit 1010 for the display unit 1004). The individual views of one or more of the displayed views include individual gesture recognizers, and the individual gesture recognizers have corresponding delegates. The processing unit 1006 is configured to detect one or more touches received by the touch sensor surface unit 1002 (eg, by the detection unit 1012). Each touch has a touch position that falls within one or more of the displayed views. The processing unit 1006 executes a delegate corresponding to an individual gesture recognizer (for example, in the execution unit 1008) to acquire the received touch value according to the application state, and if the individual touch value meets a predetermined criterion, it is individual. One by processing individual touches in the gesture recognizer (eg, in touch processing unit 1014) and conditionally sending the information corresponding to the individual touches to the software application (eg, in transmission unit 1016). It is configured to handle the individual touches of the above touches. The processing unit 1006 is configured to execute a software application (for example, in the execution unit 1008) according to the information received from the individual gesture recognizer corresponding to the individual touch.
In some embodiments, the plurality of views comprises a plurality of gesture recognizers. The processing unit 1006 identifies a set of candidate gesture recognizers among a plurality of gesture recognizers (for example, in the identification unit 1020) and determines the received touch value for each candidate gesture recognizer having an assigned delegate according to the application state. Executes the delegate assigned to retrieve (eg, in execution unit 1008) and according to the retrieved receive touch value, one or more received gesture recognizers (eg, identification unit) containing a subset of candidate gesture recognizers. Multiple gesture recognizers by identifying (on 1020) and processing individual touches (eg, on touch processing unit 1014) on each gesture recognizer of one or more of the receive gesture recognizers. A separate delegate is assigned to at least a subset of (eg, in assignment unit 1018) and configured to handle individual touches of one or more touches.
In some embodiments, the processing unit 1006 makes individual touches in the individual receiving gesture recognizers to which the delegates are assigned according to the individual gesture definitions corresponding to the individual gesture recognizers (eg, in touch processing unit 1014). Processed and executed a delegate assigned to determine one or more values according to application state (eg, in execution unit 1008), according to the result of processing individual touches by individual gesture recognizers, and assigned. Each gesture of one or more inbound gestures by conditionally sending information corresponding to an individual touch to a software application (eg, on a transmit unit 1016) according to one or more values determined by the delegate. Runs a software application (eg, on execution unit 1008) according to information received from one or more of the receiving gesture recognizers, which is configured to handle individual touches in the recognizer and corresponds to one or more of the touches. It is configured as follows.
In some embodiments, processing unit 1006 allows the corresponding state transitions in the individual gesture recognizer by the delegate to which the state transitions are assigned, provided that the detected touches are consistent with the individual gesture definitions. The individual receive gesture recognizer is configured to handle individual touches by enabling it when it is (eg, with the recognizer enable unit 1022).
In some embodiments, processing unit 1006 allows the corresponding state transitions in the individual gesture recognizers to be made by the delegate to which the state transitions are assigned, provided that the detected touches are consistent with the individual gesture definitions. It is configured to handle individual touches in a separate receiving gesture recognizer by conditionally enabling it when it is (for example, in the recognizeable unit 1022).
In some embodiments, the processing unit 1006 is individualized by simultaneously processing individual touches in the second gesture recognizer according to one or more values determined by the delegate assigned to the second gesture recognizer. The receiving gesture recognizer is configured to process individual touches (eg, on the touch processing unit 1014).
In some embodiments, the processing unit 1006 is individualized by simultaneously processing individual touches in the second gesture recognizer according to one or more values determined by the delegate assigned to the individual gesture recognizer. The receiving gesture recognizer is configured to process individual touches (eg, on the touch processing unit 1014).
According to some embodiments, FIG. 11 shows a functional block diagram of an electronic device 1100 configured according to the principles of the invention as described above. Functional blocks of a device can be implemented by hardware, software, or a combination of hardware and software to implement the principles of the invention. Those skilled in the art will appreciate that the functional blocks described in FIG. 11 may be combined or separated into sub-blocks to realize the principles of the invention as described above. Accordingly, the description herein may support any possible combination, separation or further definition of the functional blocks described herein.
As shown in FIG. 11, the electronic device 1100 includes a touch sensor surface unit 1102 configured to receive one or more touches and a processing unit 1106 coupled to the touch sensor surface unit 1102. In some embodiments, electronic device 1100 also includes display unit 1104 coupled to processing unit 1106. In some embodiments, the processing unit 1106 includes an execution unit 1108, a displayable unit 1110, a detection unit 1112, a touch processing unit 1114, a transmitting unit 1116, and a requesting unit 1118.
Processing unit 1106 is now capable of running software applications (eg, in execution unit 1108) and displaying one or more views of the software application (eg, in viewable unit 1110 with respect to display unit 1104). It is composed. One or more views displayed contain multiple gesture recognizers, and multiple gesture recognizers include at least one discrete gesture recognizer and at least one continuous gesture recognizer. Discrete gesture recognizers are configured to send a single action message in response to individual gestures, and continuous gesture recognizers send action messages in recognized contiguous subevents of recognized individual gestures. Configured to send. The processing unit 1106 detects one or more touches received by the touch sensor surface unit 1102 (eg, by the detection unit 1112) and uses one or more of the gesture recognizers to make each touch (eg, by example). It is configured to process (in the touch processing unit 1114). Processing unit 1106 processes individual touches in individual gesture recognizers (eg, in touch processing unit 1114) according to individual gesture definitions corresponding to individual gesture recognizers, and of individual touches in individual gesture recognizers. It is configured to handle individual touches by conditionally sending one or more individual action messages to the software application (eg, in transmission unit 1116) according to the result of the processing. Processing unit 1106 is configured to execute a software application (eg, on execution unit 1108) according to one or more action messages received from one or more of the gesture recognizers corresponding to one or more of the touches. Will be done.
In some embodiments, each gesture recognizer has a set of gesture recognizer states.
In some embodiments, the discrete gesture recognizer has one or more gesture-enabled states corresponding to the initial state of the discrete gesture recognizer, gesture-recognized states corresponding to the recognition of individual gestures, and one or more discrete gesture recognizers. It has a first set of gesture recognizer states, including a gesture failure state corresponding to the failure to recognize the touch as an individual gesture. The continuous gesture recognizer is a gesture-enabled state, a gesture-started state corresponding to the initial recognition of individual gestures, and a gesture-modified state corresponding to individual changes in the position of individual touches, and recognized individual gestures. Gesture finished state corresponding to the completion of, gesture canceled state corresponding to the interruption of recognition of individual gestures, and the failure of the continuous gesture recognizer to recognize one or more touches as individual gestures. It has a second set of gesture recognizer states, including the gesture failure state corresponding to.
In some embodiments, the gesture recognized state and the gesture completed state have the same gesture recognizer state value.
In some embodiments, the software application has an application state, and processing unit 1106 conditionally sends one or more operational messages (eg, in transmission unit 1116) according to the application state of the software application. It is configured to do.
In some embodiments, the processing unit 1106 requests additional information from the individual gesture recognizers (eg, in requesting unit 1118) and further follows the additional information to the software application (eg, executing unit 1108). (In) configured to run.
In some embodiments, the additional information includes the number and location of touches processed by the individual gesture recognizer.
In some embodiments, the at least one discrete gesture recognizer comprises one or more of a tap gesture recognizer and a swipe gesture recognizer. At least one continuous gesture recognizer includes one or more of a long press gesture recognizer, a pinch gesture recognizer, a pan gesture recognizer, a rotating gesture recognizer, and a transform gesture recognizer.
In some embodiments, the at least one discrete gesture recognizer includes a tap gesture recognizer and a swipe gesture recognizer. At least one continuous gesture recognizer includes a long press gesture recognizer, a pinch gesture recognizer, a pan gesture recognizer, a rotating gesture recognizer, and a transform gesture recognizer.
For the purposes of explanation, the above description has been given with reference to a specific embodiment. However, the above exemplary description is not intended to cover the invention or limit it to the exact form disclosed. Many changes and modifications are possible in light of the above teachings. In order to optimally explain the principle of the present invention and practical examples thereof, embodiments have been selected and described. This allows one of ordinary skill in the art to optimally utilize the present invention and various embodiments, including various modifications, that are suitable for the particular application considered.
29 sheets
Sheet 1 Sheet 2 Sheet 3 Sheet 4 Sheet 5 Sheet 6 Sheet 7 Sheet 8 Sheet 9 Sheet 10 Sheet 11 Sheet 12 Sheet 13 Sheet 14 Sheet 15 Sheet 16 Sheet 17 Sheet 18 Sheet 19 Sheet 20 Sheet 21 Sheet 22 Sheet 23 Sheet 24 Sheet 25 Sheet 26 Sheet 27 Sheet 28 Sheet 29
Every citation, both ways
| Document | Relation | Office | Cited during |
|---|---|---|---|
| US2008316183A1 | Cites | United States of America | Search report |
| US2008316183A1 | Cites | United States of America | Search report |
| JP2009015592A | Cites | Japan | Search report |
159 members in 12 offices
Priority claims10
| Document | Office | Kind | Date |
|---|---|---|---|
| 29853110 | United States of America | P | |
| 29853110 | United States of America | P | |
| 61298531 | United States of America | – | |
| 12789695 | United States of America | – | |
| 78969510 | United States of America | A | |
| 78969510 | United States of America | A | |
| 12789695 | – | – | – |
| 61298531 | – | – | – |
| US20100298531P | – | – | – |
| US20100789695 | – | – | – |
Members159
| Document | Office | Kind | |
|---|---|---|---|
| US2010235118A1 | United States of America | A1 | |
| CA2755443A1 | Canada | A1 | |
| CA2817648A1 | Canada | A1 | |
| CA2817890A1 | Canada | A1 | |
| CA2909730A1 | Canada | A1 | |
| WO2010107669A2 | World Intellectual Property Organization (WIPO) | A2 | |
| WO2010107669A3 | World Intellectual Property Organization (WIPO) | A3 | |
| US2011167340A1 | United States of America | A1 | |
| WO2011085118A1 | World Intellectual Property Organization (WIPO) | A1 | |
| US2011179380A1 | United States of America | A1 | |
| US2011179386A1 | United States of America | A1 | |
| US2011179387A1 | United States of America | A1 | |
| CN102135858A | China | A | |
| US2011181526A1 | United States of America | A1 | |
| WO2011094267A1 | World Intellectual Property Organization (WIPO) | A1 | |
| EP2354930A1 | European Patent Office (EPO) | A1 | |
| AU2010226120A1 | Australia | A1 | |
| KR20110130484A | Republic of Korea | A | |
| US2011310047A1 | United States of America | A1 | |
| EP2409222A2 | European Patent Office (EPO) | A2 | |
| EP2413237A1 | European Patent Office (EPO) | A1 | |
| CN102422264A | China | A | |
| DE112010001143T5 | Germany | T5 | |
| US2012159380A1 | United States of America | A1 | |
| NL2007993A | Netherlands (Kingdom of the) | A | |
| WO2012087939A1 | World Intellectual Property Organization (WIPO) | A1 | |
| NL2007993C2 | Netherlands (Kingdom of the) | C2 | |
| AU2011209720A1 | Australia | A1 | |
| JP2012521050A | Japan | A | |
| TW201237735A | Taiwan Province of China | A | |
| US8285499B2 | United States of America | B2 | |
| CN102768608A | China | A | |
| KR20120123476A | Republic of Korea | A | |
| US2013016039A1 | United States of America | A1 | |
| US8428893B2 | United States of America | B2 | |
| AU2011349513A1 | Australia | A1 | |
| JP2013518349A | Japan | A | |
| KR101278346B1 | Republic of Korea | B1 | |
| CA2755443C | Canada | C | |
| HK1177519A | Hong Kong, China | A | |
| HK1177519A1 | Hong Kong, China | A1 | |
| KR20130111615A | Republic of Korea | A | |
| US8566044B2 | United States of America | B2 | |
| US8566045B2 | United States of America | B2 | |
| AU2010226120B2 | Australia | B2 | |
| EP2656192A1 | European Patent Office (EPO) | A1 | |
| CN203287883U | China | U | |
| US2014033131A1 | United States of America | A1 | |
| CN103558983A | China | A | |
| CN103558984A | China | A | |
| JP2014504419A | Japan | A | |
| US8682602B2 | United States of America | B2 | |
| AU2011209720B2 | Australia | B2 | |
| JP5497920B2 | Japan | B2 | |
| CN102135858B | China | B | |
| CN102422264B | China | B | |
| US2014160052A1 | United States of America | A1 | |
| CN103955341A | China | A | |
| JP2014139816A | Japan | A | |
| AU2014213525A1 | Australia | A1 | |
| CN104035717A | China | A | |
| CN104035718A | China | A | |
| CN104049975A | China | A | |
| JP5594914B2 | Japan | B2 | |
| JP2014211889A | Japan | A | |
| KR20140134719A | Republic of Korea | A | |
| CA2817648C | Canada | C | |
| TWI470537B | Taiwan Province of China | B | |
| KR20150012313A | Republic of Korea | A | |
| TW201512962A | Taiwan Province of China | A | |
| AU2011349513B2 | Australia | B2 | |
| KR101519194B1 | Republic of Korea | B1 | |
| KR101523866B1 | Republic of Korea | B1 | |
| JP5747235B2 | Japan | B2 | |
| JP2015158931A | Japan | A | |
| CA2817890C | Canada | C | |
| US9223590B2 | United States of America | B2 | |
| US9244606B2 | United States of America | B2 | |
| TWI520044B | Taiwan Province of China | B | |
| JP5879395B2 | Japan | B2 | |
| US9285908B2 | United States of America | B2 | |
| US9311112B2 | United States of America | B2 | |
| US2016110230A1 | United States of America | A1 | |
| AU2014213525B2 | Australia | B2 | |
| CN102768608B | China | B | |
| US2016139747A1 | United States of America | A1 | |
| US2016162180A1 | United States of America | A1 | |
| JP5938429B2 | Japan | B2 | |
| JP2016129029A | Japan | A | |
| AU2016204905A1 | Australia | A1 | |
| KR101645685B1 | Republic of Korea | B1 | |
| KR101645839B1 | Republic of Korea | B1 | |
| KR20160095189A | Republic of Korea | A | |
| KR20160097377A | Republic of Korea | A | |
| CN103558983B | China | B | |
| JP5980368B2 | Japan | B2 | |
| JP2016177828AThis record | Japan | A | |
| US9483121B2 | United States of America | B2 | |
| CN106095418A | China | A | |
| JP2016224970A | Japan | A |
13 legal events, as the office reported them to INPADOC
Over the term
Point at a mark for the eventEvents
| Event | Code | |
|---|---|---|
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Certificate of patent or registration of utility modelJAPANESE INTERMEDIATE CODE: R150R150 | R150 | |
| First payment of annual fees (during grant procedure)JAPANESE INTERMEDIATE CODE: A61A61 | A61 | |
| Written decision to grant a patent or to grant a registration (utility model)JAPANESE INTERMEDIATE CODE: A01A01 | A01 | |
| Decision of grant or rejection writtenTRDD | TRDD | |
| Request for written amendment filedJAPANESE INTERMEDIATE CODE: A523A521 | A521 | |
| Written request for extension of timeJAPANESE INTERMEDIATE CODE: A601A601 | A601 | |
| Notification of reasons for refusalJAPANESE INTERMEDIATE CODE: A131A131 | A131 |
Numbers
- Publication
- 2016177828
- Publication, DOCDB
- 2016177828
- Publication, EPODOC
- JP2016177828
- Application
- 98113
- Application, DOCDB
- 2016098113
- Application, EPODOC
- JP20160098113
Titles2
- Japanese
- ジェスチャ認識を制御及び変更するためのデリゲートを有するジェスチャ・レコグナイザ
- English
- Gesture recognizer with delegates to control and change gesture recognition
Classification
- CPC, 4
- G06F3/04883
- G06F9/451
- G06F9/448
- G06F3/048
- IPC, 3
- G06F3 0488
- G06F3 041
- G06F3 0481