Selection and display of media associated with a geographic area based on gesture input

BACKGROUND

Digital media equipment, such as digital cameras, digital audio recorders and digital video cameras, have become increasingly popular among the general population in recent years, generating large numbers of media files. Using current media file management systems, which in general organize and search media files according to folder name, file name, and/or date and time of creation, it is often difficult to conveniently locate a desired media file. In particular, there is no suitable way with current media file management systems for a user to conveniently locate and display a media file associated with a particular geographic location, as well as time of creation, and/or other media attributes.

SUMMARY

An interactive media display system is provided to display media associated with a selected geographic area based on a gesture input of user. The interactive media display system may include a touch-sensitive display surface configured to display a geographic map and receive a gesture input generated by a user on the touch-sensitive display surface while the geographic map is displayed, the gesture input defining a selected geographic area on the geographic map. The system may additionally include a data store for storing map data and media data. The media data may include media and metadata associated with the media. The metadata may include, for example, position data associated with the media. The system may further include a controller configured to receive the gesture input from the touch-sensitive display surface, and in response to the gesture input, to filter the media data based on the position data to determine a filtered media data collection including media that are associated with the selected geographic area, and to command the touch-sensitive display surface to display the media in the filtered media data collection.

This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. Furthermore, the claimed subject matter is not limited to implementations that solve any or all disadvantages noted in any part of this disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

<figref idrefs="DRAWINGS">FIG. 1</figref> is a schematic diagram illustrating an interactive media display system having an interactive media display device with a touch-sensitive display surface for displaying media associated with a selected geographic area based on a gesture input of a user.

<figref idrefs="DRAWINGS">FIG. 2</figref> shows two representations of the touch-sensitive display surface of the interactive media display system of <figref idrefs="DRAWINGS">FIG. 1</figref>, illustrating the input of a gesture input, and the display of media associated with a selected geographic area based on the gesture input.

<figref idrefs="DRAWINGS">FIG. 3</figref> is a screen of the touch-sensitive display surface of the interactive media display system of <figref idrefs="DRAWINGS">FIG. 1</figref>, illustrating a finger-pinch drawn boundary box gesture input.

<figref idrefs="DRAWINGS">FIG. 4</figref> is a screen of the touch-sensitive display surface of the interactive media display system of <figref idrefs="DRAWINGS">FIG. 1</figref>, illustrating a press-and-hold boundary box gesture input made by a digit of a user.

<figref idrefs="DRAWINGS">FIG. 5</figref> is a screen of the touch-sensitive display surface of the interactive media display system of <figref idrefs="DRAWINGS">FIG. 1</figref>, illustrating a manually-drawn boundary gesture input drawn by a digit of a user.

<figref idrefs="DRAWINGS">FIG. 6</figref> is a screen of the touch-sensitive display surface of the interactive media display system of <figref idrefs="DRAWINGS">FIG. 1</figref>, illustrating a manually-drawn line boundary box gesture input drawn with a stylus.

<figref idrefs="DRAWINGS">FIG. 7</figref> is a flowchart of an embodiment of a method for displaying media data associated with a selected geographic area based on gesture input.

DETAILED DESCRIPTION

As illustrated in <figref idrefs="DRAWINGS">FIG. 1</figref>, an interactive media display system <b>10</b> is provided, the interactive media display system <b>10</b> including an interactive display device <b>12</b> having a touch-sensitive display surface <b>14</b> configured to display a geographic map <b>50</b> and receive a gesture input <b>52</b> generated by a user on the touch-sensitive display surface <b>14</b> while the geographic map <b>50</b> is displayed. The interactive display device <b>12</b> may further include a controller <b>16</b> configured to receive the gesture input <b>52</b> from the touch-sensitive display surface <b>14</b>, and recognize that the gesture input <b>52</b> defines a selected geographic area <b>56</b> on the geographic map <b>50</b>. In response to the gesture input <b>52</b>, the controller <b>16</b> may be further configured to filter media <b>66</b> in one or more data stores <b>18</b> based on metadata <b>68</b>, including position data, associated with the media <b>66</b> to determine a filtered media data collection <b>70</b> of media <b>66</b> that are associated with the selected geographic area <b>56</b>. The controller may further be configured to retrieve the filtered media data collection <b>70</b> from the data store <b>18</b>, and command the touch-sensitive display surface <b>14</b> to display the media <b>66</b> in the filtered media data collection <b>70</b>.

The touch-sensitive display surface <b>14</b> may be a multiple touch-sensitive display surface configured to acquire gesture input <b>52</b> at a plurality of locations on the touch-sensitive display surface <b>14</b> at a given time, enabling multiple users and/or multiple fingers of a user to interact with the touch-sensitive display surface <b>14</b>. Thus, a gesture input <b>52</b> from one or more fingers of one or more users may be recognized by the controller <b>16</b>.

Turning to the inner workings of the controller <b>16</b>, the controller <b>16</b> may be configured to execute a gesture input module <b>28</b> that is configured to receive the gesture input <b>52</b> from the touch-sensitive display surface <b>14</b> and identify a geographic boundary <b>54</b> associated with the gesture input <b>52</b> that defines the selected geographic area <b>56</b>. As illustrated in <figref idrefs="DRAWINGS">FIGS. 2-5</figref> and described below, the gesture input module <b>28</b> may be configured to recognize gesture input <b>52</b> such as a manually-drawn boundary gesture input <b>78</b>, a press-and-hold drawn boundary box gesture input <b>80</b>, finger-pinch drawn boundary box gesture input <b>82</b>, and a manually drawn line boundary box gesture input <b>84</b>.

The controller <b>16</b> may be further configured to execute a media filter module <b>30</b> that is configured to receive the geographic boundary <b>54</b> from the gesture input module <b>28</b>, and in response to the geographic boundary <b>54</b>, to filter the media data <b>62</b> stored in one or more of the data stores <b>18</b> based on the metadata <b>68</b>, such as the position data, to obtain a filtered media data collection <b>70</b> including media associated with the selected geographic area <b>56</b>.

The controller <b>16</b> may be further configured to execute a map module <b>32</b> that is configured to access the map data <b>64</b> from the data stores <b>18</b>. The map data <b>64</b> may include data for generating the geographic map <b>50</b>, including data for rendering aerial images, streets, political boundaries, buildings, geographic features, and other map features.

The controller <b>16</b> may be additionally configured to execute a display module <b>36</b>. The display module <b>36</b> may be configured to receive the map data <b>64</b> from the map module <b>32</b>, and cause the geographic map <b>50</b> encoded in the map data <b>64</b> to be displayed on the touch-sensitive display surface <b>14</b>. The display module <b>36</b> may also be configured to receive the filtered media data collection <b>70</b> from the media filter module <b>30</b>, and cause the media in the filtered media data collection <b>70</b> to be displayed on the touch-sensitive display surface <b>14</b>. The display may be effected by a display output <b>72</b> sent from the display module <b>36</b> to the touch-sensitive display surface <b>14</b>. The display of the filtered media data collection <b>70</b> may be in a suitable form, such as thumbnails, previews, lists, slideshows, audio clips, and/or video clips. The filtered media data collection <b>70</b> may be displayed as an overlay on the geographic map <b>50</b>. For example, thumbnails <b>86</b> of photographs associated with a geographic location may be overlaid on the geographic map <b>50</b> at that particular geographic location as illustrated in <figref idrefs="DRAWINGS">FIG. 6</figref>.

The controller <b>16</b> may be configured to execute an interface module <b>34</b> configured to provide an interface between the data stores <b>18</b> and each of the media filter module <b>30</b> and the map module <b>32</b>. The interface module <b>34</b> may be configured to send out a filter media data command to cause media data <b>62</b> stored in the media data store <b>40</b> of the data store <b>18</b> to be filtered based on the metadata <b>68</b>, such as the position data <b>88</b>. The filter media data command may be generated based on the geographic boundary <b>54</b> associated with the gesture input <b>52</b>. The interface module <b>34</b> may be configured to send out a filter map data command to cause map data <b>64</b> stored in the map data store <b>42</b> of the data store <b>18</b> to be filtered according to the filter map data command. The filter map data command may be generated based on the gesture input <b>52</b>.

As discussed above, one or more data stores <b>18</b> may be provided. The data stores <b>18</b> may be configured to store map data <b>64</b> for the geographic map <b>50</b> and media data <b>62</b>. The stored media data <b>62</b> may include media <b>66</b> and metadata <b>68</b> associated with the media <b>66</b>, the metadata <b>68</b> including position data indicating a geographic position at which the media was recorded. The data stores <b>18</b> may be locally provided on the interactive media display device <b>12</b>, remotely located at a media service <b>20</b> and/or map service <b>22</b>, or located on an input device <b>26</b> or intermediary computing device <b>24</b>, via a peripheral or peer-to-peer connection. Thus, it will be appreciated that the interface module <b>34</b> may be configured to communicate over a computer network <b>76</b>, such as a wide area network, with the various devices on which the data stores <b>18</b> are located. In some cases, such as when communicating with input device <b>26</b> or intermediary computing device <b>24</b>, the interface module <b>34</b> may also be configured to communicate via a wired or wireless peripheral device connection, or local area network connection, as illustrated at A in <figref idrefs="DRAWINGS">FIG. 1</figref>.

One or more of the data stores <b>18</b> may include a map data store <b>42</b> for storing the map data <b>64</b> and a media data store <b>40</b> for storing the media data <b>62</b>. In the depicted embodiment, the map data store <b>42</b> is maintained on and accessed via a map service <b>22</b> executed on a map server configured to be accessible by multiple users of the map service <b>22</b>. Further, in the depicted embodiment, the media data store <b>40</b> is maintained on and accessed via a media service <b>20</b> executed on a media server configured to store media data <b>62</b> of multiple subscribers of the media service <b>20</b>. As discussed above, in addition or as an alternative to the data store <b>18</b> on the media server, the interactive display device <b>12</b> may be configured to communicate with a data store <b>18</b> on each input device <b>26</b>, an intermediary computing device <b>24</b>, and/or on the interactive display device <b>12</b> itself, each of which are configured to store media data <b>62</b>.

To upload media data <b>62</b> to the media server on which media service <b>20</b> is executed, interactive media display system <b>10</b> may further include one or more input devices <b>26</b> and intermediary computing devices <b>24</b>, which may be used by the same or a different user than the user of interactive display device <b>12</b>. The input device <b>26</b> may include a position sensing system for determining a position of the input device at a time when media is recorded, and recording the position as the position data. The position sensing system may include a global positioning system (GPS) for tracking position of the input device <b>26</b> via a satellite network. The position sensing system of the input device <b>26</b> may further include a compass for determining a compass direction indicating the directional orientation of the input device <b>26</b>, and an altimeter for generating altitude data. Thus, the position sensing system may be configured to generate position data including longitude, latitude, altitude, and compass direction.

A clock may also be provided in input device <b>26</b> to record metadata <b>68</b> indicating the time and date at which the media were recorded. Using the time and date data, the input device <b>26</b>, or alternatively the intermediary computing device <b>24</b> or other computing device within interactive media display system <b>10</b>, may be configured to compute derivative data of the position data, including speed and/or acceleration at which the media data <b>62</b> were captured. In addition, the input device <b>26</b> may be configured to record metadata <b>68</b> indicating the type (e.g., brand, model, etc.) of input device <b>26</b> used to capture the media data <b>62</b>.

It will be appreciated that the input device <b>26</b> may be a single unitary device, or may include a plurality of separate devices. For example, a unitary GPS enabled digital camera may function as input device <b>26</b>. Alternatively, a digital camera and associated GPS unit may be provided as separate input devices <b>26</b>, and the media captured by the digital camera may be linked with position data recorded by the GPS unit at the intermediary computing device <b>24</b>, as described below.

The media data <b>62</b> captured by the input device <b>26</b> may be uploaded onto the intermediary computing device <b>24</b> via an input processor <b>74</b> of the intermediary computing device <b>24</b>. Where the media <b>66</b> include embedded metadata <b>68</b>, the input processor <b>74</b> may be configured to separate the media <b>66</b> and metadata <b>68</b> into separate data containers, and link each media file with associated metadata, and link each media file with associated metadata. The input processor <b>74</b> may also be configured to upload the separated media <b>66</b> and metadata <b>68</b> files to the data store <b>18</b> associated with media service <b>20</b>. For example, the input processor <b>74</b> may, during uploading a jpeg file that includes image data embedded with global positioning system (GPS) data, and may separate and link the image data and the GPS data and upload them to the media service <b>20</b> as separate but linked data. The image data may be uploaded as the media <b>66</b> and the GPS data may be uploaded the metadata <b>68</b> associated with the media <b>66</b>. Where separate input devices device are used to capture metadata <b>68</b> with the position data and media <b>66</b>, the input processor <b>74</b> may be configured to link the resulting separate media <b>66</b> and metadata <b>68</b> associated with the media <b>66</b>. This may be accomplished, for example, by matching time and date data encoded in each of the meta data <b>68</b> and media <b>66</b>, to link the appropriate position data and other metadata <b>68</b> with the appropriate media <b>66</b>.

In an example use-scenario of the interactive media display system <b>10</b> of <figref idrefs="DRAWINGS">FIG. 1</figref>, a user may capture photographs using an input device <b>26</b>, such as a digital camera with an integrated GPS unit, during various trips at various times around the world. The captured photographs may include image data and metadata associated with the image data, such as GPS position data. The user may subsequently upload the photographs from the digital camera onto the data store <b>18</b> of the interactive media display system <b>10</b> located on the interactive display device <b>12</b>, using an intermediary computing device <b>24</b>, such as a personal computer of the user executing appropriate software. The intermediary computing device separates out and links the image data from the metadata, stores a copy on the intermediary computing device, and uploads the images and position data to the data store <b>18</b> on media service <b>20</b>.

The user may later wish to access these images based on various metadata associated with the photographs, including position data. For example, the user may wish to access photographs take during a trip to Hawaiian Islands five years ago at sea level using the interactive media display system <b>10</b>. At an interactive media display device <b>12</b> of the interactive media display system <b>10</b>, a geographic map <b>50</b> is first displayed on a touch-sensitive display surface <b>14</b> of the interactive media display device <b>12</b>. The map data <b>64</b> encoding the geographic map <b>50</b> is accessed by a map module <b>32</b> of a controller <b>16</b> of the interactive display device <b>12</b> from a map data store <b>42</b> located on a server of a map service <b>22</b> executed on a map server.

To access the images, the user then draws a manually drawn boundary gesture input <b>78</b> encircling the Hawaiian Islands on the geographic map <b>50</b> displayed on the touch-sensitive display surface <b>14</b>. The gesture input <b>52</b> is passed to a gesture input module <b>28</b> of the controller <b>16</b> of the interactive media display device <b>12</b>. The gesture input module <b>28</b> processes the gesture input <b>52</b> of the user to identify a geographic boundary <b>54</b> that identifies the Hawaiian Islands as a selected geographic area <b>56</b>. The user may also identify the time frame of interest and the altitude of interest using the touch-sensitive display surface <b>14</b>, for example via a suitable date and time selector and a suitable altitude selector provided on a graphical user interface of the touch-sensitive display surface <b>14</b>.

The identified geographic boundary <b>54</b> is then passed to a media filter module <b>30</b> of the controller <b>16</b>. The media filter module <b>30</b> filters the media data <b>62</b> in data store <b>18</b> for images based on the metadata <b>68</b> stored in data store <b>18</b> to identify a collection of images that were taken 5 years ago in the Hawaiian Islands at sea level.

The media filter module <b>30</b> utilizes the interface module <b>34</b> to send out commands from the media filter module <b>30</b> to the relevant data stores <b>18</b> to search for the collection of images taken 5 years ago on the Hawaiian Islands at sea level. In some embodiments, the media filter module <b>30</b> may be configured to search all data stores <b>18</b> that are currently connected to the interactive media display device <b>12</b>, and download images from each that have position data within the selected geographic area. In other embodiments, the filter module <b>30</b> may be configured to only search the data store <b>18</b> at the media service <b>20</b> and in response obtain a list of the images in the collection. The interface module <b>34</b> may be configured to determine whether a cached copy of any of the images exists on the locally provided data store <b>18</b> on the interactive media display device <b>12</b>. Where no cached copy is available, the interface module <b>34</b> may then command the data store to send the images on the list if it determines that the images are not available on the local data store.

Once the collection of images is obtained from the one or more data stores <b>18</b>, it is passed from the media filter module <b>30</b> to the display module <b>36</b>. The display module <b>36</b> in response outputs a display output <b>72</b> to the touch-sensitive display surface <b>14</b> and causes the filtered collection of images to be displayed on the touch-sensitive display surface <b>14</b>.

Since the touch-sensitive display surface <b>14</b> is a multiple touch-sensitive display surface <b>14</b> configured to acquire gesture input at a plurality of locations at a given time, it will be appreciated that a second user desiring to view photographs from a different geographic area on The Hawaiian Islands may input a second gesture input at the same time as the first user described above. This second gesture input is processed in the manner described above, and a second filtered collection of images is determined and displayed to the second user on the touch sensitive display surface <b>14</b>.

<figref idrefs="DRAWINGS">FIG. 2</figref> shows two representations of the touch-sensitive display surface <b>14</b> of the interactive media display system of <figref idrefs="DRAWINGS">FIG. 1</figref>, illustrating the input of a gesture input <b>52</b> and the display of media <b>66</b> associated with a selected geographic area <b>56</b> based on the gesture input <b>52</b>. At the top in <figref idrefs="DRAWINGS">FIG. 2</figref>, the geographic map <b>50</b> is shown displayed on the touch-sensitive display surface <b>14</b>, and the manually drawn boundary gesture input <b>78</b> is shown drawn on the touch-sensitive display surface with a stylus <b>60</b>. At the bottom in <figref idrefs="DRAWINGS">FIG. 2</figref>, the filtered media data collection <b>70</b> of media <b>66</b> associated with the selected geographic area <b>56</b> are displayed on the touch-sensitive display surface <b>14</b>. In the depicted embodiment, the filtered media data collection <b>70</b> includes preview images of photographs, full versions of which can be displayed upon selection by a user.

<figref idrefs="DRAWINGS">FIG. 3</figref> is a screen of the touch-sensitive display surface of the interactive media display system of <figref idrefs="DRAWINGS">FIG. 1</figref>, illustrating a finger-pinch drawn boundary box gesture input <b>82</b>. A finger-pinch made with the thumb and the index finger of a user produces a finger-pinch drawn boundary box gesture input <b>82</b> that include a rectangular shaped geographic boundary <b>54</b> in a space between the thumb and the index finger that defines a selected geographic area <b>56</b>.

<figref idrefs="DRAWINGS">FIG. 4</figref> is a screen of the touch-sensitive display surface of the interactive media display system of <figref idrefs="DRAWINGS">FIG. 1</figref>, illustrating a press-and-hold boundary box gesture input <b>80</b> made by a finger of a user. A circular geographic boundary <b>54</b> is shown drawn when a user press and hold a finger on the touch-sensitive display surface <b>14</b>. Alternatively, the geographic boundary <b>54</b> may be formed in a shape other than a circle, such a polygon or oval. The geographic boundary <b>54</b> may grow as the finger is pressed and held on the touch-sensitive display surface <b>14</b>. The longer the hold, the bigger the geographic boundary <b>54</b> that is drawn on the touch-sensitive display surface <b>14</b>.

<figref idrefs="DRAWINGS">FIG. 5</figref> is a screen of the touch-sensitive display surface of the interactive media display system of <figref idrefs="DRAWINGS">FIG. 1</figref>, illustrating a manually-drawn boundary gesture input <b>78</b> drawn with a finger of a user. The manually-drawn boundary gesture input <b>78</b> may also be drawn with a device, such as a stylus <b>60</b> as illustrated in <figref idrefs="DRAWINGS">FIG. 2</figref>.

<figref idrefs="DRAWINGS">FIG. 6</figref> is a screen of the touch-sensitive display surface <b>14</b> of the interactive media display device <b>12</b> of the interactive media display system <b>10</b> of <figref idrefs="DRAWINGS">FIG. 1</figref>, illustrating a manually-drawn line boundary box gesture input <b>84</b> drawn with a stylus. A geographic boundary <b>54</b> may be drawn around a manually-drawn line to define a selected geographic area <b>56</b> along the manually-drawn line. Thumbnails <b>86</b> of photographs associated with a geographic location may be overlaid on the geographic map <b>50</b> at the selected geographic area <b>56</b>, as described above.

<figref idrefs="DRAWINGS">FIG. 7</figref> is a flowchart of an embodiment of a method <b>100</b> for displaying media data associated with a selected geographic area based on gesture input. The method <b>100</b> may be implemented on the interactive media display system of <figref idrefs="DRAWINGS">FIG. 1</figref>, or on other suitable computer hardware components, as desired.

The method <b>100</b> may include, at <b>102</b>, uploading media data to a media data store from one or more input devices used to capture the media data. As described above, the media data may include media and associated metadata, which in turn includes position data associated with the media. The position data may include data selected from the group consisting of longitude, latitude, altitude, and compass direction. Other metadata may also be included, as described above.

At <b>104</b>, the method may further include displaying a geographic map on a touch-sensitive display surface of an interactive media display device. Prior to displaying the geographic map, the method may further include accessing map data for the geographic map from a data store associated with a map service executed on a map server, via a computer network. As described above, the map service may be configured to be accessible by multiple users of the map service.

At <b>106</b>, the method may further include receiving a gesture input from a user while the geographic map is displayed on the touch-sensitive display surface. At <b>108</b>, the method may further include identifying a geographic boundary associated with the gesture input, the geographic boundary defining a selected geographic area on the geographic map.

At <b>110</b>, the method may further include filtering media data stored in a media data store in response to the geographic boundary associated with the gesture input to determine a filtered media data collection including media associated with the selected geographic area. As discussed above, the media data may include media and associated metadata including position data, and the determination of the filtered media data collection may be based on the position data associated with the media data. In addition, the determination of the filtered media data collection may be further based on metadata other than the position data. As discussed above, the media data store from which the media data is filtered may be associated with a media service executed on a server, the media service being configured to store media data of multiple subscribers of the media service.

At <b>112</b>, the method further includes displaying the collection of media data on the touch-sensitive display surface.

Using the above systems and methods, a user may efficiently access stored media associated with a geographic area, using a gesture input on a geographic map displayed on a touch sensitive display surface, without searching for media files in a folder hierarchy of a file system.

It will be appreciated that the computing devices described herein may be servers, personal computers, laptop computers, portable data assistant (PDA), or other type of suitable computing device, and the input devices described herein may be any suitable type of device configured to capture digital media, including a digital camera, digital video camera, digital audio recorder, camera-equipped mobile telephone, or other suitable device. These devices typically include a processor and associated volatile and non-volatile memory, and are configured to execute programs stored in non-volatile memory using portions of volatile memory and the processor.

It should be understood that the embodiments herein are illustrative and not restrictive, since the scope of the invention is defined by the appended claims rather than by the description preceding them, and all changes that fall within metes and bounds of the claims, or equivalence of such metes and bounds thereof are therefore intended to be embraced by the claims.