Generating context relevant media augmentation

GENERATING CONTEXT RELEVANT MEDIA AUGMENTATION

claim priority

This application claims the benefit of priority to U.S. Patent Application Serial No. 14/974,321, filed on December 18, 2015, and hereby claims the benefit of their respective priority, each of which is incorporated herein by reference in its entirety. The specification is incorporated by reference.

The subject matter disclosed herein relates generally to providing context relevant media augmentation. Specifically, this disclosure addresses systems and methods for a platform for publishing contextually relevant media enhancements for presentation on a user interface.

The number of digital photos and videos taken with mobile wireless devices is increasingly greater than those taken with dedicated digital and film-based cameras. Accordingly, there is a growing need to enhance the experience associated with mobile wireless digital photography and video shooting.

The various drawings in the accompanying drawings represent only exemplary embodiments of the present disclosure and should not be construed as limiting the scope thereof. 1 is a network diagram illustrating a network system having a client-server architecture configured to exchange data over a network, according to one embodiment. 2 shows a block diagram illustrating an exemplary embodiment of a messaging application. 3 shows a block diagram illustrating an exemplary embodiment of a media overlay application. 4A shows a block diagram illustrating an exemplary embodiment of a user-based media overlay publishing module. 4B shows an example of a graphical user interface for a user-based media overlay publishing module. Fig. 4c shows an operation example of the graphical user interface of Fig. 4b. 4D shows an example of publishing of a user-based media overlay. 5A shows a block diagram illustrating an exemplary embodiment of a merchant-based media overlay publishing module. 5B shows an example of a common geolocation. 5C illustrates an example graphical user interface for a merchant-based media overlay publishing module. 5D illustrates an example of a bid from a first seller using the graphical user interface of FIG. 5C . 5E illustrates an example of a bid from a second seller using the graphical user interface of FIG. 5C . 5F shows an example of the operation of a merchant-based media overlay. 6A shows a block diagram illustrating an exemplary embodiment of a predefined media overlay module. 6B shows a diagram illustrating an example of a media overlay with live data content. 6C shows a diagram illustrating an example of a media overlay with promotional content. 6D shows a diagram illustrating an example of an collectible media overlay. 6E shows a diagram illustrating an example of a media overlay with viral content. 6F shows a diagram illustrating an example of a media overlay with audio content. 7 depicts an interactive diagram representing an exemplary embodiment of the operation of a user-based media overlay publishing module. 8 depicts an interactive diagram representing an exemplary embodiment of the operation of a merchant-based media overlay publishing module. 9 depicts a flow diagram illustrating an exemplary embodiment of the operation of a user-based media overlay publishing module. 10 depicts a flow diagram illustrating an exemplary embodiment of the operation of a merchant-based media overlay publishing module. 11 depicts a flow diagram illustrating an exemplary embodiment of the operation of a live event module. 12 depicts a flow diagram illustrating an exemplary embodiment of the operation of a social network module. 13 depicts a flow diagram illustrating an exemplary embodiment of the operation of a promotion module. 14 depicts a flow diagram illustrating one exemplary embodiment of the operation of the aggregation module. 15 shows a flow diagram illustrating an exemplary embodiment of the operation of a progressive use module. 16 shows a flow diagram illustrating one exemplary embodiment of the operation of a virtual use module. 17 depicts a flow diagram illustrating one exemplary embodiment of the operation of an actionable module. 18 depicts a flow diagram illustrating an exemplary embodiment of a media overlay publishing module. 19 is a block diagram illustrating an example of a software architecture that may be installed on a machine in accordance with some demonstrative embodiments. 20 shows a schematic diagram of a machine in the form of a computer system upon which a set of instructions may be executed to cause the machine to perform any one or more of the methodologies discussed herein in accordance with one illustrative embodiment. Fig. 21 is a block diagram illustrating a mobile device according to an exemplary embodiment. 22 depicts a flow diagram illustrating an exemplary embodiment of a method of operation for a client device.

Although the present disclosure has been described with reference to specific exemplary embodiments, it will be apparent that various modifications and changes may be made to these embodiments without departing from the broader spirit and scope of the disclosure. Accordingly, the specification and drawings are to be regarded in an illustrative rather than a restrictive sense.

Adding labels, drawings, and other works of art (eg media overlays) to images (eg photos or videos) allows users to personalize the images before saving them or posting them to a wider public. It provides a powerful way to customize, supplement, and enhance. One exemplary embodiment seeks to provide users with a set of media overlays (or other enhancements and enhancements) that can be applied to images. A set of enhancements and enhancements of an exemplary form of media overlay is determined based on ambient noise or environmental acoustic (audio) sound, a location associated with the image, and other contextual information such as a particular object associated with the image. The media overlay may be presented to the user for selection and association with the image, based on the determined location of the image or the content of the image. For example, if a user takes a photo or video on a mobile device at a music concert, a media overlay showing the name of the concert event and the band playing at the current time will be presented to the user for selection and overlay on the photo or video. can In another example, when a user initiates (or has taken) a photo or video shoot at Disneyland, an image overlay indicating the name "Disneyland" is presented to the user in a specific style. A Disneyland-themed image overlay may also be presented to the user. Presentation of the image overlay may be in response to a user performing a gesture (eg, a swipe action, long press) on a screen of the mobile device. The user can then select an image overlay and apply the image overlay to the image (eg, an image taken before or after the user makes the selection), and in this way personalize and enhance the image.

In an exemplary embodiment, the media overlay may be automatically presented to the user upon detection of a particular event. For example, when a user initiates (or has taken) a photo or video, the content within the photo or video (e.g., audio, object, location, etc.) may be presented to the user by one or more media overlays for selection. It can be triggered to be displayed. For example, a user may be taking a video that includes audio. A particular audio track may be identified from the audio, and a media overlay may be created that includes (as an example) information such as the audio track name, title, artist name, album art, and the like. This media overlay may be presented to the user for integration with video content or other content taken by the user.

A third-party entity (eg, a seller, restaurant, individual, etc.), in one exemplary embodiment, seeks to have the overlays included in the set be presented for user selection at a particular geographic location. For example, a restaurant at a particular location in San Francisco may want the restaurant name and logo included in the media overlay set to be presented to the user to augment photos taken by the user near the restaurant. According to one exemplary embodiment, this third-party entity may bid (or otherwise purchase the opportunity) that certain media overlays included in the set be presented to the user for enhancement of a particular image. Various systems and methodologies that may be used to technically implement the image enhancement techniques and capabilities described above are described below.

More specifically, various examples of media overlay publishing applications are described. A media overlay publishing application runs on a server and creates a media overlay containing content based on geographic location (also called geolocation) and other contextual information (eg, characteristics of a content item). Other media enhancements or enhancements may include audio and visual content or visual effects that may be applied to augment content or media items (eg, photos or videos) on a mobile device. Media overlay publishing applications include user-based media overlay publishing platforms and merchant-based media overlay publishing platforms.

In a user-based media overlay publishing platform, a media overlay publishing application may provide a graphical user interface (GUI) for a user to upload content and select a geolocation on a map. For example, a user can upload a logo and define a boundary on a map to identify a particular geographic location associated with the logo. Once the user submits the logo and identifies the particular geolocation, the media overlay publishing application can create a media overlay that includes the logo associated with the particular geolocation. Accordingly, mobile devices within a particular geographic location can access the media overlay.

In a merchant-based media overlay publishing platform, a media overlay publishing application may provide a GUI for a merchant to upload content, select a geolocation on a map, and submit a bid for a corresponding geolocation. The bidding process may determine the seller with the highest bid amount. The seller may then preclude posting of media overlays from other sellers at the seller's selected geographic location. Thus, the top-bid seller's media overlay may be the only media overlay that can be accessed by mobile devices located in the selected geolocation.

In another example, the media overlay includes contextual data, such as the current temperature, identification of the mobile device's geolocation (eg, Venice Beach), the name of a live event associated with the mobile's geolocation, or the name of a business. do.

In some embodiments, the server-side media overlay application provides a live event media overlay to the mobile device. A live event media overlay includes live event data associated with a live event, such as a concert event, sporting event, or awards ceremony, at the geographic location of the mobile device. For example, a user watching a soccer match may access a sports media overlay that includes the current score of the soccer match. In another example, a user attending an Oscar ceremony may access an entertainment media overlay that includes the name of an Oscar winner.

In other embodiments, the server-side media overlay application provides a social network media overlay to the mobile device. The social network media overlay may be based on social network activity of a user of the mobile device. For example, if a user follows a brand such as McDonald's on a social networking service, and the user's mobile device is located at a McDonald's restaurant, the user's mobile device can access McDonald's media overlay. Other users in the same restaurant will not be able to access McDonald's media overlays unless they follow McDonald on social networking services. In another example, the order in which media overlays are presented to users at a McDonald's restaurant may be modified such that McDonald's media overlays are served in a higher order to users who follow McDonald on a social network service.

In further embodiments, the server-side media overlay application provides the promotional media overlay to the mobile device. The promotional media overlay may be based on a promotion from the seller. For example, a media overlay may be used to implement McDonald's Monopoly game by randomly selecting a media overlay each time a mobile device user walks to a McDonald's restaurant and purchases an item. Media overlays can be used to obtain Monopoly puzzle pieces that can be exchanged for prizes.

In some embodiments, the server-side media overlay application enables the mobile device to collect media overlays. For example, a mobile overlay application provides a mobile device with persistent access to an ingested media overlay. The collected media overlays may be stored in a collection portfolio for the mobile device. The mobile device can access any of the media overlays in the collection portfolio at any time.

In some embodiments, the server-side media overlay application provides a history media overlay to the mobile device. The historical media overlay may be based on geographic locations of past places visited by the user of the mobile device. For example, when the mobile device is located in one of the Seven Wonders of the World geographic locations, the mobile device is provided with a unique media overlay associated with that one of the Seven Wonders of the World.

In another embodiment, the server-side media overlay application provides a progressive use media overlay to the mobile device. The content of the progressive use media overlay depends on the number of people who have previously used the progressive use media overlay.

In another example embodiment, a user may "purchase" a geolocation for a predetermined amount of time and select a media overlay associated with that geolocation. For example, a university may purchase and select a particular media overlay associated with the geographic location of a campus.

In another embodiment, a media overlay application provides a viral media overlay to a mobile device. For example, if a user of a mobile device obtains a disseminated media overlay at a given geolocation, that user may transmit the disseminated media overlay to the mobile device outside of the original user's geolocation. Users of mobile devices that are outside the original user's geolocation may use the radio wave media overlay during the following time. These users may pass the radio wave media overlay to other users.

In another example embodiment, a media overlay application provides an actionable media overlay to a mobile device. For example, the actionable media overlay may be a link that opens a browser page on the mobile device to obtain a coupon. The actionable media overlay may trigger other functions of the mobile device.

1 is a network diagram illustrating a network system 100 having a client-server architecture configured to exchange data over a network, according to one embodiment. For example, network system 100 may be a messaging system through which clients may communicate and exchange data within network system 100 . Data may include various functions (eg, sending and receiving text and media communications, geolocation) and aspects (eg, publishing of media overlays, management of media overlays) associated with network system 100 and its users. may be related to Although network system 100 is shown herein as having a client-server architecture, other embodiments may include other network architectures, such as peer-to-peer or distributed network architectures.

The data exchange platform 102, in one example, includes a messaging application 120 and a media overlay application 122, and provides server-side functionality to one or more clients over a network 104 (eg, the Internet). can do. The one or more clients may include users exchanging data over the network 104 using the network system 100 , and more specifically, the messaging application 120 and the media overlay application 122 . These operations may include sending, receiving (communicating) and processing data about users and content of the network system 100 . Data includes, inter alia, user profiles, messaging content, messaging properties, media properties, client device information, geolocation information, media overlay content, messaging content persistence conditions, social network information, and live event data information, content and user data. can, but is not limited to.

In various embodiments, data exchange within network system 100 may depend on user-selected functionality available through one or more clients or user interfaces (UIs). The UI may be associated with a client machine, such as a client device 110 , 112 , using a programmatic client 106 , such as a client application. Programmatic client 106 may communicate with messaging application 120 and media overlay application 122 via application server 118 . Client devices 110 , 112 include mobile devices with wireless communication components and audio and optical components for capturing various forms of media, including photos and video. Client devices 110 and 112 may further include a Global Positioning System (GPS) processor 107 .

Referring specifically to messaging application 120 and media overlay application 122 , an application program interface (API) server 114 is coupled to and provides a programmatic interface to one or more application server(s) 118 . . The application server 118 hosts the messaging application 120 and the media overlay application 122 . The application servers 118 are coupled to one or more database servers 124 that enable access to one or more databases 126 .

The API server 114 transmits and receives data, inter alia, regarding messages and media overlays via various user input tools. For example, the API server 114 may serve data with an application (eg, programmatic client 106 ) running on another client machine (eg, client devices 110 , 112 or a third party server). can transmit and receive.

In one exemplary embodiment, messaging application 120 provides a messaging mechanism for users of client devices 110 , 112 to send messages including text and media content, such as pictures and videos. Client devices 110 , 112 may access and view messages from messaging application 120 for a limited period of time. For example, the client device 110 may send a message to the client device 112 via the messaging application 120 .

Once the client device 112 accesses the message from the messaging application 120 , the message is deleted after a predefined period of time from the time the client device 112 begins viewing the message. Components of messaging application 120 are described in greater detail below with respect to FIG. 2 .

In one exemplary embodiment, the media overlay application 122 provides a system and method for operating and publishing a media overlay for messages processed by the messaging application 120 . The media overlay application 122 provides a media overlay to the client device 110 based on the geographic location of the client device 110 or characteristics of the content item (eg, photo or video). In another example, the media overlay application 122 provides a media overlay to the client device 110 based on other information, such as social network information of the user of the client device 110 .

Media overlays can include audio and visual content and visual effects. Examples of audio and visual content include photos, text, logos, animations, and sound effects. Examples of visual effects include color overlays. Audio and visual content or visual effects may be applied to media content items (eg, photos) residing on client device 110 . For example, the media overlay includes text that may be overlaid on top of a photo created at the client device 110 . In another example, the media overlay includes an identification of a location overlay (eg, Venice Beach), a name of a live event, a name of a merchant overlay (eg, Beach Coffee House), or a song title. In another example, the media overlay application 122 uses the geolocation of the client device 110 to identify a media overlay that includes the name of the vendor at the geolocation of the client device 110 . The media overlay may include other indicia associated with the merchant. Examples of markings include logos and other pictures related to the seller. Media overlays may be stored in database(s) 126 and accessed through database server 124 .

In one exemplary embodiment, the media overlay application 122 includes a user-based publishing platform that allows a user to select a geolocation on a map and upload content associated with the selected geolocation. The user may also indicate other situations in which a particular media overlay may be provided. The media overlay application 122 creates a media overlay containing the uploaded content and associates the uploaded content with the selected geolocation.

In another exemplary embodiment, the media overlay application 122 includes a merchant-based publishing platform that allows merchants to select a particular media overlay associated with a geolocation through a bidding process. For example, the media overlay application 122 associates the highest-biding seller's media overlay with a corresponding geolocation for a predefined amount of time. The components of the media overlay application 122 are described in more detail below with respect to FIG. 3 .

2 shows a block diagram illustrating one embodiment of a messaging application 120 . Messaging application 120 may be hosted on a dedicated or shared server machine (not shown) that is communicatively coupled to enable communication between the server machines. The messaging application 120 and the media overlay application 122 themselves are communicatively coupled to each other and to various data sources (eg, via an appropriate interface), such that the messaging application 120 and the media overlay application 122 are communicatively coupled. ) and allow the messaging application 120 and the media overlay application 122 to share and access common data. Messaging application 120 and media overlay application 122 may also access one or more databases 126 via database server(s) 124 .

Messaging application 120 is responsible for the creation and delivery of messages between users of programmatic client 106 . Messaging application 120 may utilize any of a number of message delivery networks and platforms to deliver messages to users. For example, the messaging application 120 may include electronic mail (e-mail), instant message (IM), short message service (SMS), text, facsimile, or voice (eg, Voice over IP (VoIP)). Messages are used to deliver messages over a wired (eg, Internet), plain old telephone service (POTS), or wireless network (eg, mobile, cellular, WiFi, Long Term Evolution (LTE), Bluetooth). can

In one exemplary embodiment, the messaging application 120 includes a media receiver module 202 , a media overlay application interface 204 , a message generator module 206 , a short message access module 208 , and a short message storage module ( 210). The media receiver module 202 receives a message from the programmatic client 106 of the client device 110 . The message may include any combination of text, picture, or video. The media receiver module 202 also receives persistence metadata associated with the message. Persistence metadata defines how long a message can be viewed. For example, the user of the client device 110 may specify that the message is permanent or can only be viewed or accessed for a user-determined amount of time (eg, 10 seconds). The media overlay application interface 204 communicates with the media overlay application 122 to access and retrieve the media overlay associated with metadata in the message. The message generator module 206 applies a media overlay to the message from the programmatic client 106 to generate a short-term message, and uses the short-term message storage module 210 to temporarily store the short-term message.

The short message access module 208 notifies the recipient of the message of the availability of the short message. The short message access module 208 receives a request to access a short message from a recipient and causes the short message to be displayed on the recipient's client device for a maximum duration specified in the persistence metadata. Once the recipient has viewed the message for the maximum duration, the short message access module 208 causes the recipient's client device to stop displaying the short message and deletes the short message from the short message storage module 210 .

3 shows a block diagram illustrating one exemplary embodiment of a media overlay application 122 . The media overlay application 122 includes a media overlay publishing module 304 and a media overlay engine 306 .

The media overlay publishing module 304 provides a platform for publishing media overlays. In one exemplary embodiment, the media overlay publishing module 304 includes a user-based media overlay publishing module 314 and a merchant-based media overlay publishing module 316 . The user-based media overlay publishing module 314 allows a user of a client device (mobile or web client) to upload content and select a geolocation for a user-based media overlay. The merchant-based media overlay publishing module 316 allows merchants to upload content, select geolocations, and submit bids for merchant-based media overlays. The user-based media overlay publishing module 314 is described in more detail below with respect to FIG. 4A . The merchant-based media overlay publishing module 316 is described in greater detail below with respect to FIG. 5A .

The media overlay engine 306 creates and supplies media overlays based on the characteristics of the content item or the geographic location of the client device. The media overlay engine 306 includes a predefined media overlay module 318 , a user-based media overlay module 320 , and a vendor-based media overlay module 322 . Media overlays include predefined media overlays from the predefined media overlay module 318 , user-based media overlays from the user-based media overlay module 320 , and vendor-based media overlays from the vendor-based media overlay module 322 . It may be based on a merchant-based media overlay.

The predefined media overlay module 318 supplies one of the predefined media overlay sets to the client device. An example of a predefined media overlay is described in more detail below with respect to FIG. 6A .

The user-based media overlay module 320 supplies the user-based media overlay generated by the user-based media overlay publishing module 314 to the client device. The merchant-based media overlay module 322 supplies the merchant-based media overlay generated by the merchant-based media overlay publishing module 316 to the client device.

4A shows a block diagram illustrating an exemplary embodiment of a user-based media overlay publishing module 314 . The user-based media overlay publishing module 314 includes a user-based content upload module 402 , a user-based geolocation selection module 404 , a user-based duration selection module 406 , and a user-based publishing engine. (408).

The user-based content upload module 402 receives content uploaded from a user. Content may include media items such as photos or videos. A user-based content upload module 402 may be implemented on a web server to allow a user to upload content using the GUI shown in FIG. 4B .

The user-based geolocation selection module 404 receives geolocation identification information from the user to identify the selected geolocation. The geolocation identification information may include an address, identification of a facility already associated with the address, Global Positioning System (GPS) coordinates, or geographic boundaries. For example, an address may include a street number, a street address, a city, a state, and a country. The user may also identify a location based on an existing facility. For example, the geolocation information may include "restaurant x" of Venice Beach. A geographic boundary identifies a region or zone. For example, a geographic boundary may define an address, a point of interest, or an area located within a predetermined radius of an existing facility.

In one exemplary embodiment, the geolocation identification information may be embedded in a message or communication signal from the client device to the user-based geolocation selection module 404 . For example, a user of the client device may take a picture of a sunset at Venice Beach, and may send the picture to the user-based geolocation selection module 404 , which may then be configured to the user-based geolocation selection module 404 . ) may extract geolocation attributes or identification information from metadata associated with sunset photos. The user-based geolocation selection module 404 may be implemented on a web server to present a GUI to a user in a web page that allows the user to select a geolocation for content, as shown in FIG. 4C .

The user-based duration selection module 406 receives from the user duration information associated with the uploaded content and the selected geolocation. The duration information may identify a time period for which the uploaded content is associated with the selected geolocation. Once that period has elapsed, the uploaded content is no longer associated with the selected geolocation. For example, if a duration of 24 hours is indicated, the media overlay engine 306 makes the user-based media overlay available to client devices located at the selected geolocation. After 24 hours have elapsed, the user-based media overlay can no longer be accessed by client devices in the selected geolocation.

Other embodiments include periodic duration information or specific duration information. For example, for periodic duration information, a user-based media overlay is posted and made available to a selected geolocation every Sunday (eg, a religion-related media overlay available on a day of worship). For specific duration information, user-based media overlays are posted and made available at selected geolocations on specific holidays or dates (eg, Thanksgiving weekend, New Year's Day).

The user-based publishing engine 408 generates a user-based media overlay that associates the uploaded content from the user-based content upload module 402 with the selected geolocation from the user-based geolocation selection module 404 . do. The user-based publishing engine 408 publishes the user-based media overlay to client devices located within the geolocation selected for the duration identified by the user-based duration selection module 406 .

In another example embodiment, the user-based publishing engine 408 determines that no other user-based media overlays exist for the same selected geolocation for the same period of time. The user-based publishing engine 408 may publish only one user-based media overlay at any time for the same selected geolocation. In another exemplary embodiment, a limit may be placed on the number of user-based media overlays available at any time for the same selected geolocation. Accordingly, the user-based publishing engine 408 may publish and make available a limited number of user-based media overlays at any time for the same selected geolocation. In another example embodiment, user-based media overlays may only be posted to the uploading user's contacts or "friends".

4B shows an example of a GUI 410 for uploading content and selecting a geographic area on a map. GUI 410 includes map 412 , image upload box 414 , location selection button 416 , overlay title box 418 , and submit button 420 . The image upload box 414 enables a user to upload content (eg, photos) to the user-based content upload module 402 .

The location selection button 416 allows the user to identify a geographic location by drawing a boundary on the map 412 or by entering an address or zip code.

The identified geolocation is submitted to the user-based geolocation selection module 404 . The overlay title box 418 allows the user to submit a name for the media overlay. The user may submit the content and requested geolocation by clicking on the submit button 420 . Once the content and requested geolocation have been submitted, the user-based publishing engine 408 creates a user-based media overlay containing the uploaded content for the identified geolocation.

4C shows an example in which user-identified boundary points 424 , 426 , 428 and 430 on map 412 define geolocation 422 . The user uploaded a photo of the sun 415 displayed in the image upload box 414 . The user has entered the title of the content "Fun under the Sun!" in the overlay title box 418 . The user may submit a photo and geolocation 422 of the sun 415 by clicking the submit button 420 . Once the photo and geolocation 422 of the sun 415 have been submitted, the user-based publishing engine 408 creates a user-based media overlay.

4D shows an example of publishing a user-based media overlay. Media overlay application 122 detects that user 1816's mobile device 1802 is located at geolocation 422 . The media overlay application 122 retrieves the user-based media overlay 440 corresponding to the geolocation 422 and posts the user-based media overlay 440 to the mobile device 1802 . A user-based media overlay 440 is applied to the media content 1806 on the display 1804 of the mobile device 1802 .

5A shows a block diagram illustrating an exemplary embodiment of a merchant-based media overlay publishing module 316 . The merchant-based media overlay publishing module 316 includes a merchant-based content upload module 502 , a merchant-based geolocation selection module 504 , a merchant-based duration selection module 506 , a merchant-based bidding module ( 508 ), and a merchant-based publishing engine 510 .

The merchant-based content upload module 502 receives content from a merchant. Content may include media items such as photos, videos, graphics, or text. The merchant-based content upload module 502 may be implemented on a web server to allow merchants to upload content using a web page.

The merchant-based geolocation selection module 504 receives geolocation information from the merchant to identify the selected geolocation. The geolocation identification information may include an address of the facility, an identification of the facility already associated with the address, GPS coordinates, or geographic boundaries. For example, the address of a facility may include a street number, street address, city, state, and country. The seller may also identify a location based on an existing facility. For example, the geolocation information may include "restaurant x" of Venice Beach. A geographic boundary identifies a region or zone. For example, a geographic boundary may define an address, a point of interest, or an area located within a predetermined radius of an existing facility. Vendors can also define geographic boundaries by drawing virtual fences on the map. The merchant-based geolocation selection module 504 may be implemented on a web server to allow merchants to draw boundaries on a map of a web page.

The merchant-based duration selection module 506 receives the uploaded content and duration information associated with the selected geolocation from the merchant. The duration may identify a period of time during which the uploaded content is associated with the selected geolocation. Once that period has elapsed, the uploaded content is no longer associated with the selected geolocation. Other embodiments include periodic duration information or specific duration information. For example, for periodic duration information, a merchant-based media overlay is posted and made available (eg, at a selected geolocation (eg, a corner of two identified streets) every Saturday night). Nightclub-related media overlays available every Saturday night). For specific duration information, the selected media overlay is posted or made available at the selected geolocation near a specific date (eg, a party event date).

The seller-based bidding module 508 provides an interface that allows sellers to submit bids for a common geographic location. A common geolocation may include, for example, the same street address. For example, several businesses may have the same street address but different suit numbers within a shopping center. 5B shows an example of a common geographic location 516 . Vendor A geolocation boundaries 512 overlap with vendor B geolocation boundaries 514 to define a common geolocation 516 . Thus, sellers A and B may submit respective bids corresponding to a common geographic location 516 . In one exemplary embodiment, the merchant-based geolocation selection module 504 determines a common geolocation from the geolocations selected by the merchants. The seller-based bidding module 508 identifies the highest bidder for the common geolocation 516 and provides the highest bidder with the ability to exclude other seller-based media overlays from the common geolocation 516 for a predefined amount of time. give to

In another exemplary embodiment, the seller-based bidding module 508 prorates the bid amount based on their corresponding duration information. For example, Seller A submits a bid of $100 per day for a particular geographic location. Seller B submits a bid of $160 over two days for the same specific geographic location. The seller-based bidding module 508 prorates the bid amount from seller B over a day (eg, $80), compares both bids for the same time period (eg, 1 day), and the highest bidder can be decided

The merchant-based publishing engine 510 creates a merchant-based media overlay that associates the highest bidder's uploaded content with the geolocation identified by the highest bidder. The merchant-based publishing engine 510 publishes the merchant-based media overlay to client devices located at the geolocation selected by the highest bidder for the duration identified by the merchant-based duration duration selection module 506 .

Vendor-based media overlays of other vendors within the common geolocation 516 are excluded from publication. In another embodiment, a quota may be allocated with respect to the number of vendor-based media overlays available for a common geolocation 516 . For example, the merchant-based publishing engine 510 may publish and make available a limited number of merchant-based media overlays (eg, up to two merchant-based media overlays) to a common geolocation 516 . can

In another example embodiment, the seller-based publishing engine 510 establishes a priority relationship associating the highest bidder's uploaded content with the geolocation selected by the highest bidder. For example, the order in which the media overlays are displayed at the client device 110 may be adjusted based on results from the seller-based bidding module 508 . The seller's media overlay with the highest bid may be given priority and displayed first on the client device 110 . Media overlays from other vendors may be displayed on the client device 110 after the highest bidder's media overlay. In another exemplary embodiment, a seller may bid at any location where it maintains a presence. Accordingly, the restaurant chain may have its media overlay posted at each of the restaurant chain locations.

5C shows an example of a GUI 520 for uploading content and selecting a geographic location on a map. GUI 520 includes map 522 , image upload box 524 , location selection button 526 , overlay title box 528 , bid amount input box 530 , campaign length input box 532 , and a submit button (534). Image upload box 524 allows merchants to upload content (eg, photos, videos, or animations) to merchant-based content upload module 502 . The location selection button 526 allows the seller to identify a geographic location by drawing a boundary on the map 522 or by entering an address or zip code. Overlay title box 528 allows vendors to submit a name for the media overlay. The bid amount input box 530 allows the seller to enter a bid amount for the identified geographic location. The campaign length input box 532 allows the seller to specify the length of the campaign for which the uploaded content is associated with the identified geolocation. The seller may submit the uploaded content and the entered information by clicking the Submitted button 534 .

FIG. 5D shows an example in which merchant A defines a geolocation 540 by identifying boundary points 542 , 544 , 546 and 548 on a map 522 . Seller A uploaded the photo 525 displayed in the image upload box 524 . Merchant A entered the title "Coffee Shop A" in overlay title box 528 , a bid of $300 in bid amount input box 530 , and a campaign length of 30 days in campaign length input box 532 . Seller A submits the photo 525 , the requested geolocation 540 , and other entered information by clicking the submit button 534 . The merchant-based publishing engine 510 creates a media overlay for Merchant A.

FIG. 5E shows an example in which another vendor, vendor B, identifies boundary points 552 , 554 , 556 and 558 on map 522 to define geolocation 550 . Seller B uploaded the photo 527 displayed in the content upload box 524 . Seller B has entered "coffee shop B" in the overlay title box 528 , a bid of $500 in the bid amount input box 530 , and a campaign length of 30 days in the campaign length input box 532 . Seller B submits the photo 527 , the requested geolocation 550 , the bid amount, and the campaign length by clicking the submit button 534 . The merchant-based publishing engine 510 creates a media overlay for merchant B.

5F shows a diagram illustrating an example of a merchant-based media overlay selected based on a bidding process. Merchant A's geographic location 540 and Merchant B's geographic location 550 overlap at a common geographic location 545 . User 1816 is located at a common geographic location 545 and uses mobile device 1802 to create media content 1806 on display 1804 of mobile device 1802 (eg, user 1816 Take photo). The seller's media overlay with the highest bid for the common location 545 is posted on the mobile device 1802 . In this example, Seller B placed a higher bid than Seller A. Accordingly, Vendor B's media overlay 560 is provided and displayed on top of the media content 1806 on display 1804 . Media overlay 560 also includes uploaded content from vendor B. Further, a "seller" in the context of the present exemplary embodiment may include not only entities involved in the transaction or sale of goods, but also any other entity, including individuals, universities, non-profit organizations, student organizations, clubs, and the like. .

6A shows a block diagram illustrating one exemplary embodiment of a predefined media overlay module 318 . The predefined media overlay module 318 is, for example, a live event module 602 , a social network module 604 , a promotion module 606 , an aggregation module 608 , an incremental use module 610 , a radio wave use a module 612 , an actionable module 614 , a history recognition module 616 , and an audio module 618 .

The live event module 602 generates a media overlay based on the live event information. The live event information may relate to a live match score of a sporting event associated with a corresponding geolocation, or a live news event about an entertainment (eg, concert) or social event associated with the corresponding geolocation. For example, the user of the client device 110 attends a live sporting event (eg, a match) at a stadium. Accordingly, the media metadata from the client device 110 may identify the location of the stadium by date and time. The live event module 602 uses the information to retrieve a live event associated with the location, date and time of the stadium. The live event module 602 retrieves a current or near current match score associated with a live sporting event at the stadium (eg, via the ESPN API). The live event module 602 may also retrieve insignia or team logos associated with the live sporting event. Accordingly, the live event module 602 generates a media overlay containing the latest scores based on the news source covering the live sporting event.

In another example, the user of the client device 110 attends a live social event at a venue. Similarly, the media metadata identifies the location of the venue by date and time. The live event module 602 uses the information to retrieve a live event associated with the location, date and time of the venue, from sources such as a social network server or news media service. The live event module 602 retrieves a news feed associated with a live social event at the venue. Accordingly, the live event module 602 generates a media overlay comprising information or content based on news retrieved from a news feed associated with a live social event at the venue.

The social network module 604 generates a media overlay based on social network information of a user of the client device 110 . The social network information may include social network data retrieved from a social network service provider. Social network data may include, inter alia, the user's profile data, the user's "likes", the facilities the user follows, the user's friends, and the user's posts. For example, a media overlay associated with a restaurant may be available to a user at the restaurant's location when the user identifies himself as a fan of the restaurant or "likes" the restaurant by a social network service provider. . In another example, the rank or priority of displaying the media overlay on the user's client device 110 may be based on the user's profile or the number of times the user's "check-ins" at a restaurant.

In another example embodiment, the media overlay may be limitedly available to only the user of the client device 110 and the user's social network (eg, friends or other users in different categories). Thus, users can forward media overlays to their friends.

Promotion module 606 creates media overlays for promotions (eg, games, contests, raffles). For example, a unique set of media overlays may be created. One media overlay from a unique set of media overlays is a picture or object (eg, name, logo, product, etc.) taken by the user or when the client device 110 is at a predefined location associated with the media overlay. When recognized in the video, it may be provided to the client device 110 . For example, a user may visit a fast food restaurant. The media metadata from the client device 110 identifies the location of the fast food restaurant. The promotion module 606 retrieves a unique media overlay from the set of unique media overlays and provides it to the client device 110 . The promotion module 606 may remove the unique media overlay from the unique media overlay set after it is provided to the client device 110 . In another embodiment, the promotion module 606 removes the unique media overlay from the set of unique media overlays after the unique media overlay has been provided to another client device a predefined number of times.

In another example, a promotional media overlay may be created based on an occurrence in another application (eg, passed to the media overlay application 122 via an API). For example, the media overlay may be an image of a character from a video game, provided to the client device 110 or made available to the user when the user completes a level in the video game.

A media overlay may be available for one use, multiple uses, for a period of time, or indefinitely.

In another example, the media overlay is provided to or made available to the user on the client device 110 based on another account the user has linked with the media overlay application (eg, via an account associated with the media overlay application). can be For example, a user may have an Xbox account linked to a user account in the media overlay application.

The user is a first user and that first user can beat a second user on Xbox (eg, play a soccer match). The game or application on Xbox may communicate to the media overlay application 122 (eg, via an API) that the first user beat the second user on Xbox. A media overlay including information regarding the fact that the first user has beaten the second user may be generated and provided to the client device 110 or made available to the first user. For example, the media overlay may include the fact that the first user beats the second user, a score, an overall record of the first user versus the second user, and the like.

In another example, a user may link a frequent passenger account (eg, an American Airlines account) to a media overlay application. When a user is on a plane or is on a flight associated with a frequent passenger account (eg, American Airlines flight 10 from LAX to JFK), the user may receive a media overlay associated with that flight. For example, a user may receive a media overlay that includes an airline logo, flight routes, departure and arrival times, decorative images of departure and arrival cities (eg, Los Angeles and New York), and the like.

Media overlays contain content related to games or promotions. In another example, the media overlay may include dynamic content tailored based on a game or promotion. For example, the dynamic content may include the current number of remaining media overlays of the game or promotion. Media overlays from the promotion module 606 may be "collected" by the client device 110 . For example, the client device 110 may store the media overlay in the collection of the client device 110 . It may be redeemed for a prize upon collection of each overlay of the predefined media overlay set.

The aggregation module 608 creates an collectible media overlay.

For example, the client device 110 is provided with a media overlay associated with the geolocation of the client device 110 . The media overlay may be collected by the client device 110 and made permanently available to the client device 110 . The client device 110 may store the collected media overlays in a collection folder of the client device 110 .

The progressive use module 610 creates a media overlay with dynamic content that changes based on the number of uses of the media overlay. For example, the media overlay may be set to be used only a limited number of times. Whenever a media overlay is presented to a client device, the content of the media overlay is adjusted. For example, the media overlay may include a fundraising progress bar in which the level of the bar is raised whenever the media overlay is used. The dynamic content within the media overlay may include a countdown displaying the number of remaining uses of the media overlay.

The disseminated use module 612 creates a media overlay that can be forwarded to other users outside the geolocation associated with the media overlay. For example, the client device 110 receives the media overlay based on the geographic location of the client device 110 . The client device 110 may send the media overlay to the client device 112 outside the geolocation of the client device 110 . The forwarded media overlay may be available by the client device 112 for a predefined time limit (eg, 1 hour). Similarly, the client device 112 may forward the media overlay for use within a predefined time limit to other client devices outside the geolocation of the client device 110 .

The actionable module 614 creates a media overlay having an action associated with the content item of the media overlay. For example, the media overlay may launch a browser of the client device 110 and open a predetermined website in the browser. In another embodiment, the media overlay may open another function (eg, a payment application) or execute another program on the client device 110 . For example, a user may tap the media overlay to download or display a coupon associated with the media overlay on the client device 110 .

The historical recognition module 616 generates a media overlay based on the geolocation of the client device 110 and historical events associated with the geolocation. For example, the media overlay may include a pyramid picture associated with the geographic location of the client device 110 . Media overlays may be collected based on historical events, or for each of the Seven Natural Wonders of the World, for example. For example, when a user visits a national park, a media overlay associated with the national park may be collected. The client device 110 may collect all media overlays associated with the national park.

The audio module 618 generates a media overlay based on audio content (eg, an audio track) associated with the content item received from the client device 110 . For example, the media overlay may include one or more of a title associated with the audio content, an artist name associated with the audio content, an album name associated with the audio content, album artwork associated with the audio content, and lyrics associated with the audio content.

In another embodiment, the media overlay may open other functions (eg, purchasing a song associated with the audio content, obtaining more information about the song, etc.). For example, the user may tap the media overlay to download or display a coupon associated with the media overlay on the client device 110 .

6B shows a diagram illustrating an example of a media overlay 1820 with live data content. The media overlay 1820 includes live data associated with the geolocation of the mobile device 1802 . For example, live data includes live weather conditions 1822 associated with the geolocation of mobile device 1802 and latest score updates 1824 of sporting events. The mobile device 1802 displays the media overlay 1820 (eg, as a transparent overlay) on top of the media content 1806 on the display 1804 . In one exemplary embodiment, the media overlay 1820 may be implemented with the live event module 602 of FIG. 6A .

6C shows a diagram illustrating an example of a media overlay 1830 with promotional content. For example, the media overlay 1830 includes a digital coupon 1832 that can be redeemed for merchandise at a coffee shop. Media overlay 1830 can include dynamic content 1834 . For example, dynamic content 1834 may include a remaining number of times a coupon may be used. Media overlay 1830 may also include an actionable area 1836 associated with an executable function. For example, if the user taps the actionable area 1836 , the media overlay 1830 is forwarded to the user's friend's mobile device. Mobile device 1802 displays media overlay 1830 on top of media content 1806 on display 1804 . In one exemplary embodiment, the media overlay 1830 may be implemented with the social network module 604 , the promotion module 606 , the progressive use module 610 , and the actionable module 614 of FIG. 6A .

6D shows a diagram illustrating an example of an collectible media overlay 1840 . The collectible media overlay 1840 may be randomly supplied to the mobile device 1802 in response to detecting the mobile device 1802 at a geolocation associated with the collectible media overlay 1840 . Collectable media overlay 1840 can be stored on mobile device 1802 . Once the mobile device 1802 detects that the associated collectable media overlay has been saved, the mobile device 1802 may cause the associated collectible media overlay or a corresponding unique media overlay to be displayed on the display 1804 .

Mobile device 1802 displays media overlay 1840 on top of media content 1806 on display 1804 . In one exemplary embodiment, the media overlay 1840 may be implemented with the ingestion module 608 of FIG. 6A .

6E shows a diagram illustrating an example of a propagation media overlay 1850 . The disseminated media overlay 1850 can include dynamic content 1854 and an actionable area 1852 . For example, dynamic content 1854 shows a progress bar and goals of a fundraising event. The progress bar is adjusted based on the amount that has recently occurred. The actionable area 1852 is to cause a financial transaction (eg, a donation) and communication (eg, a message to another device using the messaging application 120) to another mobile device 1802 . ) can be triggered. The mobile device 1802 displays a disseminated media overlay 1850 on top of the media content 1806 on the display 1804 . In one exemplary embodiment, the disseminated media overlay 1850 may be implemented with the progressive usage module 610 , the disseminated usage module 612 , and the actionable module 614 of FIG. 6A .

6F shows a diagram illustrating an example of a media overlay 1860 with audio content. Media overlay 1860 may include dynamic content 1864 including, for example, a song title 1862 and a band name. Dynamic content 1864 may also include an actionable area 1866 . The actionable area 1866 may provide additional information about the song, how to purchase the song, and the like. Mobile device 1802 can display media overlay 1860 on top of media content 1806 (eg, a content item) on display 1804 . In one example, media overlay 1860 may be implemented with audio module 618 and actionable module 614 of FIG. 6A . Alternatively, or in addition, the actionable area 1866 may be at another location (eg, below the media content 1806 ). In another example, a user gesture (eg, a finger swipe up on media content 1806) may cause an action to occur (eg, purchase a song, listen to an entire track, or contact an artist). learn more about it, etc.).

7 depicts an interactive diagram representing an exemplary embodiment of the operation of the user-based media overlay publishing module 314 . In operation 711 , the first user's client device 110 uploads the content and sends the requested geolocation and requested duration to the media overlay application 122 . In operation 713 , the media overlay application 122 creates a media overlay based on the uploaded content and associates the media overlay with the requested geolocation for the requested duration. In one exemplary embodiment, operations 711 and 713 may be implemented with the user-based media overlay publishing module 314 of FIG. 3 .

In operation 714 , the second user's client device 112 sends geolocation information to the messaging application 120 . In operation 715 , the messaging application 120 identifies, from the media overlay application 122 , a media overlay based on the geolocation information of the client device 112 . At operation 718 , the media overlay application 122 provides the identified media overlay to the client device 112 . In one embodiment, operations 716 and 718 may be implemented with the media overlay engine 306 of FIG. 3 .

8 shows an interactive diagram representing an exemplary embodiment of the operation of the merchant-based media overlay publishing module 316 . At operation 808 , merchant A's client device 802 uploads the content along with geolocation information (eg, geolocation X) and bid amount (eg, bid A) to media overlay application 122 , Form overlay A. At operation 810 , Merchant B's client device 804 uploads the content to the media overlay application 122 along with the same geolocation information (eg, geolocation X) and bid amount (eg, bid amount B) to media Form overlay B. At operation 812 , the media overlay application 122 determines the highest bidder and associates the highest bidder's media overlay with geolocation X. For example, if bid A is greater than bid B, then media overlay A is provided to client devices at geolocation X. In one embodiment, operations 808 , 810 , and 812 may be implemented with the merchant-based media overlay publishing module 316 of FIG. 3 .

In operation 814 , the client device 806 at geolocation X sends its geolocation information to the messaging application 120 . At operation 816 , the messaging application 120 identifies, from the media overlay application 122 , a media overlay associated with geolocation X. In operation 818 , the media overlay application 122 supplies the media overlay A to the client device 806 . In one embodiment, operations 816 and 818 may be implemented with the media overlay engine 306 of FIG. 3 . In another exemplary embodiment, media overlay application 122 is media overlay A with instructions to client device 806 to display media overlay A first before media overlay B because seller A is the highest bidder. and B both to the client device 806 .

9 shows a flow diagram illustrating an exemplary embodiment of a method 901 of the user-based media overlay publishing module 314 . At operation 903 , the user-based media overlay publishing module 314 receives the uploaded content and the requested geolocation from the first client device. In one exemplary embodiment, operation 903 may be implemented with the user-based content upload module 402 , the user-based geolocation selection module 404 , and the user-based duration selection module 406 of FIG. 4A . have.

At operation 904 , the user-based media overlay publishing module 314 forms a user-based media overlay that includes the uploaded content and is associated with the requested geolocation. In one exemplary embodiment, operation 904 may be implemented with the user-based publishing engine 408 of FIG. 4A .

At operation 906 , the user-based media overlay publishing module 314 receives geolocation information from the second client device. At operation 908 , the user-based media overlay publishing module 314 determines whether a geolocation of the second client device is within a requested geolocation from the first client device. In operation 909 , the user-based media overlay publishing module 314 is configured to, in response to the geolocation of the second client device being within the requested geolocation from the first client device, from the first client device to the second client device Publish user-based media overlays. In one exemplary embodiment, operation 909 may be implemented with the user-based media overlay module 320 of FIG. 3 .

In operation 911 , the media overlay engine 306 is configured to, in response to the geolocation of the second client device not being within the geolocation requested from the first client device, a predefined media corresponding to the geolocation of the second client device. Supply the overlay to the second client device. In one exemplary embodiment, operation 911 may be implemented with the predefined media overlay module 318 of FIG. 3 .

10 shows a flow diagram illustrating one exemplary embodiment of a method 1000 of operation for the merchant-based media overlay publishing module 316 . In operations 1002 and 1004 , the merchant-based media overlay publishing module 316 receives the uploaded content, geolocation information, and a corresponding bid amount from the merchants. For example, in operation 1002 , the merchant-based content upload module 502 receives content A from merchant A. Vendor-based geolocation selection module 504 receives geolocation X from merchant A. The seller-based bidding module 508 receives bid amount A from seller A.

At operation 1004 , the merchant-based content upload module 502 receives content B from merchant B. Vendor-based geolocation selection module 504 receives geolocation X from merchant B. The seller-based bidding module 508 receives the bid amount B from the seller B.

At operation 1006 , the highest bid amount is determined. In one exemplary embodiment, operation 1006 may be implemented with the seller-based bidding module 508 of FIG. 6A . If bid A is greater than bid B, then merchant-based publishing engine 510 generates merchant-based media overlay A based on content A and geolocation X in operation 1008 . At operation 1010 , the merchant-based media overlay module 322 supplies the merchant-based media overlay A to client devices at geolocation X.

If the bid amount B is greater than the bid amount A, then in operation 1014 , the merchant-based publishing engine 510 generates a merchant-based media overlay B based on the content B and the geolocation X. At operation 1016 , the merchant-based media overlay module 322 supplies the merchant-based media overlay B to client devices at geolocation X.

11 shows a flow diagram illustrating one exemplary embodiment of a method 1100 of operation for a live event module 602 . At operation 1104 , the live event module 602 receives geolocation information from the client device. At operation 1106 , the live event module 602 identifies a live event associated with geolocation information. At operation 1108 , the live event module 602 accesses live event data related to the live event. In operation 1110 , the live event module 602 generates a live event media overlay based on the live event data. At operation 1112 , the live event module 602 supplies the live event media overlay to the client device.

12 shows a flow diagram illustrating an exemplary embodiment of a method 1200 of operation for a social network module 604 . In operation 1202 , the social network module 604 receives social network information from the client device. In operation 1204 , the social network module 604 accesses social network data from the social network service provider based on the social network information from the client device. At operation 1206 , the social-networking module 604 identifies a geolocation from geolocation information of the client device. At operation 1208 , the social network module 604 generates a social network-based media overlay based on the geolocation and social network data of the client device. At operation 1210 , the social network module 604 supplies the social network-based media overlay to the client device.

13 shows a flow diagram illustrating an exemplary embodiment of a method 1300 of operation for a promotion module 606 . At operation 1302 , the promotion module 606 creates a set of media overlays for the merchant for the predefined geolocation. At operation 1304 , the promotion module 606 receives geolocation information from the client device. At operation 1306 , the promotion module 606 identifies a geolocation of the client device from the geolocation information. At operation 1308 , the promotion module 606 accesses a set of media overlays for the merchant associated with the geolocation. In operation 1310 , the promotion module 606 randomly selects at least one media overlay from the set of media overlays. At operation 1312 , the promotion module 606 supplies the randomly selected media overlay(s) to the client device.

14 depicts a flow diagram illustrating an exemplary embodiment of a method 1400 of operation for a collection module 608 . At operation 1402 , the aggregation module 608 receives geolocation information from the client device. At operation 1404 , the aggregation module 608 determines a geolocation of the client device from the geolocation information. At operation 1406 , the aggregation module 608 accesses media overlays associated with the geolocation of the client device. At operation 1408 , the aggregation module 608 stores the media overlay in a media overlay collection associated with the client device. In operation 1410 , the aggregation module 608 provides the media overlays in the media overlay collection to the client device for use.

15 depicts a flow diagram illustrating one exemplary embodiment of a method 1500 of operation for the incremental use module 610 . At operation 1502 , the progressive use module 610 creates an incremental use media overlay for a given geographic location. At operation 1504 , the progressive use module 610 receives geolocation information from the first client device at the geolocation. In operation 1506 , the progressive use module 610 provides the progressive use media overlay to the first client device, and generates a first modified media overlay based on the progressive use media overlay. At operation 1508 , the progressive use module 610 receives geolocation information from the second client device at the geolocation. In operation 1510 , the progressive use module 610 supplies the first modified media overlay to the second client device, and generates a second modified media overlay based on the first modified media overlay.

16 shows a flow diagram illustrating an exemplary embodiment of a method 1600 of operation for the radio wave utilization module 612 . In operation 1602 , the radio wave use module 612 generates a media overlay for a given geographic location. At operation 1604 , the radio wave utilization module 612 receives media metadata from the first client device at the geolocation. At operation 1606 , the radio wave use module 612 provides the media overlay to the first client device at the geographic location. At operation 1608 , the radio wave use module 612 receives a request from the first client device to forward the media overlay to a second client device outside its geolocation. At operation 1610 , the radio wave use module 612 provides the media overlay to a second client device outside its geolocation for a limited time.

17 shows a flow diagram illustrating one exemplary embodiment of a method 1700 of operation for an actuable module 614 . At operation 1702 , the actionable module 614 creates an actionable media overlay having an actionable portion associated with the function. At operation 1704 , the actionable module 614 provides the actionable media overlay to the first client device. At operation 1706 , the actionable module 614 receives a media item (eg, a photo) with a media overlay from the first client device. At operation 1708 , the actionable module 614 supplies the media item with the media overlay to the second client device. At operation 1710 , the actionable module 614 identifies a selection of the actionable portion from the second client device. At operation 1712 , the actionable module 614 executes a function associated with the actionable portion at the second client device.

18 depicts a flow diagram illustrating an exemplary embodiment of a method 1800 of operation for the media overlay publishing module 304 . At operation 1882 , the media overlay publishing module 304 may receive a content item (eg, photo, video, etc.) from the client device 110 (eg). For example, a user of the client device 110 may take a photo or video using the client device 110 . While the user is taking the photo or video, or after the user takes the photo or video, the client device 110 may display a portion of the video, audio related to the photo or video (eg, playing in the background), or The captured photo or video may be sent to the media overlay posting module 304 .

At operation 1884 , the media overlay publishing module 304 may determine whether one or more characteristics of the content item match a trigger for the media overlay. Exemplary characteristics or triggers include a geolocation associated with the content item or client device 110 , audio content within the content item, a velocity of the client device 110 based on movement data from the client device 110 , and a time associated with the content item. stamps, data associated with machine-readable codes (eg, barcodes, QR codes), objects within the content item, adding new recipients to the content item, and the like. If one or more characteristics of the content item match a trigger for the media overlay, then at operation 1886 , the media overlay publishing module 304 may generate a media overlay associated with the content item. Media overlays may include creative overlays, geo lenses, audio filters, music clips, creative tools, access to additional features or functionality , audio clips, and the like.

For example, the media overlay publishing module 304 may detect audio content within the content item and identify an audio track based on the audio content within the content item (eg, a known fingerprint of the audio content). identifying an audio track by matching with fingerprints). The audio track may trigger the creation of a media overlay associated with the audio track. A media overlay can be created that includes at least one of a title associated with the audio track, an artist name associated with the audio track, an album name associated with the audio track, album artwork associated with the audio track, lyrics associated with the audio track, and the like (eg, For example, see Figure 6f).

In one example, the audio track may be playing at a live event, such as a concert. For example, different media overlays may be created based on which band is playing at a concert, which song is playing, and the like. The specific song or band being played, the next song or band to be played, the previous song or band played, the song and/or band highlighted in the list of songs and/or bands playing or to be played during the event, when and when the song will be played A media overlay may be created that includes one or more timestamps of when it will end, and the like. The media overlay publishing module 304 may use other information associated with the event (eg, schedule information) to supplement information received from the client device 110 .

In another example, the media overlay may provide additional audio content for inclusion in the content item. For example, a content item may include one or more photos or videos with a particular audio track playing in the background. The media overlay publishing module 304 may identify the audio track as described above and provide a set of consecutive versions of the audio track to the photo(s) or video.

In another example, the media overlay publishing module 304 can detect a particular object (eg, in a photo or video) in a content item and identify the object (eg, using object recognition technology). have. A media overlay associated with that object may be created. For example, a media overlay may be provided to the client device 110 in response to a user taking a photo or video of an object (eg, an incentive or reward for taking a photo or video). A media overlay may be associated with the object or may have additional information about the object. For example, the media overlay publishing module 304 may recognize an object indicating that the user is in Disneyland and present a Disneyland-themed media overlay. In another example, the media overlay publishing module 304 may recognize objects such as logos, brand names, products, specific people (eg, celebrities), and the like.

In another example, the media overlay publishing module 304 can detect data from a machine-readable code (eg, barcode, QR code, etc.) in the content item. A media overlay may be created or selected based on the provided code. For example, machine-readable code may be generated for a music artist, and a media overlay may be generated for the music artist. The user may use the client device 110 to scan a machine-readable code associated with a music artist, and the media overlay publishing module 304 may create a media overlay associated with the music artist. Additionally or alternatively, additional accessories or features (eg, tools for photo or video editing, etc.) may be provided to the user.

In another example, the media overlay publishing module 304, as discussed in more detail above, detects a geolocation associated with a content item, and a media overlay associated with the geolocation (eg, having a real-time score of a game). media overlay, media overlay of concert information, etc.),

At operation 1888 , the media overlay publishing module 304 may send the media overlay to the client device 110 . A media overlay may be displayed on the client device 110 (eg, overlaid over a photo or image), or may have one or more media overlays (eg, audio media overlay, creative tools, etc.) for the photo or video. A notification indicating that it is available may be displayed to the user. A user may access one or more media overlays by interacting with the client device 110 (eg, pressing a button, long-pressing the screen, swiping, or other gesture, etc.). Additionally or alternatively, the media overlay may already be stored locally on the client device 110 , and the media overlay publishing module 304 displays an indication to "unlock" the media overlay to provide to the client device ( 110) can be transmitted.

A user may choose to use a media overlay and send an item of content containing the media overlay to one or more recipients. The media overlay publishing module 304 may receive a content item including a media overlay in operation 1890 . The media overlay posting module 304 may transmit the content item with the media overlay to the second client device 112 (or one or more client devices) in operation 1892 . The content item with the media overlay may be displayed on the second client device 112 .

Prior to sending the content item with the media overlay to the second client device, the media overlay publishing module 304 may generate a second media overlay for the content item. The media overlay publishing module 304 can send the content item with the media overlay (eg, the first media overlay) and the second media overlay to the second client device 112 .

For example, a first media overlay may include audio track information, and a second media overlay may include other information about an audio track that may be displayed and/or accessed by a user of the second client device ( For example, detailed information about a track, artist, album or lyrics, how to purchase an audio track or album, etc.) or a shortcut. In one example, the second media overlay may be an actionable media overlay as described above.

Characteristics of a content item may trigger other things in addition to or instead of a media overlay. For example, geolocation information indicating that a user is in a particular store may trigger access to a particular eCommerce store related to the actual store entered by the user.

The example above describes a media overlay publishing module 304 that determines whether one or more characteristics of a content item match a trigger for the media overlay. In another example embodiment, the client device 110 may determine (eg, via one or more applications) whether one or more characteristics of the content item match a trigger for the media overlay. 22 shows a flow diagram illustrating an exemplary embodiment of a method 2200 of operation for a client device 110 .

At operation 2202 , the client device 110 generates a request for one or more media overlay(s). The request may simply be a request for available media overlays, or the request may include additional information. For example, the request may include a location associated with the client device 110 , a time of day, data related to a content item, and the like. For example, a user of the client device 110 may take a photo or video using the client device 110 . While the user is taking the photo or video, or after the user takes the photo or video, the client device 110 may display a portion of the video, audio related to the photo or video (eg, playing in the background), or The captured photo or video may be sent to the media overlay posting module 304 .

In operation 2204 , the client device 110 sends a request received by the media overlay publishing module 304 to the data exchange platform 102 . The media overlay publishing module 304 receives a request from the client device 110 and returns an associated media overlay.

In one example, the media overlay publishing module 304 may return all available media overlays. For example, the media overlay publishing module 304 may return all available media overlays and an associated trigger for each media overlay. In another example, the media overlay publishing module 304 can determine which media overlay(s) have not yet been sent to the client device 110 . For example, there may be one or more new media overlays since the client device 110 last requested the media overlay.

In another example, the media overlay publishing module 304 may return only relevant media overlays using the information provided in the request. For example, the media overlay publishing module 304 can determine which media overlays are appropriate for the location, time, content item, etc. of the client device 110 .

At operation 2206 , the client device 110 receives one or more media overlays. The client device 110 may store one or more media overlays in a database or other storage structure on the client device 110 . Client device 110 may also receive trigger(s) associated with one or more media overlays and store the one or more trigger(s) in a database or other storage structure on client device 110 .

At operation 2208 , the client device 110 determines whether one or more characteristics of the content item match a trigger for the media overlay. Exemplary characteristics or triggers include a geolocation associated with the content item or client device 110 , audio content within the content item, a velocity of the client device 110 based on movement data from the client device 110 , and a time associated with the content item. stamps, data associated with machine-readable codes (eg, barcodes, QR codes), objects within the content item, adding new recipients to the content item, and the like. If one or more characteristics of the content item match a trigger for the media overlay, then in operation 2210 , the client device 110 generates a media overlay associated with the content item from one or more media overlays received from the media overlay publishing module 304 . do. For example, the client device 110 may determine which of the media overlay(s) received from a media overlay publication on the module 304 is associated with the trigger. As described above, media overlays include creative overlays, geo lenses, audio filters, music clips, creative tools, access to additional features or functions, audio clips, lenses that magnify faces or other objects within a content item, content items. It can include lenses that apply animations or other effects to the .

A trigger may be associated with a single media overlay or more than one media overlay. For example, the first trigger may be associated with a single media overlay to be displayed to the user. In another example, the second trigger may be associated with two media overlays, both of which will be displayed to the user. The user may select one or more of the media overlays. Furthermore, there may be more than one trigger associated with a single media overlay. For example, a third trigger or a fourth trigger may be associated with a single media overlay. A single media overlay is displayed to the user when a third trigger or a fourth trigger (or both) is detected.

The client device 110 may determine whether one or more characteristics of the content item match a trigger for the media overlay, as described above. For example, the client device 110 may detect audio content within the content item and identify an audio track based on the audio content within the content item (eg, a fingerprint of the audio content with a known fingerprint). to identify an audio track by matching it with a field). The audio track may trigger the creation of a media overlay associated with the audio track. A media overlay can be created that includes at least one of a title associated with the audio track, an artist name associated with the audio track, an album name associated with the audio track, album artwork associated with the audio track, lyrics associated with the audio track, and the like (eg, For example, see Figure 6f).

In one example, the audio track may be playing at a live event, such as a concert. For example, different media overlays may be created based on which band is playing at a concert, which song is playing, and the like. The specific song or band being played, the next song or band to be played, the previous song or band played, the song and/or band highlighted in the list of songs and/or bands playing or to be played during the event, when and when the song will be played A media overlay may be created that includes one or more timestamps of when it will end, and the like. The client device 110 may generate a media overlay using other information (eg, schedule information) associated with the event.

In another example, the media overlay may provide additional audio content for inclusion in the content item. For example, a content item may include one or more photos or videos with a particular audio track playing in the background. The client device 110 may identify the audio track as described above and provide a continuous version of the set of audio tracks to the photo(s) or video.

In another example, client device 110 can detect a particular object (eg, in a photo or video) in a content item and identify the object (eg, using object recognition technology). A media overlay associated with that object may be created. For example, a media overlay may be provided to the client device 110 in response to a user taking a photo or video of an object (eg, an incentive or reward for taking a photo or video). A media overlay may be associated with the object or may have additional information about the object. For example, the client device 110 may recognize an object indicating that the user is in Disneyland and present a Disneyland-themed media overlay. In another example, client device 110 may recognize objects such as logos, brand names, products, specific people (eg, celebrities), and the like.

In another example, the client device 110 may detect data from a machine-readable code (eg, barcode, QR code, etc.) in the content item. A media overlay may be created or selected based on the provided code. For example, machine-readable code may be generated for a music artist, and a media overlay may be generated for the music artist. A user may use the client device 110 to scan a machine-readable code associated with a music artist, and the client device 110 may generate a media overlay associated with the music artist. Additionally or alternatively, additional accessories or features (eg, tools for photo or video editing, etc.) may be provided to the user.

In another example, the client device 110 detects a geolocation associated with a content item, and a media overlay associated with the geolocation (eg, a media overlay with a real-time score of a game, as discussed in more detail above). , media overlay of concert information, etc.),

At operation 2212 , a media overlay may be displayed on the client device 110 (eg, overlaid over a photo or image), or one or more media overlays for the photo or video (eg, an audio media overlay, creation A notification may be displayed to the user indicating that a tool, etc.) is available. A user may access one or more media overlays by interacting with the client device 110 (eg, pressing a button, long-pressing the screen, swiping, or other gesture, etc.).

A user, via the client device 110 , may elect to use the media overlay and send the content item comprising the media overlay to one or more recipients. The media overlay publishing module 304 may receive a content item that includes a media overlay. The media overlay publishing module 304 may send the content item with the media overlay to the second client device 112 (or one or more client devices). The content item with the media overlay may be displayed on the second client device 112 .

Certain embodiments are described herein as comprising logic or a number of components, modules, or mechanisms. A module may constitute a software module (eg, (1) code embodied on a non-transitory machine-readable medium or (2) in a transmitted signal) or a hardware-implemented module. A "hardware-implemented module" is a tangible unit capable of performing certain operations, and may be constructed or arranged in a certain manner. In an exemplary embodiment, one or more computer systems (eg, stand-alone, client, or server computer systems) or one or more processors are configured by software (eg, an application or application portion) as described herein. may be configured as a hardware-implemented module that operates to perform the operations of

In various embodiments, a hardware-implemented module may be implemented mechanically or electronically. For example, a hardware-implemented module may include dedicated circuitry or logic permanently configured to perform certain operations (eg, as a special-purpose processor such as a field programmable gate array (FPGA) or application specific integrated circuit (ASIC)). may include. A hardware-implemented module may also include programmable logic or circuitry (eg, included within a general purpose processor or other programmable processor) temporarily configured by software to perform certain operations. It will be understood that the decision to implement a hardware-implemented module mechanically, in dedicated and permanently configured circuitry, or in temporarily configured circuitry (eg, configured by software) may be dictated by cost and time considerations. will be.

Thus, the term "hardwired-implemented module" means that it is physically configured, permanently configured (eg, hardwired) or temporarily configured to operate in a certain manner or to perform certain operations described herein. It should be understood to encompass a tangible entity that is constructed (eg, programmed) on a temporary or temporary basis. Considering embodiments in which hardware-implemented modules are temporarily configured (eg, programmed), each of the hardware-implemented modules need not be configured or instantiated at any one point in time. For example, if the hardware-implemented modules include a general-purpose hardware processor configured using software, the general-purpose processor may be configured as different hardware-implemented modules, respectively, at different times. The software may accordingly configure the hardware processor to, for example, configure a particular hardware-implemented module at one point in time and a different hardware-implemented module at a different point in time.

A hardware-implemented module may provide information to, and receive information from, other hardware-implemented modules. Accordingly, the described hardware-implemented modules may be considered communicatively coupled. When a plurality of such hardware-implemented modules exist concurrently, communication may be achieved through signal transmission (eg, via appropriate circuitry and bus connecting the hardware-implemented modules). In embodiments where a plurality of hardware-implemented modules are configured or instantiated at different times, communication between such hardware-implemented modules may be, for example, within a memory structure accessed by the plurality of hardware-implemented modules. This can be achieved through storage and retrieval of information. For example, one hardware-implemented module may perform certain operations and store the output of those operations in a communicatively coupled memory device. Another hardware-implemented module can then, at a later time, access the memory device to retrieve and process the stored output. A hardware-implemented module may also initiate communication with an input or output device and may operate on a resource (eg, a collection of information).

The various operations of the example methods described herein may be performed, at least in part, by one or more processors temporarily configured (eg, by software) or permanently configured to perform the related operations. Whether temporarily or permanently configured, such processors may constitute processor-implemented modules that operate to perform one or more operations or functions. Modules referred to herein may, in some demonstrative embodiments, include processor-implemented modules.

Similarly, the methods described herein may be at least partially processor-implemented. For example, at least some of the operations of the method may be performed by one or more processors or processor-implemented modules. Performance of certain operations may be distributed among one or more processors, either residing within a single machine or distributed across multiple machines. In some demonstrative embodiments, the processor or processors may be located in a single location (eg, within a home environment, office environment, or server farm), while in other embodiments the processors may be located in multiple locations. It can also be spread across.

In addition, one or more processors may also operate as a "software as a service" (SaaS) or to support performance of related operations in a "cloud computing" environment. For example, at least some of the operations may be performed by a group of computers (as an example of a machine including processors), the operations being performed over the network 104 (eg, the Internet) and one or more suitable It is accessible through an interface (eg, API).

Example embodiments may be implemented in digital electronic circuitry, or computer hardware, firmware, software, or a combination thereof. Illustrative embodiments include a computer program product (eg, a machine-readable medium, etc., for execution by a data processing device such as a programmable processor, computer, plurality of computers, etc. may be implemented using a computer program tangibly embodied in an information carrier of

A computer program may be written in any form of programming language, including compiled or interpreted languages, and may be in any form, including stand-alone programs or modules, components, subroutines, or other units suitable for use in a computing environment. can be placed as The computer program may be arranged to be executed on one computer, or on a plurality of computers at at least one site or distributed over a plurality of sites and interconnected through a communications network.

In example embodiments, the operations may be performed by one or more programmable processors executing a computer program that performs a function by operating on input data and generating output. Method operations may also be performed by the apparatus of the exemplary embodiments, and the apparatus of the exemplary embodiment may be implemented as a special purpose logic circuit (eg, FPGA or ASIC).

A computing system may include a client and a server. A client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises because of computer programs running on each computer and having a client-server relationship to each other. It will be appreciated that in embodiments deploying a programmable computing system, both hardware and software architectures are worth considering. Specifically, a function is implemented with permanently configured hardware (eg, ASIC), temporarily configured hardware (eg, a combination of software and a programmable processor), or a combination of permanently and temporarily configured hardware. It will be appreciated that the choice as to whether to implement may be a design choice. Disclosed below are hardware (eg, machine) and software architectures that may be deployed in various example embodiments.

19 is a block diagram 700 illustrating an architecture of software 702 that may be installed on any one or more of the devices described above. For example, in various embodiments, client devices 110 and 112 and data exchange platform 102 (eg, application server 118 , database server 124 , etc.) are components of software 702 . It may be implemented using some or all of them. It will be appreciated that FIG. 19 is only a non-limiting example of a software architecture, and that many other architectures may be implemented to enable the functionality described herein. In various embodiments, software 702 is implemented by hardware, such as machine 900 of FIG. 20 , including processor 910 , memory 930 , and I/O components 950 . In this exemplary architecture, software 702 may be conceptualized as a stack of layers, each layer capable of providing a particular function. For example, software 702 includes layers such as operating system 704 , libraries 706 , frameworks 708 , and applications 710 . Operationally, applications 710 initiate API calls 712 through the software stack and receive messages 714 in response to API calls 712 , according to some embodiments.

In various implementations, operating system 704 manages hardware resources and provides common services. Operating system 704 includes, for example, a kernel 720 , services 722 , and drivers 724 . Kernel 720 acts as an abstraction layer between hardware and other software layers in accordance with some embodiments. For example, kernel 720 provides, among other things, memory management, processor management (eg, scheduling), component management, networking, and security settings. Services 722 may provide other common services for different software layers. Drivers 724 are responsible for controlling or interfacing with the underlying hardware, in accordance with some embodiments. For example, drivers 724 may include a display driver, a camera driver, a BLUETOOTH® or BLUETOOTH® Low Energy driver, a flash memory driver, a serial communication driver (eg, a universal serial bus (USB) driver), WI-FI ® drivers, audio drivers, power management drivers, etc.

In some embodiments, libraries 706 provide a low-level common infrastructure used by applications 710 . Libraries 706 may include system libraries 730 (eg, C standard library) that may provide functions such as memory allocation functions, string manipulation functions, math functions, and the like. In addition, the libraries 706 include a media library (eg, Moving Picture Experts Group-4 (MPEG4), Advanced Video Coding (H.264 or AVC), Moving Picture Experts Group Layer-3 (MP3), Advanced Audio). A library that supports the presentation and manipulation of various media formats, such as Coding (AAC), Adaptive Multi-Rate (AMR) audio codecs, Joint Photographic Experts Group (JPEG or JPG), or Portable Network Graphics (PNG)), graphics libraries); (e.g., an OpenGL framework used to render graphical content in two dimensions (2D) and three dimensions (3D) on a display), database libraries (e.g. SQLite, which provides various relational database functions), web library (e.g. An API library 732 such as WebKit) providing a web browsing function may be included. Libraries 706 may also include a wide variety of other libraries 734 that provide many other APIs to applications 710 .

Frameworks 708 provide a high-level common infrastructure that can be used by applications 710 in accordance with some embodiments.

For example, frameworks 708 provide various graphical user interface (GUI) functions, high-level resource management, high-level location services, and the like. Frameworks 708 may provide a broad spectrum of other APIs that may be used by applications 710 , some of which may be specific to a particular operating system 704 or platform.

In one exemplary embodiment, applications 710 include home application 750 , contacts application 752 , browser application 754 , book reader application 756 , location application 758 , media application 760 . , messaging applications 762 , gaming applications 764 , and a wide variety of other applications such as third party applications 766 . According to some embodiments, the applications 710 are programs that execute functions defined in the program. Various programming languages for creating one or more of the applications 710 structured in various ways, such as an object-oriented programming language (Objective-C, Java, or C++) or a procedural programming language (eg, C or assembly language). can be used In a particular example, third-party application 766 (eg, an application developed using an ANDROID or IOS software development kit (SDK) by an entity other than a vendor of a particular platform) is an IOS, ANDROID , mobile software running on a mobile operating system, such as WINDOWS® Phone or another mobile operating system. In this example, the third-party application 766 can invoke API calls 712 provided by the operating system 704 to enable the functionality described herein.

Some embodiments may specifically include a content gallery application 767 . In certain embodiments, this may be a standalone application that operates to manage communications with a server system, such as data exchange platform 102 . In other embodiments, this functionality may be integrated with another application, such as a media application 760 or another such application. The content gallery application 767 is configured to collect content using the camera device of machine 900 , communicate with a server system via I/O components 950 , and receive content and memory 930 of the received content. storage can be managed. Presentation of the content and user input associated with the content is sent to the content gallery application 767 using different frameworks 708 , libraries 706 elements or operating system 704 elements running on the machine 900 . can be managed by

20 is capable of performing any one or more of the methodologies discussed herein by reading instructions from a machine-readable medium (eg, a machine-readable storage medium), in accordance with some embodiments. A block diagram illustrating the components of the machine 900 . Specifically, FIG. 20 shows a schematic diagram of a machine 900 in an exemplary form of a computer system, within the machine, causing the machine 900 to perform any one or more of the methodologies discussed herein. Instructions 916 (eg, software, program, applications 710, applet, app, or other executable code) may be executed to do so. In alternative embodiments, machine 900 may operate as a standalone device or be coupled (eg, networked) to another machine. In a networked deployment, machine 900 operates in the capacity of a server machine (eg, application server 118 , database server 124 ) or client devices 110 , 112 , or peers in a server-client network environment. It can operate as a peer machine in a peer-to-peer (or distributed) network environment. Machine 900 is a server computer, client computer, personal computer (PC), tablet computer, laptop computer, netbook, set top box (STB), personal digital assistant (PDA), entertainment media system, cellular phone, smart phone, mobile by device, wearable device (eg, smart watch), smart home device (eg, smart appliance), other smart device, web appliance, network router, network switch, network bridge, or machine 900 . It may include, but is not limited to, any machine capable of executing instructions 916 specifying actions to be taken, sequentially or otherwise. Further, although only one machine 900 is illustrated, the term "machine" also refers to individually or jointly executing instructions 916 to perform any one or more of the methodologies discussed herein. should be considered to include a collection of machines 900 that

In various embodiments, machine 900 includes a processor 910 , memory 930 , and I/O components 950 that may be configured to communicate with each other via a bus 902 . In one exemplary embodiment, a processor 910 (eg, a central processing unit (CPU), a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, a graphics processing unit (GPU) ), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a radio-frequency integrated circuit (RFIC), another processor, or any suitable combination thereof. ) includes, for example, a processor 912 and a processor 914 that can execute instructions 916 . The term "processor" is meant to include multi-core processor 910, which may include two or more independent processors 912, 914 (also referred to as "cores") capable of executing instructions 916 concurrently. intend Although FIG. 19 shows multiple processors 910 , the machine 900 is a single processor 910 having a single core, a single processor 910 having multiple cores (eg, a multi-core processor 910 ). , multiple processors with a single core 912 , 914 , multiple processors with multiple cores 912 , 914 , or any combination thereof.

According to some embodiments, memory 930 includes main memory 932 , static memory 934 , and storage unit 936 , accessible by processor 910 via bus 902 . . The storage unit 936 can include a machine-readable medium 938 having stored thereon instructions 916 implementing any one or more of the methodologies or functions described herein. Instructions 916 may also, during their execution by machine 900 , fully or at least partially, in main memory 932 , static memory 934 , in at least one of processors 910 (eg, for example, in the cache memory of the processor), or any suitable combination thereof. Accordingly, in various embodiments, main memory 932 , static memory 934 , and processors 910 are considered machine-readable medium 938 .

As used herein, the term "memory" refers to a machine-readable medium 938 capable of temporarily or permanently storing data, including random access memory (RAM), read-only memory (ROM), buffer memory. , flash memory, and cache memory. Although machine-readable medium 938 is depicted as being a single medium in the exemplary embodiment, the term "machine-readable medium" refers to a single medium or multiple media (eg, , centralized or distributed databases, or associated caches and servers). The term "machine-readable medium" also refers to instructions 916 that, when executed by one or more processors (eg, processor 910 ) of machine 900 , cause machine 900 to be used herein. Any medium or plurality of mediums capable of storing instructions (eg, instructions 916 ) for execution by a machine (eg, machine 900 ) to cause performing any one or more of the described methodologies. Therefore, "machine-readable medium" means a single storage device or device, as well as a "cloud-based" storage system or storage network comprising a plurality of storage devices or devices. say The term "machine-readable medium" means, accordingly, solid-state memory (eg, flash memory), optical media, magnetic media, other non-volatile memory (eg, erasable and programmable read-only memory (EPROM)), or one or more data stores in the form of any suitable combination thereof. The term "machine-readable medium" specifically excludes non-statutory signals per se.

I/O components 950 include various components that receive input, provide output, generate output, send information, exchange information, capture measurements, and the like. In general, it will be appreciated that I/O components 950 may include many other components not shown in FIG. 20 . I/O components 950 are grouped according to functionality merely to simplify the discussion below, and this grouping is not limiting in any way. In various demonstrative embodiments, I/O components 950 include an output component 952 and an input component 954 . The output components 952 may include a visual component (eg, a display such as a plasma display panel (PDP), a light emitting diode (LED) display, a liquid crystal display (LCD), a projector, or a cathode ray tube (CRT)), an acoustic component (eg, a speaker), a haptic component (eg, a vibration motor), other signal generators, and the like. Input components 954 include an alphanumeric input component (eg, a keyboard, a touch screen configured to receive alphanumeric input, a photo-optical keyboard, or other alphanumeric input component), a point-based input component (eg, a mouse). , touchpad, trackball, joystick, motion sensor, or other pointing tool), tactile input components (e.g., physical buttons, touch screens that provide the location and force of touch or touch gestures, or other tactile input components), audio input components (eg, microphones); and the like.

In some further exemplary embodiments, I/O components 950 include, inter alia, a biometric component 956 , a motion component 958 , an environment component 960 , or a location component 962 . . For example, the biometric component 956 can detect an expression (eg, a hand expression, a facial expression, a voice expression, a gesture, or an eye tracking) and a biosignal (eg, blood pressure, heart rate, body temperature, sweat or brain waves) and identify a person (eg, voice identification, retina identification, face identification, fingerprint identification, or electroencephalogram-based identification). Motion component 958 includes an acceleration sensor component (eg, an accelerometer), a gravity sensor component, a rotation sensor component (eg, a gyroscope), and the like. Environment component 960 may include, for example, a lighting sensor component (eg, a photometer), a temperature sensor component (eg, one or more thermometers to detect ambient temperature), a humidity sensor component, a pressure sensor component (eg, For example, a barometer), an acoustic sensor component (eg, one or more microphones to detect background noise), a proximity sensor component (eg, an infrared sensor to detect a nearby object), a gas sensor component (eg, a machine olfactory sensors, gas detection sensors that, for safety purposes, detect concentrations of hazardous gases or measure airborne contaminants), or other components that can provide indications, measurements, or signals corresponding to the surrounding physical environment. may include Position component 962 may include a location sensor component (eg, a Global Positioning System (GPS) receiver component), an altitude sensor component (altimeter or barometer that detects barometric pressure from which an altitude can be derived), an orientation sensor component (eg, for example, magnetometer) and the like.

Communication may be implemented using a variety of technologies. I/O components 950 may include a communication component 964 operable to couple machine 900 to network 980 or devices 970 via coupling 982 and coupling 972, respectively. can For example, communication component 964 includes a network interface component or another device suitable for interfacing with network 980 . In further examples, communication component 964 may include a wired communication component, a wireless communication component, a cellular communication component, a near field communication (NFC) component, a BLUETOOTH® component (eg, BLUETOOTH®Low Energy), a WI-FI® component, and other communication components for providing communication via other aspects. Devices 970 may be another machine 900 or any of a variety of peripheral devices (eg, peripheral devices coupled via Universal Serial Bus (USB)).

Moreover, in some embodiments, communication component 964 detects an identifier or includes a component operable to detect an identifier. For example, the communication component 964 may be a radio frequency identification (RFID) tag reader component, an NFC smart tag detection component, an optical reader component (eg, a one-dimensional barcode such as a Universal Product Code (UPC) barcode, a Quick Response) code, Aztec code, Data Matrix, Dataglyph, MaxiCode, PDF417, Ultra code, UCC RSS (Uniform Commercial Code Reduced Space Symbology)-2D bar code, optical sensor that detects multidimensional barcodes such as other optical codes); an acoustic detection component (eg, a microphone that identifies the tagged audio signal), or any suitable combination thereof. In addition, various information such as location via Internet Protocol (IP) geo-location, location via WI-FI® signal triangulation, location via BLUETOOTH® or NFC beacon signal detection, which may indicate a specific location can be derived via communication component 964 .

In various demonstrative embodiments, one or more portions of network 980 may include an ad hoc network, an intranet, an extranet, a virtual private network (VPN), a local area network (LAN), a wireless LAN (WLAN) (WAN), a wide area network (WAN). ), wireless WAN (WWAN), Metropolitan Area Network (MAN), the Internet, part of the Internet, part of the Public Switched Telephone Network (PSTN), plain old telephone service (POTS) networks, cellular telephone networks, wireless networks, It may be a WI-FI® network, another type of network, or a combination of two or more of these networks. For example, network 980 or a portion of network 980 may include a wireless or cellular network, and combination 982 may include a Code Division Multiple Access (CDMA) connection, a Global System for Mobile communications (GSM) connection, or other types of cellular or wireless coupling. In this example, combining 982 is 1xRTT (Single Carrier Radio Transmission Technology), EVDO (Evolution-Data Optimized) technology, GPRS (General Packet Radio Service) technology, EDGE (Enhanced Data rates for GSM Evolution) technology, 3G, 4G (fourth generation wireless) networks, Universal Mobile Telecommunications System (UMTS), High Speed Packet Access (HSPA), Worldwide Interoperability for Microwave Access (WiMAX), Long Term Evolution (LTE) standards, as defined by various standards-setting organizations 3GPP (third Generation Partnership Project), other long-distance protocols, including others; or any of various types of data transfer technologies, such as other data transfer technologies.

In example embodiments, the instructions 916 may be transmitted over a network interface device (eg, a network interface component included in communication component 964 ) using a transmission medium and among a number of well-known protocols. transmitted or received over network 980 using any one (eg, Hypertext Transfer Protocol (HTTP)). Similarly, in other example embodiments, the instructions 916 are transmitted or received using a transmission medium via coupling 972 (eg, peer-to-peer coupling) to devices 970 . do. The term "transmission medium" refers to a digital or analog communication signal or other intangible medium capable of storing, encoding, or carrying instructions 916 for execution by machine 900 , and facilitates delivery of such software. It is considered to include any intangible medium that contains it.

Further, machine-readable medium 938 is non-transitory (ie, has no transitory signal) in that it does not embody a propagated signal. However, labeling machine-readable medium 938 as "non-transitory" should not be construed to mean that the medium is immovable; Media 938 should be considered as capable of being moved from one physical location to another. Additionally, since machine-readable medium 938 is tangible, medium 938 may be considered a machine-readable device.

21 is a block diagram illustrating an exemplary mobile device 1900 in accordance with an exemplary embodiment. Mobile device 1900 can include a processor 1902 . Processor 1902 may include a variety of different types of commercially available processors 1902 suitable for mobile device 1900 (e.g., XScale architecture microprocessors, microprocessor without interlocked pipeline stages (MIPS) architecture processors, or other types of processors) 1902)). Memory 1904 , such as random access memory (RAM), flash memory, or other type of memory, is typically accessible by processor 1902 . The memory 1904 is configured to store an operating system (OS) 1906 as well as an application 1908 , such as a mobile location capable application that may provide a location-based service (LBS) to a user. can be The processor 1902 may be coupled, directly or through suitable intermediate hardware, to the display 1910 and to one or more input/output (I/O) devices 1912 , such as a keypad, touch panel sensor, microphone, or the like. Similarly, in some embodiments, the processor 1902 may be coupled to a transceiver 1914 that interfaces with an antenna 1916 . Transceiver 1914 may be configured to transmit and receive cellular network signals, wireless data signals, or other types of signals via antenna 1916 , depending on the characteristics of mobile device 1900 . Also, in some configurations, GPS receiver 1918 may also receive GPS signals using antenna 1916 .

Throughout this specification, multiple instances may implement components, operations, or structures described as a single instance. Although individual acts of one or more methods are illustrated and described as separate acts, one or more of the individual acts may be performed concurrently, and the acts need not be performed in the order illustrated. Structure and functionality presented as separate components in an example configuration may be implemented as a combined structure or component. Similarly, structure and functionality presented as a single component may be implemented as separate components. These and other variations, modifications, additions, and improvements are within the scope of the subject matter herein.

Although the summary of the subject matter of the present invention has been described with reference to specific exemplary embodiments, various modifications and changes may be made to these embodiments without departing from the broader scope of the embodiments of the present disclosure.

These embodiments of the subject matter of the present invention, although in fact more than one invention or inventive concept is disclosed herein, without the intention of voluntarily limiting the scope of the present application to any single disclosure or inventive concept. They may be referred to individually or collectively by the term "invention" merely for convenience.

The embodiments illustrated herein have been described in sufficient detail to enable any person skilled in the art to practice the teachings disclosed herein. Other embodiments may be utilized and may be derived from the present disclosure as structural and logical substitutions and changes may be made without departing from the scope of the present disclosure. Accordingly, this detailed description is not to be considered in a limiting sense, and the scope of various embodiments is defined only by the full scope of the appended claims and their equivalents.

As used herein, the term "or" may be interpreted in an inclusive or exclusive sense. Moreover, multiple instances may be provided for a resource, operation, or structure described herein as a single instance. Additionally, boundaries between various resources, operations, modules, engines, and data stores are somewhat arbitrary, and specific operations are illustrated in the context of specific example configurations. Other assignments of functionality are contemplated and may be within the scope of various embodiments of the present disclosure. In general, structures and functionality presented as separate resources in an example configuration may be implemented as a combined structure or resource. Similarly, structure and functionality presented as a single resource may be implemented as separate resources. These and other variations, modifications, additions and improvements are within the scope of the embodiments of the present disclosure as expressed in the appended claims. Accordingly, the specification and drawings are to be regarded in an illustrative rather than a restrictive sense.

The Abstract of the Disclosure is provided in accordance with 37 CFR Section 1.72(b), which requires an abstract that allows the reader to quickly ascertain the nature of the technical disclosure. It is suggested that this abstract should not be used to interpret or limit the scope or meaning of the claims. Also, in the above detailed description, it can be seen that various features are grouped together in one embodiment for the purpose of streamlining the present disclosure. This method of disclosure is not to be construed as reflecting an intention that the claimed embodiments require more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive subject matter lies in less than all features of a single disclosed embodiment. Accordingly, the following claims are incorporated into the Detailed Description, with each claim representing a separate embodiment in its own right.