Periodic ambient waveform analysis for dynamic device configuration

0001The present disclosure relates generally to the use of a sensor subsystem of a mobile device to periodically capture waveform data and generate waveform fingerprints for dynamic device configuration.

0002Social networking systems, such as social networking websites, allow users to interact with their social networking systems and between users through the system. Social networking systems generate and store records, often called user profiles, in association with users. User profiles include user demographic information, communication channel information, and personal interests. Social networking systems also create and store records of relationships between users and other users within the social networking system (eg, social graphs) and provide services (wall posts, photo sharing, instant messaging, etc.). It can also encourage social dialogue between users within a social networking system. Geosocial networking systems are social networking systems that enable additional social dialogue using geographic services and geographic features. User-provided location data or geolocation techniques (eg, mobile phone location tracking) allow geosocial networks to connect and link users to local people and local events that are of interest to them. To do so. For example, a user can check in to any location using a mobile client application by providing the name of the location (or by selecting a location from a preset list of locations). The geosocial networking system can record information about the presence of a user, among other things, and in some cases provide that information to other users of the geosocial networking system.

0003<figref num="1">The figure which shows the example social networking environment.</figref><figref num="2">The figure which shows the example user device.</figref><figref num="3">A diagram illustrating an exemplary data source that a social networking system can use to determine whether or not to adjust settings on a mobile device and what settings to adjust.</figref><figref num="4">Diagram showing an example of three users of a social networking system with self-reported geographic locations in the same neighborhood.</figref><figref num="4A">The figure which shows the example part of the social graph representing three users of FIG.</figref><figref num="5">Diagram showing an exemplary method of adjusting device settings based on captured ambient sounds retrieved from an online database.</figref><figref num="6">A diagram illustrating an exemplary method of adjusting device settings based on social information.</figref><figref num="7">The figure which shows the example computer system.</figref>

0004A particular embodiment relates to a geosocial networking system that includes the feature of automatically adjusting mobile device settings based on ambient sound conditions. The disclosure also relates to adjusting the settings of a mobile device based on social data calculated from periodic uploads of waveform data from multiple users of a geosocial networking system. The above and other features, embodiments and advantages of the present disclosure will be described in more detail with the accompanying drawings in the following detailed description.

0005The present invention will be described in detail below with reference to some embodiments shown in the accompanying drawings. In the following description, many specific details are provided so that the present disclosure can be fully understood. However, it will be apparent to those skilled in the art that this disclosure is feasible in the absence of some or all of these particular details. Also, well-known process steps and / or structures are not described in detail so that the disclosure is not unnecessarily obscured. Further, although the present disclosure is described with particular embodiments, it will be understood that the following description is not intended to limit this disclosure to the described embodiments. Conversely, the following description is intended to cover alternatives, modifications, and equivalents contained within the spirit and scope of the present disclosure as specified by the appended claims.

0006Social networking systems, such as social networking websites, allow users to interact with and interact with each other through the system. Generally, in order to become a registered user of a social networking system, a human or non-human entity registers an account with the social networking system. The registered user can then log in to the social networking system, for example by providing the correct login ID or username and password through an account. As used herein, a "user" is an individual (human user), entity (eg, a company, establishment or third party application), group that interacts or communicates with or through such a social networking environment. It may be (eg, an individual or a group of entities).

0007When a user registers for an account on a social networking system, the social networking system can create a record, often called a "user profile," to associate with and store the user. A user profile may include information provided by that user and information about a user's activities or behaviors collected by various systems, including social networking systems. For example, users may want to include their name, profile image, contact information, birthday, gender, marital status, family relationships, work, educational background, hobbies, interests, and other demographics that they want to include in their user profile. Information can be provided. Users can also identify other users of social networking systems that they consider to be friends. A list of the user's friends or primary contacts (primary contacts) may be included in the user's profile. Connections within a social networking system may be bidirectional or simply unidirectional. For example, if both Bob and Joe are users and are connected to each other, then Bob and Joe are each connected to the other. On the other hand, if Bob wants to connect with Sam to view Sam's posted content items, but Sam doesn't choose to connect with Bob, he can form a one-way connection, in which case Sam can connect with Bob. But Bob is not Sam's connection. In some embodiments of social networking systems, connections may be indirect through one or more connection levels (eg, friends of a friend). Connections are either explicitly added by the user, for example by the user selecting a particular other user as a friend, or social based on the user's common characteristics (eg, users who are alumni of the same institution). It may be created automatically by the networking system. Users identify or bookmark frequently visited websites or web pages

0008The user may provide information about various aspects of the user, such as contact information and interests, when or after the user registers for an account. In addition, the user can update his / her profile information at any time. For example, a user is moving or calling When changing the talk number, the user can update his contact information. In addition, the user's interests may change over time, and the user may update his or her interests in the profile at any time. The user's activity on the social networking system, such as how often he or she accesses certain information on the social network, may also provide information that can be included in the user's profile. Again, such information may be updated from time to time to reflect the user's latest activity. Furthermore, other users, friends, or contacts can also perform activities that affect or update a user's profile. For example, a contact can add a user as a friend (or remove a user from friends). Contacts can also write a message on the user profile page (commonly known as a wall post). The user can also enter a status message posted on the user's profile page.

0009Social network systems are capable of maintaining social graph information, which can generally model relationships between groups of individuals, ranging from mere acquaintances to intimate family relationships. Can be included. Social networks can be represented using a graph structure. Each node in the graph corresponds to a member of the social network. The edge that connects the two nodes represents the relationship between the two users. In addition, the degree of separation between two nodes is defined by the minimum number of hops required to move on the graph from one node to the other. The degree of separation between the two users can be regarded as the degree of relevance between the two users represented by those nodes in the graph.

0010Social networking systems can support a variety of applications such as photo sharing, online calendars, and events. For example, a social networking system may have a media sharing function. For example, a social networking system may allow a user to post photos and other multimedia files to a user's profile, such as in a wall post or in a photo album, both wall posts and photo albums. It may be accessible to other users of the social networking system. Social networking systems may also allow users to configure events. For example, a first user may configure an event by attributes that include the date and time of the event, the location of the event, and other users invited to the event. Invited users can receive invitations to the event and respond (by accepting or declining the invitation, etc.). In addition, social networking systems may allow users to maintain a personal calendar. Like events, calendar items can include time, date, location, and other user identification information.

0011Social networking systems can also support privacy models. Users may or may not want to share their information with other users or third party applications, or users may only want to share their information with specific users or third party applications. Sometimes. Users can control whether their information is shared with other users or third-party applications through privacy settings associated with their user profile. For example, a user selects a privacy setting for each user data associated with the user and / or a setting that applies (globally) to the general application or applies to a category or type of user profile information. Can be selected. Privacy settings define or identify a set of entities that can access user data (eg, other users, user connections, friends of friends, or third-party applications). Privacy settings are specific entities within a social network (eg other users), predefined groups of user connections, specific types of connections, all user connections, user connections. Specify at various levels of granularity, such as specifying all primary connections, the entire social network, and even the entire Internet (for example, to make posted content items indexable or searchable on the Internet). Can be done. Users can also choose default privacy settings for all user data they post. In addition, the user can explicitly exclude certain entities from viewing user data or certain types of user data. Social networking systems may maintain a database of information about geographic locations or locations. Locations can correspond to various physical locations such as restaurants, bars, train stations, airports, etc. Some locations may themselves correspond to a larger area that includes the location, such as near a restaurant or gate in an airport. In one embodiment, each location is maintained as a hub node within a social graph or other data structure maintained by a social networking system, as described in U.S. Patent Application No. 12/863181 incorporated herein. can do. Social networking systems allow users to access information about each location using client applications (eg browsers) hosted on wired or wireless stations such as laptops, desktops or mobile devices. You may. For example, a social networking system may provide a web page (or other structured document) to a user requesting information about a location. In addition to user profiles and location information, social networking systems may track or maintain other information about users. For example, a social networking system can accommodate geosocial networking system features that include one or more location-based services that record a user's location. For example, the user is You can access the geosocial networking system using a dedicated client application (or a web-based or network-based application that uses a browser client) hosted on The Mobile Device. Client applications can automatically access the Global Positioning System (GPS) or other geolocation features that are compatible with mobile devices and report the user's current location to social networking systems. In addition, the client application may support a geosocial networking system that allows a user to check in at various locations and communicate that location to other users. Check-in to a given location may occur when the user is physically located anywhere, uses a mobile device to access the geonetworking system, and registers the user's presence at that location. The user can select a location from the list of existing locations near the user's current location or create a new location. The user can also provide a string comment when checking in at a given location. Users can also identify one or more users (such as a user's friends) in connection with check-in and associate those users with check-in. U.S. Patent Application No. 12/584614, incorporated herein by reference for all purposes, describes a system that allows a first user to check in another user at any location. A comment and a description including a time stamp corresponding to the time when the user checked in may be displayed to other users. For example, a record of user check-in activity may be stored in a database. Social networking systems select one or more records associated with a user's check-in activity at any location and such a check on a web page (or other structured document) that corresponds to a given location. Can include cook-in activities. For example, a social networking system may select a check-in activity associated with a friend or other social networking contact of a user requesting a page corresponding to a location. U.S. Patent Application No. 12/858817, which is incorporated herein by reference in its entirety for all purposes, describes an example of a geosocial networking system that can be used in connection with various embodiments of the invention. ing. Check-in activity is provided to users on user profile pages and social networking systems May be displayed in the news feed.

0012In addition, a dedicated client application hosted on the user's mobile device can be configured to continuously capture the location data of the mobile device and send that location data to a social networking system. In this way, the social networking system may track the user's location and provide the user with various recommendations regarding where to approach the user's path or where the user frequently visits. In one embodiment, the user may subscribe to this recommended service, thereby causing the client application to periodically post the user's location data to the social networking system.

0013Certain embodiments of a dedicated client application running on a user's mobile device are configured to use a microphone or sensor subsystem within the mobile device to capture ambient sonic data continuously or periodically. be able to. In certain embodiments, the portable device can perform waveform analysis of the ambient sound and adjust device settings such as ring volume or vibration mode based on the ambient waveform data. In certain embodiments, the client device automatically adapts the ring tones to match or slightly exceed the ambient sound level, which allows the ring tones to be audible or quiet even in noisy environments. It can increase the likelihood of reaching the appropriate level in the environment. In certain embodiments, the self-determined position of the mobile device can be utilized to adjust device settings. For example, if the user's GPS indicates that the user is in a movie theater, the device may automatically turn off the ring.

0014In certain embodiments, the portable device can generate a waveform fingerprint of the captured ambient waveform for compact, band-efficient transmission to a social networking system. In certain embodiments, the received waveform fingerprint may be matched against a database of object waveforms. For example, if the user is in a specific location and the waveform fingerprint uploaded by the user's mobile device matches the stored fingerprint of a particular song in the sound database, the social networking system is that the user is currently You may decide that you are listening to the song.

0015In certain embodiments, a combination of database information, waveform matching, and the user's GPS data can be used to adjust the user's device settings. For example, GPS devices often do not work indoors. Therefore, the user's last reported location may be near the cinema, not the cinema. If the reported location of the user device is near the cinema and the user's waveform fingerprint matches the soundtrack of a particular movie, the social networking system will run the cinema's show time online near the user. You may access the database. Based on timestamps from uploaded waveform fingerprints, matching movies, and GPS locations, social networks allow users to watch, for example, a 5:30 pm screening of "True Grit" on AMC30. You can decide that you are doing it. Therefore, the device can automatically turn off the ringing tone.

0016Further, in certain embodiments, the social networking system may receive waveform fingerprints from a client device or mobile device, analyze the waveform fingerprints, and provide users of the device with improved social capabilities. In certain embodiments, the accuracy of the client device determination position determined by the client device can be improved by waveform analysis. For example, if the user is in a particular field, the waveform fingerprint captured by the user's mobile device will match the waveform fingerprint uploaded by another user. Any of the other user's positions Social networking systems if their location is concentrated, or if one of the users makes an explicit network statement that they are on the scene by checking in, registering for an event, or commenting. Can update the user's inaccurate determined position to improve its accuracy. In certain embodiments, the mobile device can adjust the device settings based on the location received from the social networking system.

0017In certain embodiments, information from social networking systems may be leveraged to adjust device settings. In certain embodiments, information from the user's profile and behavior on social networks can be used to adjust the user's device settings. For example, when a user checks in to a public library, the device can automatically turn off the ring. Alternatively, the device may turn up the ring volume if the user is checked in or tagged for a sporting event. In addition, information from other users of the social networking system can be leveraged. For example, waveform analysis determines that multiple users are in the same location, and one of those users makes a call about the type of event by commenting, checking in, updating their status, tagging, etc. , All users who have been determined to be in that location may automatically change their device settings.

0018Figure 1 shows an exemplary social networking system. In certain embodiments, the social networking system can store user profile data and social graph information in the user profile database 101. In certain embodiments, the social networking system can store user event data in event database 102. For example, the user can register a new event by accessing the client application and defining the event name, time, and position, and store the newly created event in the event database 102. In certain embodiments, the social networking system can store user privacy policy data in privacy database 103. In certain embodiments, the social networking system can store geographic and location data in location database 104. In certain embodiments, the social networking system can store sound waves, i.e. sound wave fingerprints, of various songs, television programs, soundtracks, movies, performances, etc. in the sound database 105. In certain embodiments, databases 101, 102, 103, 104 and 105 may be operably connected to the front end of a social networking system. In certain embodiments, the social networking system also includes a waveform matching application 118 that matches the uploaded waveform fingerprint with the waveform stored in the sound database 105, i.e., the waveform fingerprint.

0019In certain embodiments, the front end 120 can interact with the client device 122 through the network cloud 121. The client device 122 is generally a computer or computing device that has the ability to communicate (eg, remote) through a computer network. The client device 122 may be a suitable computing device such as a desktop computer, laptop computer, personal digital assistant (PDA), in-vehicle or out-of-vehicle navigation system, smartphone, or other cellular or mobile phone, mobile gaming device. .. Client device 122 can be a web browser (eg, Microsoft Windows® Internet Explorer), Modira Firefox (Mozilla Firefox), Apple Safari (Apple Safari), Google Chrome (Google Chrome), And Opera (such as Opera) or You can run one or more client applications, such as a dedicated client application (such as Facebook for iPhone (Facebook® for iPhone®)) to access and browse content through your computer network. it can. The front end 120 can include web or HTTP server functionality and other features to allow users access to social networking systems. The network cloud 121 generally represents a network or set of networks through which client devices 122 can access social network systems, such as the Internet and / or a combination of corporate intranets.

0020In certain embodiments, the location database 104 is a location information base where each location contains a name, geographic location, and meta information, such as the user who originally created the location, reviews, comments, check-in activity data, and so on. Can be memorized. The location may be created by the system administrator and / or the system user. For example, a user can access a client application and define a location name to register a new location, provide a geographic location, and have the newly created location registered in the location database 104. As mentioned above, the location created may correspond to a hub node, and the administrator needs the hub node for the purpose of augmenting information about the location and for creating advertisements or other suggestions to deliver to users. In certain embodiments, the front end 120 of the system can build and serve a location web page at the request of the user. In some embodiments, the location web page may include components that the user can select to like or check in to the location. In certain embodiments, the location database 104 may store geolocation data that identifies the user's actual geolocation associated with check-in. For example, the geographic location of an Internet-connected computer can be identified by the computer's IP address. For example, the geographic location of a mobile phone with cellular, Wi-Fi and / or GPS capabilities can be identified by base station triangulation, Wi-Fi positioning, and / or GPS positioning. In certain embodiments, the location database 104 may store geographic locations and additional information for multiple locations. For example, the place is a local company, a tourist attraction (for example, rice)Union Square, San Francisco, California), universities, cities, national parks, etc. For example, the geographic location of a location (eg, a local coffee shop) can be an address, a set of geographic coordinates (latitude and longitude), or a reference to another location (eg, "a coffee shop next to a station"). Good. For example, the geographical location of a large area (eg Yosemite National Park) may be a shape (eg a circle or polygon) that approximates the boundaries of the location and / or the center of that shape. For example, additional information about a location may be business hours, photos, or user reviews of that location. In certain embodiments, the location database 104 can store user location data. For example, a user can create a location (eg, a new restaurant or coffee shop) and store the location where the social networking system was created in the location database 104. For example, location database 104 can store user check-in activity. For example, the location database 104 can store the user's geographic location provided by the user's GPS-equipped mobile device. In certain embodiments, the social networking system navigates one or more routes of the user based on the user's user profile information, check-in activity, and / or geographic location data reported by the client application (see above). It can be calculated and one or more of its routes stored in the location database 104. For example, social networking systems use map service applications such as Google Maps, or use geographic location data points from a user's GPS-equipped mobile phone while the user drives to work. It is described in the user's user profile information (stored in the user profile database 101). ) Can calculate the user's "commuting route" between home and work.

0021The waveform matching application 118 collates the waveform, or waveform fingerprint, uploaded by the client device 122 with the waveform, or waveform fingerprint, in the sound database 105. In certain embodiments, the waveform matching application utilizes feature detection using a Fast Fourier Transform (FFT) or a Direct Cosine Transform (DCT). In certain embodiments, frequency domain or time domain cross-correlation is used for waveform matching. In certain embodiments, dynamic waveform matching (DWM) can be used to displace the waveform on the time axis. In certain embodiments, the waveform matching application 118 utilizes the sound fingerprint of the waveform file in the sound database 105. The present disclosure contemplates a waveform, i.e., any method or algorithm suitable for waveform fingerprint matching.

0022FIG. 2 shows an exemplary client device 122. In certain embodiments, the client device 122 is a smartphone (eg, iPhone or Blackberry), which is a mobile phone that offers higher computing performance and connectivity than traditional mobile phones. It may be. It is considered a portable computer integrated with a mobile phone. In certain embodiments, the client device 122 may be a netbook or tablet computer (eg, an iPad®). In certain embodiments, the client device 122 can connect to the network through a wireless connection.

0023In certain embodiments, the client device 122 can include hardware 210 and software 220. In certain embodiments, the hardware 210 includes, for example, processor 211, memory 212, storage 213, transmitter / receiver 214, input / output devices 215 (eg, display, touch screen, keypad, microphone, speaker, etc.), camera 216, all. Earth Positioning System (GPS) Sensor 217, Sensor Hub 218, Notification Control Switch 219, RFID Reader 241, RFID Sensor 242, Accelerator 243, Optical Sensor 244, Microphone 245 (may be part of I / O Block 215), etc. It can have any number of hardware components, but is not limited to these. The present disclosure contemplates any suitable hardware component. In certain embodiments, some or all of the user's user data can be stored in storage 213.

0024In certain embodiments, software 220 may include operating system 221 which is any number corresponding to kernel 231 and / or some of the hardware components available on client device 122. Device driver 232 can be included. Operating system 221 may be selected for client device 122 based on the actual type of client device 122. For example, if the client device 122 is a mobile device (eg a smartphone), the operating system 221 may be, for example, Microsoft's Windows Mobile, Google's Android (Android®), Nokia ( Mobile operating systems such as, but are not limited to, Nokia's Symbian, Apple's iOS, and Samsung's Bada.

0025In certain embodiments, one or more software applications 223 can be hosted and / or run on client device 122. Specific Embodiment So, these are native applications that are installed and resident on client device 122. For example, one application (eg Google Maps (Google) Maps)) may allow the user of the device to browse the map, search for addresses and offices, and get directions, while a second application allows the user of the device to read and send emails, It may be able to receive, a third application (eg, a web browser) may allow the user of the device to browse and search the Internet, and a fourth application may allow the user of the device to camera 216. May be used to take pictures and record, and a fifth application may allow the user of the device to send and receive VoIP and / or cellular network calls. In certain embodiments, there may be software applications (eg, notification control 241) that allow users of the device to manage notifications that are pushed to client device 122. Each software application 220 has a user interface and may implement one or more specific features. Each software application 220 may include one or more software modules that implement individual functions. The executable code of software application 220, including notification control 241 may be stored on a computer-readable and non-temporary medium (eg, storage 213 or memory 212) on client device 122.

0026A sound capture application 246 is operably connected to the microphone 245 and the sensor hub 218. In certain embodiments, the sound capture application 246 continuously captures sound data and transforms it into a waveform fingerprint using a sound feature detection algorithm (such as FFT / DCT). In certain embodiments, the sound capture application 246 captures waveform data only periodically and converts the data into waveform fingerprints. In certain embodiments, the sensor hub 218 captures the waveform data through the microphone 245 and stores the waveform in storage 213 for later use in fingerprint generation by the sound capture application 245. In certain embodiments, the sound capture application 246 runs only when the user changes position. In certain embodiments, the sound capture application 246 is only executed when the detected volume exceeds a predetermined number of decibels. The present disclosure contemplates any suitable method of capturing ambient sound data.

0027FIG. 3 shows an exemplary data source that can be used by the client device 122 or social networking system to determine whether or not to adjust the settings of the client device 122 and how to adjust the settings. Data sources 301-306 may be utilized by the client device 122 alone or in any combination with each other to adjust the device settings of the client device 122. The ambient sound 301 and the self-determined device position 302 may be utilized without the client device 122 interacting with the social networking system. For example, the basic method of adjusting the settings of client device 122 is based solely on ambient sound 301. In certain embodiments, the client device 122 captures the ambient sound wave form and calculates the average ambient noise level for the environment (eg, 95 decibels). In certain embodiments, the client device 122 adjusts the ringing volume to match the ambient noise level or to be slightly higher than the ambient noise level. In certain embodiments, the client device 122 turns on vibration mode when the ambient noise is greater than or less than a predetermined threshold.

0028In certain embodiments, the self-determined device location 302 is used in combination with the online database 306 to adjust device settings. For example, if the client device 122 reports the longitude / latitude position associated with a particular site, the client device 122 may adjust the settings based on the site type. For example, if the self-determined position of client device 122 is associated with a cinema, then the client device 122 turns off the ring or enables vibration mode. In certain embodiments, the online database 306 may be one or more servers in a social networking system. In certain embodiments, the online database 306 is Google Maps (Google). It may be an external third party database such as Maps, Yelp or FourSquare. In certain embodiments, the client device 122 accesses the online database 306 without action from the social networking system. In certain embodiments, the client device 122 accesses the online database 306 through a social networking system. The present disclosure contemplates any suitable method of access to the online database 306 and any type of online database 306. In certain embodiments, explicit position 303 may be utilized when adjusting the device settings of client device 122. The explicit location may be self-reported by the user of the client device 122 by check-in, comment, attendance reply request, tag, etc., or by another user by checking in the user, tagging the user with a photo, etc. It may be a declared one. For example, a social networking system determines from a user's actions on a social network that the client device 122 is in a particular location or is attending a particular event, and adjusts the device settings based on that decision. can do. For example, users can check in for events on social networks, such as weddings or nightclubs. Accordingly, the social networking system can issue commands to the client device 122 to reduce or increase the ringing volume, respectively. The present disclosure contemplates any method of utilizing actions on the network to adjust device settings.

0029In certain embodiments, waveform-based position 304 may be used to adjust device settings. For example, if the sonic fingerprint uploaded by client device 122 matches the sonic fingerprint uploaded by another user, the social networking system corrects the position of client device 122 and the corrected position. You can adjust the device settings based on. In certain embodiments, device settings may be adjusted using the explicit location information 303 of other detected users. This particular embodiment will be further described with reference to FIGS. 4, 4A and 5.

0030In certain embodiments, various combinations of data sources 301-306 can be utilized to adjust the settings of client device 122. For example, the ambient sound 301 is used to collate with an object in the sound object database 105 to generate a detected sound object 305. This detected sound object can be used in conjunction with a self-determined position 302, an explicit position 303, or a waveform-based position 304 to further improve the position accuracy of the client device 122. Alternatively or additionally, the online database 306 can be utilized to further improve the position accuracy of the client device 122 and then adjust the device settings of the client device 122. For example, a user indicated by a self-determined position 302 of a client device to be near a park may capture and upload a sonic fingerprint that matches the music of the band playing in the park. Client device 122 or an alternative social networking system can access the online database 306, which contains a list of performances near the park. If it finds a match between the detected sound object 305 and one of the performances in the online database 306, the social networking system knows that the user is actually in the park and listening to a live performance, and sets the ringing volume to " You can adjust it to "high" and turn on the vibration mode. The present disclosure contemplates any suitable combination of data sources 301-306 for adjusting the configuration of client device 122.

0031FIG. 4 shows an exemplary geographic area 400 and self-reported locations on maps of various users 410, 420 and 430 of a social networking system. In certain embodiments, the self-reported position is calculated by a GPS receiver chip within the client device 122. In certain embodiments, the position is calculated by TDOA or other positioning algorithm. In certain embodiments, the geographic area can be segmented into different quadrants and the user waveform fingerprint can be searched against waveform fingerprints uploaded by other users in the same quadrant. In certain embodiments, the search area is based on the amplitude or absolute volume of the uploaded fingerprint. For example, if the uploaded fingerprint indicates that the ambient sound in the user's environment is extremely loud, it is likely that many users are reporting the same sound from different locations, thus narrowing the search area. You may. In certain embodiments, the social networking system first searches for fingerprints uploaded simultaneously by the user's friends, based on the assumption that those friends are likely to be together. In certain embodiments, a spatial partitioning algorithm may be used to determine the search space. In a particular embodiment, the ocree space partitioning algorithm is used to address the search space. In certain embodiments, kd trees are used. In certain embodiments, quadtrees or any other grid-based method is used. The present disclosure contemplates any suitable method of generating a search area for fingerprints uploaded by the user.

0032In FIG. 4, the self-determined positions of the three users are marked on the map. Two users 410 and 420 report their location within Fox Plaza within a few meters of each other. Another user, 430, reports that his location is near the "Marvelous" restaurant across the street. Users 410, 420 and 430 may or may not have explicit edge connections between each other on the social graph. In this example, the social networking system matches the waveform fingerprints generated by all three users.

0033FIG. 4A shows an example of a portion of a social graph that includes user 410 represented by user node 1, user 420 represented by user node 2, and user 430 represented by user node 3. Since the waveform fingerprints from all three users match, social networking systems may assume that these users are generally around the same. In certain embodiments, the social networking system takes into account the amplitudes of all three waveform fingerprints. For example, if all three waveform fingerprints are at the same volume and the volume is low, the social network will assume that the user position of user 430 is incorrect and will determine the corrected position of client device 122 of user 430. You may send it. On the other hand, if all three waveform fingerprints are at the same volume and the volume is extremely high, the social network does not have to send the corrected position.

0034In Figure 4A, user node 1 sends an attendance reply request to the event, checks in at any location, or tags the location or event for the event page, in this case "John and Jane's wedding". You are taking an explicit action on the net, such as being attached. Therefore, if the social networking system finds a waveform fingerprint that matches the fingerprint sent by user node 1, the user sending the matching fingerprint is in the same position and is participating in the same event. Know instantly. Therefore, the user node 2 and the user node 3 can be arranged at the same position, and the corrected position can be transmitted to the user node 3 (user 430 in FIG. 4). In certain embodiments, event types can be used to determine whether to change a user's device settings. For example, social networking systems -If you learn that the node 1, user node 2 and user node 3 are attending the wedding, the social networking system will send a command to automatically change the client device settings to "mute" or "vibrate". You may.

0035Figure 5 shows an exemplary method of adjusting device settings based on ambient waveform capture. At step 501, microphone 245 on client device 122 records the sound wave shape. As mentioned above, the waveform may be recorded by processor 221 running sound capture application 246, or by a low power processor in sensor hub 218. In certain embodiments, the client device 122 constantly records ambient sound and stores it in storage 213. In certain embodiments, the client device 122 periodically records ambient sounds for analysis. In certain embodiments, the client device 122 records ambient sound each time a position change is detected through a GPS sensor 217 or other positioning algorithm. In certain embodiments, the client device 122 captures ambient sound each time the sound exceeds a certain volume. In certain embodiments, sound capture step 501 is performed triggered by user interaction with a social network, such as using a social networking application, browsing a social networking website, or checking in at any location. The present disclosure contemplates any suitable method of initiating or timing sound capture.

0036In step 502, the sound capture application 246 performs feature detection on the waveform and analyzes waveform characteristics, also known as "waveform DNA". Feature detection step 502 may include, in certain embodiments, identifying sonic spikes, predominance or lack of a particular frequency band, or other trends in waveform amplitude. In certain embodiments, feature detection is implemented by FFT or DCT. In certain embodiments, other frequency domain transformations may be utilized to identify important characteristics of the waveform, such as how much treble or bass is present in the waveform. In certain embodiments, the sound capture application 246 uses a discrete wavelet transform. In certain embodiments, the sound capture application 246 utilizes a short-time Fourier transform. In certain embodiments, the sound capture application 246 detects the fundamental frequency of the waveform. In certain embodiments, the sound capture application 246 filters and removes noise components from the captured waveform. In certain embodiments, the sound capture application 246 utilizes a form of event start detection to create a beat model of the captured waveform. In certain embodiments, the sound capture application 246 produces a beat histogram of the waveform. The present disclosure contemplates any suitable method or algorithm for sound wave feature detection.

0037In step 503, generate a sound "fingerprint" for one or more waveforms. A fingerprint is a waveform, a small, robust representation that aggregates a set of waveforms. For example, in certain embodiments, the waveform may be periodically captured and analyzed at one sample per second. Each captured waveform can be analyzed for feature detection, and the sound capture application 246 integrates several waveforms with similar features (perhaps 100-200 waveforms) to create a song, video, or movie. Generates an integrated waveform fingerprint that represents the soundtrack of. In certain embodiments, sound capture application 246 uses vector quantization to generate a representative vector as a waveform fingerprint. In certain embodiments, the sound capture application 246 utilizes spectrogram peaks as captured waveform fingerprints, such as those used in Shazam's music recognition software. Techniques for generating corrugated fingerprints are well known in the art. The present disclosure contemplates any suitable method of generating a fingerprint of one or more captured waveforms.

0038At step 504, client device 122 has fins generated in step 503. Send the garprint to the server. In certain embodiments, the server is the system front end 120 of a social networking system. In certain embodiments, the client device 122 sends the fingerprint directly to one or more servers running the sound matching application 118. The present disclosure contemplates any suitable mechanism for transmitting waveform fingerprints to sound matching application 118. At step 505, one or more servers running the sound matching application 118 receive the transmitted fingerprint.

0039In step 506, the received fingerprint is searched against the database of fingerprints stored in the sound database 105. In certain embodiments, the sound database 105 stores various different types of fingerprints so that even fingerprints generated by different methods can be compared to the stored fingerprints. In certain embodiments, the sound database 105 stores the waveform and dynamically generates a fingerprint of the stored waveform. If no match is found, the process ends at step 512. If a match is found, in step 507, information related to the object associated with the matching fingerprint is retrieved from the social networking system. In certain embodiments, this information is stored in the sound database 105. In certain embodiments, this information may be stored in a separate object database or node database within the social networking system. In certain embodiments, this information may be metadata added to the waveform stored in the sound database 105. This information includes general characteristics such as artist, song name, album name, recording date, and how many of your friends like, comment, or otherwise. It can include characteristics on the social network, such as how it interacted with the object. In certain embodiments, this information may be other music played by the artist or other types of music of the genre. The present disclosure contemplates any kind of information associated with an object associated with a matching waveform fingerprint.

0040At step 508, the social networking system searches its own internal database or a third party online database for location information about the discovered object. For example, because the social networking system knows the self-determined location of client device 122, the social networking system matches the detected sound object with an item in the online sound database associated with a common neighborhood. , The accuracy of the position of the client device 122 can be improved. For example, in the absence of a clear network statement that a user is in a cinema, traditionally, social networking systems have no way of determining whether a user is in or near a cinema. It was. However, a social networking system matches the sonic fingerprint captured by the client device 122 with a particular movie, and the particular movie and the client device 122 location are located near the location of the client device 122. When compared to a list of show times, the social networking system can ensure that a user is watching a particular movie in a particular cinema.

0041At step 509, the server sends this information to client device 122, and at step 510 this information is received at client device 122. The client device 122 can then adjust the device settings of the client device itself, such as lowering the ringing volume or enabling vibration mode in step 511.

0042The sound object database 105 is not limited to music only. For example, social networking systems determine whether a particular user is watching a movie or TV show. Can be determined. In certain embodiments, the sound object database 105 stores a fingerprint for each voice of a user on a social network. Therefore, it is possible to adjust the device settings based on the voice of a particular user detected based on predefined rules. For example, a user of client device 122 may set an explicit rule that sets his client device 122 to mute mode where the user's employer is. This allows the device to automatically enter mute mode when the server matches the detected voice with the voice of the user's employer in the sound database 105.

0043FIG. 6 shows an exemplary method of adjusting device settings based on the user's position with improved accuracy based on the detected waveform. This method is almost the same as the method shown in FIG. 5 up to step 606. In certain embodiments, steps 606-610 can be performed substantially simultaneously with or in parallel with steps 505-509.

0044In step 606, sound matching algorithm 246 pulls the uploaded waveform from the same approximate time and area as the uploading user. The location may be ascertained by self-reported location such as GPS coordinates, TDOA, or check-in. In certain embodiments, the geographic search area is smaller for areas where a large number of fingerprints have been uploaded. In certain embodiments, the search area is based on the amplitude or absolute volume of the uploaded fingerprint. For example, if the uploaded fingerprint indicates that the ambient sound in the user's environment is extremely loud, the search area should be small because it is likely that many users are reporting the same sound from different locations. You may. In certain embodiments, the social networking system first searches for fingerprints uploaded by the user's friends at the same time, based on the assumption that the user's friends who uploaded the fingerprints at the same time are more likely to be together. In certain embodiments, individual users uploading fingerprints are plotted on a map or grid. In such an embodiment, the search space may be determined by using a space division algorithm. In certain embodiments, an ocree space partitioning algorithm is used to address the search space. In certain embodiments, kd trees are used. In certain embodiments, quadtrees or any other grid-based method is used. The present disclosure contemplates any suitable method of generating a search area for fingerprints uploaded by other users.

0045Upon obtaining the matching fingerprints in step 608, the social networking system retrieves the location data associated with the matching fingerprints. For example, if a particular user's uploaded fingerprint matches the uploaded fingerprints of three other users, and one of the three other users checks in to a particular location. The check-in location data is retrieved and sent to the client devices of the three users who did not check in. In certain embodiments, the social networking system averages the geographic locations of users uploading matching waveforms, or points that correspond to all locations of users uploading matching waveforms. May be sought. For example, if the sound fingerprints uploaded by three other users match the fingerprints uploaded by that particular user, the social networking system will calculate one point that corresponds to the position of the three other users. You may. In certain embodiments, social networking systems search for outliers, for example, three out of four users with matching uploaded fingerprints are very close, and one is hundreds of feet (tens). If they are metric) apart, the social networking system may infer that the outlier user's location is inaccurate. At step 609, the location data from step 608 is sent to the client device 122, and at step 610 the client device Received by chair 122.

0046At step 611, the client device 122 adjusts the device settings of the client device itself based on the received position data. In certain embodiments, this setting includes adjusting ring volume, enabling or disabling vibration, adjusting screen brightness, various wireless connections (Bluetooth (Bluetooth®), Wi-Fi), It can include enabling or disabling WiMax, near field communication, or cell radio, locking or unlocking devices, enabling sound recording, and so on. The present disclosure contemplates any suitable method of changing device settings.

0047Certain embodiments may be implemented on one or more computer systems. FIG. 7 shows an exemplary computer system 700. In certain embodiments, one or more computer systems 700 perform one or more steps of one or more of the methods described and illustrated in the present disclosure. In certain embodiments, one or more computer systems 700 provide the functionality described and illustrated in the present disclosure. In certain embodiments, software running on one or more computer systems 700 performs one or more steps of one or more of the methods described and illustrated herein, or the present specification. Provides the functionality described and illustrated in the book. Certain embodiments include one or more parts of one or more computer systems 700.

0048The present disclosure contemplates any suitable number of computer systems 700. The present disclosure contemplates a computer system 700 having any suitable physical form. As an example, the computer system 700 includes embedded computer systems, system-on-chip (SOC), single-board computers (SBC), such as computer-on-module (COM) or system-on-module (SOM), desktop computer systems, wraps. Top or notebook computer systems, interactive kiosks, mainframes, mesh computer systems, mobile phones, mobile information terminals (PDAs), servers, or a combination of two or more of these. Not limited. If appropriate, the computer system 700 can include one or more computer systems 700, either standalone or distributed, installed across multiple locations, installed across multiple machines, or one or more. It may be resident in the cloud, including one or more cloud computers in the network. If appropriate, one or more computer systems 700 perform one or more steps of one or more of the methods described and illustrated herein without substantial spatial or temporal limitation. can do. By way of example, one or more computer systems 700 may, but is not limited to, perform one or more steps of one or more of the methods described or illustrated herein in real-time or batch mode. .. One or more computer systems 700 may, where appropriate, perform one or more steps of one or more of the methods described or illustrated in the present disclosure at different times or at different locations.

0049In certain embodiments, the computer system 700 includes a processor 702, memory 704, storage 706, input / output (I / O) interface 708, communication interface 710, and bus 712. Although the present disclosure describes and illustrates a particular computer system having a particular number of particular components of a particular configuration, the present disclosure describes any suitable configuration of any suitable number of any configuration. Any suitable computer system with elements is intended.

0050In certain embodiments, the processor 702 comprises hardware for executing instructions, such as instructions that make up a computer program. If appropriate, processor 702 can contain one or more arithmetic logic units (ALUs) and is a multi-core processor. It may include one or more processors 702. Although this disclosure describes and illustrates a particular processor, this disclosure contemplates any suitable processor.

0051In certain embodiments, memory 704 includes main memory for storing instructions for execution by processor 702 or data for operation by processor 702. One or more memory buses, each of which may include an address bus and a data bus, allow the processor 702 to be coupled to memory 704. Bus 712 may include one or more memory buses, as described below. In certain embodiments, one or more memory management units (MMUs) reside between the processor 702 and the memory 70, facilitating access to the memory 704 required by the processor 702. In certain embodiments, memory 704 includes random access memory (RAM). This RAM may be a volatile memory if appropriate. If appropriate, this RAM may be dynamic RAM (DRAM) or static RAM (SRAM). Further, if appropriate, this RAM may be single-port or multi-port RAM. This disclosure contemplates any suitable RAM. The memory 704 can include one or more memories 704, as appropriate. Although this disclosure describes and illustrates specific memory, the present disclosure contemplates any suitable memory.

0052In certain embodiments, storage 706 includes mass storage for data or instructions. As an example, storage 706 may include HDDs, floppy (registered trademark) disk drives, flash memory, optical disks, magnetic optical disks, magnetic tape or universal serial bus (USB) drives, or a combination of two or more of these. Not limited to. Storage 706 can include removable or non-removable (or fixed) media, as appropriate. The storage 706 may be internal storage or external storage of the computer system 700, as appropriate. In certain embodiments, the storage 706 is a non-volatile solid-state memory. In certain embodiments, storage 706 includes read-only memory (ROM). The present disclosure contemplates mass storage 706 in any suitable physical form. The storage 706 may optionally include one or more storage control units that facilitate communication between the processor 702 and the storage 706. If appropriate, the storage 706 can include one or more storage 706s. Although this disclosure describes and illustrates specific storage, this disclosure contemplates any suitable storage.

0053In certain embodiments, the I / O interface 708 provides hardware, software, or both that provide one or more interfaces for communication between the computer system 700 and one or more I / O devices. Including. The computer system 700 may include one or more of these I / O devices, as appropriate. One or more of these I / O devices can allow communication between a human and the computer system 700. As an example, I / O devices include keyboards, keypads, microphones, monitors, mice, printers, scanners, speakers, still image cameras, stylus, tablets, touch screens, trackballs, camcorders and other suitable I / O devices. Or can include, but is not limited to, a combination of two or more of these. The I / O device may include one or more sensors. The present disclosure contemplates any suitable I / O devices and any suitable I / O interface 708 for them. If appropriate, the I / O interface 708 may include one or more device or software drivers that allow the processor 702 to drive one or more of these I / O devices. The I / O interface 708 can include one or more I / O interfaces 708, as appropriate. This disclosure describes and illustrates specific I / O interfaces, but the book. The disclosure is intended for any suitable I / O interface.

0054In certain embodiments, the communication interface 710 is one or more for communication between the computer system 700 and one or more other computer systems 700 or one or more networks (eg, packet-based communication). Includes hardware, software, or both that provide the interface. As an example, the communication interface 710 is a network interface controller (NIC) or network adapter for communication with Ethernet (registered trademark) (Ethernet (registered trademark)) or other wired networks, or wireless such as a Wi-Fi network. It may include, but is not limited to, a network interface controller (NIC) or wireless adapter for communication with a wireless network interface controller (WNIC) for communication with the network. The present disclosure contemplates any suitable network and any suitable communication interface 710 for it. As an example, the computer system 700 can be an ad hoc network, a personal area network (PAN), a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), or one or more parts of the Internet, or It can communicate with two or more combinations of these, but is not limited to these. One or more parts of one or more of these networks may be wired or wireless. As an example, the computer system 700 is a wireless PAN (WPAN), (eg Bluetooth WPAN, etc.), WI-FI network, WIMAX network, cellular telephone network (eg, Global System for Mobile Communication: It can communicate with GSM® networks), or other suitable wireless networks, or a combination of two or more of these. The computer system 700 may include any suitable communication interface 710 for any of these networks, if appropriate. The communication interface 710 may include one or more communication interfaces 710, as appropriate. Although this disclosure describes and illustrates a particular communication interface, this disclosure contemplates any suitable communication interface.

0055In certain embodiments, bus 712 includes hardware, software, or both that connect the components of computer system 700 to each other. Bus 712 can include one or more buses 712, as appropriate. Although this disclosure describes and illustrates a particular bus, the disclosure contemplates any suitable bus or interconnect.

0056When used herein as a computer-readable storage medium, it includes one or more non-transitory tangible computer-readable storage medium process structures. As an example, computer readable storage media can be semiconductor-based or other integrated circuits (ICs), such as field programmable gate arrays (FPGAs) or special purpose ICs (ASICs), hard disks, HDDs, hybrid hardware, if appropriate. Drives (HHD), optical disks, optical disk drives (ODD), magnetic optical disks, magnetic optical drives, floppy disks, floppy disk drives (FDD), magnetic tapes, holographic storage media, semiconductor drives (SSD), RAM drives, secure digital ( It can include, but is not limited to, SD) cards, SD drives, or other suitable computer-readable storage media, or a combination of two or more of them. As used herein, the term computer-readable storage medium does not include media that are not eligible for protection under Section 101 of the US Patent Act (35 U.SC § 101). As used herein, the term computer-readable storage medium does not include temporary forms of signal transmission (such as propagation of electrical or electromagnetic signals themselves) unless they are eligible for the protection of Section 101 of US Patent Act. Computer-readable non-temporary storage media, if appropriate, are volatile, non-volatile, or volatile and non-volatile. It may be an outgoing combination.

0057The present disclosure contemplates one or more computer-readable storage media that implement any suitable storage. In certain embodiments, the computer-readable storage medium is, as appropriate, one or more parts of processor 702 (eg, one or more internal registers or caches), one or more parts of memory 704, storage 706. Implement one or more parts, or a combination of these. In certain embodiments, the computer-readable storage medium implements RAM or ROM. In certain embodiments, the computer-readable storage medium implements volatile or persistent memory. In certain embodiments, one or more computer-readable storage media embody the software. As used herein, the term software, as appropriate, refers to one or more applications, bytecode, one or more computer programs, one or more executable files, one or more imperatives, logic, machine code, One or more scripts, Or it can include source code and vice versa. In certain embodiments, the software includes one or more application programming interfaces (APIs). The present disclosure contemplates any suitable programming language or any suitable software described or otherwise represented in any suitable programming language or combination of programming languages. In certain embodiments, the software is represented as source code or object code. In certain embodiments, the software is represented in a high-level programming language, such as C, Pearl, or a suitable extension thereof. In certain embodiments, the software is represented in a low-level programming language such as assembly language (or machine code). In certain embodiments, the software is represented in Java®, C or C ++. In certain embodiments, the software is represented in hypertext markup language (HTML), extended markup language (XML), or other suitable markup language.

0058As used herein, "or" is inclusive and not exclusive unless otherwise specified or otherwise indicated in the context. Accordingly, "A or B" as used herein means "A or B, or both," unless otherwise stated or otherwise indicated in the context. Also, "and" are both joint and individual, unless otherwise stated or otherwise indicated by context. Accordingly, in the present disclosure, "A and B" are used "in collaboration with A and B, or individually with A and B," unless otherwise stated or otherwise indicated in the context. It means that there is.

0059The present disclosure includes all modifications, alternatives, modifications, modifications and modifications to the exemplary embodiments described in this disclosure that will be appreciated by those skilled in the art. Similarly, where appropriate, the appended claims include all modifications, alternatives, modifications, modifications and modifications to the exemplary embodiments described in this disclosure that will be appreciated by those skilled in the art. In addition, within the appended claims, certain functions are adapted to, arranged to, capable of, and perform. Configured to, enabled to, operable to, or operative to perform) When we say a device or system, or a component of a device or system, it is the device, system, or component, regardless of whether the device, system, component, or a particular feature thereof is activated, turned on, or unlocked. Includes the device, system, and component as long as the component is so adapted, configured, possible, configured, enabled, operational, or operational.