Applying an image overlay to an image based on relationship of the people identified in the image

BACKGROUND

00011. Field

0002The disclosure relates generally to image processing and more specifically to selecting, recommending, and applying an image overlay to an image based on a relationship between the people identified in the image.

00032. Description of the Related Art

0004In various social media applications, users are able to post their photographs online. In addition, various social media applications may provide a user profile to identify persons that the user knows and how a user is associated with another person.

SUMMARY

0005According to one illustrative embodiment, a computer-implemented method for applying an image overlay is provided. A data processing system determines a relationship type and a relationship strength between two or more people identified in a first image based on profile data corresponding to each of the two or more people and a set of data elements determined to be common to the profile data corresponding to each of the two or more people. The data processing system selects an image overlay to apply to the first image based on the relationship type and the relationship strength between the two or more people and the set of data elements determined to be common to the profile data corresponding to each of the two or more people. The data processing system applies the image overlay to the first image generating a second image that includes the first image and the applied image overlay. According to other illustrative embodiments, a data processing system and computer program product for applying an image overlay are provided.

BRIEF DESCRIPTION OF THE DRAWINGS

0006<figref idref="DRAWINGS">FIG. 1</figref> is a pictorial representation of a network of data processing systems in which illustrative embodiments may be implemented;

0007<figref idref="DRAWINGS">FIG. 2</figref> is a diagram of a data processing system in which illustrative embodiments may be implemented;

0008<figref idref="DRAWINGS">FIG. 3</figref> is an example of a best friends image with overlay in accordance with an illustrative embodiment;

0009<figref idref="DRAWINGS">FIG. 4</figref> is an example of a just married image with overlay in accordance with an illustrative embodiment;

0010<figref idref="DRAWINGS">FIGS. 5A-5B</figref> are a flowchart illustrating a process for applying an image overlay to an image in accordance with an illustrative embodiment; and

0011<figref idref="DRAWINGS">FIG. 6</figref> is a flowchart illustrating a process for searching for an image overlay to apply to an image based on criteria in accordance with an illustrative embodiment.

DETAILED DESCRIPTION

0012The present invention may be a system, a method, and/or a computer program product at any possible technical detail level of integration. The computer program product may include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present invention.

0013The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium may be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of the computer readable storage medium includes the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.

0014Computer readable program instructions described herein can be downloaded to respective computing/processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and/or a wireless network. The network may comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers. A network adapter card or network interface in each computing/processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing/processing device.

0015Computer readable program instructions for carrying out operations of the present invention may be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, configuration data for integrated circuitry, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++, or the like, and procedural programming languages, such as the “C” programming language or similar programming languages. The computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) may execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present invention.

0016Aspects of the present invention are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer readable program instructions.

0017These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, a programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer readable storage medium having instructions stored therein comprises an article of manufacture including instructions which implement aspects of the function/act specified in the flowchart and/or block diagram block or blocks.

0018The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process, such that the instructions which execute on the computer, other programmable apparatus, or other device implement the functions/acts specified in the flowchart and/or block diagram block or blocks.

0019The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the Figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts or carry out combinations of special purpose hardware and computer instructions.

0020With reference now to the figures, and in particular, with reference to <figref idref="DRAWINGS">FIG. 1</figref> and <figref idref="DRAWINGS">FIG. 2</figref>, diagrams of data processing environments are provided in which illustrative embodiments may be implemented. It should be appreciated that <figref idref="DRAWINGS">FIG. 1</figref> and <figref idref="DRAWINGS">FIG. 2</figref> are only meant as examples and are not intended to assert or imply any limitation with regard to the environments in which different embodiments may be implemented. Many modifications to the depicted environments may be made.

0021<figref idref="DRAWINGS">FIG. 1</figref> depicts a pictorial representation of a network of data processing systems in which illustrative embodiments may be implemented. Network data processing system <b>100</b> is a network of computers, data processing systems, and other devices in which the illustrative embodiments may be implemented. Network data processing system <b>100</b> contains network <b>102</b>, which is the medium used to provide communications links between the computers, data processing systems, and other devices connected together within network data processing system <b>100</b>. Network <b>102</b> may include connections, such as, for example, wire communication links, wireless communication links, and fiber optic cables.

0022In the depicted example, server <b>104</b> and server <b>106</b> connect to network <b>102</b>, along with storage <b>108</b>. Server <b>104</b> and server <b>106</b> may be, for example, server computers with high-speed connections to network <b>102</b>. Server <b>104</b> and server <b>106</b> may provide a set of one or more services to registered users of client devices connected to network <b>102</b>. For example, server <b>104</b> and server <b>106</b> may provide a set of social media services to a plurality of registered users. Also, it should be noted that server <b>104</b> and server <b>106</b> may represent a plurality of different servers hosting a plurality of different social media websites.

0023Client <b>110</b>, client <b>112</b>, and client <b>114</b> also connect to network <b>102</b>. Clients <b>110</b>, <b>112</b>, and <b>114</b> are clients of server <b>104</b> and server <b>106</b>. Further, server <b>104</b> and server <b>106</b> may provide information, such as boot files, operating system images, and software applications to clients <b>110</b>, <b>112</b>, and <b>114</b>.

0024In this example, clients <b>110</b>, <b>112</b>, and <b>114</b> are illustrated as desktop or personal computers with wire or wireless communication links to network <b>102</b>. However, it should be noted that clients <b>110</b>, <b>112</b>, and <b>114</b> are meant as examples only. In other words, clients <b>110</b>, <b>112</b>, and <b>114</b> may include other types of data processing systems, such as, for example, laptop computers, handheld computers, smart phones, cellular phones, smart watches, personal digital assistants, gaming devices, kiosks, set top boxes, cameras, and the like. Registered users of clients <b>110</b>, <b>112</b>, and <b>114</b> may utilize clients <b>110</b>, <b>112</b>, and <b>114</b> to access the social media services provided by server <b>104</b> and server <b>106</b>.

0025Further, users of clients <b>110</b>, <b>112</b>, and <b>114</b> may utilize client <b>110</b>, client <b>112</b>, and client <b>114</b> to capture images using imaging device <b>116</b>, imaging device <b>118</b>, and imaging device <b>120</b>, respectively. Imaging devices <b>116</b>, <b>118</b>, and <b>120</b> may be an integral part of clients <b>110</b>, <b>112</b>, and <b>114</b> or may be coupled to clients <b>110</b>, <b>112</b>, and <b>114</b>. Each of imaging devices <b>116</b>, <b>118</b>, and <b>120</b> may be, for example, a camera capable of capturing images. The images may be, for example, still photographs and video clips.

0026Furthermore, clients <b>110</b>, <b>112</b>, and <b>114</b> may select, recommend, and apply image overlays to captured images. An image overlay is a decorative layer of textual content and/or graphical content added to a captured image to create a composite image comprising the original captured image and the applied image overlay. The textual content may be, for example, “I love my best friend” and the graphical content may be, for example, a “♥”. Clients <b>110</b>, <b>112</b>, and <b>114</b> may select, recommend, and apply image overlays based on clients <b>110</b>, <b>112</b>, and <b>114</b> identifying people in a captured image. Clients <b>110</b>, <b>112</b>, and <b>114</b> may utilize, for example, facial recognition technology to identify the people in a captured image. Moreover, clients <b>110</b>, <b>112</b>, and <b>114</b> may access data posted on one or more social media websites and/or data stored in user profiles to identify relationships between the people in the captured image. Clients <b>110</b>, <b>112</b>, and <b>114</b> also may generate new image overlays by combining two or more stored image overlays. In addition, clients <b>110</b>, <b>112</b>, and <b>114</b> may generate new image overlays by removing textual content from or adding textual content to stored image overlays, along with graphical content.

0027Storage <b>108</b> is a network storage device capable of storing any type of data in a structured format or an unstructured format. In addition, storage <b>108</b> may represent a set of one or more network storage devices. Storage <b>108</b> may store, for example, names and identification information for a plurality of different registered users; profiles corresponding to each of the registered users; images captured by the plurality of registered users; image overlays to be applied to captured images; history of image overlay selections by each of the registered users; et cetera. Further, storage <b>108</b> also may store other data, such as authentication or credential data that may include user names, passwords, and biometric data associated with the plurality of different registered users, for example.

0028In addition, it should be noted that network data processing system <b>100</b> may include any number of additional server devices, client devices, and other devices not shown. Program code located in network data processing system <b>100</b> may be stored on a computer readable storage medium and downloaded to a computer or data processing system for use. For example, program code may be stored on a computer readable storage medium on server <b>104</b> and downloaded to client <b>110</b> over network <b>102</b> for use on client <b>110</b>.

0029In the depicted example, network data processing system <b>100</b> may be implemented as a number of different types of communication networks, such as, for example, an internet, an intranet, a local area network (LAN), a wide area network (WAN), or any combination thereof. <figref idref="DRAWINGS">FIG. 1</figref> is intended as an example, and not as an architectural limitation for the different illustrative embodiments.

0030With reference now to <figref idref="DRAWINGS">FIG. 2</figref>, a diagram of a data processing system is depicted in accordance with an illustrative embodiment. Data processing system <b>200</b> is an example of a data processing system, such as client <b>110</b> in <figref idref="DRAWINGS">FIG. 1</figref>, in which computer readable program code or program instructions implementing processes of illustrative embodiments may be located. In this illustrative example, data processing system <b>200</b> includes communications fabric <b>202</b>, which provides communications between processor unit <b>204</b>, memory <b>206</b>, persistent storage <b>208</b>, imaging unit <b>210</b>, communications unit <b>212</b>, input/output unit <b>214</b>, and display <b>216</b>.

0031Processor unit <b>204</b> serves to execute instructions for software applications and programs that may be loaded into memory <b>206</b>. Processor unit <b>204</b> may be a set of one or more hardware processor devices or may be a multi-processor core, depending on the particular implementation. Further, processor unit <b>204</b> may be implemented using one or more heterogeneous processor systems, in which a main processor is present with secondary processors on a single chip. As another illustrative example, processor unit <b>204</b> may be a symmetric multi-processor system containing multiple processors of the same type.

0032Memory <b>206</b> and persistent storage <b>208</b> are examples of storage devices <b>218</b>. A computer readable storage device is any piece of hardware that is capable of storing information, such as, for example, without limitation, data, computer readable program code in functional form, and/or other suitable information either on a transient basis and/or a persistent basis. Further, a computer readable storage device excludes a propagation medium. Memory <b>206</b>, in these examples, may be, for example, a random access memory, or any other suitable volatile or non-volatile storage device. Persistent storage <b>208</b> may take various forms, depending on the particular implementation. For example, persistent storage <b>208</b> may contain one or more devices. For example, persistent storage <b>208</b> may be a hard drive, a flash memory, a rewritable optical disk, a rewritable magnetic tape, or some combination of the above. The media used by persistent storage <b>208</b> may be removable. For example, a removable hard drive may be used for persistent storage <b>208</b>.

0033In this example, persistent storage <b>208</b> stores image overlay manager <b>220</b>, image <b>222</b>, profile <b>224</b>, overlays <b>226</b>, and composite image <b>228</b>. Image overlay manager <b>220</b> selects, recommends, and applies image overlays to captured images based on image overlay manager <b>220</b> identifying relationships between people in the captured images. It should be noted that even though image overlay manager <b>220</b> is illustrated as residing in persistent storage <b>208</b>, in an alternative illustrative embodiment image overlay manager <b>220</b> may be a separate component of data processing system <b>200</b>. For example, image overlay manager <b>220</b> may be a hardware component coupled to communication fabric <b>202</b> or a combination of hardware and software components. In another alternative illustrative embodiment, image overlay manager <b>220</b> may be located in a server device, such as server <b>104</b> in <figref idref="DRAWINGS">FIG. 1</figref>. In a further alternative illustrative embodiment, a first set of components of image overlay manager <b>220</b> may be located locally in data processing system <b>200</b> and a second set of components may be located remotely in a server device.

0034In this example, image overlay manager <b>220</b> includes facial recognition component <b>230</b>, profile analyzing component <b>232</b>, social media search component <b>234</b>, relationship analyzing component <b>236</b>, overlay selector <b>238</b>, and overlay generator <b>240</b>. However, it should be noted that illustrative embodiments are not limited to such. In other words, image overlay manager <b>220</b> may include more or fewer components than illustrated.

0035Image overlay manager <b>220</b> utilizes facial recognition component <b>230</b> to identify people <b>246</b> in image <b>222</b>. People <b>246</b> represent two or more individuals captured in image <b>222</b>. Image <b>222</b> may be, for example, a photograph or a video.

0036Image overlay manager <b>220</b> utilizes profile analyzing component <b>232</b> to access and analyze the relationship and life event data contained in profile <b>224</b>. Profile <b>224</b> represents a collection of data corresponding to a user of data processing system <b>200</b>. The data contained in profile <b>224</b> may include, for example, names of people <b>248</b> that the user knows and is associated with. Profile <b>224</b> also includes relationships <b>250</b>, which represent how the user is associated with people <b>248</b>. For example, the user may identify a person in people <b>248</b> as a family member, such as a spouse, a child, a parent, or a grandparent, a friend, a best friend, a fiancé or fiancée, a co-worker, a classmate, a traveling companion, and the like. In other words, image overlay manager <b>220</b> utilizes profile analyzing component <b>232</b> to help identify the relationships between people <b>246</b> in image <b>222</b>.

0037Image overlay manager <b>220</b> also utilizes social media search component <b>234</b> to help identify the relationships between people <b>246</b> in image <b>222</b>. Social media search component <b>234</b> searches one or more social media websites to find relationship and life event data posted on the social media websites by people <b>246</b>, which were identified by facial recognition component <b>230</b>, in image <b>222</b>. Image overlay manager <b>220</b> utilizes relationship analyzing component <b>236</b> to analyze the relationship and life event data found by profile analyzing component <b>232</b> in profile <b>224</b> and the relationship and life event data found by social media search component <b>234</b> in the social media websites.

0038In this example, relationship analyzing component <b>236</b> includes commonality identifier <b>242</b> and relationship identifier <b>244</b>. Commonality identifier <b>242</b> determines data elements that are common to people <b>246</b> in image <b>222</b>. Relationship identifier <b>244</b> determines a relationship type and a relationship strength between people <b>246</b> in image <b>222</b>. In various illustrative embodiments, relationship strength may be determined from the length of time two persons have been associated, the number other persons the two persons have a common relationship with, the number of interactions between the two persons, a schedule of relationship strengths where the two persons have been explicitly identified by a user as having a particular relationship (e.g., a person identified as a spouse may be assigned a high relationship strength on the schedule while a person identified as being a member of a common organization may be assigned a low relationship strength), and other similar factors. In various illustrative embodiments, two persons may be explicitly identified by a user as having a particular relationship, such as, for example, sibling, spouse, family member, et cetera.

0039Image overlay manager <b>220</b> utilizes overlay selector <b>238</b> to analyze the output of relationship analyzing component <b>236</b> and select one or more image overlays from overlays <b>226</b>. Overlays <b>226</b> represent a plurality of stored image overlays that may be applied to captured images, such as image <b>222</b>, based on the relationship type and strength between people <b>246</b> and the data elements determined to be in common between people <b>246</b>. Alternatively, image overlay manager <b>220</b> may utilize overlay generator <b>240</b> to analyze the output of relationship analyzing component <b>236</b> and generate a new image overlay for image <b>222</b> based on selected content from two or more different overlays in overlays <b>226</b>.

0040Composite image <b>228</b> comprises image <b>252</b> and image overlay <b>254</b>. Image <b>252</b> is the same as image <b>222</b>. Image overlay manager <b>220</b> generates composite image <b>228</b> by combining image <b>252</b> with image overlay <b>254</b>. After generating composite image <b>228</b>, image overlay manager <b>220</b> displays composite image <b>252</b> in display <b>216</b> for review by the user of data processing system <b>200</b>. Image overlay manager <b>220</b> also may include an accept element and a reject element in the display for selection by the user. If the user selects the accept element, then image overlay manager <b>220</b> saves composite image <b>228</b>. If the user selects the reject element, then image overlay manager <b>220</b> removes image overlay <b>254</b> from image <b>252</b>. After removing image overlay <b>254</b> from image <b>252</b>, image overlay manager <b>220</b> selects and recommends a new set of image overlays for image <b>252</b>.

0041Imaging unit <b>210</b> is a hardware device capable of capturing still and/or video images. Imaging unit <b>210</b> may be, for example, a camera. Also, it should be noted that imaging unit <b>210</b> may include a set of two or more imaging devices.

0042Communications unit <b>212</b>, in this example, provides for communication with other computers, data processing systems, and devices via a network, such as network <b>102</b> in <figref idref="DRAWINGS">FIG. 1</figref>. Communications unit <b>212</b> may provide communications using both physical and wireless communications links. The physical communications link may utilize, for example, a wire, cable, universal serial bus, or any other physical technology to establish a physical communications link for data processing system <b>200</b>. The wireless communications link may utilize, for example, shortwave, high frequency, ultra high frequency, microwave, wireless fidelity (WiFi), bluetooth technology, global system for mobile communications (GSM), code division multiple access (CDMA), second-generation (2G), third-generation (3G), fourth-generation (4G), 4G Long Term Evolution (LTE), LTE Advanced, or any other wireless communication technology or standard to establish a wireless communications link for data processing system <b>200</b>.

0043Input/output unit <b>214</b> allows for the input and output of data with other devices that may be connected to data processing system <b>200</b>. For example, input/output unit <b>214</b> may provide a connection for user input through a keypad, keyboard, and/or some other suitable input device. Display <b>216</b> provides a mechanism to display information to a user and may include touch screen capabilities to allow the user to make on-screen selections through user interfaces or input data, for example. Display <b>216</b> also may display images, such as photographs and videos with or without image overlays, to the user.

0044Instructions for the operating system, applications, and/or programs may be located in storage devices <b>218</b>, which are in communication with processor unit <b>204</b> through communications fabric <b>202</b>. In this illustrative example, the instructions are in a functional form on persistent storage <b>208</b>. These instructions may be loaded into memory <b>206</b> for running by processor unit <b>204</b>. The processes of the different embodiments may be performed by processor unit <b>204</b> using computer implemented program instructions, which may be located in a memory, such as memory <b>206</b>. These program instructions are referred to as program code, computer usable program code, or computer readable program code that may be read and run by a processor in processor unit <b>204</b>. The program code, in the different embodiments, may be embodied on different physical computer readable storage devices, such as memory <b>206</b> or persistent storage <b>208</b>.

0045Program code <b>256</b> is located in a functional form on computer readable media <b>258</b> that is selectively removable and may be loaded onto or transferred to data processing system <b>200</b> for running by processor unit <b>204</b>. Program code <b>256</b> and computer readable media <b>258</b> form computer program product <b>260</b>. In one example, computer readable media <b>258</b> may be computer readable storage media <b>262</b> or computer readable signal media <b>264</b>. Computer readable storage media <b>262</b> may include, for example, an optical or magnetic disc that is inserted or placed into a drive or other device that is part of persistent storage <b>208</b> for transfer onto a storage device, such as a hard drive, that is part of persistent storage <b>208</b>. Computer readable storage media <b>262</b> also may take the form of a persistent storage, such as a hard drive, a thumb drive, or a flash memory that is connected to data processing system <b>200</b>. In some instances, computer readable storage media <b>262</b> may not be removable from data processing system <b>200</b>.

0046Alternatively, program code <b>256</b> may be transferred to data processing system <b>200</b> using computer readable signal media <b>264</b>. Computer readable signal media <b>264</b> may be, for example, a propagated data signal containing program code <b>256</b>. For example, computer readable signal media <b>264</b> may be an electro-magnetic signal, an optical signal, and/or any other suitable type of signal. These signals may be transmitted over communication links, such as wireless communication links, an optical fiber cable, a coaxial cable, a wire, and/or any other suitable type of communications link. In other words, the communications link and/or the connection may be physical or wireless in the illustrative examples. The computer readable media also may take the form of non-tangible media, such as communication links or wireless transmissions containing the program code.

0047In some illustrative embodiments, program code <b>256</b> may be downloaded over a network to persistent storage <b>208</b> from another device or data processing system through computer readable signal media <b>264</b> for use within data processing system <b>200</b>. For instance, program code stored in a computer readable storage media in a data processing system may be downloaded over a network from the data processing system to data processing system <b>200</b>. The data processing system providing program code <b>256</b> may be a server computer, a client computer, or some other device capable of storing and transmitting program code <b>256</b>.

0048The different components illustrated for data processing system <b>200</b> are not meant to provide architectural limitations to the manner in which different embodiments may be implemented. The different illustrative embodiments may be implemented in a data processing system including components in addition to, or in place of, those illustrated for data processing system <b>200</b>. Other components shown in <figref idref="DRAWINGS">FIG. 2</figref> can be varied from the illustrative examples shown. The different embodiments may be implemented using any hardware device or system capable of executing program code. As one example, data processing system <b>200</b> may include organic components integrated with inorganic components and/or may be comprised entirely of organic components excluding a human being. For example, a storage device may be comprised of an organic semiconductor.

0049As another example, a computer readable storage device in data processing system <b>200</b> is any hardware apparatus that may store data. Memory <b>206</b>, persistent storage <b>208</b>, and computer readable storage media <b>262</b> are examples of physical storage devices in a tangible form.

0050In another example, a bus system may be used to implement communications fabric <b>202</b> and may be comprised of one or more buses, such as a system bus or an input/output bus. Of course, the bus system may be implemented using any suitable type of architecture that provides for a transfer of data between different components or devices attached to the bus system. Additionally, a communications unit may include one or more devices used to transmit and receive data, such as a modem or a network adapter. Further, a memory may be, for example, memory <b>206</b> or a cache such as found in an interface and memory controller hub that may be present in communications fabric <b>202</b>.

0051In the course of developing illustrative embodiments, it was discovered that a way to recommend image overlays to a user of a device capable of taking images based on the relationship of the people captured in an image does not exist. The prior art does not discuss selecting an image overlay from a database of possible overlay graphics based on the relationship of the people identified in an image, such as a photograph or video. Illustrative embodiments generate and/or recommend image overlays to be applied an image using retrieved information regarding the type and strength of relationship between the people that illustrative embodiments identify in the image.

0052Illustrative embodiments receive an image and perform an analysis to determine whether two or more people are captured in the image. If the image contains two or more people, then illustrative embodiments identify the people in the image. Illustrative embodiments may perform this identification using various technologies, such as, for example, facial recognition. Alternatively, illustrative embodiments may rely on manual identification of people in the image by a user. In addition, illustrative embodiments may attach the identification of people in the image as metadata to the image.

0053Subsequently, illustrative embodiments identify the relationship between the people identified in the image by retrieving relationship data from a set of one or more online social media sources and/or stored user profiles. The relationship data may include, for example, how long a friendship has lasted, how many friends the people have in common, whether the people are explicitly identified as family members or as a married couple, dates when life events, such as marriages, anniversaries, and birthdays, occur, locations where the people attended school or college, groups or organizations, such as scouts, the people are members of, and the like. Then, illustrative embodiments analyze the relationship data to determine the type and strength of the relationship between the people captured in the image. Afterward, illustrative embodiments generate and/or recommend image overlays for the user to select from based on identified relationships and life events.

0054Further, illustrative embodiments may determine whether too many or too few people exist in the image in order to apply an overlay. For example, if illustrative embodiments identify two siblings and one non-sibling in an image, then illustrative embodiments do not recommend a sibling-related overlay or if illustrative embodiments identify only one person in an image, then illustrative embodiments do not recommend a relationship-related overlay to the image. After the user selects a desired image overlay, then illustrative embodiments apply the selected image overlay to the image to generate a composite image. Furthermore, it should be noted that illustrative embodiments may apply an image overlay to a video. Moreover, illustrative embodiments may apply animated image overlays, which are actively in motion and not static, to images.

0055With reference now to <figref idref="DRAWINGS">FIG. 3</figref>, an example of a best friends image with overlay is depicted in accordance with an illustrative embodiment. Best friends image with overlay <b>300</b> may be, for example, composite image <b>228</b> in <figref idref="DRAWINGS">FIG. 2</figref>. In this example, illustrative embodiments receive an image capturing two people in the foreground. Illustrative embodiments identify the two people as best friends based on relationship data retrieved from online social media sources and/or stored user profiles. Illustrative embodiments then apply a best friends forever or “BFF” image overlay containing two heart graphics to the image. As a result, best friends image with overlay <b>300</b> includes best friends image <b>302</b> and applied overlay <b>304</b>. However, the users may not like applied overlay <b>304</b> and reject its application to best friends image <b>302</b>. Consequently, illustrative embodiments remove applied overlay <b>304</b> from best friends image <b>302</b> and display recommended overlays <b>306</b> to the users for selection and application to best friends image <b>302</b>.

0056With reference now to <figref idref="DRAWINGS">FIG. 4</figref>, an example of a just married image with overlay is depicted in accordance with an illustrative embodiment. Just married image with overlay <b>400</b> may be, for example, composite image <b>228</b> in <figref idref="DRAWINGS">FIG. 2</figref>. In this example, illustrative embodiments receive an image capturing two people. Illustrative embodiments identify the two people as a just recently married couple based on life event data retrieved from online social media sources and/or stored user profiles. Illustrative embodiments then apply a just married image overlay containing two heart graphics to the image. As a result, just married image with overlay <b>400</b> includes just married image <b>402</b> and applied overlay <b>404</b>. However, the users may not like applied overlay <b>404</b> and reject its application to just married image <b>402</b>. Consequently, illustrative embodiments remove applied overlay <b>404</b> from just married image <b>402</b> and display recommended overlays <b>406</b> to the users for selection and application to just married image <b>402</b>.

0057With reference now to <figref idref="DRAWINGS">FIGS. 5A-5B</figref>, a flowchart illustrating a process for applying an image overlay to an image is shown in accordance with an illustrative embodiment. The process shown in <figref idref="DRAWINGS">FIGS. 5A-5B</figref> may be implemented in a data processing system, such as, for example, client <b>110</b> in <figref idref="DRAWINGS">FIG. 1</figref> or data processing system <b>200</b> in <figref idref="DRAWINGS">FIG. 2</figref>.

0058The process begins when the data processing system receives an input to capture an image using an imaging device of the data processing system (step <b>502</b>). The imaging device may be, for example, imaging unit <b>210</b> in <figref idref="DRAWINGS">FIG. 2</figref>. In response to receiving the input at step <b>502</b>, the data processing system captures the image, such as image <b>222</b> in <figref idref="DRAWINGS">FIG. 2</figref> (step <b>504</b>).

0059Subsequently, the data processing system makes a determination as to whether people are in the image (step <b>506</b>). If the data processing system determines that no people are in the image, no output of step <b>506</b>, then the data processing system stores the image (step <b>508</b>). Thereafter, the process terminates.

0060If the data processing system determines that people are in the image, yes output of step <b>506</b>, then the data processing system identifies the people in the image (step <b>510</b>). In addition, the data processing system searches a set of one or more data sources to find one or more commonalities between the people in the image (step <b>512</b>). The set of data sources may be, for example, online social media data sources and/or user profile data sources.

0061Afterward, the data processing system makes a determination as to whether the data processing system is able to identify a relationship between the people in the image based on finding one or more commonalities between the people (step <b>514</b>). If the data processing system is able to identify a relationship between the people in the image based on finding one or more commonalities between the people, yes output of step <b>514</b>, then the data processing system selects an image overlay to apply to the image based on the relationship between the people in the image (step <b>516</b>). Further, the data processing system applies the selected image overlay to the image generating a composite image, such as composite image <b>228</b> in <figref idref="DRAWINGS">FIG. 2</figref> (step <b>518</b>).

0062Then, the data processing system displays the composite image in a display of the data processing system with an accept element and a reject element (step <b>520</b>). The data processing system makes a determination as to whether the accept element was selected (step <b>522</b>). If the data processing system determines that the accept element was selected, yes output of step <b>522</b>, then the process returns to step <b>508</b> where the data processing system saves the composite image. If the data processing system determines that the reject element was selected, no output of step <b>522</b>, then the data processing system removes the selected image overlay from the image (step <b>524</b>). Thereafter, the process proceeds to step <b>526</b>.

0063Returning again to step <b>514</b>, if the data processing system is not able to identify a relationship between the people in the image, no output of step <b>514</b>, then the data processing system recommends a set of image overlays to apply to the image (step <b>526</b>). The recommended set of image overlays may be, for example, a set of general or generic image overlays or a set of default image overlays. Alternatively, the recommended set of image overlays may be based on one or more other factors, such as, for example, geographic location of where the image was captured. Subsequently, the data processing system receives a selection of one of the set of image overlays to apply to the image (step <b>528</b>). Thereafter, the process returns to step <b>518</b> where the data processing system applies the selected image overlay to the image.

0064With reference now to <figref idref="DRAWINGS">FIG. 6</figref>, a flowchart illustrating a process for searching for an image overlay to apply to an image based on criteria is shown in accordance with an illustrative embodiment. The process shown in <figref idref="DRAWINGS">FIG. 6</figref> may be implemented in a data processing system, such as, for example, client <b>110</b> in <figref idref="DRAWINGS">FIG. 1</figref> or data processing system <b>200</b> in <figref idref="DRAWINGS">FIG. 2</figref>.

0065The process begins when the data processing system identifies two or more people captured in a first image (step <b>602</b>). Afterward, the data processing system retrieves profile data corresponding to each of the two or more people from a storage device of the data processing system (step <b>604</b>).

0066Further, the data processing system determines a set of one or more data elements common to the profile data corresponding to each of the two or more people (step <b>606</b>). Furthermore, the data processing system determines a relationship type and a relationship strength between the two or more people based on the profile data corresponding to each of the two or more people and the set of one or more data elements determined to be common to the profile data corresponding to each of the two or more people (step <b>608</b>). Then, the data processing system searches the storage device for an image overlay to apply to the first image using search criteria based on the relationship type and the relationship strength between the two or more people and the set of one or more data elements determined to be common to the profile data corresponding to each of the two or more people (step <b>610</b>). Subsequently, the data processing system applies the image overlay to the first image generating a second image comprising the first image and the applied image overlay (step <b>612</b>). Thereafter, the process terminates.

0067Thus, illustrative embodiments of the present invention provide a computer-implemented method, data processing system, and computer program product for selecting, recommending, and applying an image overlay to an image based on a relationship between the people identified in the image. The descriptions of the various embodiments of the present invention have been presented for purposes of illustration, but are not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiment. The terminology used herein was chosen to best explain the principles of the embodiment, the practical application or technical improvement over technologies found in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed here.