Computer-implemented systems and methods for comparing and associating objects
Abstract
Computer-implemented systems and methods for comparing and associating objects are disclosed. In some embodiments, a method is provided for associating a first object with one or more objects within a plurality of objects, each object having a first plurality of characteristics, each characteristic having data reflecting a characteristic of a compound represented by the object entity wherein the associated objects have matching data in corresponding characteristics for a second plurality of properties. The method may include that, for each object within the plurality of objects and for the first object, the following is carried out: generating a Slug for the object, wherein the slug comprises the second plurality of characteristics of the object; and enter the slug on the object in a Bloom filter. Further, the method may include that, for a position in the Bloom filter, corresponding to the Slug for the first object, an association between objects is generated whose Slugs position correspond if the Slugs for these objects match.

Term
Projected expiry 14 March 2034.
- Priority
- Filed
- Published
- Today
- Projected expiry
12 claims: 12 independent, 0 dependent
- 1A method for identifying unikalen objects in a plurality of objects, each object having a first plurality of characteristics, each characteristic having data reflecting a characteristic of a compound represented by the object entity, the method comprising the following operations or by a multiple processors are running:Performing, for each object in the plurality of objects, of the following: Generating a Slug for the object, wherein the slug comprises a second plurality of characteristics of the object;and Entering the Slug for the object in a counting Bloom Filter;Identifying, for each generated Slug, its corresponding position in the counting Bloom filter has a count equal to 1, of the Slug associated object as unikal in the plurality of objects;Inputting, for each generated slug, the slug and its associated object into a Multimap, if a position in the counting bloom filter corresponding to the Slug, a count value of greater than 1, wherein said slug is a key to the Multimap and the object a value to the Multimap is;and Identifying, for each key, the Multimap with a single value of the object that is associated with the stored key as Slug, as unikal in the plurality of objects. Verfahren zum Identifizieren von unikalen Objekten in einer Mehrzahl von Objekten, wobei jedes Objekt eine erste Mehrzahl von Eigenschaften aufweist, jede Eigenschaft Daten aufweist, die ein Kennzeichen einer durch das Objekt repräsentierten Entität widerspiegeln, wobei das Verfahren die folgenden Operationen umfasst, die durch einen oder mehrere Prozessoren ausgeführt werden: Ausführen, für jedes Objekt in der Mehrzahl von Objekten, des Folgenden: Erzeugen eines Slug für das Objekt, wobei der Slug eine zweite Mehrzahl von Eigenschaften von dem Objekt aufweist;und Eingeben des Slug für das Objekt in einen zählenden Bloomfilter;Identifizieren, für jeden erzeugten Slug, dessen entsprechende Position in dem zählenden Bloomfilter einen Zählwert gleich 1 aufweist, des dem Slug zugeordneten Objekts als unikal in der Mehrzahl von Objekten;Eingeben, für jeden erzeugten Slug, des Slug und dessen zugehörigen Objekts in eine Multimap, falls eine Position in dem zählenden Bloomfilter, die dem Slug entspricht, einen Zählwert größer als 1 aufweist, wobei der Slug ein Schlüssel zu der Multimap ist und das Objekt ein Wert zu der Multimap ist;und Identifizieren, für jeden Schlüssel der Multimap mit einem einzigen Wert, des Objekts, das dem als Schlüssel gespeicherten Slug zugeordnet ist, als unikal in der Mehrzahl von Objekten.
- 2The method of claim 1, further comprising:Setting the size of the counting Bloom filter, for a predetermined error rate, and number of objects within the plurality of objects. Verfahren nach Anspruch 1, weiter umfassend: Festlegen der Größe des zählenden Bloomfilters, für eine vorbestimmte Fehlerrate und Anzahl von Objekten innerhalb der Mehrzahl von Objekten.
- 5The method of claim 4, wherein each 2-bit counter is a saturation counter. Verfahren nach Anspruch 4, wobei jeder 2-Bit-Zähler ein Sättigungszähler ist.
- 6Method according to one of claims 1 to 5, wherein the slug comprises a concatenation of character strings which are separated by a delimiter, the delimiter having a character that is not otherwise present in the character strings that have been concatenated. Verfahren nach einem der Ansprüche 1 bis 5, wobei der Slug eine Verkettung von Zeichenketten aufweist, die durch einen Abgrenzer getrennt sind, wobei der Abgrenzer ein Zeichen aufweist, das ansonsten nicht in den Zeichenketten vorhanden ist, die verkettet wurden.
- 7Method according to one of claims 1 to 5, wherein the slug comprises a concatenation of character strings which are separated by a delimiter, the delimiter comprises a sequence of two or more characters and the sequence of two or more characters not in any of the two or is more strings available that have been chained. Verfahren nach einem der Ansprüche 1 bis 5, wobei der Slug eine Verkettung von Zeichenketten aufweist, die durch einen Abgrenzer getrennt sind, wobei der Abgrenzer eine Abfolge von zwei oder mehr Zeichen aufweist und die Abfolge von zwei oder mehr Zeichen nicht in irgendeiner der zwei oder mehr Zeichenketten vorhanden ist, die verkettet wurden.
- 8A method according to any one of claims 1 to 7, wherein the number of properties in the first plurality of properties is the same as the number of properties in the second plurality of characteristics. Verfahren nach einem der Ansprüche 1 bis 7, wobei die Anzahl von Eigenschaften in der ersten Mehrzahl von Eigenschaften gleich groß wie die Anzahl von Eigenschaften in der zweiten Mehrzahl von Eigenschaften ist.
- 9A method according to any one of claims 1 to 7, wherein the number of properties in the first plurality of characteristics is larger than the number of attributes in the second plurality of characteristics. Verfahren nach einem der Ansprüche 1 bis 7, wobei die Anzahl von Eigenschaften in der ersten Mehrzahl von Eigenschaften größer als die Anzahl von Eigenschaften in der zweiten Mehrzahl von Eigenschaften ist.
- 10A method according to any one of claims 1 to 9, wherein the operation of inputting, Slug generated for each, of the slug and its associated object into the Multimap using at least one processor is executed. Verfahren nach einem der Ansprüche 1 bis 9, wobei die Operation des Eingebens, für jeden erzeugten Slug, des Slug und dessen zugehörigen Objekts in die Multimap unter Verwendung mindestens eines Prozessors ausgeführt wird.
- 11System, comprising:a storage device which stores a set of instructions;and at least one processor executing the set of instructions to perform operations which include the operations of any one of claims 1 to 10th System, aufweisend: eine Speichervorrichtung, die einen Satz von Anweisungen speichert;und mindestens einen Prozessor, der den Satz von Anweisungen ausführt, um Operationen durchzuführen, welche die Operationen nach einem der Ansprüche 1 bis 10 beinhalten.
- 12The computer-readable medium that stores instructions, which when executed by a plurality of processors, the one or more processors cause carrying out operations which include the operations of any one of claims 1 to tenth or an Computerlesbares Medium, das Anweisungen speichert, die, wenn sie durch einen oder mehrere Prozessoren ausgeführt werden, den einen oder die mehreren Prozessoren veranlassen, Operationen durchzuführen, welche die Operationen nach einem der Ansprüche 1 bis 10 beinhalten.
Independent claims12
65 paragraphs, as filed
This application claims the benefit of US provisional patent application Ser. No. 61 / 801.297, filed March 15, 2013. US patent application Ser. No. 14 / 099,661, filed December 6, 2013 and US patent application Ser. No. 14 / 140.415 filed on December 24, 2013, the disclosures of which are hereby incorporated by reference in full in this document.
Many organizations, including industry and government agencies recognize that important conclusions can be drawn when large data sets can be analyzed to identify patterns of behavior that suggest danger to public safety or prove illegal acts. These analyzes often include that data, which are associated with an interest person or thing to be compared with other data, which are associated with the same person or thing, to determine that the same person or thing has been involved in several acts that security concerns cause or criminal concerns.
However, it can improve the quality of the analytical results, resulting from a use of technically advanced analysis tools, be limited by the quality of the data used by the tool. For certain types of analyzes an acceptable error rate has literally or almost be zero, so that a drawn from the data of analytical conclusion is well founded. An achievement of this zero or almost-zero error rate for data sets that include a two- or three-digit million value of each record can be problematic. Current data comparison tools are not well suited to solve these problems.
The above problems are particularly acute for analyzes that involve data concerning available public safety in relation to identifying persons or property for investigations. For example, the acceptable error rate is in analysis tools for identifying potential security threats in general not greater than zero, because the price that is recognized by mistake to a presence of a security threat (ie "false positive") may be completed or that a non-detection of a security threat (ie ), is unacceptably high "false negative". It is therefore necessary that serves to support public safety use tools data associated with interest any person or property to data, are in the same person or thing in relation, in a correct manner with respect.
There are some tools to make an accurate comparison of data, but these are records that contain millions of individual entries, no practical use to the computer analysis. For example, one solution is to determine whether two special properties are associated with the same interest person or thing is to compare each element of an object with a corresponding element in the second object. For example, for objects that include M elements, a first element in the first object with a corresponding first element in the second object are compared, and appropriate comparisons can be made for each of the remaining M-1 elements, which together form the first and second objects are. If the elements in each object are altogether suitable to uniquely identify the represented person or thing with certainty, and match the corresponding elements in the first and second objects, it can be concluded in reasonable manner the fact that the objects reflect the same person or thing. Alternatively, each object may be converted into a single string (serialized) are reflecting the content of each element to be compared.
Thereafter, could be a character string, which is generated from the one object with a character string, which is generated from another object, as a shape of an object are compared with the comparison.
For certain records, it is possible that the solutions described above consume little memory or system resources, since the objects or their serialized strings on disk instead can be stored in main memory. Quickly, however, the above-described solutions in large or non-trivial data sets can practically no longer be usable. As the number of objects to be compared, the number of comparisons, and therefore the processing time of the comparisons increases exponentially; that is proportional to n<sup>2</sup>/ 2, where n is the number of objects to be compared represents. Thus, a comparison of 500 objects using a serialized approach, its processing time can be assumed to be approximated as the time to 125,000 string comparisons to perform, be manageable by computer analysis. However, a comparison of 100 million (100 M) records can be performed using this approach, the computation time approximated as the time to carry out 5 trillion (5E15) string comparison can be assumed to be intractable by computer analysis. In addition, a reading strings from a disk, instead of this oneread memory, add additional computation time.
Another solution for identifying matching objects in a corpus of objects is to store each object in a Multimap. This Multimap is an associative array that stores a plurality of values for each key. The import of the objects in the Multimap causes objects with the same element data is stored in a single entry on Multimap. Thus is accomplished by use of a Multimap associating identical objects.
A drawback with using a Multimap of object comparison is that the Multimap is typically stored in the main memory, having due concerned of the algorithm considerations, which relates to a organize keys in Multimap, and therefore has an object comparison means sufficient main memory to a the keeping entire body comprehensive Multimap in memory. Therefore, can not possibly be a Multimap solution for records of 100 M or more objects. Similar disadvantages exist for each approach when applied to other object comparison problems, for example, an efficient identification of unikalen (one occurring) objects in a corpus of objects and efficient comparing a single object with all objects in a corpus of objects.
None of the solutions is feasible for records that include the approximately or more than 100 M objects. However, it is object data records that include 100 M or more objects, not unusual nowadays. Therefore, the problems described here are quite real, and there is a need for improved object comparison means.
The present invention is set out in the independent claims. The dependent claims concern optional features of some embodiments of the invention.
Subsequently, reference is made to the accompanying drawings which illustrate exemplary embodiments of the present application, wherein:
<figref>1</figref> a flow chart of an exemplary process for comparing a target object to at least some objects in a corpus shows, according to some embodiments of the present disclosure.
<figref>2</figref> a flow chart of an exemplary process for comparing all the objects showing in a corpus with all the other objects in the body to determine matching items within the corpus, according to some embodiments of the present disclosure.
<figref>3</figref> a flow chart of an exemplary process for comparing all the objects showing in a corpus with all the other objects in the body to determine Unique objects within the body, according to some embodiments of the present disclosure.
<figref>4</figref> an exemplary computing environment displays can be implemented within embodiments of the present disclosure.
Hereinafter, detailed reference is made to the embodiments, examples of which are illustrated in the accompanying drawings. Whenever possible, the same reference numbers will be used throughout the drawings to refer to the same or like parts.
Embodiments of the present disclosure can avoid the disadvantages of conventional object comparison means, characterized that computer-implemented systems and methods are provided to compare objects in a way that allows for greater computing throughput and an acceptable memory usage without requiring a comparison accuracy is reduced, as well as for record sizes previously virtually impossible, or could not be met at an acceptable calculation throughput level.
Embodiments of the present disclosure relate to a class of computational problems, relating to a property comparison. A member of this class includes an efficient object comparison in a particular object with a corpus of objects. Another member of this class includes an efficient comparison of each object in a body with all other objects in the body. An additional element of this class involves efficiently identifying unikalen objects in a corpus of objects.
The following detailed description begins with a general overview of an object comparison. Some examples of being compared or analyzed objects are delivered. The description then explained an exemplary embodiment, which is concerned with the previously discussed first problem class (ie an efficient comparing a single object with all objects in a corpus). The description then expands the solution for the first problem class, to address the above-explained second problem class (ie in an efficient comparison of each object in a body with all other objectsCorpus). The detailed description then discloses a solution for the third problem class (ie an efficient identification of unikalen objects in a corpus of objects). An introduction to objects and an overview of Object Compare follows.
Several types of objects exist in the field of computer science. One in the field of computer science of well-known type of object is an object in the object-oriented sense. Wikipedia describes an object of this type as a set of elements (ie, data structures), and methods, the functions are similar. Without necessarily this simplistic description concur, are embodiments that implement the discussed here Property reference solutions, compatible with a comparing objects of this type.
Another type of object in the field of computer science is a data structure that reflects the characteristics of a person or thing that is relevant for a specific task or data processing environment. In some embodiments, these characteristics are reflected by strings (strings). In further embodiments, characteristics by strings, integers, real numbers, time or dates, binary values, structures in the sense of the C programming, variables lists and / or other forms of data can be mirrored. In some embodiments, properties of any object type to be converted before a compare to strings. In other embodiments possibly some properties be strings or may be able to be converted to strings, however, may possibly be other properties not strings and may not be converted to strings. The embodiments of the present disclosure can work with Zeichenkette- or non-string properties.
Moreover, the idea of a "data structure" in this context is very flexible. The term "data structure" can reflect any type of structured data, from a database of stored information (with columns that reflect the elements in an object or a data structure, and table rows, the instances of the object or the data structure to reflect), and further formatted text in a text file (for example, data in an XML structure), to data stored in a running computer program. Accordingly include, as a data structure comprising the above-described types of structured data grossly, grossly objects also these types of structured data. In addition, those listed here Property reference solutions are also compatible with a comparing objects of these types.
In some embodiments, includes an effective object comparison that you consider what characteristics of relevant objects to be compared for performing the comparison, since the entities (eg. As people or things), which are reflected by these objects, in different environments may have different relevant properties. For example, an object properties of an automobile store, which may be relevant for a motor vehicle office of a State, by storing the following information: Vehicle Identification Number (VIN), year, make, model, expiration date of the vehicle registration and a direct or indirect indication of the person is the owner of the vehicle.
For cars that are sold on an auction site such as eBay, however, the relevant characteristics of an automobile may be different from those that are relevant for the motor vehicle office of the state. For example, a data structure for storing characteristics of an automobile, which is listed for sale on eBay include: VIN, year, make, model, mileage, condition of the automobile, minimum auction bid and a direct or indirect indication of the person for the sale vehicle has stopped. Thus, characteristics of an entity (eg. As a person or thing) that is relevant to an environment different from properties of the entity that are relevant to a different environment. Accordingly, properties of an object that will be considered during an object comparison in an environment different from those that will be considered during an object comparison in a second environment.
In some embodiments, an effective data comparison can include that into account, what properties it aimed to distinguish (a person or thing, for. Example) of other instances of the entity an entity. For example, should, according to interpretation, be a VIN for an automobile unikal (one occurring) for this automobile. However, sometimes there may be situations where a VIN is not unikal for a particular automobile. Such situations may result from deliberate errors or accidental errors. An example of a deliberate error is an attempt of fraudulent registration of a stolen vehicle with a VIN alleged. An example of a random error occurs when an areas addressing smog-check employee a VIN incorrectly enters into a computer at a smog inspection station, leading toa smog-verification data with incorrect VIN lead, which is then passed on to a database of a state. Data errors occur in real data processing environments, and therefore result in some embodiments of the present disclosure, a minimize or eliminate errors by, namely the fact that objects are identified by a combination of several object properties, rather than to identify objects by using a single object property.
In some embodiments, one or more identifying characteristics of an object are extracted from the object and saved in a data structure. This data structure is referred to as "slug"; it contains information that may be sufficient to identify an entity (eg. as a person or thing) with a certain degree of information redundancy unique in order to permit detecting errors in the properties of the slug. In some embodiments, the slug has a concatenation of strings separated by a delimiter. In some embodiments, the separator is a null character, however, in other embodiments, the separator be a sign that otherwise does not exist in the concatenated string. In some embodiments, the concatenated strings can through a separation string (eg. "-") Instead of a separator to be limited. In embodiments that use a separator string, the separation string can be any string that might not otherwise occur in the strings that were chained. In further embodiments, the slug has a data structure such as an object, a data field (array), a structure or an associative data array (array of values).
For example, included in an embodiment of a slug for an automobile features that include a VIN, a brand, a model and a year specification for the automobile. Characterized that the properties of the make, model and year for the automobile is contained in the Slug, a capability for detecting errors in the VIN property is provided as the VIN property is not the only object property that is being compared. Thus Slugs, which are associated with two cars, if there is an error in the VIN property of an automotive object, match, an automotive object with the same VIN property like incorrect VIN has also the same brand, model have and year properties.
The chances of these random match several characteristics between two or more objects may be negligible. Therefore, an inclusion of a certain degree should be avoided to information redundancy erroneous matches with an object comparison or at least considerably reduce, with respect to object comparisons, where only a single property between objects is compared, irrespective of the fact, that it was intended that single property its associated entity (eg. B. person or thing) clearly identified.
Exemplary embodiments are described below, which solve the above-mentioned first problem, ie an efficient comparing a specific object (hereinafter referred to as "target") having all of the objects in a corpus. The disclosed embodiments use a bloom filter to identify Slugs, which are associated with objects in the body that do not match the slug for the target object. This rapid detection is performed by that Slugs are discarded, which are associated with a position in the Bloom filter, which is different from the position that is associated with the slug for the target object.
Bloom filters have the property that two slugs that fall in different positions in the Bloom filter, with certainty have different properties and thus reflect different objects. Therefore true when the slug for the target object does not fall in the same position as the slug for a specific object in the body, the target object does not match the specific object in the body and thus can be excluded from future consideration in such embodiments.
<figref>1</figref> shows a flowchart of an exemplary process <figref>100</figref> for comparing a target object with at least some of objects in a body, according to some embodiments of the present disclosure. In some embodiments, the target object to be compared with at least some objects in the body, part of the corpus. In these embodiments, a comparison between the target object and all other objects is carried out in the corpus. In further embodiments, the object which is to be compared with at least some objects in the body, is not part of the corpus. In these further embodiments, a comparison between the target object and all the objects is performed in the corpus.
As shown, at step <figref>102</figref> set the size of a Bloom filter, and this created taking into account the error rate, which is obtained for the body size to be processed. For example, a increasing the number of positions inaiming a Bloom filter intended to reduce the error rate for a specific body size, however, a decrease in the number of positions in a Bloom filter designed to increase the error rate for a particular body size. Methods for determining the size of a Bloom filter to achieve a target error rate for a specific body size, are well known in the art, and therefore these methods are not discussed here.
At step <figref>104</figref> a slug of the target object (ie the object, are compared against all of the objects in the body) generated. Considerations which properties are to be included on an object in a Slug were previously discussed. At step<figref>106</figref> determining a Bloom filter-position corresponding to the Slug for the target object. In some embodiments, a Bloom filter position for a Slug be determined that the slug is inputted to a bloom filter and the bloom filter is instructed to reveal the position at which the slug has been added.
In further embodiments, a Bloom filter position for a Slug be determined that the slug is presented as an input for a software function, which is associated with the Bloom filter without saving the Slug in Bloom filters. In further embodiments, a position of a slug may be determined by the fact that the slug is input to a software function that reflects a position selection algorithm for a Bloom filter, in the absence of use of actual and / or total Bloom filter, and the Bloom filter position as the output value of this software function is obtained. In other embodiments other approaches may be used to provide a Bloom filter-position of a slug. In these approaches for identifying a Bloom filter for a Slug, according to the embodiments explained above, will total in steps<figref>106</figref>. <figref>108</figref> Referring. The particular bloom filter position is used to identify matches of Slug-comparison, some of which can be "false positive", said explained below Bloom filter is used.
At step <figref>108</figref> a slug for each object is created in the body. At step<figref>110</figref> is determined in the corpus a Bloom filter position for each object. In some embodiments, a Bloom filter position for an object are determined by the fact that the slug of the object is entered in the Bloom filter and the bloom filter is instructed to reveal the position at which the slug has been added.
After completion of step <figref>110</figref> Slugs reflect that of in step <figref>108</figref> meet identified position matches the slug for the target resist. Some of these matches can be false-positive matches, instead of real matches. Therefore filtering steps<figref>112</figref> and <figref>114</figref> this false positive matches from by using a Multimap.
In step <figref>112</figref> is, for each Slug, which is associated with an object in the body and its position in the Bloom filter is the same position as that of the slug for the target object, which added to an object in the body associated Slug and its associated object in the body of a Multimap. When adding the Slug and its associated object to the Multimap Slug represents the key to the Multimap, and the object in the body represents the value to the Multimap. This Multimap is used to remove false positives resulting from the treatment. In step<figref>114</figref> the process will be completed by selecting the real-positive matches identified in the Multimap. These non-false positive matches can be retrieved from the Multimap, characterized that, be read with the Slug for the target object as a key, data from the Multimap.
In some embodiments, the process <figref>100</figref> are distributed across multiple processors. For example, a Bloom filter on each of multiple processors may be present, and steps<figref>102</figref> to <figref>114</figref> can be run on each of the plurality of processors. The corpus of objects can be distributed among the different processors so that all objects are processed by a processor, but no object is processed by more than one processor. In such embodiments, each of the plural processors performs outputting of part of the matching with the target object by objects in the body.
Exemplary embodiments are described below, which solve the above-explained second problem, ie an efficient Compare all properties with all the objects in a corpus. These embodiments use a counting bloom filter to make a quick identification of Slugs that are associated with objects in the body, which does not match the slug for the target object. Counting Bloom filter are well known in the art, and therefore their structure and their construction will not be discussed here.
Specifically, even assuming a position in the counting Bloom filter has a value of zero or one after Slugs were entered for all objects in the body in the Bloom filter, no object whose Slug is associated with this position, coincide with another slug, and therefore this Slugs are excluded from further consideration. These slugs can be excluded asis understood by those skilled that Bloom filters may have false positive results, but they may have no false negatives. Therefore reflects a counting Bloom filter, whose count is less than two, an accurate determination reflected that no agreement between Slugs exists associated with this position because any agreement would produce a count of at least two. However, false-positive results occur with objects whose Slugs are associated with the same bloom filter position, and therefore false positives may be removed by an additional processing, as will be explained later.
<figref>2</figref> shows a flowchart of an exemplary process <figref>200</figref> for comparing all objects in one body with all other objects in the body to determine matches within the body, according to some embodiments of the present disclosure. As shown, in step<figref>202</figref> set the size of a counting Bloom filter and this created taking into account the error rate that results for the to be processed corpus size. For example, an increasing number of positions in a counting Bloom filter designed to reduce the error rate for a specific body size, however, a decrease in the number of positions in a counting Bloom filter designed to increase the error rate for a particular body size. Methods for determining the size of a counting Bloom filter to achieve a target error rate for a specific body size, are well known in the art, and therefore these methods are not discussed here.
In some embodiments, the counting bloom filter may comprise an N-bit counters, and these counters may be implemented as a two-bit counter (ie, N = 2). In further embodiments, these counters one-bit counter or counter comprises more than two bits can be. In still further embodiments, these meters may be saturation counter; ie these counters count up to a maximum value high and then not exceed this value.
In step <figref>204</figref> a slug for each object is created in the body. In step<figref>206</figref> is input to the counting bloom filter, which causes a counter is incremented in a position corresponding to a Slug Slug each. After completion of step<figref>206</figref> reflect positions whose counters have a value greater than one, one or more matching Slugs resist. Some of these matches may false positive matches, instead of his real matches. Therefore filtering steps<figref>208</figref> and <figref>210</figref> the false-positive matches from using Multimap.
In step <figref>208</figref> are added for Slugs, which are associated with positions in counting Bloom filter whose counters have a value greater than one, the Slug and its associated object of Multimap. When adding the Slug and its associated object to the Multimap Slug represents the key to the Multimap, and the object in the body represents the value to the Multimap. This Multimap is used to remove false positives resulting from the treatment. In step<figref>210</figref> is the process <figref>200</figref> finished, by outputting a value for any key in the Multimap having two or more values. The output values reflect objects that matched their Slugs Slugs with at least one other object in the body. Thus, the displayed objects identify such objects where selected properties, as reflected in a slug of an object in an unambiguous manner consistent with at least one other object in the body.
In some embodiments, the process <figref>200</figref> are distributed across multiple processors. For example, a counting Bloom filter on each of multiple processors exist, and steps<figref>202</figref>. <figref>204</figref> and <figref>206</figref> can be run on each of the plurality of processors. The corpus of objects can be distributed among the different processors so that all objects are processed by a processor, but no object is processed by more than one processor. In such embodiments, takes place, prior to carrying out step<figref>208</figref>, A summing up of counters for each position in the counting Bloom filter with counters for the same position in counting Bloom Filter on other processors. Thereafter, the process moves<figref>200</figref> with a performing steps <figref>208</figref> and <figref>210</figref> on a single processor continues.
Exemplary embodiments are described below, solve the above-explained third problem, that is an efficient unikalen identifying objects in a corpus. These embodiments use a counting Bloom filter and Multimap to perform rapid identifying unikalen objects. If Slugs are entered for all objects in the body in the counting Bloom filter, each position reflects a count of one resist includes a unique object because Bloom filters do not produce false negatives. Additionally could, to the extent that these positions counts reflect counts of two or more false-positive results. Therefore Multimap allows determination,whether the matches, which are reflected in the counts, false or true-positive.
<figref>3</figref> shows a flowchart of an exemplary process <figref>300</figref> for comparing all objects in one body with all other objects in the body to determine Unique objects inside the body, according to some embodiments of the present disclosure. As shown, in step<figref>302</figref> set the size of a counting Bloom filter and this created taking into account the error rate that results for the to be processed corpus size. For example, an increasing number of positions in a counting Bloom filter designed to reduce the error rate for a specific body size, however, a decrease in the number of positions in a counting Bloom filter designed to increase the error rate for a particular body size. Methods for determining the size of a counting Bloom filter to achieve a target error rate for a specific body size, are well known in the art, and therefore these methods are not discussed here.
In some embodiments, the counting bloom filter may comprise an N-bit counter and this counter can be implemented as a two-bit counter (ie, N = 2). In further embodiments, these counters one-bit counter or counter comprises more than two bits can be. In still further embodiments, these meters may be saturation counter; ie these counters count up to a maximum value high and then not exceed this value.
In step <figref>304</figref> a slug for each object is created in the body. In step<figref>306</figref> is input to the counting bloom filter, which causes a counter is incremented in a position corresponding to a Slug Slug each. As previously discussed, after Slugs were entered for all objects in the body in the counting Bloom filter, reflected in any position with a count of one includes a unique object in the body, since the counting Bloom filter produces no false negatives. Therefore, in step<figref>308</figref> for each Slug whose count is one in counting Bloom filter, output the corresponding object as the Slug unikales object in the body.
After completion of step <figref>308</figref> reflect positions whose counts have a value greater than one, one or more matching Slugs resist; ie Slugs not unikal. Some of these matches can be false-positive matches instead of real matches, due to the nature of Bloom filters, as previously discussed. Therefore, in steps<figref>310</figref> and <figref>312</figref> the false-positive matches are filtered using a Multimap.
In steps <figref>310</figref> and <figref>312</figref> it is determined whether the counting Bloom filter the presence of other objects unikalen masked because of the Bloom filter allows false positive results. In step<figref>310</figref> is, for each Slug, its associated position one count of greater than one, entered the slug as the key to Multimap, and the Slug associated object is entered as the value for this key. In step<figref>312</figref> the process ends after each of the value in the Multimap is issued for keys that only have a single value. Unique properties in the body are represented by the collection of objects of step<figref>308</figref> are issued, and the collection of objects which by step <figref>312</figref> are issued, reflected, since the former objects reflects the Slugs were the sole Slug in a position in counting Bloom filter, and therefore were unikal under Slugs, which are associated with objects in the body, whereas the latter Slugs reflects that false-positive results in counting Bloom filter were, for however by the Multimap uniqueness was prepared.
In some embodiments, the process <figref>300</figref> are distributed across multiple processors. For example, a counting Bloom filter on each of multiple processors exist, and steps<figref>302</figref>. <figref>304</figref> and <figref>306</figref> can be run on each of the plurality of processors. The corpus of objects can be distributed among the different processors so that all objects are processed by a processor, but no object is processed by more than one processor. In such embodiments, takes place, prior to carrying out step<figref>308</figref>, A summing up of counters for each position in the counting Bloom filter with counters for the same position in counting Bloom Filter on other processors. Thereafter, the process moves<figref>300</figref> with a performing steps <figref>308</figref>. <figref>310</figref> and <figref>312</figref> on a single processor continues.
<figref>4</figref> shows an exemplary computing environment in which the embodiments of the present disclosure can be implemented.
computer system <figref>400</figref> includes a bus <figref>402</figref> or other communication mechanism for forwarding information, and a hardware processor <figref>404</figref>Coupled to the bus <figref>402</figref> is connected for processing information. In some embodiments, the hardware processor<figref>404</figref> for example, be a general purpose microprocessor, or it may be a microprocessor with reduced instruction set.
The computer system <figref>400</figref> also includes a main memory <figref>406</figref>, For example, a RAM (Random Access Memory) or other dynamic storage device, coupled to the bus <figref>402</figref> is connected to store information and instructions by the processor <figref>404</figref> are to be executed. The main memory<figref>406</figref> can also be used to store temporary variables or other intermediate information during execution of instructions by the processor <figref>404</figref>, make such statements when they are stored in non-transitory storage media processor the<figref>404</figref> are available, the computer system <figref>400</figref> to a special machine that is customized to perform the specified in the instructions operations.
In some embodiments, the computer system includes <figref>400</figref> Further, a ROM (read only memory) <figref>408</figref> or other static storage device coupled to bus <figref>402</figref> is connected to static information and instructions for processor <figref>404</figref> save. A memory device<figref>410</figref>Such as a magnetic disk or optical disk, is provided and the bus <figref>402</figref> connected to store information and instructions.
The computer system <figref>400</figref> can via the bus <figref>402</figref> with a display <figref>412</figref> be connected, for example, a cathode ray tube (CRT) or an LCD screen to display a user of a computer information. An input device<figref>414</figref>That includes alphanumeric and other keys, is coupled to the bus <figref>402</figref> connected to information and command selections to processor <figref>404</figref> to be transmitted. Another type of user input device is cursor control<figref>416</figref>Such as a mouse, trackball, or cursor direction keys to the processor <figref>404</figref> communicate direction information and command selections and cursor movement on the display device <figref>412</figref> control. This input device typically has two degrees of freedom in two axes, a first axis (e.g., x) and a second axis (e.g., y), which allows the device to designate positions in a plane.
In computer system <figref>400</figref> , the processes and methods described herein using customized hard-wired logic circuits, one or more ASICs or FPGAs, firmware and / or program logic to be implemented, the cause or program in combination with the computer system, the computer system <figref>400</figref> a special machine. In some embodiments, the processes and methods described herein are by the computer system<figref>400</figref> responsive performed thereon, that the processor <figref>404</figref> one or more sequences of one or more in the main memory <figref>406</figref> executes instructions. Such instructions may in the main memory<figref>406</figref> from another memory medium, such as the storage device <figref>410</figref>are read. An embodiment of the main memory<figref>406</figref> Instruction sequences contained causes the processor <figref>404</figref>To carry out the process steps described herein. In further embodiments, hard-wired circuitry may be used instead of software instructions or in combination with these.
The term "storage media" as used herein, refers to any non-transitory media that store data and / or instructions that cause a machine to operate in a specific manner. Such storage media may comprise non-volatile media and / or volatile media. Non-volatile media include, for example, optical or magnetic disks, a, for example, the storage device<figref>410</figref>, Volatile media include dynamic memory, such as the main memory<figref>406</figref> on. Common forms of storage media include, for example, a floppy disk, a flexible disk, a hard disk, a semiconductor disk drive, a magnetic tape or any other magnetic data storage medium, a CD-ROM, any other optical data storage medium, any physical medium with patterns of holes, a RAM, a PROM, and EPROM, a flash-EPROM, NVRAM, and any other memory chip or cassette.
Storage media differ from one transmission media, but may be used in conjunction with these. Transmission media are participating in a transfer of information between storage media. For example, transmission media includes coaxial cables, copper wire and fiber optics, including the wires that the bus<figref>402</figref> includes. Transmission media can also take the form of acoustic or light waves, such as those generated during radio and infrared data communications.
Various forms of media may be involved in executing one or more sequences of one or more instructions by the processor <figref>404</figref> are to be executed. For example, the instructions may be stored on a magnetic disk or a semiconductor drive (solid-state drive) a remotely located computer initially. The remote computer can load the instructions into its dynamic memory and send the instructions over a telephone line using a modem. A computer system at the<figref>400</figref> befindliches modem can receive the data on the telephone line and use an infrared transmitter to convert the data to an infrared signal. An infrared detector can receive the data carried in the infrared signal and appropriate circuitry can use the data on the bus<figref>402</figref> lay. The bus<figref>402</figref> conveys the data to main memory <figref>406</figref>From which the processor <figref>404</figref> the instructions fetch and execute. The from main memory<figref>406</figref> received instructions can optionally in the memory device <figref>410</figref> be stored either before or after executing by the processor <figref>404</figref>,
The computer system <figref>400</figref> also includes a communication interface <figref>418</figref>That the bus <figref>402</figref> connected is. The communication interface<figref>418</figref> provides a two-way data communication connection to a network link <figref>420</figref> provided that a local area network <figref>422</figref> connected is. For example, a communication interface<figref>418</figref> be an ISDN card (ISDN = Integrated Services Digital Network), a cable modem, a satellite modem, or a modem to provide a data communication connection to a corresponding type of telephone line. As another example, a communication interface<figref>418</figref> be a LAN card (LAN = Local Area Network) to provide a data communication connection to a compatible LAN. It is also possible wireless connections are implemented. In any such implementation, transmits and receives a communication interface<figref>418</figref> electrical, electromagnetic or optical signals that carry digital data streams representing various types of information.
The network link <figref>420</figref> typically provides a data communication to other data devices via one or more networks. For example, the network link<figref>420</figref> over a local network <figref>422</figref> a connection to a host computer <figref>424</figref> provide or to data equipment operated by an Internet Service Provider (ISP) <figref>426</figref> operate. The ISP<figref>426</figref> in turn provides data communication services through the world wide packet data communication network ready, the now commonly referred to as "Internet" <figref>428</figref> referred to as. Both the local network<figref>422</figref> and the web <figref>428</figref> use electrical, electromagnetic or optical signals that can carry digital data streams. The current through the various networks signals and the signals on the network link<figref>420</figref> and the communication interface <figref>418</figref> run, which the digital data to the computer system <figref>400</figref> back and carry away from, are exemplary forms of transmission media.
The computer system <figref>400</figref> can send messages and data, including program code, send and receive, via the / the network (s), the network link <figref>420</figref> and the communication interface <figref>418</figref>, In the example of the Internet could be a server<figref>430</figref> a requested code for an application program through Internet <figref>428</figref>, The ISP <figref>426</figref>, The local area network <figref>422</figref> and the communication interface <figref>418</figref> send. The received code may by processor<figref>404</figref> be carried out, namely unchanged as received, and / or it may in the storage device <figref>410</figref> or other non-volatile storage for later execution are stored.
5 sheets
Sheet 1 Sheet 2 Sheet 3 Sheet 4 Sheet 5
Every citation, both ways
| Document | Relation | Office | Cited during |
|---|---|---|---|
| US10877654B1 | Cited by | United States of America | Applicant |
| US10515109B2 | Cited by | United States of America | Applicant |
| US10579950B1 | Cited by | United States of America | Applicant |
| US11227344B2 | Cited by | United States of America | Applicant |
| US10817655B2 | Cited by | United States of America | Applicant |
| US9898528B2 | Cited by | United States of America | Applicant |
| US10726507B1 | Cited by | United States of America | Applicant |
| US10346410B2 | Cited by | United States of America | Applicant |
| US11521096B2 | Cited by | United States of America | Applicant |
| US9984428B2 | Cited by | United States of America | Applicant |
| US10885456B2 | Cited by | United States of America | Applicant |
| US10635276B2 | Cited by | United States of America | Applicant |
| US10552998B2 | Cited by | United States of America | Applicant |
| US11250027B2 | Cited by | United States of America | Applicant |
| US9671776B1 | Cited by | United States of America | Applicant |
| US10249033B1 | Cited by | United States of America | Applicant |
| US11308117B2 | Cited by | United States of America | Applicant |
| US10484407B2 | Cited by | United States of America | Applicant |
| US9846731B2 | Cited by | United States of America | Applicant |
| US10114884B1 | Cited by | United States of America | Applicant |
| US10275778B1 | Cited by | United States of America | Applicant |
| US10762102B2 | Cited by | United States of America | Applicant |
| US10636097B2 | Cited by | United States of America | Applicant |
| US10762471B1 | Cited by | United States of America | Applicant |
| US10452678B2 | Cited by | United States of America | Applicant |
| US11182204B2 | Cited by | United States of America | Applicant |
| US9652139B1 | Cited by | United States of America | Applicant |
| US10373099B1 | Cited by | United States of America | Applicant |
| US10795909B1 | Cited by | United States of America | Applicant |
| US10732803B2 | Cited by | United States of America | Applicant |
| US10127289B2 | Cited by | United States of America | Applicant |
| US10198515B1 | Cited by | United States of America | Applicant |
| US12079887B2 | Cited by | United States of America | Applicant |
| US11693877B2 | Cited by | United States of America | Applicant |
| US11928211B2 | Cited by | United States of America | Applicant |
| US10970261B2 | Cited by | United States of America | Applicant |
| US10706434B1 | Cited by | United States of America | Applicant |
| US11281726B2 | Cited by | United States of America | Applicant |
| US10585883B2 | Cited by | United States of America | Applicant |
| US11715167B2 | Cited by | United States of America | Applicant |
| US10866936B1 | Cited by | United States of America | Applicant |
| US10474326B2 | Cited by | United States of America | Applicant |
| US10356032B2 | Cited by | United States of America | Applicant |
| US9734217B2 | Cited by | United States of America | Applicant |
| US10459619B2 | Cited by | United States of America | Applicant |
| US10444940B2 | Cited by | United States of America | Applicant |
| US10606872B1 | Cited by | United States of America | Applicant |
| US10235533B1 | Cited by | United States of America | Applicant |
| US11048706B2 | Cited by | United States of America | Applicant |
| US11074277B1 | Cited by | United States of America | Applicant |
| US10162887B2 | Cited by | United States of America | Applicant |
| US10552002B1 | Cited by | United States of America | Applicant |
| US9996595B2 | Cited by | United States of America | Applicant |
| US10437450B2 | Cited by | United States of America | Applicant |
| US10180977B2 | Cited by | United States of America | Applicant |
| US11138279B1 | Cited by | United States of America | Applicant |
| US10430444B1 | Cited by | United States of America | Applicant |
| US10853454B2 | Cited by | United States of America | Applicant |
| US11269906B2 | Cited by | United States of America | Applicant |
| US12056718B2 | Cited by | United States of America | Applicant |
| US10956406B2 | Cited by | United States of America | Applicant |
| US10721262B2 | Cited by | United States of America | Applicant |
| US9946738B2 | Cited by | United States of America | Applicant |
| US10360238B1 | Cited by | United States of America | Applicant |
| US11080296B2 | Cited by | United States of America | Applicant |
| US11126638B1 | Cited by | United States of America | Applicant |
| US9792020B1 | Cited by | United States of America | Applicant |
| US9785317B2 | Cited by | United States of America | Applicant |
| US11106638B2 | Cited by | United States of America | Applicant |
| US10664490B2 | Cited by | United States of America | Applicant |
| US12079357B2 | Cited by | United States of America | Applicant |
| US11119630B1 | Cited by | United States of America | Applicant |
| US10769171B1 | Cited by | United States of America | Applicant |
| US10552994B2 | Cited by | United States of America | Applicant |
| US10795749B1 | Cited by | United States of America | Applicant |
| US9836694B2 | Cited by | United States of America | Applicant |
| US10977279B2 | Cited by | United States of America | Applicant |
| US10192333B1 | Cited by | United States of America | Applicant |
| US11874850B2 | Cited by | United States of America | Applicant |
| US9817563B1 | Cited by | United States of America | Applicant |
| US11263382B1 | Cited by | United States of America | Applicant |
| US10691662B1 | Cited by | United States of America | Applicant |
| US11829928B2 | Cited by | United States of America | Applicant |
| US11507657B2 | Cited by | United States of America | Applicant |
| US10885021B1 | Cited by | United States of America | Applicant |
| US9891808B2 | Cited by | United States of America | Applicant |
| US11150629B2 | Cited by | United States of America | Applicant |
| US10877984B1 | Cited by | United States of America | Applicant |
| US10444941B2 | Cited by | United States of America | Applicant |
| US10783162B1 | Cited by | United States of America | Applicant |
| US10728262B1 | Cited by | United States of America | Applicant |
| US11892901B2 | Cited by | United States of America | Applicant |
| US10176482B1 | Cited by | United States of America | Applicant |
| US10242072B2 | Cited by | United States of America | Applicant |
| US10140664B2 | Cited by | United States of America | Applicant |
| US9727622B2 | Cited by | United States of America | Applicant |
| US10157200B2 | Cited by | United States of America | Applicant |
| US10120857B2 | Cited by | United States of America | Applicant |
| US9727560B2 | Cited by | United States of America | Applicant |
| US10452651B1 | Cited by | United States of America | Applicant |
22 members in 6 offices
Priority claims12
| Document | Office | Kind | Date |
|---|---|---|---|
| 201361801297 | United States of America | P | |
| 61801297 | United States of America | – | |
| 14099661 | United States of America | – | |
| 201314099661 | United States of America | A | |
| 14140415 | United States of America | – | |
| 201314140415 | United States of America | A | |
| 14099661 | – | – | – |
| 14140415 | – | – | – |
| 61801297 | – | – | – |
| US201314099661 | – | – | – |
| US201314140415 | – | – | – |
| US201361801297P | – | – | – |
Members22
| Document | Office | Kind | |
|---|---|---|---|
| GB201404486D0 | United Kingdom | D0 | |
| GB201404489D0 | United Kingdom | D0 | |
| CA2846330A1 | Canada | A1 | |
| NL2012417A | Netherlands (Kingdom of the) | A | |
| NL2012421A | Netherlands (Kingdom of the) | A | |
| DE102014204830A1 | Germany | A1 | |
| DE102014204834A1This record | Germany | A1 | |
| US2014280155A1 | United States of America | A1 | |
| US2014280252A1 | United States of America | A1 | |
| AU2014201540A1 | Australia | A1 | |
| AU2014201595A1 | Australia | A1 | |
| GB2513720A | United Kingdom | A | |
| GB2513721A | United Kingdom | A | |
| US8924388B2 | United States of America | B2 | |
| US8924389B2 | United States of America | B2 | |
| US2015106379A1 | United States of America | A1 | |
| US9286373B2 | United States of America | B2 | |
| US2016179927A1 | United States of America | A1 | |
| NL2012417B1 | Netherlands (Kingdom of the) | B1 | |
| NL2012421B1 | Netherlands (Kingdom of the) | B1 | |
| US10152531B2 | United States of America | B2 | |
| CA2846330C | Canada | C |
2 legal events, as the office reported them to INPADOC
Over the term
Point at a mark for the eventEvents
| Event | Code | |
|---|---|---|
| Application deemed withdrawn due to failure to request examinationWithdrawnR005 | R005 | |
| Amendment of ipc main classR079 | R079 |
Numbers
- Publication
- 102014204834
- Publication, DOCDB
- 102014204834
- Publication, EPODOC
- DE102014204834
- Application
- 10204834
- Application, DOCDB
- 102014204834
- Application, EPODOC
- DE201410204834
Titles2
- English
- Computer-implemented systems and methods for comparing and associating objects
- German
- Computerimplementierte Systeme und Verfahren zum Vergleichen und Assoziieren von Objekten
Classification
- CPC, 4
- G06F17/30949
- G06F16/9014
- G06F17/30985
- G06F16/90344
- IPC, 1
- G06F17 30