System and method for processing messages stored in multiple message stores

System for efficiently processing messages stored in memories (32) of multiple messages, comprising: a processor (20) of messages iteratively copying metadata identifying a range of topically identical messages extracted from a plurality of message memories that store an multiplicity of messages to be processed and categorizing the metadata for the extracted range of topically identical messages, further comprising message processing: a selector (61) of duplicate messages that identify those messages containing content substantially duplicated within the range extracted as duplicate messages; a selector (62) series of lengths that matches those non-duplicate messages within the extracted range into an ordering of conversation thread length; a selector (63) almost duplicate messages classifying those messages whose content is recursively-included content within another of the tallied non-duplicate messages as near-duplicate messages; and a selector (64) only messages designated as unique messages remaining messages containing substantially unduplicated content.