Question

I’m aggregating concert listings from several different sources, none of which are both complete and accurate. Some of the data comes from users (such as on last.fm), and may be incorrect. Other data sources are highly accurate, but may not contain every event. I can use attributes such as the event date, and the city/state to try to match listings from disparate sources. I’d like to be reasonably certain that the events are valid. It seems like it would be a good strategy to consume as many different sources as possible to validate listings on error-prone sources.

I’m not sure what the technical term for this is, as I’d like to research it further. Is it data mining? Are there any existing algorithms? I understand a solution will never be completely accurate.

Answer 1

I believe the term you are looking for is Record Linkage –

the process of bringing together two or more records relating to the same entity(e.g., person, family, event, community, business, hospital, or geographical area)

This presentation (PDF) looks like a nice introduction to the field. One algorithm you might use is Fellegi-Holt – a statistical method for editing records.