Question

I’m a little confused on how to connect BI with data mining. Can BI be termed as some kind of a manifestation of data mining?

How different is a BI tool like Microsoft Analysis Services from a data mining tool like Weka?

I guess BI involves more of reporting and analysis of data, where in the data undergoes some kind of aggregation and is represented in the form of cubes, but data mining also involves different algorithms to perform clustering, no?

Any pointers?

cheers

Answer 1

BI small is generating a detail report (list of today’s sales). Very little math involved, maybe counting rows and summing sales. This is where you see reporting tools called “BI”

BI medium is generating a metric (profit margin for the quarter). It’s just simple algebra, but producing it on a frequent basis is a challenge on account of the sheer amount of data. This is the world of cubes and olap.

BI large is doing mathematical modeling. This may be anything from linear regression to statistics models, you name it. The key here is the models are using large quantities of data. Real statisticians use the phrase “data mining” in a derogatory sense because people untrained in the use of statistics are likely to mine the data until they find a spurious correlation. The bigger your data set the more likely you are to find relationships due to chance instead of there really being such a relationship in reality.

Because the customer for BI are line of business managers, not PhD grad students, vendors like Microsoft et al. have dumbed it down by providing us with black box “Data Mining” tools, many are the same as what you’d find in SAS and the like.

The only thing I see connecting all of these applications of the phrase BI is that they all are using large quantities of data to make a business decision.