Question

We’ve got a data warehouse design with four dimension tables and one fact table:

• dimUser id, email, firstName, lastName
• dimAddress id, city
• dimLanguage id, language
• dimDate id, startDate, endDate
• factStatistic id, dimUserId, dimAddressId, dimLanguageId, dimDate, loginCount, pageCalledCount

Our problem is: We want to build the fact table which includes calculating the statistics (depending on userId, date range) and filling the foreign keys.

But we don’t know how, because we don’t understand how to use natural keys (which seems to be the solution to our problem according to the literature we read).

I believe a natural key would be the userId, which is needed in all ETL jobs which calculate the dimension data.

But there are many difficulties:

• in the ETL jobs load(), we do bulk inserts with INSERT IGNORE INTO to remove duplicates => we don’t know the surrogate keys which were generated
• if we create meta data (including a set of dimension_name, surrogate_key, natural_key) this will not work because of the duplicate elimination

The problem seems to be the duplicate elimination strategy. Is there a better approach?

We are using MySQL 5.1, if it makes any difference.

Answer 1

If your fact table is tracking logins and page calls per user, then you should have set of source tables which track these things, which is where you’ll load your fact table data from. I would probably build the fact table at the grain of one row per user / login date – or even lower to persist atomic data if at all possible.

Here you would then have a fact table with two dimensions – User and Date. You can persist address and language as dimensions on the fact as well, but these are really just attributes of user.

Your dimensions should have surrogate keys, but also should have the source “business” or “natural” key available – either as an attribute on the dimension itself, or through a mapping table as your colleague suggested. It’s not “wrong” to use a mapping table – it does make things easier when there are multiple sources.

If you store the business keys on a mapping table, or in the dimension as an attribue, then for each row to load in the fact, it’s a simple lookup (usually via a join) against the dim or mapping table to get the surrogate key for the user (and then from the user to get the user’s “current” address / language to persist on the fact). The date dimension usually hase a surrogate key stored in a YYYYMMDD or other “natural” format – you can just generate this from the date information on your source record that you’re loading into the fact.