Question

I’m trying to figure out a way to store metadata about a column without repeating myself.

I’m currently working on a generic dimension loading SSIS package that will handle all my dimensions. It currently does :

1. Create a temporary table identical to the given table name in parameters (this is a generic stored procedure that receive the table name as parameter, and then do : select top 0 * into ##[INSERT ORIGINAL TABLE NAME HERE] from [INSERT ORIGINAL TABLE NAME HERE]).
2. ==> Here we insert custom code for this particular dimension that will first query the data from a datasource and get my delta, then transform the data and finally loads it into my temporary table.
3. Merge the temporary table into my original table with a T-SQL MERGE, taking care of type1 and type2 fields accordingly.

My problem right now is that I have to maintain a table with all the fields in it to store a metadata to tell my scripts if this particular field is type1 or type2… this is nonsense, I can get the same data (minus type1/type2) from sys.columns/sys.types.

I was ultimately thinking about renaming my fields to include their type in it, such as :
FirstName_T2, LastName_T2, Sex_T1 (well, I know this can be type2, let’s not fall into that debate here).

What do you guyz would do with that? My solution (using a table with that metadata) is currently in place and working, but it’s obvious that repeating myself from the systables to a custom table is nonsense, just for a simple type1/type2 info.

UPDATE: I also thought about creating user defined types like varchar => t1_varchar, t2_varchar, etc. This sounds like something a bit sluggy too…

Answer 1

I know this is bad, but I will post an answer to my own question… Thanks to GBN for the help tho!

I am now storing “flags” in the “description” field of my columns. I, for example, can store a flag this way : “TYPE_2_DATA”.

Then, I use this query to get the flag back for each and every column :

select columns.name as [column_name]
      ,types.name as [type_name]
      ,extended_properties.value as [column_flags]
  from sys.columns 
 inner join sys.types
         on columns.system_type_id = types.system_type_id
  left join sys.extended_properties
         on extended_properties.major_id = columns.object_id 
        and extended_properties.minor_id = columns.column_id  
        and extended_properties.name = 'MS_Description'
 where object_id = ( select id from sys.sysobjects where name = 'DimDivision' )
   and is_identity = 0
 order by column_id

Now I can store metadata about columns without having to create a separate table. I use what’s already in place and I don’t repeat myself. I’m not sure this is the best possible solution yet, but it works and is far better than duplicating information.

In the future, I will be able to use this field to store more metadata, where as : “TYPE_2_DATA|ANOTHER_FLAG|ETC|OH BOY!”.

UPDATE :

I now store the information in separate extended properties. You can manage extended properties using sp_addextendedproperty and sp_updateextendedproperty stored procedures. I have created a simple store procedure that help me to update those values regardless if they currently exist or not :

create procedure [dbo].[UpdateSCDType]
    @tablename nvarchar(50),
    @fieldname nvarchar(50),
    @scdtype  char(1),
    @dbschema nvarchar(25) = 'dbo'
as
begin

    declare @already_exists int;

    if ( @scdtype = '1' or @scdtype = '2' )
    begin

         select @already_exists = count(1)
          from sys.columns 
         inner join sys.extended_properties 
                 on extended_properties.major_id = columns.object_id 
                and extended_properties.minor_id = columns.column_id 
                and extended_properties.name     = 'ScdType' 
         where object_id = (select sysobjects.id from sys.sysobjects where sysobjects.name = @tablename) 
           and columns.name = @fieldname

        if ( @already_exists = 0 )
        begin

            exec sys.sp_addextendedproperty 
                 @name       = N'Scd_Type', 
                 @value      = @scdtype, 
                 @level0type = N'SCHEMA',
                 @level0name = @dbschema,
                 @level1type = N'TABLE',
                 @level1name = @tablename,
                 @level2type = N'COLUMN',
                 @level2name = @fieldname

        end
        else
        begin

            exec sys.sp_updateextendedproperty 
                 @name       = N'Scd_Type', 
                 @value      = @scdtype, 
                 @level0type = N'SCHEMA',
                 @level0name = @dbschema,
                 @level1type = N'TABLE',
                 @level1name = @tablename,
                 @level2type = N'COLUMN',
                 @level2name = @fieldname

        end

    end

end

Thanks again