Can someone speak to ‘common practice’ or general “acceptable practice” on handling data that is highly transient for short periods of time – yet has to have historical presence for years…
Take for instance a small tour company – booking tours.
The passenger manifest would be highly fluid, up until the tour..
once the tour completed – the data is technically obsolete, but useful for reporting or trending…
With HIGH volume – the ‘search’ or ‘creation’ of a tour – the db would become THICK with data that is only going to change rarely.
Is it common to have tables with similar structure – that “move” data from one state to another (PRE/POST event)… as say an interim before data warehousing it to a semi-flat structure for ‘pure’ reporting
Is that advisable, common or correct? Is there a better way to do that… or is dome DBA gonna come in and go “WTF were you thinking”
Standard would be to have some sort of flag to indicate record obsolescence. There are several methods available that would allow you to deal well with performance issue such as pruning and indexing techniques. These would probably also include some kind of archiving strategy. You would likely move old data from the table (how often and when would depend on performance requirements) either with some sort of bulk insert into a history table with the same structure (how you actually implement this would depend on your DBMS but choose the most robust method) or better with some sort or partitioning strategy if your DBMS has a robust partitioning system. You also might think about some sort of multi-database archiving strategy if the analyst who need the historic data are not the same as the people who run the operational systems.