A) Is it possible to have Grandfather-Father-Son archiving? For example we would like to have the following precalculated at all times, and nothing else precalculated:
- Daily totals of last week
- Weekly totals of previous 5-6 weeks
- Monthly totals of all previous months
Note that we don’t want daily totals of a day that was 2 months ago for example. We want that daily total to be deleted.
Will indexed views be good enough for this purpose? We want all the fields to be precalculated and stored.
B) We would like to have some features like StackExchanges (and generally Wikis) versioning. Are there ways to archive the older versions somehow on the production environment, and keep the newer versions of stuff more readily available? We have been looking into Partitioning but it doesn’t seem to handle such intricate scenario (we don’t want ALL posts prior to X date to be partitioned, rather than we need all versions that are older than the newest version)
What are the best practices on these matter?
A: You are describing the storage of time-series data with a varying retention period/graularity, features SQL Server does not provide in any form natively. You can of course build this strategy yourself fairly easily, and luckily you have a few great examples in open source projects for guidance:
B: Its hard to speak in such general terms, but perhaps you can maintain an Version column on your table, where 0 always reflects the latest version. Each time you promote a new version you demote all the other versions by incrementing this value. Having versions increase as they get older allows you to create a filtered index on a deterministic value (Version = 0) making it very performant to get the current version.
You can then purge based on number of versions ago rather than date. Just
delete yourTable where Version > 5. A partitioning scheme could work using this method as well.