I have a huge database that is meant to be indexed, into which I am inserting a huge amount of information into it, broken up into a thousand chunks. Each time one is inserted the index is updated. Problem is, the time it takes to insert the information is becomes insanely long after around the 100th one or so, due to these huge transactions that cause the journal file to climb to over 300 megs.
Is there a way to postpone updating indices (possibly with a FREEZE INDEX command or something) whereby the index would stay defined in the schema, and after all the tables have been populated, the it may be unfrozen and updated.
I know that this can already be accomplished by DROP INDEX and CREATE INDEX, but I was looking for a somewhat more elegant solution, or possibly a trick.
Unfortunately, I cannot change DBMSes easily, or I would investigate using PostgreSQL.
Apart from it not being “elegant” is there any other reason why you do not want to use DROP INDEX/CREATE INDEX. I assume you are doing this bulk load during a scheduled maintenance? DROP INDEX/CREATE INDEX is pretty much standard procedure in bulk data loads. By recreating the non-clustered indexes at the end you ensure that the DB can create the most efficient index structure (thus making queries against the data run faster).
Some databases have additional features to hide these details from you (SQL server allows you to “disable/enable” an index) but behind the scenes they are still effectively executing DROP INDEX/CREATE INDEX.
Again, you are not saving anything even if there were a “FREEZE INDEX”. If you are entering a lot of data it would be best to completely rebuild the index at the end anyway (which effectively acts like a “DROP INDEX/CREATE INDEX”).
If your journal is getting too big perhaps you should split the insert into even smaller transactional chunks?