Question

A game that I play stores all of its data in a .DAT file. There has been some work done by people in examining the file. There are also some existing tools, but I’m not sure about their current state. I think it would be fun to poke around in the data myself, but I’ve never tried to examine a file, much less anything like this before.

Is there anything I should know about examining a file format for data extraction purposes before I dive headfirst into this?

EDIT: I would like very general tips, as examining file formats seems interesting. I would like to be able to take File X and learn how to approach the problem of learning about it.

Answer 1

1. You’ll definitely want a hex editor before you get too far. It will let you see the raw data as numbers instead of as large empty blocks in whatever font notepad is using (or whatever text editor).
2. Try opening it in any archive extractors you have (i.e. zip, 7z, rar, gz, tar etc.) to see if it’s just a renamed file format (.PK3 is something like that).
3. Look for headers of known file formats somewhere within the file, which will help you discover where certain parts of the data are stored (i.e. do a search for ‘IPNG’ to find any (uncompressed) png files somewhere within).
4. If you do find where a certain piece of data is stored, take a note of its location and length, and see if you can find numbers equal to either of those values near the beginning of the file, which usually act as pointers to the actual data.
5. Some times you just have to guess, or intuit what a certain value means, and if you’re wrong, well, keep moving. There’s not much you can do about it.
6. I have found that http://www.wotsit.org is particularly useful for known file type formats, for help finding headers within the .dat file.