Question

So I’m designing a 2D game in C++ with tons of data– basically, it features randomly generated, blocky terrain (in the form of a very large byte array), much in the style of Terraria, and will also need to save state for monsters (among other various game objects). As the player runs around, sections of the game world will be loaded/saved to the disk, making for an infinitely expanding game world (similar to Minecraft).

What’s my best option for saving everything to the disk? I’m mostly concerned with efficiency (so the user never has to wait on content to load), but ease of use and maintainability are a close second. I’m very familiar with Minecraft’s method of saving, which has a separate file for each “Chunk”, but I was wondering– should I designate my own files like Minecraft, or go with a database? What other possibilities are out there? If I use a file, would I get a significant performance boost by asking the OS for raw access and handling that myself (much like a database does)?

Also, I’d like to be able to port the project from PC to Mac and console– alternatives for individual platforms are fine, I’ll just have to swap methods depending on the platform.

Thanks!

Answer 1

A single file would be much cleaner and could take less physical disk space than relying on the filesystem for organization. Just maintain a table of offsets at the beginning of the file to link to each of the chunks.

As far as compression, applying a run length encoding would definitely deflate the file size, and if it is like Minecraft, you’ll have a lot of identical blocks in sequence. As a bonus, it will load faster on computers using platter drives, as a couple of bytes can equate to thousands of blocks, instead of having to read a byte for each of those blocks. Here’s a quick tip for an effective run length encoding: use a command byte, say 0x00. All other bytes are associated with a block, but the command byte would signify a non-block bit of info. The traditional run length encoding looks like this:

Source: AAAABBBBBBBCDDDEFA
Destination: 0x0441074201430344014501460161
Plaintext: .A.B.C.D.E.F.A

Using a command byte, omitting sequences that would increase size:

Source: AAAABBBBBBBCDDDEFA
Destination: 0x00044100074243444444454641.
Plaintext: ..A..BCDDDEFA

This has a distinct advantage in “marbled” sections of data.

One other thing, I recommend that you use a system for storing numbers similar to the midi format, although in little endian format. To save you the googling, all it really means is that you sacrifice a bit for each byte you use to store the number. If the 8th bit is set, it means there’s another byte in the number, and it cascades. Example:

Storing: 65
Stored as: 0x41 (01000001) //The eighth bit is not set, so this number only occupies 7 bits.

Storing: 192
Stored as: 0xC081 (11000000 10000001) //The eighth bit of the first byte is set, so the following byte is part of the number. The second byte doesn't have the eighth bit set, so the number ends there.

Storing: 612453
Stored as: 0x72D825 (11100101 1011000 00100101) //The eighth bit of the first two bytes are set, and the third is not, so this number occupies 3 bytes.

I hope that I helped provide some insight.