Question

I’m currently working with a lot of different file types (txt, binary, office, etc). I typically use a byte[] or string to hold the file data in memory (while it is being written/parsed) and in order to read/write it into files I write the entire data using a FileStream after the data has been completely processed.

• Should I be using a TextStream instead of a string while generating data for a text file?
• Should I be using a FileStream instead of a byte[] while generating data for a binary file?
• Would using streams give me better performance instead of calculating the entire data and outputting it in one go at the end?
• Is it a general rule that File I/O should always use streams or is my approach fine in some cases?

Answer 1

The advantage of a byte[]/string vs a stream may be that the byte[]/string is in memory, and accessing it may be faster. If the file is very large, however, you may end up paging thus reducing performance. Another advantage of the byte[]/string approach is that the parsing may be a little easier (simply use File.ReadAllText, say).

If your parsing allows (particularly if you don’t need to seek randomly), using a FileStream can be more efficient especially if the file is rather large. Also, you can make use of C#’s (4.5) async/await features to very easily read/write the file asynchronously and process chunks that you read in.

Personally, I’d probably just read the file into memory if I’m not too worried about performance, or the file is very small. Otherwise I’d consider using streams.

Ultimately I would say write some simple test programs and time the performance of each if you’re worried about the performance differences, that should give you your best answer.