Question

I was reading others’ code and saw so many people using something like BUFFER_SIZE as a macro. The thing is, many programs could be written without this buffer thing. So when do we need buffer and when not? I mean, why we need a buffer? And how to use it properly?

Answer 1

From wikipedia:

a buffer is a region of a physical memory storage used to temporarily hold data while it is being moved from one place to another.

With that being said, I feel there are a few concrete uses of buffers:

• Transforming an asynchronous data source into a synchronous data source:
  This is a big one, and a lot of APIs are built with this mindset. For example, imagine you’re reading a data source which is inherently prone to failure. Asynchronously you have a stream which can fail at certain times, but you can request the data be read again. This handling of data is a very low level detail, and you wouldn’t want programmers at a high level to have to worry about it. The solution, write a low level handler which manages the stream and put data in a buffer once it’s safely been read in. For example, you see this use of a buffer in file systems, network protocols, etc…

• Passing large amounts of data around: If you want to share data between multiple people, you need a temporary place to store data to mediate it between people.

• Copying things / doing destructive manipulations: If you have a situation where you need to free one pointer and move something around in memory (for whatever reason), you can put the data in a temporary holding place. One common case is where I’m doing something like destructively manipulating a string: I can’t manipulate the original string, I need to make a copy of it, so I don’t corrupt the pointer if other people are holding on to it.