Question

Some background:

I working on some java packages that need to take data, divide it and distribute to many Servers online. According to the user code (person using my packages), I will know which kind of buffer to use. I have seen The Netty Project that has some interesting concepts in mind and other solutions suggested here in the site, but not exactly what I’m looking for. I guess I can implement my requirements, but I like reusing better.

The actual Issue: Byte Buffer Requirements

Separate implementation from interface.
Change between the two implementation.

Fixed Size:

1. Direct memory usage.
2. Zero Copy IO on distribution.

Growing Size

1. Can grow to up to a certain Size.
2. Keeps balance between reallocation and amount of memory in use.

Update 1: Decided to go with the Netty project library due to a lot of other benefits.

I wanted to give some details about my system. I need to distribute large amounts of data throughout a network of servers. I have very low amount of knowledge regarding the size of my buffers. Sometimes it can be fixed sizes, sometimes it can be in unknown size mode. I want to be able to develop a mechanism that enjoys the benefit of both worlds. I do have a defined event when I receive the final buffer for inspection.

I understand that buffers can perform better when they are directly mapped to memory. Netty provides this for me, but I can’t make this buffer grow. I have dynamic buffer which can grow -Netty provides that too.

Couple of questions:

1. should I copy the buffer to from dynamic to direct in case it is changed? Is there a way to switch modes?

2. What do you think of implementing a class that holds an array of ChannleBuffers and exposes the same interface like ChannleBuffer while containing “inside” an array of buffers allocating new direct memory as needed.

3. Do you have other solutions in mind ?

Answer 1

I found out that Netty supplied me with all the things I need and more.