Question

In Silverlight, the VideoSink.OnSample method is used to get raw webcam data. The function signature is as follows:

protected abstract void OnSample(
    long sampleTimeInHundredNanoseconds,
    long frameDurationInHundredNanoseconds,
    byte[] sampleData
)

Documentation for sampleData says

A byte stream that contains video data. The byte stream should be processed into samples according to the relevant video format information.

Going to the documentation of the VideoFormat class, one gets to the VideoFormat.PixelFormat property which can only have two possible values according to the documentation:

• Unknown – The format is unknown.
• Format32bppArgb – The format uses 32 bits of color information per pixel and reports color information by using alpha, red, blue, and green channels.

I want to understand how the sampleData parameter for the VideoSink.OnSample method looks like. Is it simply an array of pixels like the following?

sampleData[0] = image[x = 0, y = 0].R;
sampleData[1] = image[x = 0, y = 0].G;
sampleData[2] = image[x = 0, y = 0].B;
sampleData[3] = image[x = 0, y = 0].A;
sampleData[4 to 7] = image[x = 0, y = 1];
sampleData[8 to 11] = image[x = 0, y = 2];
sampleData[4*Height-4 to 4*Height-1] = image[x = 0, y = Height];
sampleData[4*Height to 4*Height+3] = image[x = 1, y = 0];
// ...

If it is that easy: Is the ordering correct?

• R before G before B before A for each pixel
• All pixels of first column before all pixels of second column and so on

Answer 1

The byte stream looks as follows for the Format32bppArgb pixel format:

sampleData[0- 3] = image[x=0, y=0]
sampleData[4- 7] = image[x=1, y=0]
sampleData[8-11] = image[x=2, y=0]
      ...                ...
                   image[x=w, y=0]
                   image[x=0, y=1]
                         ...
                   image[x=w, y=h]

So, it is stored row by row.

For each pixel which is stored in bytes i to i+3 the information comes as follows:

sampleData[i+0] = BLUE
sampleData[i+1] = GREEN
sampleData[i+2] = RED
sampleData[i+3] = ALPHA

All channels are stored as an integer in the range [0; 255] where 255 indicates the maximum intensity for the color channels and opaque for the alpha channel. So, besides the ordering it works like expected.