Question

i have a binary file which has 4 KB of header information and then 28 bytes of data and then 24 byte which i want to read. How can i loop every 24 and 28 bytes and read(or extract) every first 8 bytes of data of those 28 and 24 bytes.. In python i did something like this. Not sure how to do for variable length

import sys
import struct
f = open(sys.argv[1],"rb")
f.seek(4096)
byte = f.read(28)
while byte != "":   
    ticks = struct.unpack("<ll",byte[:8]) #not sure how to read 8 bytes 
    byte = f.read(28)
f.close()

Here is what it looks like after the header.

Length
(bytes) Field Name
8   TS_INCR
4   SEQID
2   OP
2   LUN
4   NBLKS
8   LBA


Length
(bytes) Field Name
8   TS_INCR
4   SEQID
2   OP
2   LUN
4   LATENCY_TICKS
2   HOST_ID
2   HOST_LUN

If you guys can help with this please. Python or PERL does not matter. Thanks!!!!

Answer 1

The Endianness of the data you are reading matters here. You seem to be unpacking the 8 octets as two longs stored in little endian order. Are you sure it is not a single 64-bit quantity (which would make the q or Q formats more appropriate)? Unfortunately, I am on 32-bit machine so my perl does not support Q.

However, the following should point you in the right direction:

#!/usr/bin/env perl

use strict; use warnings;
use autodie;

use Fcntl qw(:seek);
use List::Util qw( sum );

my ($input_file) = @ARGV;
die "Need input file\n" unless defined $input_file;

my $HEADER_SIZE = 4_096;

my @typedef = (
    {
        fields => [
            qw(
                TS_INCR_LO
                TS_INCR_HI
                SEQID
                OP
                LUN
                NBLKS
                LBA_LO
                LBA_HI
            )
        ],
        tmpl => 'LLLSSLLL',
        start => 0,
        size => 28,
    },
    {
        fields => [
            qw(
                TS_INCR_LO
                TS_INCR_HI
                SEQID
                OP
                LUN
                LATENCY_TICKS
                HOST_ID
                HOST_LUN
            )
        ],
        tmpl => 'LLLSSLSS',
        start => 28,
        size => 24,
    },
);

open my $input, '<:raw', $input_file;

seek $input, $HEADER_SIZE, SEEK_SET;

my $BLOCK_SIZE = sum map $_->{size}, @typedef;
read $input, my($buffer), $BLOCK_SIZE;

my @structs;

for my $t ( @typedef ) {
    my %struct;
    @struct{ @{ $t->{fields}} } = unpack(
        $t->{tmpl},
        substr($buffer, $t->{start}, $t->{size})
    );
    push @structs, \%struct;
}

use Data::Dumper;
print Dumper \@structs;