Question

I used Ruby to read an image file and save that into a string.

partial_image100 = File.read("image.tga")
partial_image99 = File.read("image.tga")
partial_image98 = File.read("image.tga")
...

I read those images at one end of a distributed system. In another system I want to do a Tail operation. The system receives just the images.

I have around a 100 partial images. I want to do a Tail operation, like this:

tail -c +19 image100 >> image99
tail -c +19 image99 >> image98
tail -c +19 image97 >> image96
...

Basically it just removes the first 18 bytes of the partial image and append what is left to the next image.

The problem is that this is slow. Calling 100 unix commands from Ruby is slow. I want to refactor this so that this happen in Ruby world. Just in memory. No files.

How can I do this in Ruby?

Thanks

edit:

The images are stored in a hash like this:

{"27"=>"\u0000\u0000\u0002\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u000E\u0001\xD0\a\xD0\a\u0018 \xFF\xFF\u0000\xFF\xFF\u0000\xFF\xFF\u0000\xFF\xFF\u0000\xFF\xFF\u0000\xFF\xFF\u0000\xFF\xFF\u0000\xFF\xFF\u0000\xFF\xFF\u0000\xFF\xFF\u0000\xFF\xFF\u0000\xFF\xFF\u0000\xFF\xFF\u0000\xFF\xFF\u0000\xFF\xFF\u0000\xFF\xFF\u0000\xFF\xFF\u0000\xFF\xFF\u0000\xFF\xFF\u0000\xFF\xFF\u0000\xFF\xFF\u0000\xFF\xFF\u0000\xFF\xFF\u0000\xFF\xFF\u0000\xFF\xFF\u0000\xFF\xFF\u0000\xFF\xFF\u0000\xFF\xFF\u0000\xFF\xFF\u0000\xFF\xFF\u0000\xFF\xFF\u0000\xFF\xFF\u0000\xFF\xFF\u0000\xFF\xFF\u0000\xFF\xFF\u0000\xFF\xFF\u0000\xFF\xFF\u0000\xFF\xFF\u0000\xFF\xFF\u0000\xFF\xFF\u0000\xFF\xFF\u0000\xFF\xFF...

EDIT:

You have all the relevant code here: https://gist.github.com/989563

There are two files. The code and a hash object encoded in json in a file. When you run the code there will be two image files created at /tmp

• /tmp/image-tail-merger.tga – The output from the tail-merge algorithm
• /tmp/image-/time/.tga – the output from the in-memory-tail algorithm

Currently the in-memory algorithm fails because the generated image is a Picasso.

If you manage to make the in-memory-algorithm generate the same image that the tail-merge algorithm do then you have succeeded.

EDIT:

I got it right finally!!!

Here is the code
https://gist.github.com/989563

Answer 1

You might try something like this

image100 = "some image string"
image99 = "some other image string"

image99 += image100.slice(0,19)

EDIT: In your specific example you could do this to iterate through the entire image

(image_hash.size..1).each do i
  # Here we use slice to select everything *except* the first 19 bytes
  # Note: To select just the first 19 bytes we could do slice(0,19)
  #       To select just the last 19 bytes we could do slice(-19,19)

  # We then append this result to the next image down the line
  image_hash[i-1] += image_hash[i].slice(19,image_hash[i].size-19)
end

If you want to remove the “tailed” bits permanently you can use slice! to do an inline replace.

Maybe a bit cleaner:

# Strip the headers
image_hash.each { |k,v| v.slice!(0,19) }

# Append them together
(image_hash.keys.sort).collect{ |i| image_hash[i] }.join

EDIT: Working code example https://gist.github.com/989563