Question

I made a very big script to feel my initial datas into my rails app. I have about 3000 lines in my CSV and 10000 images.

After maybe 300 upload i got this message :

/usr/local/lib/ruby/gems/1.9.1/gems/activesupport-3.0.9/lib/active_support/core_ext/kernel/agnostics.rb:7:in ``': Cannot allocate memory - identify -format %wx%h '/tmp/stream20111104-14788-1hsumv7.jpg[0]' (Errno::ENOMEM)

My upload script :

if (row[28] != nil)
   hotelalbum = HotelAlbumPhoto.find_or_create_by_title(h.title)
   hotelalbum.description = "Album photo de l'hotel " + h.title.capitalize
   hotelalbum.hotel_id = h.id
   hotelalbum.save

   files =  Dir.glob('IMAGES/' + row[28].gsub(/\\/,'/') + '/*.jpg')
   i =0
   for file in files
      i += 1
      photopath = File.expand_path('../../import', __FILE__) + '/' + file
      photoname = file.split('/').last
      if (i==1)
        hotelalbum.thumbnail = open(photopath)
        hotelalbum.save
      end
      if (i==1)
        h.thumbnail = open(photopath)
      end
      photo = HotelImage.find_or_create_by_image_file_name_and_hotel_album_photo_id(photoname,hotelalbum.id)
      if (photo.image_file_size == nil || photo.image_file_name != photoname)
          photo.image = open(photopath)
          photo.activated = true
          photo.alt = "Photo de l'hotel " + h.title
          photo.save
      else
         puts photopath + ' already updated'
      end
   end
end

When i check my memory with top command, i see ruby process use more memory on each upload. How can i manage this?

Thank you for help

ps : My server is a virtual machine with 512Mb memory, one solution is to inscrease this memory but i hope to find an other.

Answer 1

I don’t know where the open function is defined, but I’m suspicious that I don’t see a corresponding close…

update Better idea, change photo.image = open(photopath) to photo.image = File.read(photopath)

According to the docs, read:

Opens the file, optionally seeks to the given offset, then 
returns length bytes (defaulting to the rest of the file). 
read ensures the file is closed before returning.