Question

This is an error that I’m getting when I run a script.

repo nikhil$ ruby repobuilder.rb
Resuming from package:20
Now Processing 21
repobuilder.rb:16:in `gets': closed stream (IOError)
    from repobuilder.rb:16
    from repobuilder.rb:15:in `open'
    from repobuilder.rb:15

My platform is

repo nikhil$ ruby -v
ruby 1.8.7 (2009-06-12 patchlevel 174) [universal-darwin10.0]
repo nikhil$ uname -a
Darwin Radha Krishna 10.8.0 Darwin Kernel Version 10.8.0: Tue Jun  7 16:33:36 PDT 2011; root:xnu-1504.15.3~1/RELEASE_I386 i386 

and the script that causes this is

require 'rubygems'
require 'nokogiri'
require 'open-uri'


# additional code to make sure that we can resume after a break seamlessly
last_package = 0
File.open('lastbreak','r') { |fptr| last_package = fptr.gets.to_i }
puts "Resuming from package:#{last_package}" if last_package != 0

# to read each package from packageslist.txt and fetch the required info
# also to store this into a file that can easily be read by the c++ program
BASE_URL = "http://packages.debian.org/stable/"

File.open('packages_list.txt','r') do | fptr |
  while line = fptr.gets
    package_id = line.split[0].to_i
    package = line.split[1]
    dependencies = ""
    url = BASE_URL + package
    if package_id >= last_package
      doc = doc = Nokogiri::HTML(open(url))
      doc.css(".uldep a").each do |dependency|
        dependencies << dependency.text + ","
      end
      dependencies = dependencies.split(',').uniq.join(',')
      description = doc.css('#pdesc').text.strip
      if doc.css('#content h1').text 
         version = doc.css('#content h1').text.strip.scan( /\(([^>]*)\)/).last.first
      end

      File.open("packages/#{package}","w") do |wfptr|
      wfptr.puts "PackageId:#{package_id}"
      wfptr.puts "Name:#{package}"
     # wfptr.puts "Version:#{version}"
      wfptr.puts "Deps:#{dependencies}"
      end
      File.open("packages/#{package}.description",'w') {|wf| wf.write(description.capitalize)}

      package_id += 1
      puts "Now Processing #{package_id}"
      File.open('lastbreak','w') { |fptr| fptr.puts "#{package_id}" }
    end
  end

end

Answer 1

I am on a different system so I cannot check your code. Anyway I suggest using File.foreach:

File.foreach("packages_list.txt") do |line|
  package_id = line.split[0].to_i
  package = line.split[1]
  dependencies = ""
  url = BASE_URL + package
  if package_id >= last_package
    doc = Nokogiri::HTML(open(url))
    doc.css(".uldep a").each do |dependency|
      dependencies << dependency.text + ","
    end
    dependencies = dependencies.split(',').uniq.join(',')
    description = doc.css('#pdesc').text.strip
    if doc.css('#content h1').text 
       version = doc.css('#content h1').text.strip.scan( /\(([^>]*)\)/).last.first
    end

    File.open("packages/#{package}","w") do |wfptr|
      wfptr.puts "PackageId:#{package_id}"
      wfptr.puts "Name:#{package}"
      wfptr.puts "Deps:#{dependencies}"
    end
    File.open("packages/#{package}.description",'w') {|wf| wf.write(description.capitalize)}

    package_id += 1
    puts "Now Processing #{package_id}"
    File.open('lastbreak','w') { |fptr| fptr.puts "#{package_id}" }
  end
end