Question

Help me brainstorm how I would solve this problem.

I have a file of dates with corresponding data, the format looks like this:

Date,data,data,data,data,data
Date,data,data,data,data,data

It’s a plain csv file, only commas being used.

I need to be able to select a beginning date. And then get the data for the next 20 days beginning with the date selected.

Date format:
2007.05.21 (y,m,d)

So I think it would be best to search for the date. Either loading the entire file first into memory or read line by line. The file is only 1 megabyte, however I might want to do this with a 100 megabyte file as well. Is that still little?

Also I will want to do this very many times. I think I may want to keep the file in memory for the entire run of the program. So I can repeatedly access it.

After finding the date. I need to be able to get column 2 day 1, column 4 day 4. Ect. However there is always the same amount of columns for each day. So I guess if this is loaded into some kind of array I can always know in what array number the next and next day starts.

Any help would be greatly appreciated. Also any code examples provided would really help. This is not a homework problem or anything like that and I’m really new to programming.

Answer 1

You can user csv library to parse your file like this line by line

require 'csv'
date_to_search = Date(2009, 10, 10)
CSV.read('yourfilename.txt', :col_sep => ',') do |row|
  # row will be an array of strings which you can parse
  cur_date = Date.parse(row[0])
  if cur_date == date_to_search
    # you are set to read next 19 lines
    # you can keep a counter and increment it after parsing each line (row here)
  end
  # compare and check if you need this line (and next 19)
  # other calculations
end