Question

First, Let me apologize as I know NOTHING about ruby. I can read through the code and understand whats going on, but have never written any code.

I have a script we use to parse some log files that we obtained from someone a long time ago.

# format the csv data into an sql insert query
def FormatToSQL(file_name)
  $logger.info(caller) { "creating SQL insert q's" }
  formatted_data = []
  data_lines = []
  open("#{LOCAL_DIR}/#{file_name}") { |f| data_lines = f.readlines }
  data_lines.each do |r|
    data = []
    rdata = r.split(' ')
    rdata.each { |e| data.push("'#{e}'") }
    data.unshift('DEFAULT')
    sql_data = data.join(',')
    formatted_data.push(sql_data)
  end
  return(formatted_data)
end
# -------------------------------------------------------------------
# -------------------------------------------------------------------
# M  A   I    N

$logger = Logger.new("//var/www/metaquery/calllogger.log", 3, 1024000)
$logger.info(caller) { "start" }
now = Time.now().gmtime() - 3600  ## files are name using UTC
file_name = sprintf("calllog_%s_%02d_%02d_%02d.log", now.year, now.month.to_i,     now.day.to_i, now.hour.to_i)
if(doSFTPPull(file_name)) then
  ConnectDB()
  formatted_sql = FormatToSQL(file_name)
  formatted_sql.each { |sql| $local_conn.query("insert into #{DB_TABLE} values(#{sql})") }
else
  # we did not download the new file, report to ???
  false
end
$logger.info(caller) { "normal end" }
# -------------------------------------------------------------------

The log contains thousands of rows of records, like so

2xx3xx2xx7 2xx3xx56xx 07/28/11.19:55:45 19:55:46 20:00:00 2 4092 - - N - - TER - A T -

However, we enabled some new QoS Stats, and it adds lines like this,

VQM: 2xx3xx00xx 08/12/11.13:02:07 - - - - 20ms 0 0

I want to add a statement to ignore anylines beginning with VQM, as this throws the column count off on the MySQL Inserts, ultimately causing the query to fail, thus causing the script to fail.

How would I acheive this? Again, sorry for being a complete Noob with Ruby. Im only a php guy, and not even good at that. lol

I do know, the area probably has to be in
data_lines.each do |r|

where the code would be, and likely a if/then with some regex. thanks for your help!

Answer 1

Try something like:

data_lines.each do |r|
  next if r.match(/^VQM/) #will skip this one if the line starts VQM
  #everything else as before
end