Home/ Questions/Q 9136179
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-06-17T08:55:09+00:00

I have to work with a csv file that is not directly useable for

  • 0

I have to work with a csv file that is not directly useable for my needs of generating a simple chart. I need to manipulate the file into something “cleaner” and am running into issues and unsure if my overall strategy is correct, as I’m just learning to parse through files with ruby…. My issues here mainly related to me looking for data that is offset from where I’ve found or haven’t found matches. After I find a line which meets criteria, I need to read info from 2 lines after it and manipulate some of it (move something from the last column to the second).

Here’s the original csv file:

component
quantity header,design1,design2,design3,Ref,Units
quantity type,#,#,#,ref#,unit value
component
quantity header,design1,design2,design3,Ref,Units
quantity type,#,#,#,ref#,unit value
component
quantity header,design1,design2,design3,Ref,Units
quantity type,#,#,#,ref#,unit value

Desired output:

Component Header,Quantity type Header,Units Header,design1 header,design2 header,design3 header,Ref header
component,quantity type,unit value,#,#,#,n/a
component,quantity type,unit value,#,#,#,n/a
component,quantity type,unit value,#,#,#,n/a
component,quantity type,unit value,#,#,#,n/a
component,quantity type,unit value,#,#,#,n/a

My ruby script at the moment:

require 'csv'
f = File.new("sp.csv")
o = CSV.open('output.csv', 'w')

f.each_line do |l| #iterate through each line
    data = l.split
    if l !~ /,/ #if the line does not contain a comma it is a component
        o << [data,f.gets] #start writing data, f.gets skips next line but need to skip 2 and split the line to manipulate columns
    else
        o << ['comma'] #just me testing that I can find lines with commas
    end
end

f.gets skips the next line and the documentation isn’t clear to me how to use it to skip 2. After that I THINK I can split that line by commas and manipulate row data with array[column]. Aside from this offset issue I’m also unsure if my general approach is a good strategy

EDIT

Here’s some lines from the real file…. I’ll work through the answers provided and see if I can make it all work. The idea I’ve had is to read and write line by line, vs. converting the whole file to an array and then reading and writing. My thought is that when these files get big, and they do, it’ll take less memory doing it line by line.

THANKS for the help, I’ll work through answers and get back to you.

DCB
Result Quantity,BL::BL,BL::BL_DCB-noHeat,DC1::DC1,DC2::DC2,noHS::noHS,20mmHS::20mmHS,Reference,Units
Avg Temperature,82.915,69.226,78.35,78.383,86.6,85.763,N/A,Celsius
RCB
Result Quantity,BL::BL,BL::BL_DCB-noHeat,DC1::DC1,DC2::DC2,noHS::noHS,20mmHS::20mmHS,Reference,Units
Avg Temperature,76.557,68.779,74.705,74.739,80.22,79.397,N/A,Celsius
Antenna
Result Quantity,BL::BL,BL::BL_DCB-noHeat,DC1::DC1,DC2::DC2,noHS::noHS,20mmHS::20mmHS,Reference,Units
Avg Temperature,69.988,65.045,69.203,69.238,73.567,72.777,N/A,Celsius
PCBA_fiberTray
Result Quantity,BL::BL,BL::BL_DCB-noHeat,DC1::DC1,DC2::DC2,noHS::noHS,20mmHS::20mmHS,Reference,Units
Avg Temperature,66.651,65.904,66.513,66.551,72.516,70.47,N/A,Celsius

EDIT 2

Using some regexp from answers below I developed a line by line strategy to parse through this. I’ll post it as an answer for completeness.

Thanks for helping out and exposing me to methods to develop a solution

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    Code I’m using which creates csv file with everything manipulated… Thanks to those that contributed some help.

    require 'csv'

file_in = File.new('sp1.csv')
file_out = CSV.open('output.csv', 'w')

header = []
row = []


file_in.each_line do |line|

  case line
  when /^[^,]+$/ #Find a component (line with no comma)
    comp_header = file_in.gets.split(',') #header is after component and is split into an arry

    if header.empty? #header
      header.push("Component", comp_header[0], comp_header[-1].strip)
      comp_header[1..-3].each do |h|
        header.push(h)
      end
      file_out << header 

    end
    @comp = line.to_s.strip
    next
  when /,/ #when a row had commas
    puts @comp
    vals = line.split(',') #split up into vals array
    row.push(@comp, vals[0], vals[-1].strip) #add quantity and unit to row array
    vals[1..-3].each do |v| #for values (excluding quanity, units, reference info)
      row.push(v) #add values to row array
    end

  end
    file_out << row #write the current row to csv file
    row = [] #reset the row array to move on to the next component set

end
    • 0