Home/ Questions/Q 7078417
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-05-28T06:32:42+00:00

I’m pretty familiar with PHP including command line, semi-familiar with BASH scripting, and no

  • 0

I’m pretty familiar with PHP including command line, semi-familiar with BASH scripting, and no experience with Perl or other languages but willing to use whatever works.

The HTML file I am trying to parse is 700,000+ lines, 61MB. I cannot change the source that builds the HTML table, only download the entire table via wget http://10.1.1.2/file.pl.

Here’s an example format of the HTML code that I’m trying to parse:

<HTML>
  <HEAD>
    <TITLE>Objects</TITLE>
    <STYLE type="text/css">
    a:hover
    {
    color:red
    }
    </STYLE>
    </HEAD>
  <BODY>
  <IMG src="http://10.1.1.2/images/logo.gif"/>
  <BR/><BR/>
  <TABLE border="0">
    <TR>
      <TH>Objects</TH>
    </TR>
    <TR>
      <TD><HR style="width:227px"></TD>
    </TR>
  </TABLE>
  <table border=1 cellpadding=5 cellspacing=0><tr><th><b>Subtype</b></th><th><b>Object</b>    </th></tr>
    <tr><td>10GigEthernet</td><td>SNFCCAMK34T-TenGigE0/10/0/0</td></tr>
    <tr><td>10GigEthernet</td><td>SNFCCAMK34T-TenGigE0/13/0/0</td></tr>
    <tr><td>10GigEthernet</td><td>SNFCCAMK34T-TenGigE0/13/3/0</td></tr>
    <tr><td>10GigEthernet</td><td>SNFCCAMK34T-TenGigE0/3/0/0</td></tr>
    <tr><td>10GigEthernet</td><td>SNFCCAMK34T-TenGigE0/3/0/0-5</td></tr>
    ... 700,000 more lines ...
  </table>        </BODY>
</HTML>

What I’d like in the CSV:

Subtype,Object
10GigEthernet,SNFCCAMK34T-TenGigE0/10/0/0
10GigEthernet,SNFCCAMK34T-TenGigE0/13/0/0
10GigEthernet,SNFCCAMK34T-TenGigE0/13/3/0
10GigEthernet,SNFCCAMK34T-TenGigE0/3/0/0
10GigEthernet,SNFCCAMK34T-TenGigE0/3/0/0-5

I’d appreciate any help you can give! Thanks in advance.

Result from @shellter’s code:

# wget http://10.1.1.2/reports/file.pl
--2012-01-19 06:56:59--  http://10.1.1.2/reports/file.pl
Connecting to 10.1.1.2... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified
Saving to: `file.pl'

    [          <=>                          ] 61,000,000  1.01M/s   in 58s     

2012-01-19 06:58:00 (1.01 MB/s) - `file.pl' saved [61000000]

# sed -n '/<\/td>/{
>            s@<tr><td>@@;
>            s@</td>@XaYbZc@;
>            s@<td>@@;
>            s@</td></tr>@@;
>            s/XaYbZc/,/
>            s/^    //
>            p
>           }' file.pl > routerList.csv
# ls -l
total 203408
-rw-r--r--   1 root     root     61000000 Jan 19 06:58 file.pl
-rw-r--r--   1 root     root     42708247 Jan 19 06:58 routerList.csv
# head routerList.csv
10GigEthernetn,SNFCCAMK34T-TenGigE0/10/0/0
10GigEthernetn,SNFCCAMK34T-TenGigE0/13/0/0
10GigEthernetn,SNFCCAMK34T-TenGigE0/13/3/0
10GigEthernetn,SNFCCAMK34T-TenGigE0/3/0/0
10GigEthernetn,SNFCCAMK34T-TenGigE0/3/0/0-5
Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    While I have to agree with most of the comments like ‘use a DOM, or XPATH, etc.’,
    you are lucky in this case that all data you want to process is on one line. If there are ever linebreaks anywhere in that data, then this will not work AND it will be essentially impossible to get a working solution is sed. So forwarned of these issues, try this

     wget http://10.1.1.2/file.pl

 sed -n '/<\/td>/{
           s@<tr><td>@@;
           s@</td>@XaYbZc@;
           s@<td>@@;
           s@</td></tr>@@;
           s/XaYbZc/,/
           s/^    //
           p
          }' file.pl > routerList.csv

cat routerList.csv
10GigEthernet,SNFCCAMK34T-TenGigE0/10/0/0
10GigEthernet,SNFCCAMK34T-TenGigE0/13/0/0
10GigEthernet,SNFCCAMK34T-TenGigE0/13/3/0
10GigEthernet,SNFCCAMK34T-TenGigE0/3/0/0
10GigEthernet,SNFCCAMK34T-TenGigE0/3/0/0-5

    The sed script is using the ‘@’ char as the match/replace section delimiter.

    First we take the first <tr><td> on the line and delete it,

    We then take the first </td> and replace it with XaYbZc as a temp marker.

    Remove the remaining opening <td>.

    Remove the trailing </td></tr>

    Replace the temporary XaYbZc with the ‘,’

    Remove 4 spaces at the front of the line.

    Print the buffer. (Done!)

    I hope this helps.

    • 0