Question

How would i go about using c to write a program to strip a webpage inputted in via the terminal in Linux and remove all the HTML tags and images and put the remaining content into a file.

I am just confused on how to get the webpage from the terminal.

Thanks for your help.

Answer 1

• You can get a web page in terminal by various programs such as curl, wget, aria2c etc. Download webpage using those program use write your C program to strip tags.

• If you want to download webpage using C. You can use libcurl. To get sample code how to use libcurl to download http://stackoverflow.com use the following command.

  curl --libcurl downloadstackoverflow.c  http://stackoverflow.com
  

  This will generate a file downloadstackoverflow.c which contains sample code.

• This is a simple program that strips tags from html. It does not support tags in quotes ". But you should get the idea.

  int main() {
     int in_tag = 0;
     char c;
  
     while ((c = getchar()) != EOF) {
         if (c == '<' || c == '>') {
             in_tag = (c == '<') ? 1 : 0;
         } else {
             if (!in_tag) {
                 putchar(c);
             }
         }
     } }