Sign Up

Sign Up to our social questions and Answers Engine to ask questions, answer people’s questions, and connect with other people.

Sign In

Login to our social questions & Answers Engine to ask questions answer people’s questions & connect with other people.

Forgot Password

Lost your password? Please enter your email address. You will receive a link and will create a new password via email.

You must login to ask a question.

Please briefly explain why you feel this question should be reported.

Please briefly explain why you feel this answer should be reported.

Please briefly explain why you feel this user should be reported.

Search

Ask A Question

0

Editorial Team

Asked: June 17, 20262026-06-17T02:43:55+00:00 2026-06-17T02:43:55+00:00

I want to crawl a page and I need to take only the HTML

0

I want to crawl a page and I need to take only the HTML itself, avoiding all images/videos etc…
Is it possible to do this?
Thanks in advance.

Report

Leave an answer
Cancel reply

You must login to add an answer.

Need An Account,

1 Answer

Editorial Team
2026-06-17T02:43:56+00:00Added an answer on June 17, 2026 at 2:43 am
Check regex-urlfilter.txt file.

You can include the extensions of the file extensions which you dont want to index. e.g.

# skip image and other suffixes we can't yet parse # for a more extensive coverage use the urlfilter-suffix plugin -\.(gif|GIF|jpg|JPG|png|PNG|ico|ICO|css|CSS|sit|SIT|eps|EPS|wmf|WMF|zip|ZIP|ppt|PPT|mpg|MPG|xls|XLS|gz|GZ|rpm|RPM|tgz|TGZ|mov|MOV|exe|EXE|jpeg|JPEG|bmp|BMP|js|JS)$
0

Reply

Share
Share

Share on Facebook

Share on Twitter

Share on LinkedIn

Share on WhatsApp

Report