Home/ Questions/Q 4119822
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-05-20T23:08:24+00:00

I would like to use a web crawler and crawl a particular website. The

  • 0

I would like to use a web crawler and crawl a particular website. The website is a learning management system where many student upload their assignments,project presentations and so on. My question is that can i use a web crawler and download the files that have been uploaded in the learning management system. After i download them i would like to create an index on them so as to query the set of documents. User can use my application as a search engine. Can a crawler does this? I know about webeater ( Crawler written in Java )

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team
    1. Download the files in Java SingleThread.
    2. Parse the files (you can get idea from parse plugins of nutch).
    3. Create index with lucene
    • 0