I’m developing a webcrawler, but often after a short time executing (minutes), some threads

Question

0

Asked: May 23, 20262026-05-23T11:57:43+00:00 2026-05-23T11:57:43+00:00

I’m developing a webcrawler, but often after a short time executing (minutes), some threads

0

I’m developing a webcrawler, but often after a short time executing (minutes), some threads stop to do their work. Running a debugger, I found that it stop in SocketRead0.

This occurs when the thread will download the content of a page with a HttpURLConnection.getInputStream().

I don’t know what causes this, but I think that is associated to the multithreading.

Someone knows how to solve or avoid this?

I’m not using a pool of HttpURLConnection yet beucase I don’t know how to do.

conn = (HttpURLConnection) new URL(url).openConnection();
conn.setInstanceFollowRedirects(true);
conn.connect();
CountingInputStream content;

try {
    content = new CountingInputStream(conn.getInputStream());
    //processing of content
    content.close();
    return true;
    } catch (Exception e) {
        return false;
    }