Home/ Questions/Q 521167
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-05-13T08:13:18+00:00

The scenario: We have a python script that checks thousands of proxys simultaneously. The

  • 0

The scenario: We have a python script that checks thousands of proxys simultaneously.
The program uses threads, 1 per proxy, to speed the process. When it reaches the 1007 thread, the script crashes because of the thread limit.

My solution is: A global variable that gets incremented when a thread spawns and decrements when a thread finishes. The function which spawns the threads monitors the variable so that the limit is not reached.

What will your solution be, friends?

Thanks for the answers.

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    You want to do non-blocking I/O with the select module.

    There are a couple of different specific techniques. select.select should work for every major platform. There are other variations that are more efficient (and could matter if you are checking tens of thousands of connections simultaneously) but you will then need to write the code for you specific platform.

    • 0