Home/ Questions/Q 9001449
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-06-16T00:26:16+00:00

Look, people. We have a question about gevent.pool class and pool.wait_available() method, both code

  • 0

Look, people. We have a question about gevent.pool class and pool.wait_available() method, both code snippets

1. 

def fetch(url):
    print 'start fetching...', url
    data = urllib2.urlopen(url)
    print url,':',data.code

urls = ['http://www.google.ru', 'http://www.s-str.ru', 'http://www.vk.com', 'http://www.yandex.ru', 'http://www.xxx.com']

pool = Pool(2)

def producer():
    for url in urls:
        pool.spawn(fetch, url)
    pool.join()

p = gevent.spawn(producer)
p.join()

2. 

def fetch(url):
    print 'start fetching...', url
    data = urllib2.urlopen(url)
    print url,':',data.code

urls = ['http://www.google.ru', 'http://www.s-str.ru', 'http://www.vk.com', 'http://www.yandex.ru', 'http://www.xxx.com']

pool = Pool(2)

def producer():
    for url in urls:
        pool.wait_available()
        pool.spawn(fetch, url)
    pool.join()

p = gevent.spawn(producer)
p.join()

give us similar results:

start fetching... http://www.google.ru
start fetching... http://www.s-str.ru
http://www.google.ru : 200
start fetching... http://www.vk.com
http://www.s-str.ru : 200
start fetching... http://www.yandex.ru
http://www.yandex.ru : 200
start fetching... http://www.xxx.com
http://www.vk.com : 200
http://www.xxx.com : 200

Can anyone explain the meaning of wait_available() method? And possible cases of it’s usage.

=======update========
I already monkey pathched it, it works correctly, all I want to know – is the difference between theese two code snippets.

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    Working with gevent you need to patch standard module before.

    >>> import gevent.monkey
>>> gevent.monkey.patch_all()
>>> ...
>>> p = gevent.spawn(producer)
>>> p.join()
start fetching... http://www.google.ru
start fetching... http://www.s-str.ru
http://www.google.ru : 200
start fetching... http://www.vk.com
http://www.vk.com : 200
start fetching... http://www.yandex.ru
http://www.yandex.ru : 200
start fetching... http://www.xxx.com
http://www.xxx.com : 200
http://www.s-str.ru : 200

    You can see, that pool.wait_available() works predictable.

    Update

    Pool works the same way only for spawn function (it will wait for available “slot” in pool). If you need to provide other functionality based on Pool state (logging, tracing, monitoring) – you definitely will use functions like wait_available, free_count etc. If you only need to spawn new green thread – you can rely on Pool implementation.

    • 0